|
In today's newsletter:
- The operating system for AI research labs!
- Why agent crashes are nothing like database crashes.
- [Hands-on] Deploy a Qwen 3 Agentic RAG.
|
|
|
OPEN-SOURCE
Transformer Lab is an open-source ML platform that orchestrates GPUs across any cloud and runs any training or eval workflow you define:
- supports LoRA, QLoRA, DPO, ORPO, SIMPO
- use it from a GUI, CLI, or agent skill.
- works with MLX, vLLM, Ollama, and HF Transformers
- one-click convert between HF, GGUF, and MLX
- diffusion training and inference built in
- LLM-as-a-judge evals plus the EleutherAI harness
- submits jobs to Slurm and SkyPilot from the same UI
- runs on Apple Silicon, NVIDIA, and AMD
The same interface a solo researcher uses on a MacBook also drives a 64-GPU cluster. Nothing else does that today.
Find the GitHub repo here → (don’t forget to star it ⭐️)
|
|
|
AGENTS
When a database crashes and restarts, it replays the write-ahead log and reconstructs the exact same state, down to every transaction, row, and index entry.
This deterministic recovery is a solved problem in databases and has been for decades.
Agent crashes don’t work this way, and just restarting it is not the ideal option.
Let’s dive in to understand the problem and also see how Google Cloud’s new Agent Platform has engineered the infra to solve it.
The decision drift problem
Consider an agent that processed financial records over five days.
- On day 1, it ingested 4,000 records and normalized formats. One record has an ambiguous date field (“
03/04/2026”). The agent interprets it as March 4th. Every downstream decision, the cross-referencing on day 2, the anomaly detection on day 3, builds on that interpretation.
- On day 3, the agent crashes.
A stateless restart would involve re-ingesting all 4,000 records from scratch.
Now, the LLM may interpret that same ambiguous date as April 3rd.
That single different decision influences every subsequent step. This is what makes agent state fundamentally different from database state.
Checkpoint-and-resume strategy for Agents
The fix is conceptually simple.
The agent should periodically save its intermediate state, including progress, accumulated decisions, and the reasoning chain that led to them. If it crashes, it reloads the last checkpoint and continues with every prior decision intact.
Technically, here’s what you’d need to do:
- Implement persistent state serialization across sessions
- Manage storage and retrieval
- Reconstruct the agent’s full context window exactly as it was before the crash
- Handle edge cases like human-in-the-loop approvals, where the agent needs to pause indefinitely without consuming compute
If you want to see it in practice, Google Cloud’s Agent Platform now actually ships this as a native platform capability through three mechanisms that work together.
- Memory Bank gives agents a persistent state that accumulates across sessions. So the agent on day 5 can read what it decided on day 1 without re-processing. Under the hood, it uses scoped memory extraction, converting conversation history and intermediate decisions into searchable memories that persist and consolidate automatically.
- Resume Agents handle checkpoint-and-resume natively within the ADK. When an agent is interrupted (due to a crash, timeout, or human approval gate), Agent Runtime saves the full state. When it resumes, the context reloads exactly as it was. In the ADK, a basic resumable agent definition looks like this:
- Ambient Agents make execution event-driven. Instead of waiting for a user prompt, the agent activates when new data arrives. For instance, an invoice processing agent can trigger on incoming records, not on a human typing a query.
Why care?
The deeper point here isn’t about Google’s specific implementation.
It’s that the industry has been treating agent memory as a retrieval problem (RAG, vector search, context windows) when the harder problem is actually a consistency problem.
An agent that processes data over days needs the same guarantees that databases have provided for decades, i.e., if it crashes, the recovered state must be identical to the pre-crash state.
LLM non-determinism makes that impossible without explicit checkpointing.
Agent Platform is the first infrastructure that treats this as a platform-level concern rather than leaving it to application developers to solve with custom Redis pipelines and state serialization code.
You can try the Agent Platform here →
👉 Over to you: what’s the longest-running agent workflow you’ve tried to build, and where did state management bite you?
Thanks to Google Cloud for working with us on today’s issue!
|
|
|
HANDS-ON
Deploy a Qwen 3 Agentic RAG
Let's learn how to deploy an Agentic RAG powered by Alibaba's Qwen 3.
Here's our tool stack:
- CrewAI for Agent orchestration.
- Firecrawl for web search.
- LightningAI's LitServe for deployment.
The diagram shows our Agentic RAG flow:
- The Retriever Agent accepts the user query.
- It invokes a relevant tool (Firecrawl web search or vector DB tool) to get context and generate insights.
- The Writer Agent generates a response.
Next, let's implement and deploy it!
Here's the entire code to serve our Agentic RAG.
- The
setup method orchestrates the Agents.
- The
decode_request method prepares the input.
- The
predict method invokes the Crew.
- The
encode_response method sends the response back.
Let's understand it step by step below.
Set up LLM
CrewAI seamlessly integrates with all popular LLMs and providers.
Here's how we set up a local Qwen 3 via Ollama.
Define Research Agent and Task
This Agent accepts the user query and retrieves the relevant context using a vectorDB tool and a web search tool powered by Firecrawl.
Again, put this in the LitServe setup() method:
Define Writer Agent and Task
Next, the Writer Agent accepts the insights from the Researcher Agent to generate a response.
Yet again, we add this in the LitServe setup method:
Set up the Crew
Once we have defined the Agents and their tasks, we orchestrate them into a crew using CrewAI and put that into a setup method.
Decode request
With that, we have orchestrated the Agentic RAG workflow, which will be executed upon an incoming request.
Next, from the incoming request body, we extract the user query.
Check the highlighted code below:
Predict
We use the decoded user query and pass it to the Crew defined earlier to generate a response from the model.
Check the highlighted code below:
Encode response
Here, we can post-process the response & send it back to the client.
Note: LitServe internally invokes these methods in order: decode_request → predict → encode_request.
Check the highlighted code below:
With that, we are done with the server code.
Next, we have the basic client code to invoke the API we created using the requests Python library:
Done!
We have deployed our fully private Qwen 3 Agentic RAG using LitServe. Here's a recording of our deployed Qwen3 Agentic RAG:
That said, we started a crash course to help you implement reliable Agentic systems, understand the underlying challenges, and develop expertise in building Agentic apps on LLMs, which every industry cares about now.
Here’s what we have done in the crash course (with implementation):
- In Part 1, we covered the fundamentals of Agentic systems, understanding how AI agents act autonomously to perform tasks.
- In Part 2, we extended Agent capabilities by integrating custom tools, using structured outputs, and we also built modular Crews.
- In Part 3, we focused on Flows, learning about state management, flow control, and integrating a Crew into a Flow.
- In Part 4, we extended these concepts into real-world multi-agent, multi-crew Flow projects.
- In Part 5 and Part 6, we moved into advanced techniques that make AI agents more robust, dynamic, and adaptable, like Guardrails, Async execution, Callbacks, Human-in-the-loop, Multimodal Agents, and more.
- In Part 7, we covered Knowledge of agentic Systems.
- In Part 8 and Part 9, we primarily focused on 5 types of Memory for AI agents, which help agents “remember” and utilize past information.
- In Part 10, we implemented the ReAct pattern from scratch.
- In Part 11, we implemented the Planning pattern from scratch.
- In Part 12, we implemented the Multi-agent pattern from scratch.
- In Part 13 and Part 14, we covered 10 practical steps to improve Agentic systems.
- And in Part 15, Part 16 and Part 17, we covered practical ways to optimize the Agent’s memory in production use cases.
Of course, if you have never worked with LLMs, that’s okay. We cover everything in a practical and beginner-friendly way.
You can find the code in this GitHub repo →
|
|
|
|
|
PARTNER WITH US
ADVERTISE TO 950k+ AI Professionals
Our newsletter puts your products and services directly in front of an audience that matters, including thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., around the world.
Get in touch today by replying to this email.
|
|
|
