Companies that have tried to build their own SRE agents often ask us how we stop the agent from starting every investigation from square one. Their agent explores, runs queries that mostly come back empty, and does the same exploration again on the next incident because nothing it learns is captured. Every run costs time and tokens, and lands somewhere slightly different.
What those agents are missing is a persistent understanding of the environment they work in. Most people call that a context graph. The term gets used loosely, and a principal engineer on a call pushed me on the term. Did I mean a service dependency graph extracted from his code? And, if so, was I getting there through static analysis or with an agent?
An agent, I told him, and what it produces is a logical picture of the system. It represents the application logic and the architecture: what components exist, what depends on what, where each one runs, what changed recently. It knows that checkout depends on a specific database. It does not track how many pods are running behind them right now, and it does not need to. When an alert fires, the agent uses that picture to cut the search space down to the components that could plausibly be involved.
Teams that build their own SRE agent rarely think about the context graph. It also turns out to be both the hardest and most important part for getting accurate RCAs. They wire the agent to a few MCP servers and it returns something useful on an early incident. Even if the agent got lucky, nothing in the setup tells anyone that. Run it across enough incidents and the agent clearly misses things and hallucinates. Most of the time, teams end up running their agent (or even ones they've paid for) in the background during an incident while they investigate by hand, then compare results at the end. The hand investigation is the one they act on. Teams stay curious about the agent but don't trust it, because it isn't right often enough. I recently wrote about the one job a home-built agent does reasonably well, which is simple knowledge retrieval from a single system.
So the context graph is the thing to build, but most teams don't. If you go for it, here are the three areas our own engineering team initially struggled with when building an AI SRE product. For each, I'll define the problem, why the obvious approach doesn't work, and how we handle it now.
Deciding what counts as one thing
Your systems can refer to the same database by many names. For example, the application code calls it payments-db. The Terraform calls it payments-postgres. The GCP console calls it cloud-sql-payments. An engineer looking at those three strings knows they are one database. There is no string comparison that reaches the same conclusion.
Get this wrong and the graph contains one database as three separate components. The agent traces a failure, reports that the affected service depends on payments-db, and finds no relationship to cloud-sql-payments, which is where the problem actually is. The output is specific, confident, and wrong.
Merging too aggressively causes the opposite failure. Two components that are genuinely distinct become one node, and the agent reasons about something that does not exist.
So identity has to be resolved on meaning rather than spelling. Our approach is to over-discover on purpose. Every name found anywhere becomes a candidate component. Candidates get verified in parallel, and duplicates get merged so that one entity produces one node. Every merge stores the evidence that justified it, because a bad merge is a difficult bug to find later.
This is the same class of problem as entity resolution in a customer database. Bob Smith, Robert Smith, and rsmith@acme.com are one person. Every report you run before you resolve that is incorrect.
Keeping the graph true to what is running
The graph is accurate on the day you build it. Then someone deploys.
A graph built once starts drifting immediately. It works for a few weeks. The drift is hard to perceive at first, because a stale graph returns answers rather than errors. The answers were correct at some point.
An alert fires. The agent finds a PR that touched the relevant code four hours ago and reports it as the likely cause. That PR merged to main and never shipped, because the release process batches deploys, or the pipeline failed, or someone held it. The agent has produced a plausible answer pointing at code that is not running, and the on-call engineer spends twenty minutes reading a diff that has nothing to do with the incident. Knowing that a PR was merged comes from the source control integration. Knowing that the code is live comes from CI/CD, a different system with different data. Teams building their own tend to wire up the first and then assume it covers the second.
Three things address this. First, the graph has to rebuild itself on a schedule without anyone triggering it. Second, it has to be versioned, because a large share of incidents trace to a recent change and a diff against the previous deploy is one of the most useful inputs an investigation can have. Third, failed tool calls have to feed back into it. A query against a label that no longer exists is a failure and also a precise signal that the graph is wrong in a specific place.
Storage has to match the questions
On a different call, an engineer asked me whether he could export the context graph. There is no single export, because the graph lives across four systems: text search, vector search, a graph structure inside a relational database, and key value storage.
Different questions about the same graph need different access patterns. "What depends on this service?" is a graph traversal. "What have we recorded about this service?" is semantic search. "What is this service's current deployment state?" is a key lookup. Force all those through one store and they get slow.
The engineer who asked had built an internal tool at his own company, with his own vector store and several backends behind it. When I explained the split, he said he had run into the same thing. His tool does offline processing rather than joining an incident as it happens, so it answers a different set of questions, and still needs more than one store to answer them.
Where this leaves you
All three of the challenges I described can be solved by a competent team with enough effort. The real questions are how much effort it takes to get there, what the ongoing maintenance looks like, and what your team stops doing to make room for it. What we've found is that maintaining an AI SRE requires continuous work. The models underneath you change, the systems you're describing change, and every change is a correction someone has to make or automate.
If your team is building its own AI SRE, I'm happy to chat about how to make it work better or whether Herald could make sense for you. Just reach out!