Credential Sentinel: An Agent That Reads Widely and Acts Narrowly

Most of the noise about AI agents is about what they can do. The part I find more interesting is what they cannot do, by design, and Credential Sentinel is the project where I tried to actually build that idea instead of just writing about it.

The problem it solves is unglamorous. Every company has credentials that no rotation service is tracking. The forgotten token, the service account key left over from a migration, the cert nobody will admit to owning. The rotation itself is a solved problem, since AWS Secrets Manager and Vault and cert-manager all handle that fine. The hard part is the layer around it: working out which credentials exist, which ones are about to bite you, and dealing with that messy tail without doing anything you cannot take back. So the agent reads widely and acts narrowly. It looks at almost everything, and changes almost nothing without a human saying yes first.

Under the hood it is a LangGraph state machine that walks each credential through a fixed pipeline. Discover, reconcile against what the rotation services already manage, assess the risk, prioritize, draft a plan, and only then touch anything. Built into that graph are two hard stops, interrupt gates where the whole run pauses and waits for a person. One before it stages a new credential, one before it cuts over to it. The trick I like most here is that the API in front of the agent is completely stateless while the graph behind it is stateful. Each run’s id doubles as its thread id, so every call just resumes from the last saved checkpoint. You can walk away in the middle of a rotation, come back an hour later, and the agent picks up exactly where it paused.

The full control flow. Everything down the left edge is the safe exit, a managed and rotating credential gets deferred and logged. Everything on the right is the careful path to a rotation, with a human gate before staging and another before cutover.

Here is a small detail I am unreasonably fond of. The checkpointer and the live event log live in two separate SQLite files, on purpose. The checkpointer holds a write lock across those long human pauses, and if the event log shared the same file the two would eventually deadlock waiting for it. Splitting them gives each file exactly one writer, so they never fight. That is the kind of bug you find once, at a bad hour, and design around forever.

I also wanted at least one piece of discovery to be genuinely real rather than simulated. So the agent does an actual TLS handshake against live endpoints, reads the expiry date straight off the certificate, and feeds it into the urgency score. To prove it works, I point it at expired.badssl.com, a real site that intentionally serves an expired certificate, and watch the agent flag it as already dead. The reconciliation that follows is a small, honest decision tree. Managed and rotating means defer and leave it alone. In a store but not rotating means stale, and it needs owning. Absent everywhere means unmanaged and at risk. And anything the agent cannot classify gets marked unknown and escalated to a human, never quietly assumed safe.

The write path is where I was most careful, because that is the part with teeth. When the agent finally rotates something, it follows a delayed-revoke order: promote the new credential, repoint the consumers, verify everything is healthy, and only then revoke the old one. The old credential stays valid as a safety net until the new one has proven itself. If verification fails after cutover, the agent rolls back on its own, repoints everyone to the still-good old credential, and escalates. Nothing gets stranded halfway, and getting that automatic rollback working cleanly end to end was one of the more satisfying parts of the build.

There is also a memory that survives across runs, which is what lifts it above a one-shot script. Every sweep writes a compact summary of what it found, and the next run diffs against it to surface what changed. Newly discovered unmanaged credentials, coverage that shifted, and my favorite, items stuck across multiple cycles. A credential that keeps showing up unrotated week after week is exactly what a tired on-call engineer stops seeing, so I wanted the agent to keep seeing it for them. That same event log doubles as an audit trail of the run.

None of this lands if you cannot watch it happen, so the frontend exists to make the agent’s reasoning legible. It is a Next.js and shadcn interface: a live activity feed streamed over SSE, a reconciliation table, the two approval gates carrying the full context a human needs to decide, and dedicated panels for staging, cutover, drift, and the report. The SSE stream replays its event log on reconnect, so refreshing mid-run loses nothing. There is also a real-browser Playwright test that boots both servers, clicks through both gates, and watches one credential cut over cleanly while another deliberately rolls back.

The whole stack in one view. The discovery and inventory sources are mostly simulated, with one honest exception: the TLS checker does a real handshake. The agent sits in the middle with its two interrupt gates, and the SQLite layer holds the checkpointer, the event log, and the audit trail.

There was one more gap I did not want to leave open. Getting an agent to reason well is one problem; getting it to run the way real software runs is a separate one I find just as absorbing, so I kept going into the less glamorous half: making it run anywhere, hold up under concurrent traffic, and stay watchable while it does. Both services are containerized with multi-stage Docker builds, so the frontend image ships only its compiled output and lands about ten times smaller than a naive one. The agent reaches its LLM through an OpenAI-compatible client pointed at an environment variable, which means swapping the hosted Nebius API for a self-hosted vLLM server on my own GPU is one line of config and zero code changes. I wrote the Kubernetes manifests too, with the awkward real-world bits handled honestly: the backend pinned to a single replica because SQLite has one writer, GPU nodes selected and tolerated for the vLLM pod, and an ingress tuned so the minutes-long SSE stream does not get quietly killed by a proxy timeout. Prometheus scrapes a metrics endpoint so I can watch P95 latency instead of guessing at it. And the whole thing runs live on Fly.io, because a system that only exists in a repo never quite feels finished to me. That last stretch turned into its own body of work, enough that it earned its own writeup.

Put together, it is one stubborn idea repeated at every layer. Give the agent wide eyes and narrow hands. Let it find everything, decide carefully, and make a human bless anything it cannot undo. That is the version of an agent I would actually point at production secrets.

Then I spent the next stretch trying to break it on purpose and score it honestly, but that turned into its own post.