A while back I spent a few weeks building an AI observability product from scratch. We pivoted before it shipped. But the system itself was the most interesting thing I have built, so here is how it came together.
01 The problem: nobody sees the whole user
A SaaS company's picture of its users is split across a dozen tools.
PostHog. Events, funnels, heatmaps, where people stall.
Stripe. Plan, MRR, failed charges, who is actually worth money.
Sentry. What broke, how often, and for whom.
Intercom. What people complain about in their own words.
Each tool is good at its own slice. None of them sees the user.
So the churn signal sits in four places at once. A customer hits the same Sentry error three days running, opens an Intercom ticket about it, stops using the feature in PostHog, and then downgrades in Stripe. Four systems each saw a quarter of that story. Nobody saw the story.
The goal was to merge all of it into one profile, and put an AI agent on top that continuously understands what is happening with each user.
02 Integrations: don't rebuild OAuth
The obvious first wall is the integration layer. Every tool has its own OAuth dance, its own token refresh, its own rate limits, its own idea of what a "user" is. Building and maintaining that per tool is a full-time job that produces no product.
So we used Composio for the integration layer instead. A company connects any of 980+ tools in one click, and we never touch a refresh token.
Not everything lives in Composio, though. For providers outside it -- Braintrust, for example -- there is a separate API-key path: a provider registry that describes each one, plus live key validation so a bad key fails at connect time instead of silently at 3am inside a background job.
The integration layer is the part everyone underestimates and the part that decides whether you ship. Buy it if you can, and keep a hand-rolled escape hatch for the providers nobody has wrapped yet.
03 The Go analysis pipeline
Two languages, split on what each is actually good at. Node/Express handles the public API, auth, and everything the dashboard talks to. Go handles the heavy analysis pipeline. Jobs run through Asynq on Redis.
Composio + API keys → Asynq queue on Redis → one goroutine per adapter → identity resolution → canonical profile in JSONB
Each connected source has its own adapter, and they all run in parallel. This is where Go earns its place: pulling six APIs at once is the whole job, and the concurrency is free.
JSONB instead of a table per tool
Data lands in a flexible JSONB model rather than per-tool tables. Every integration would otherwise mean a migration, and the shape of a Stripe customer has nothing in common with the shape of a Sentry issue. JSONB means adding a source is config, not schema work.
Aggregate before you ingest
High-volume sources like PostHog get aggregated before ingestion. Storing every raw event per user is how you end up paying more for your observability tool than for the product it observes.
Identity resolution
Records across sources get resolved into one canonical user profile, carrying health scores, churn-risk flags, and power-user signals. This is the step that turns four quarter-stories into one.
And because the resolved profile is stored, re-analysis runs against it instead of re-querying every source. That path is about 5x faster. Which matters more than it sounds -- it is the difference between an agent that can re-check a user on every event and one that can only run nightly.
04 An agent behind every user
This became the core idea. Every user gets an AI agent, built on the OpenAI Agents SDK.
The agent watches product activity, PostHog behaviour and heatmaps, payments, errors, and support conversations. It produces an ongoing analysis: what this user is doing, where they are dropping off, whether they are at risk, and what could be done about it.
Responses stream over WebSockets. Tools are scoped per tenant, and tenant isolation is enforced during the socket handshake rather than inside each tool -- if you check it per tool, you will eventually add a tool and forget.
A separate agent scans GitHub, X, Hacker News, and Reddit and writes a cited sentiment memo about the company. Cited, because an uncited memo about what the internet thinks of you is just a vibe.
05 An AI PM team
Per-user agents tell you about one user. Teams need someone watching a cohort. So teams can create autonomous PM agents -- Churn, Growth, Activation, Reliability, or one they define themselves.
The rating loop is the interesting part. An agent that produces reports nobody grades will produce the same mediocre report forever. Putting the rating button inside Slack, next to the report, is what makes the feedback actually arrive.
06 Triage: turning signals into issues
An observation that nobody acts on is not observability. So Sentry errors, PostHog drop-offs, and Intercom support load automatically become draft incidents.
| Signal | Becomes | Routed to |
|---|---|---|
| Sentry error spike | Draft incident with root-cause analysis | GitHub issue |
| PostHog drop-off | Draft incident with affected users | Linear issue |
| Intercom support load | Draft incident with revenue at risk | GitHub or Linear |
Each draft carries a root-cause analysis, the affected users, and the estimated revenue at risk. That last number is the one that gets an issue prioritised, and it only exists because payments and product usage finally sat in the same profile.
From there it goes to GitHub for engineering issues or Linear for product issues. The agent does not close the loop by itself. It just makes the first draft, which is the part humans procrastinate on.
07 And then we pivoted
The same agent also lives in Slack, with memory and reminders, so you can just ask it about a user instead of opening a dashboard.
Then we pivoted, and none of this shipped.
Which is fine. I got to build the thing properly -- Composio, Go with Asynq, JSONB modelling, identity resolution, the OpenAI Agents SDK, per-tenant WebSocket isolation -- and I learned more in those few weeks than in a quarter of maintaining something safe.
The idea has stayed with me, though. Not "one dashboard for everything." That has been tried and it always ends up as a twelfth tool nobody opens. The idea worth keeping is smaller and stranger: give every single user their own agent, and let it watch.