Team Memory: A Shared, Permission-Aware Memory for Developers and Their AI Agents

Written by Tyler La Fronz on October 23, 2026

The Problem

Agent sessions start cold. A decision made in one developer’s session (“we chose Calico over Cilium, and here’s why”) is invisible to the next developer’s agent, and to my own agent on a different machine. Chat history and per-user memory files don’t travel across people, repositories, or clients.

There’s a second problem that bothered me more. When an agent does write notes, nothing distinguishes a guess from an official standard. One agent invents a convention, writes it down, and every other agent treats it as fact. Memory that agents can write to and trust equally is a way to launder guesses into policy.

AGENTS.md and CLAUDE.md files cover part of this, and they’re shared through git. What they don’t give you is searchable cross-session knowledge, explicit review, or access control that works across clients and projects.

What I Built

Team Memory is a self-hosted shared memory service, written in Go and served over MCP (Model Context Protocol). Claude Code, Codex, or any MCP-compatible client connects to it and reads and writes one body of team knowledge.

  • Versioned memories, not transcripts. Five kinds (fact, decision, convention, lesson, context), each with immutable revisions, lifecycle events, optimistic concurrency, and supersession instead of silent overwrite.
  • Trust levels. Every memory is unverified, cited (a source link was supplied, not checked), or approved. Search results always carry the trust level, author, revision, and source.
  • Hierarchical scopes. Organization → team → project. Knowledge flows down only: a project sees its team’s and organization’s memories, labeled with where they came from, and never the reverse. Moving a memory up takes an explicit, human-approved promotion, one level at a time.
  • Hybrid search. PostgreSQL full-text plus pgvector similarity, fused with Reciprocal Rank Fusion and filtered to the caller’s scope chain before ranking. If the embedding model is unreachable, search falls back to full-text and says so in the response.
  • A web portal for browsing, searching, and reviewing, where maintainers approve or reject what agents propose.
  • Optional one-way git mirror. Every exportable lifecycle event becomes a commit in a private repo, written with go-git so the container needs no git binary.

The point is that a developer in Claude Code can record a decision, a maintainer can approve that exact revision in the portal, and a teammate in Codex can later ask “why did we choose PostgreSQL?” and get the decision, rationale, source, and approval status.

Architecture diagram: Claude Code and Codex connect to the /mcp endpoint, where memory_create lands memories as unverified and memory_search returns trust level, origin scope and source. A maintainer in the browser approves exact revisions through the /portal, the only path that can change trust. Both entry points go to the Go server, which does auth and role checks, the memory engine and hybrid search, backed by PostgreSQL with pgvector, forced row-level security, an audit log and a downward-only scope tree of organization, team and project. An embedding worker claims jobs from PostgreSQL and calls a local Ollama.

The Design Decision I Care Most About

Agents can propose, but only a person can approve. memory_approve over MCP only requests review of an exact revision. It never changes trust. Approval, rejection, and promotion happen only in the browser portal, bound to the specific revision a person looked at. If an agent edits a memory after review is requested, the approval no longer applies.

Related to that, everything coming back from memory is treated as untrusted. Memory is written by people and agents, so it can contain instructions. Every search result that carries content starts with an UNTRUSTED DATA notice, and the agent policy I wrote for my own repositories says to treat memory as a claim to verify against the code, not an order to follow. This repo’s AGENTS.md tells Claude exactly that, which is how the project is used day to day.

Architecture

  • Runtime: Go, a single binary containing the server, the embedding worker, and the operator CLI.
  • Storage: PostgreSQL 17 with pgvector. No separate vector database.
  • Transport: real MCP over Streamable HTTP (JSON-RPC, capability negotiation, tools, resources, OAuth discovery), not a REST API relabelled as MCP. Eight tools and four resource views.
  • Embeddings: a local Ollama (nomic-embed-text by default), through a transactional outbox in PostgreSQL and a worker with retries, backoff, and dead-lettering. A slow or down model server never blocks a write.
  • Auth: personal access tokens (shown once, stored hashed, expiring, revocable) for solo use, plus optional SSO where Team Memory is the OAuth 2.1 authorization server for MCP clients and delegates sign-in to any OpenID Connect provider.
  • Deployment: one distroless container image and a Helm chart with NetworkPolicies and a migration hook, Prometheus metrics, OpenTelemetry traces, example alerts, and pgBackRest backups with a measured restore drill.

Security in Layers

I wanted tenant isolation to hold even if I wrote a bug in the application code, so most of it lives in the database:

  • Row-level security forced on every tenant table. The server refuses to start as the database owner or any role that bypasses RLS.
  • Separate database roles for the server, the worker, and the operator.
  • Role checks per tool (reader, writer, maintainer), scoped by grant, with an optional cascade to descendants.
  • An append-only audit log, secret scanning on writes, request limits, and rate limiting per principal.
  • Adversarial tests for cross-tenant reads and for downward-only inheritance, plus a written threat model with residual risks listed.

Making Search Honest

I wrote an evaluation harness before tuning anything: 67 synthetic memories across 8 scopes and 42 labeled queries (identifiers, acronyms, repository names, paraphrases, plus unanswerable queries and conflict pairs). A few findings changed what I built:

  • Hybrid beat either retriever alone with the default model (recall@5 of 0.917 against 0.893 for full-text and 0.810 for vector). Full-text was perfect on identifiers like DB_POOL_MAX_CONNS; vector search was weak there and helped on paraphrases.
  • The biggest single gain wasn’t vectors. My first full-text version required every query term, which scored 0.059 recall on paraphrases. OR-ing the terms took overall full-text recall from 0.560 to 0.893.
  • A similarity floor matters. Without one, unanswerable queries returned ten confident-looking results.
  • A larger embedding model was better on this set (0.988 vs 0.810 vector recall@5), but I kept the smaller default. I wrote the corpus and labels myself, tuned the floors on the same queries, and the set is small, so the report says plainly that the numbers compare modes and not real-world quality.

Ranking is still mediocre: relevant memories usually land in the top 5, but often not first.

Built in Stages, With Claude and Codex

I built it with both Claude Code and Codex, which suited a project whose whole point is that different agents share one memory.

The project went from first commit to a tagged release in four days, October 20–23, about 32,000 lines of Go, broken into seven build stages: foundation, retrieval, identity, team security, web portal, operational hardening, and release. Each stage had a written prompt, checkable acceptance criteria, and a handoff document recording what was delivered and what wasn’t.

What worked, as with my other AI-assisted builds:

  • A proposal and a review-decisions file attached to every stage, so each new session started from the agreed design instead of rediscovering it.
  • Acceptance criteria that could fail. Isolation, inheritance, and approval rules are enforced by tests against a real PostgreSQL. Each integration test creates and drops its own database.
  • Measuring, not promising. The load-test report states which targets were met and which weren’t. Search p95 excluding query embedding was 8 / 154 / 60 ms at concurrency 1 / 8 / 32, against a 500 ms target. Including query embedding on a CPU-only laptop it was over 2 seconds, which the report calls out instead of burying. One write-latency run measured 261 ms and an earlier run of the same configuration measured 550 ms, and both numbers are in the report.
  • Review before release. An external-style review pass caught real problems, including OAuth registration validation and a release workflow that tagged the image differently from what the Helm chart deployed.

Honest Status

This is a personal proof of concept, and the repository says so at the top. It has not had an external security review, none is planned, and it hasn’t been used by more than one real person. The multi-identity evidence comes from synthetic test identities, which I don’t count as team adoption. I’ve used both Claude Code and Codex to build it and connected both to it as clients, and verified both work.

What I Learned

The hard part of shared agent memory is deciding what it’s allowed to claim. Storing and retrieving text is easy; the design work was in trust levels, revision-bound approval, and downward-only inheritance, so the system can say “an agent wrote this yesterday” differently from “a maintainer approved this exact text.”

Putting isolation in the database also changed how I wrote everything else. Once PostgreSQL enforces who can see what, application bugs become less scary and tests become simpler to write.

What’s Next

A real pilot is the main gap: using it on an actual project with more than one person, then rerunning the evaluation on real queries instead of my synthetic ones. After that, better ranking (relevant results should come first, not just appear), and making the portal’s role management writable instead of CLI-only.

More Information

  • Team Memory on GitHub, including the README, threat model, and operational runbooks
  • Standards API, an earlier AI-assisted project built around feeding standards to code-review agents