[{"slug":"orchestack-architecture","title":"Orchestack: Multi-tenant Orchestration for Agent Systems","date":"2026-05-22","summary":"What Orchestack is, how it's built, and what it integrates with. The orchestration layer for everything else CDR ships.","tags":["orchestack","infrastructure","multi-tenancy","agents","architecture"],"author":"Coastal Digital Research","unlisted":false,"content_md":"Most agent demos run as a single Python script with hard-coded API keys, a single tool, and a single user. That works for proving an idea. It does not work when you have multiple agents running on behalf of multiple operators, talking to multiple platforms, executing actions that touch real systems, with auditability and rate limits and budget controls and a policy layer. The gap between \"demo agent\" and \"production agent infrastructure\" is the gap Orchestack fills.\n\n## Purpose\n\nOrchestack is a multi-tenant orchestration platform for agent systems. It runs on Kubernetes, talks to operators through chat connectors (Discord, Slack, Email, Telegram, Webchat), executes work in isolated sandboxes, routes requests through a policy-aware model router, and keeps a hierarchical memory system that scales from hot in-cluster cache to cold archive storage.\n\nThe design goal we kept coming back to: handle 100+ concurrent agent sessions with sub-500ms response latency, while making it easy to onboard new agents in under 30 seconds and supporting 80% automation of routine infrastructure tasks. Internal operators first. Selected external users in v1.5. Multi-tenant SaaS is a v2 problem.\n\n## Architecture\n\nAt the top level, Orchestack decomposes into a control plane, an execution plane, a connector layer, and a tiered memory system. Everything talks through NATS for events and Postgres for authoritative state.\n\n```mermaid\nflowchart TB\n    subgraph Connectors[\"Connector Layer\"]\n        Discord[Discord]\n        Slack[Slack]\n        Email[Email]\n        Telegram[Telegram]\n        Webchat[Webchat]\n    end\n\n    subgraph Bus[\"NATS + JetStream\"]\n        Events[(Event Bus)]\n    end\n\n    subgraph Control[\"Control Plane\"]\n        Scheduler[Session Scheduler]\n        Dispatcher[Task Dispatcher]\n        Policy[Policy Evaluator]\n        Loop[Loop Runner]\n        Router[Model Router]\n        Budget[Budget Accounting]\n    end\n\n    subgraph Execution[\"Execution Plane\"]\n        Daytona[Daytona Sandboxes]\n        Agents[Agent Workloads]\n    end\n\n    subgraph State[\"State + Memory\"]\n        Postgres[(Postgres)]\n        Hot[(Hot: in-cluster)]\n        Warm[(Warm: NAS)]\n        Cold[(Cold: object store)]\n        Archive[(Archive: git)]\n    end\n\n    Connectors --> Events\n    Events --> Scheduler\n    Scheduler --> Dispatcher\n    Dispatcher --> Policy\n    Policy --> Loop\n    Loop --> Router\n    Loop --> Daytona\n    Daytona --> Agents\n    Router -.->|tracks cost| Budget\n    Control <--> Postgres\n    Agents <--> Hot\n    Hot --> Warm\n    Warm --> Cold\n    Cold --> Archive\n```\n\nA request enters through a connector (someone messages an agent on Discord, say). The connector publishes a normalized message envelope to NATS. The Session Scheduler picks it up, assigns a session, and the Task Dispatcher decomposes the work. The Policy Evaluator decides what the agent is allowed to do for this tenant. The Loop Runner drives the agent's reasoning loop, with the Model Router picking the right model based on data sensitivity, latency budget, and cost. Execution happens in Daytona sandboxes, one per active session. Everything emits audit events back to NATS for observability.\n\n## Design decisions we made on purpose\n\n**Progressive security disclosure.** Defaults are strict: deny-all RBAC, local embeddings only, ephemeral sandboxes that get torn down between tasks. Power users get documented escape hatches when they need them. IT compliance is happy. AI research isn't blocked.\n\n**Agent-centric ownership.** The unit of ownership is the agent, not the team or project. This matches how operators actually think about agents, and it sidesteps a class of cross-team permission problems that we kept hitting in earlier designs.\n\n**GitOps for everything.** Runbooks, policies, agent definitions, all in git. Memory tier L3 is a cache. Git is source of truth. This gives us auditability for free and lets agents move fast against a stable substrate.\n\n**Privacy-first model routing.** Sensitive data routes to local models by default. External APIs only for non-sensitive, quality-critical tasks. The Policy Evaluator gates the routing decision so this isn't a convention, it's enforced.\n\n**Tiered memory matches tiered storage economics.** Hot in-cluster for the working set. Warm NAS for the recent past. Cold object storage for historical. Archive in compressed git for the deep cold. The agent doesn't know the difference. The bill does.\n\n## Integration points with other CDR projects\n\nOrchestack is meant to be the harness that other CDR work runs on top of, or feeds into.\n\n- [**CDRcache**](/page/projects#cdrcache) sits inside the Loop Runner as the semantic cache for agent outputs. When an agent produces the same result for the same input, Orchestack short-circuits the rerun and serves from cache. Hash-addressed, deterministic, audit-friendly.\n- [**CDRnext**](/page/projects) (private; successor to CDRmem and CDRdistill) is where the architectural-memory work lands. The memory access gates and fast-weight modules from that research are what we want eventually plugged in behind the Hot tier of Orchestack's memory system.\n- [**mae**](/page/projects#mae) and [**rlm-linux**](/page/projects#rlm-linux) are exactly the kind of system agents the platform was designed to host. They run as long-lived agents inside dedicated sandboxes, with the connector layer giving operators a way to direct them through chat.\n- [**TopoLI**](/page/projects#topoli) and [**fpre**](/page/projects#fpre) feed into the Model Router as candidate components: pruned retrieval and typed-primitive reasoning are both routing decisions that depend on the task at hand.\n- [**Pachyterm**](/page/projects#pachyterm) and [**CDRbrowser**](/page/projects#cdrbrowser) are end-user surfaces that can act as clients of Orchestack agents, but they don't run inside the orchestrator itself.\n\n## Status\n\nv1 is in active development. The architecture is set; the RFCs (event and state model, isolation and policy, identity, payments, extensibility) are written and reviewed. Implementation is monorepo and polyglot: Go for the high-concurrency control plane, Python for connectors and ML, TypeScript for any web UI.\n\nWhat's targeted for v1 Phase 1 (the core that has to ship and stabilize first):\n\n- Session Scheduler, Task Dispatcher, Policy Evaluator, Loop Runner as separate services\n- Model Router with dynamic registration and size-plus-locality fallback\n- Budget Accounting per model and per provider\n- All five connectors: Discord, Slack, Email, Telegram, Webchat\n- NATS event bus with JetStream for durable streams\n- Postgres for authoritative state, Atlas for schema migrations\n- Daytona for sandbox execution\n- Vault for secrets, OIDC plus LDAP for identity, SPIFFE for service identity\n\nWhat's deferred to Phase 2 (built on top of the proven core):\n\n- On-chain trust and payments (RFC-004) for cross-tenant economics\n- Modularized extensibility surface (RFC-005) for third-party plugins\n\nDeployment is via a Kubernetes Operator for clusters and Podman or Docker Compose for local development. CI/CD targets Tekton primary, with support for GitLab CI and GitHub Actions for upstream contributors.\n\n## Open questions we're working through\n\nThe honest part. These are unresolved in the architecture docs and we don't have clean answers yet.\n\n- How aggressive should default budget caps be before we surprise an operator with an out-of-budget pause? Strict defaults are friendlier to IT compliance but unfriendlier to research workflows.\n- Where exactly does DLP (data-loss prevention) plug in? Inline at the policy evaluator, async on the audit stream, or both? The mydlp tooling can do either; the right answer depends on latency budget.\n- For the on-chain piece in Phase 2, the unresolved question is whether we want a public ledger or a permissioned one. Both have legitimate use cases. We have not committed to either.\n- What's the right boundary between a \"system agent\" (long-lived, run by the platform itself) and a \"user agent\" (spawned per session, torn down after)? mae and rlm-linux blur this line, and the policy implications are different in each direction.\n\nIf any of these turn into a research thread on their own, we'll write it up."},{"slug":"cdr-home-architecture","title":"cdr-home: A Website for Humans and Agents at the Same Time","date":"2026-05-21","summary":"How this site serves the same content as HTML for people and as JSON or Markdown for agents, without compromising either side.","tags":["cdr-home","dual-mode","fastapi","content-negotiation","agents"],"author":"Coastal Digital Research","unlisted":false,"content_md":"Every website has two audiences now. The visitor reading on a laptop and the agent fetching the page on someone else's behalf are not going to engage with the same payload well. Most sites pick one and bolt the other on later, badly. cdr-home is the inverse: we picked both at the same time, on the same routes, with content negotiation deciding what each visitor sees.\n\n## Purpose\n\ncdr-home is the public site for Coastal Digital Research. It's also the reference implementation for the dual-mode pattern: human-readable HTML pages and machine-readable JSON/Markdown endpoints, both served from the same source of truth, with proper content negotiation so each request gets what it asked for. The codebase is small enough that we use it as a working example of how the rest of our agent infrastructure should think about web surfaces.\n\nEvery page on this site is reachable three ways. The HTML at `/page/mission` is what a browser sees. The Markdown at `/agent/page/mission.md` is what an LLM crawler gets when it sends `Accept: text/markdown` or shows up with a known LLM user-agent. The JSON at `/agent/page/mission.json` carries the same content plus metadata, useful for programmatic consumers.\n\n## Architecture\n\nThe stack is intentionally minimal. FastAPI for routing. Jinja2 templates. Python's `markdown` library for server-side rendering. No database. Content lives in `content/pages/*.md` and `blog/*.md` with YAML frontmatter; agent manifests live in `agents/manifests/*.yaml`. The app reads from disk on startup and serves the results.\n\n```mermaid\nflowchart LR\n    subgraph Source[\"Source of Truth\"]\n        Pages[content/pages/*.md]\n        Posts[blog/*.md]\n        Manifests[agents/manifests/*.yaml]\n    end\n\n    subgraph Loaders[\"Loaders\"]\n        ContentPy[content.py]\n        BlogPy[blog.py]\n        RegistryPy[registry.py]\n    end\n\n    subgraph App[\"FastAPI Application\"]\n        Router[Content Negotiation]\n        Pages --> ContentPy\n        Posts --> BlogPy\n        Manifests --> RegistryPy\n        ContentPy --> Router\n        BlogPy --> Router\n        RegistryPy --> Router\n    end\n\n    Router --> HTML[HTML for humans]\n    Router --> MD[Markdown for LLM crawlers]\n    Router --> JSON[JSON for programmatic clients]\n\n    HTML --> Caddy[Caddy reverse proxy]\n    MD --> Caddy\n    JSON --> Caddy\n    Caddy --> Public[coastaldigital.ai]\n```\n\nThe interesting bit is the content negotiation layer. Each `/page/{slug}` request runs through `_wants_format()`, which checks (in order) an explicit `?format=` query parameter, the `Accept` header, then the User-Agent string. LLM crawlers like GPTBot, ClaudeBot, and PerplexityBot auto-get Markdown because that's a cleaner payload for them. Search-engine crawlers like Googlebot get HTML because their rankings should reflect what humans see. Everyone else gets HTML by default.\n\nEvery response carries a `Link` header advertising the alternate representations, so any client that pays attention to standards can discover the other formats without guessing URLs. The `Vary: Accept, User-Agent` header keeps caches honest.\n\n## Design decisions we made on purpose\n\n**Disk is the database.** Content authoring happens in plain Markdown files, committed to git, deployed in a container image. No CMS to operate, no schema migrations, no editorial state machine. The version history is the audit trail. The cost is that publishing requires a deploy. We're fine with that.\n\n**One source of truth, three views.** The Markdown body of a page is rendered server-side into HTML for humans, served raw to agents that ask for Markdown, and packaged into JSON (alongside frontmatter metadata) for programmatic clients. We never maintain three versions of the same content.\n\n**Auto-detection over explicit opt-in.** Most LLM crawlers don't bother to send a proper Accept header. We sniff the User-Agent and serve them Markdown anyway. They're better served, we save bytes, the human visit isn't affected.\n\n**Standard headers over custom mechanisms.** `Link: rel=\"alternate\"`, `Vary`, `Accept`, query-string overrides. Nothing in here is exotic. If someone reads this and wants to do it on their own site, none of the moving parts are CDR-specific.\n\n## Integration with other CDR projects\n\ncdr-home is the surface, not infrastructure that other agents depend on. The integration story runs in the other direction: this site documents and links to the rest of the work.\n\n- [**Orchestack**](/blog/orchestack-architecture) documents itself through the `/.well-known/agent.json` discovery pattern that cdr-home demonstrates. Any external operator integrating with a CDR-hosted agent platform can start from the same convention.\n- [**The agent registry**](/agents) at `/agents` and the manifests under `agents/manifests/` are a reference for how CDR-style agent declarations look in YAML. Agents we run on Orchestack publish manifests in the same shape.\n- The dual-mode pattern itself is something we hope other CDR projects adopt for their own admin or status surfaces. There's no library yet; the right move is probably to write one once we've used the pattern in two or three more places.\n\n## Status\n\nIn production at https://coastaldigital.ai. Source at github.com/CoastalDigitalResearch/cdr-home.\n\nWhat's built:\n\n- HTML pages with server-side rendered Markdown\n- Blog posts with Mermaid diagram support\n- Agent endpoints for pages, blog posts, and agent manifests\n- Content negotiation by Accept header, query parameter, and User-Agent\n- Discovery via `/.well-known/agent.json`\n- A voice-check workflow that runs against drafts before publish\n- Auto-deploy pipeline via Forgejo Actions, with Forgejo as canonical source and GitHub as a public mirror\n\nWhat's intentionally not built:\n\n- Comments. They're a moderation burden and add nothing to a small site like this.\n- User accounts. Everything is public.\n- Analytics beyond standard server logs.\n- A search index. The site is small enough that the agent endpoints are a better search interface than anything we'd ship.\n\n## Open questions we're working through\n\n- The HEAD-request 405 issue. FastAPI's GET-only routes return 405 for HEAD requests, which trips up some scrapers and uptime monitors that use HEAD by default. We could add HEAD support explicitly, or let Caddy synthesize HEAD from GET. Probably the latter.\n- Streaming. The full Mermaid library is loaded only on blog post pages, which is fine, but the load is still a few hundred KB. A self-hosted slimmer build would be faster. Worth doing once we have more than five posts.\n- The `unlisted: true` flag on the legacy launch post hides it from the index but the file stays in the image. Should unlisted posts be removed from the agent JSON listing as well, or kept discoverable? Currently they're served by direct slug but absent from `/agent/blog.json`. We're leaving them out of the listing for now and revisiting if that's wrong.\n- We use a smarty markdown extension that renders straight quotes into curly quotes. This is correct for humans and slightly annoying for agents (curly quotes mess up some downstream tooling). The agent Markdown endpoint serves the raw body, but the JSON endpoint serves the metadata only. Should we also offer a \"raw text\" view that strips smarty? Probably yes; nobody's asked yet."},{"slug":"cdrcache-architecture","title":"CDRcache: Hash-Addressed Caching for Agent Pipelines","date":"2026-05-20","summary":"If an agent produces the same output for the same input, you should only have to run it once. CDRcache is the layer that makes that true.","tags":["cdrcache","caching","agents","infrastructure"],"author":"Coastal Digital Research","unlisted":false,"content_md":"LLM inference is expensive. Tool invocations are expensive. Multi-step agent pipelines are extremely expensive. And a meaningful fraction of those calls are redundant: the same prompt with the same context, the same tool with the same arguments, fed back through the system over and over because nothing remembered the answer. CDRcache is the layer that fixes that.\n\n## Purpose\n\nCDRcache is a semantic caching and retrieval system for AI agent pipelines. The premise is straightforward. If an agent step is a deterministic function of its inputs (model + prompt + context + tool args), the output can be hashed, stored, and returned on the next identical call without re-running anything. Cost goes down. Latency goes down. Audit trail goes up.\n\nThe \"semantic\" part matters. A naive cache that keys only on the exact byte-for-byte input will miss a lot of opportunities, because real-world prompts include timestamps, request IDs, and whitespace differences that don't change the semantic content. CDRcache normalizes inputs before hashing so functionally identical calls hit the cache, while genuinely different calls don't.\n\n## Architecture\n\nCDRcache sits in the middle of an agent's call path. The agent (or its harness) asks the cache for a result first. On a hit, the cached output is returned and the underlying model or tool call is skipped. On a miss, the agent executes the call, and the result is written back to the cache keyed by the input hash.\n\n```mermaid\nflowchart LR\n    Agent[Agent step] --> Norm[Input normalizer]\n    Norm --> Hash[Hash key generator]\n    Hash --> Lookup{Cache lookup}\n    Lookup -->|hit| Return[Return cached output]\n    Lookup -->|miss| Exec[Execute call]\n    Exec --> Model[(Model or Tool)]\n    Model --> Write[Write to cache]\n    Write --> Return\n    Write --> Audit[(Audit log)]\n\n    subgraph Storage[\"Cache Storage\"]\n        Local[Local store]\n        Remote[Optional remote store]\n    end\n\n    Lookup <--> Local\n    Write --> Local\n    Local <--> Remote\n```\n\nStorage is content-addressed, so the cache key is derived from the input itself. Entries are immutable once written, which makes the cache double as a tamper-evident audit trail. You can always replay exactly what an agent received as input and exactly what it returned, at exactly what time.\n\n## Design decisions we made on purpose\n\n**Content addressing, not key namespacing.** The hash of the normalized input *is* the key. There's no separate naming scheme to coordinate, no risk of two agents accidentally colliding on a name, no need to manage cache invalidation through key rewrites.\n\n**Immutable entries.** Once written, a cache entry doesn't get overwritten. If a model version changes and produces a different output for the same input, that's a new entry. The old entry stays. This is essential for the audit-trail use case: you should never wonder whether a stored output was tampered with after the fact.\n\n**Local-first storage.** CDRcache works as a process-local cache, scales to a host-local cache, and can federate to a shared cache across hosts when that's needed. Default is local because the latency win is largest there and most workloads don't need cross-host sharing.\n\n**Caching is opt-in per step.** Some agent steps shouldn't be cached because their outputs aren't actually deterministic (time-sensitive queries, randomized generation). The harness decides per-step whether to consult the cache, rather than CDRcache trying to guess.\n\n## Integration with other CDR projects\n\nCDRcache is foundational infrastructure. It plugs in underneath most other CDR work that does meaningful inference or tool execution.\n\n- [**Orchestack**](/blog/orchestack-architecture) uses CDRcache inside the Loop Runner. Every agent step in a session is a candidate for cache lookup before execution; the Policy Evaluator can mark steps as non-cacheable when needed (e.g., for time-sensitive tasks or audit-required reruns).\n- [**mae**](/page/projects#mae) caches its long-running task intermediates through CDRcache. When mae is reasoning about a multi-step system change, repeated subqueries against the same machine state hit the cache rather than burning tokens.\n- [**rlm-linux**](/page/projects#rlm-linux) uses CDRcache to memoize conductor workflow outputs. When the same kind of system event recurs (a package upgrade, a config drift detection), the conductor's planning step often produces the same workflow, and the cache short-circuits.\n- [**CDRdistill**](/page/projects#cdrdistill) and [**TopoLI**](/page/projects#topoli) use CDRcache during research runs where the same embedding computations and the same retrieval scores get recomputed across experiments. Caching them is the difference between \"rerun in seconds\" and \"rerun in minutes.\"\n\n## Status\n\nEarly but in active use. Source at github.com/CoastalDigitalResearch/CDRcache.\n\nWhat's built:\n\n- Content-addressed storage with normalized input hashing\n- Immutable entry semantics\n- Local-process and local-host backends\n- A pluggable storage interface (so a Redis or PostgreSQL backend is straightforward to add)\n- Audit log of all cache reads and writes\n\nWhat's not yet built:\n\n- A remote shared-cache backend with proper concurrency semantics for multi-host setups\n- Eviction policies beyond manual purge (LRU, size cap, age cap)\n- A management UI for inspecting and pruning entries\n- Tooling to detect \"cache-hostile\" inputs (inputs that look deterministic but actually aren't, like prompts containing a `now()` timestamp the agent forgot to strip)\n\n## Open questions we're working through\n\n- **Determinism in practice.** Most models are nominally deterministic at temperature 0, but in practice GPU kernel non-determinism, batched inference effects, and tokenizer edge cases produce small output variations for identical inputs. How tolerant should CDRcache be? A strict byte-match policy maximizes correctness; a tolerance window (Levenshtein, embedding similarity) maximizes hit rate. We don't have a clean answer yet.\n- **Cache invalidation when the model changes.** New model version equals new entry under content addressing, which is correct but also means the entire cache is functionally invalidated by a model swap. Should there be a \"translation\" pass that detects equivalent outputs across model versions and links the entries? Probably not, but it's a real cost question for production agents.\n- **Sensitive data.** Cache entries can contain anything that flows through an agent. We say \"no PII in cache\" as a hard rule, but enforcement is currently the caller's job. Is there a sensible scanning policy that catches obvious mistakes (credit card numbers, SSN patterns) without false-positive-storming legitimate use cases? Worth exploring.\n- **The right granularity.** Cache a whole agent step? A whole tool invocation? A model call? An embedding lookup? We currently support all three but don't have strong guidance on which to use when. This is the kind of thing that should turn into a write-up once we have more real-world data."},{"slug":"mae-architecture","title":"mae: A Minimal Agentic Environment for Linux Maintenance","date":"2026-05-19","summary":"Most Linux maintenance is repetitive and error-prone. mae is the persistent agent we built to take over the toil.","tags":["mae","linux","system-management","agents"],"author":"Coastal Digital Research","unlisted":false,"content_md":"There's a category of work that every Linux operator does every week, and it never gets more interesting the second or fiftieth time you do it. Apply the security updates. Reconcile the configuration drift. Check the log for the thing that's been creeping. Audit the SELinux rules. Rebuild the dotfiles after the laptop reset. Each individual task is straightforward, the cumulative time spent on it is enormous, and none of it is the reason you sit down at the keyboard. mae is the persistent agent that takes that work off the human.\n\n## Purpose\n\nmae is short for \"Minimal Agentic Environment.\" It runs on a Linux box (laptop, workstation, server, anything systemd-based) and keeps the system healthy: applies OS and package updates, enforces declarative configuration, audits security posture, and executes whatever ad-hoc tasks you hand it through a web UI. The goal is a self-managing workstation where the human focuses on work, not maintenance.\n\nmae is multi-provider by design. It runs against Anthropic Claude, OpenAI, OpenRouter, OpenCode Zen, or fully local models via Ollama. The model choice is a setup decision, not an architectural one, and you can run the whole thing offline against a local 7B if that's what your environment requires.\n\n## Architecture\n\nmae has three layers: the agent core that does reasoning, the action layer that touches the system, and the operator interface that lets a human watch and direct what's happening.\n\n```mermaid\nflowchart TB\n    subgraph Operator[\"Operator Surface\"]\n        WebUI[Next.js Web UI]\n        Terminal[Built-in Terminal]\n        Palette[Command Palette]\n    end\n\n    subgraph Agent[\"Agent Core\"]\n        Loop[Reasoning Loop]\n        Provider[Provider Adapter]\n        Memory[Session Memory]\n    end\n\n    subgraph Actions[\"Action Layer\"]\n        Shell[Shell Execution]\n        Files[File System Operations]\n        Pkg[Package Management]\n        Config[Configuration Enforcement]\n        Sec[Security Audit + Remediation]\n    end\n\n    subgraph Models[\"Model Providers\"]\n        Claude[Anthropic Claude]\n        OpenAI[OpenAI]\n        Router[OpenRouter]\n        Zen[OpenCode Zen]\n        Local[Ollama local]\n    end\n\n    Operator --> Loop\n    Loop <--> Memory\n    Loop --> Provider\n    Provider --> Models\n    Loop --> Actions\n    Actions --> Loop\n```\n\nThe reasoning loop pulls a task from the operator, plans the steps, executes through the action layer, observes results, and decides what to do next. The provider adapter abstracts which model backend is running, so the same loop works against Claude or a local Ollama instance. Session memory carries context across the reasoning loop's iterations, and through MEMORY.md persists across mae restarts.\n\nThe operator surface is a Next.js app that gives you a chat with the agent, a built-in terminal for direct system interaction when you want to bypass the agent, and a command palette for common operations. The terminal is there because sometimes you just want to type `ls -la` without a model intermediating.\n\n## Design decisions we made on purpose\n\n**Multi-provider, not vendor-locked.** mae is useful only if it works against the model an operator can actually run. That includes operators with strict data-locality requirements who need a fully local model. The provider adapter is a thin interface; adding a new backend is an afternoon's work.\n\n**Web UI plus terminal, not one or the other.** Some tasks are easier to describe in natural language; others are easier to type directly. mae assumes both, and the operator switches between them per task.\n\n**Setup wizard before configuration files.** On first launch with no provider configured, mae opens a setup wizard at `/init` instead of failing or demanding you edit a config file. This is the friction we'd actually want to remove for ourselves; we removed it.\n\n**System changes are auditable.** Every action mae takes is logged in enough detail to reconstruct what happened, with a clear distinction between \"agent decided to\" and \"operator explicitly asked for.\" This is the same audit-trail principle that runs through CDRcache and Orchestack.\n\n## Integration with other CDR projects\n\nmae is one of the canonical \"system agents\" we want to be runnable on top of Orchestack, but it stands alone too.\n\n- [**Orchestack**](/blog/orchestack-architecture) is the platform that mae naturally fits into. As an Orchestack system agent, mae gets a connector-mediated chat surface, policy-enforced action permissions, and a sandboxed execution plane. Running mae standalone is fine; running it inside Orchestack adds multi-tenant safety.\n- [**CDRcache**](/blog/cdrcache-architecture) sits underneath mae's reasoning loop. Repeated planning calls for the same kind of system event (e.g., a kernel upgrade on Fedora) hit cache after the first time, which makes mae cheaper to run on similar tasks.\n- [**rlm-linux**](/page/projects#rlm-linux) is a sibling project. Where mae focuses on running an agent on an existing system, rlm-linux focuses on composing and maintaining the system itself. The two complement each other: rlm-linux builds the substrate, mae operates on it.\n- [**Pachyterm**](/page/projects#pachyterm) is a natural client for mae. Type `p` in Pachyterm to send a prompt to a local model; that local model can be a mae instance running on the same box.\n\n## Status\n\nIn daily use as a personal Linux maintenance agent. Source at github.com/CoastalDigitalResearch/mae. Originally forked from VibeOS and rebuilt around how we think about agent-managed systems.\n\nWhat's built:\n\n- Reasoning loop with the five provider backends\n- Web UI for chat, terminal, and command palette\n- File operations, shell execution, package management, configuration enforcement\n- Security hardening audits and remediations\n- Setup wizard for first-run provider configuration\n- MEMORY.md as persistent context across sessions\n\nWhat's deferred to later:\n\n- Multi-machine coordination (one mae managing a small fleet)\n- Declarative system-state files (think NixOS-style declarations, but agent-driven)\n- A test suite for the destructive operations (the ones that touch package installs and system configs are hard to test without VMs)\n- Plugin hooks so users can extend the action layer without forking\n\n## Open questions we're working through\n\n- **The trust boundary.** mae has the keys to your system. How do we make it observably safer than a script that does the same things, without making it so locked down that operators just go around it? The pattern we keep landing on is \"ask for confirmation on destructive actions\" plus \"log everything,\" but the confirmation prompts get repetitive fast.\n- **Local-model sufficiency.** The provider abstraction is solid; the open question is whether a 7B local model is actually good enough for the full range of system-management tasks, or whether the operator needs to know which tasks reliably work with which provider. We have anecdotes; we don't have a clear matrix.\n- **Configuration drift detection.** \"Did this file change because the operator did something, or because some service rewrote it, or because someone else logged in and edited it?\" is harder than it sounds. Filesystem audit hooks help. They don't fully solve it.\n- **Boundaries with rlm-linux.** Where exactly does \"operating a system\" stop and \"composing a system\" begin? We've drawn the line based on \"is the system already installed?\" but that's pragmatic rather than principled. Worth a write-up of its own once we've used both more."},{"slug":"rlm-linux-architecture","title":"rlm-linux: Recursive Orchestration for Fedora-Based Systems","date":"2026-05-18","summary":"Build and maintain your distro from first principles with a Sakana-style RL conductor over a tiered pool of local and remote LLM workers.","tags":["rlm-linux","linux","fedora","orchestration","agents"],"author":"Coastal Digital Research","unlisted":false,"content_md":"A Linux installation is a million tiny decisions baked into a thousand files. Most of those decisions are reasonable defaults you'd never change. Some are personal preferences you re-apply every time you reinstall. Some are workflow optimizations that took years to arrive at and exist only in your muscle memory. The standard answer is dotfiles plus a kickstart plus some shell scripts plus, increasingly, a Nix or Ansible setup. rlm-linux is the agent-driven answer to the same problem.\n\n## Purpose\n\nrlm-linux does two things. First, it composes a custom Fedora 44-based image from a profile system: package set, kickstart, systemd services, desktop environment, dotfiles, all driven by layered config with sensible defaults. Second, once installed, it becomes the resident customization layer on the system, orchestrating package upgrades, configuration drift, log triage, RPM authoring, and the lifecycle of isolated AI project sandboxes.\n\nThe architecture leans on a Sakana-style RL Conductor pattern: a small local model emits full agentic workflows in natural language, which then get executed against a tiered pool of LLM workers based on the latency and capability budget of each subtask.\n\n## Architecture\n\n```mermaid\nflowchart TB\n    subgraph Profile[\"Profile System (layered config)\"]\n        Base[base]\n        Desktop[desktop]\n        Site[site]\n        User[user]\n    end\n\n    subgraph Conductor[\"Conductor (local 7B)\"]\n        Plan[Workflow Planner]\n        Assign[Worker Assignment]\n        Access[Access List Generator]\n    end\n\n    subgraph Workers[\"Tiered Worker Pool\"]\n        GPU[Interactive GPU-resident]\n        CPU[Larger CPU/RAM via llama.cpp]\n        Remote[Remote API]\n    end\n\n    subgraph Evaluators[\"Evaluators\"]\n        Tests[pytest / ruff / mypy]\n        RPM[rpmbuild]\n        Systemd[systemctl]\n        SELinux[SELinux audit]\n    end\n\n    subgraph Sandboxes[\"Project Sandboxes\"]\n        Pod1[Podman + Quadlet env A]\n        Pod2[Podman + Quadlet env B]\n    end\n\n    Profile --> Conductor\n    Conductor --> Workers\n    Workers --> Evaluators\n    Evaluators -.->|score| Conductor\n    Conductor --> Sandboxes\n```\n\nThe Conductor is the heart of it. It's a small (7B) local model whose job is not to do the work but to plan the work. It takes a system event (\"kernel update available\", \"config file changed\", \"operator asked for X\"), emits an agentic workflow as natural-language subtask plans, assigns workers based on latency budget, and produces an access list scoped to what the workflow needs.\n\nWorkers are tiered. The interactive GPU-resident model handles things that need to be fast and small. Larger CPU/RAM-resident models via llama.cpp handle deeper reasoning tasks that can wait a few seconds. Remote API models are the long tail for anything that exceeds local capability.\n\nEvaluators are where the system gets honest. Every workflow produces verifiable signals (pytest pass/fail, ruff lint, mypy strict, rpmbuild success, systemctl status, SELinux audit). Those signals score the workflow and feed an eventual GRPO fine-tune of the Conductor. The model gets better at planning over time because it gets to learn from what actually worked.\n\nSandboxes are per-project Podman+Quadlet environments. Each AI project runs in its own sandbox with a scoped Conductor instance, an isolated trace store, and a shared model cache. This is how you run five experiments at once without them stepping on each other's CUDA, their dependencies, or their assumptions about the file system.\n\n## Design decisions we made on purpose\n\n**Profile layering.** Configuration is composed in layers: base → desktop → site → user. The same tool ships an opinionated GNOME default for adoption, while power users compose their own desktop, services, and worker pools by overlaying. This is the closest equivalent to Nix's overlay model in the RPM world.\n\n**Local conductor, tiered workers.** The Conductor stays local for privacy, latency, and cost. Workers are tiered because not every subtask needs a 70B model, and most subtasks shouldn't pay for one.\n\n**Verifiable evaluators.** Every workflow ends with a measurement, not a vibe-check. rpmbuild either succeeds or it doesn't. Pytest either passes or it doesn't. The Conductor's learning signal is grounded in those outcomes.\n\n**Sandboxes are first-class.** AI project work is messy. CUDA versions conflict. Python environments rot. Models get downloaded and forgotten. Per-project sandboxes turn the mess into something containable.\n\n## Integration with other CDR projects\n\nrlm-linux is system-level infrastructure. It connects to the rest of CDR work in a few specific ways.\n\n- [**mae**](/blog/mae-architecture) is the sibling project. rlm-linux composes and maintains the substrate; mae operates the agent on top of it. Running both gives you a Fedora system that builds itself from a profile and then keeps itself healthy on autopilot.\n- [**Orchestack**](/blog/orchestack-architecture) is where the Conductor pattern eventually wants to live. The Conductor + tiered workers design is exactly what Orchestack's Model Router is intended to do at scale; rlm-linux's implementation is a smaller-scoped predecessor that informed the design.\n- [**CDRcache**](/blog/cdrcache-architecture) memoizes Conductor planning outputs. Many system events recur (\"apply weekly security update\"), and the resulting workflows are nearly identical each time. Cache hits short-circuit the planning step.\n- [**fpre**](/page/projects#fpre)'s typed-primitive reasoning is a candidate component for the Conductor when a workflow needs grounded symbolic structure (e.g., dependency resolution, package version compatibility checks).\n\n## Status\n\nEarly. Source at github.com/CoastalDigitalResearch/rlm-linux. Personal project of [@AdamPippert](https://github.com/AdamPippert) per the README.\n\nWhat's built:\n\n- Profile system with base/desktop/site/user layers\n- Conductor with workflow emission\n- Three-tier worker pool with local-GPU, local-CPU, and remote API backends\n- Evaluators wired up to pytest, ruff, mypy, rpmbuild, systemctl, SELinux audit\n- Podman+Quadlet sandbox lifecycle\n- Shared model cache across sandboxes\n\nWhat's not yet built:\n\n- The GRPO fine-tune of the Conductor. The evaluator scores are being collected; the actual training pass against those scores is on the list, not done.\n- A web UI. Right now everything is CLI-driven, which is fine for the maintainer but a barrier for anyone else.\n- Drift remediation policies (the system can detect drift; how it decides what to do about it is currently per-handler logic rather than declarative policy).\n- Multi-machine support. One conductor per system today.\n\n## Open questions we're working through\n\n- **The right size for the Conductor.** 7B is small enough to run interactively on a desktop GPU. Is 7B big enough to actually plan well, or is it leaning on the worker pool to compensate? The GRPO experiment will tell us, eventually.\n- **Profile boundaries.** Where do you draw the line between \"base layer\" (everyone gets this) and \"desktop layer\" (only people running a desktop)? We started with reasonable splits and they're already getting tangled. A clearer ontology is warranted.\n- **Worker pool elasticity.** The tiered pool design assumes workers are roughly always available. What happens when the local GPU is occupied or the remote API is rate-limited? The Conductor currently queues; whether it should fail fast or fall back to a smaller worker is unclear.\n- **Sandbox proliferation.** Per-project sandboxes are great until you have 40 of them. There's no garbage collection policy yet beyond manual cleanup. Worth a write-up once the pattern is more proven."},{"slug":"pachyterm-architecture","title":"Pachyterm: A GPU-Accelerated Terminal with Native AI Integration","date":"2026-05-17","summary":"Sub-50ms time-to-first-frame, POSIX-compliant, and a `p` prefix that turns the rest of any line into an LLM prompt.","tags":["pachyterm","terminal","rust","gpu","agents"],"author":"Coastal Digital Research","unlisted":false,"content_md":"The terminal is where developers spend the most time and where the AI tooling story is the worst. The options today are either fast (Alacritty, Kitty, WezTerm) with no AI integration, or AI-integrated (Warp) with proprietary lock-in and macOS-mostly availability. Pachyterm is the answer to \"why does this have to be a tradeoff.\"\n\n## Purpose\n\nPachyterm is a cross-platform, GPU-accelerated terminal emulator with native AI agent integration, built in Rust. It targets sub-50ms time-to-first-frame, sub-300ms AI response times against local 7B 4-bit models, and 100% POSIX compatibility so it works with every shell and TUI you already use. The model integration is local-first: GGUF, MLC, vLLM all supported, with remote fallback when you want it.\n\nThe activation pattern is one keystroke. Type `p` at the beginning of any line and the rest of that line becomes an LLM prompt. No context switch, no menu, no separate window. The AI shows up exactly where your typing already lives.\n\n## Architecture\n\nPachyterm is modular in the way fast terminals usually are: a tight core for input, output, and rendering, with extension surfaces for the AI layer and the plugin system.\n\n```mermaid\nflowchart TB\n    subgraph Core[\"Core Runtime\"]\n        TTY[Async PTY Engine]\n        Renderer[wgpu GPU Renderer]\n        Input[Input + Keymap]\n    end\n\n    subgraph Agent[\"Agent Subsystem\"]\n        Parser[Command Parser p-prefix]\n        Host[Model Host with warm pools]\n        StreamUI[Streaming Response UI]\n    end\n\n    subgraph Models[\"Model Backends\"]\n        GGUF[GGUF local]\n        MLC[MLC local]\n        vLLM[vLLM local]\n        Remote[Remote fallback]\n    end\n\n    subgraph Config[\"Configuration\"]\n        TOML[pachyterm.toml live-reload]\n        Profile[Per-model Profile Cache]\n    end\n\n    subgraph Plugins[\"Plugin SDK\"]\n        WASM[Rust + WASM plugins]\n    end\n\n    Input --> TTY\n    Input --> Parser\n    TTY --> Renderer\n    Parser --> Host\n    Host --> Models\n    Host --> StreamUI\n    StreamUI --> Renderer\n    Config -.->|live-reload| Core\n    Config -.->|live-reload| Agent\n    Plugins --> Core\n    Plugins --> Agent\n```\n\nThe TTY engine is async PTY management with zero-copy input and output buffers. The renderer is wgpu-backed with hardware-accelerated text and ligature support, targeting 144 FPS with sub-1ms round-trip echo under load. The input layer handles configurable keybindings that preserve native shell behavior, so your existing `Ctrl-R` muscle memory still works.\n\nThe agent subsystem watches input. When a line starts with `p`, the parser routes the rest of the line to the model host instead of the shell. The model host manages local and remote backends with warm pools, so first-token latency stays low. The streaming UI renders the response inline with markdown support.\n\nConfiguration is plain TOML with live-reload. Edit `pachyterm.toml`, save, the change applies without a restart. Profiles cache per-model settings (system prompts, temperature, max tokens) so switching between models is a single flag rather than a config dance. Optional encryption on the profile cache for stored API keys.\n\nPlugins are Rust or WASM, sandboxed, with a defined SDK surface. The plugin system is meant to handle the \"add a new command type\" case without anyone forking the terminal itself.\n\n## Design decisions we made on purpose\n\n**POSIX compliance is non-negotiable.** Warp's biggest problem isn't its proprietary licensing; it's that it isn't a real terminal, so a huge slice of TUI applications break inside it. Pachyterm is a real terminal first, an AI surface second. Every shell, every editor, every TUI tool you already use works unmodified.\n\n**Local-first AI.** Cloud-only AI in your terminal means latency, data egress, and a hard dependency on someone else's uptime. Local-first means the response time stays under 300ms and your prompts stay on your machine by default.\n\n**Plain-text configuration.** No GUI settings panel, no proprietary format. Edit a TOML file, live-reload picks it up. This is how power users actually want to configure tools.\n\n**GPU acceleration without GPU dependency.** The wgpu renderer uses whatever's available (Vulkan, Metal, DirectX) but degrades gracefully on systems without a real GPU. Targets a 144 FPS render path, falls back to a sane software path otherwise.\n\n## Integration with other CDR projects\n\nPachyterm is end-user surface, not infrastructure. It connects to the rest of CDR in a few directions.\n\n- [**mae**](/blog/mae-architecture) can be the model behind the `p` prefix. A local mae instance running on the same box becomes the agent that responds when you type `p`. You get system-aware answers without leaving the terminal.\n- [**Orchestack**](/blog/orchestack-architecture) is an obvious \"remote backend\" target for the model host. When the `p` prompt warrants a heavier model or a multi-step agent flow, the model host can route to an Orchestack-hosted agent and stream the response back into the terminal.\n- [**CDRcache**](/blog/cdrcache-architecture) is a candidate plugin: caching the response for an identical `p` prompt across sessions. Most terminal-level prompts are short and recurring (`p what does this error mean`), so the hit rate would be high.\n- [**rlm-linux**](/blog/rlm-linux-architecture) ships Pachyterm in its base profile as the default terminal. The combination is the workstation experience we're building toward.\n\n## Status\n\nSource at github.com/CoastalDigitalResearch/Pachyterm. Last significant push was August 2025; this is an active project but not the most-recently-touched.\n\nWhat's built:\n\n- TTY engine with async PTY and zero-copy buffers\n- wgpu renderer with ligature support\n- Input/keymap layer with configurable bindings\n- Command parser for the `p` prefix activation\n- Model host with GGUF, MLC, vLLM backends\n- Plain-text TOML configuration with live-reload\n- Profile cache for per-model settings\n\nWhat's deferred:\n\n- Cross-platform parity. Native macOS and Linux work; Windows is on the list but not done.\n- Streaming markdown render quality. Currently markdown is rendered post-stream; ideally we'd update incrementally as tokens arrive.\n- Plugin SDK formalization. The internal extension surface exists; making it a real SDK that third parties can build against takes more work.\n- A test suite that actually covers the GPU paths reliably. Headless GPU tests are notoriously flaky.\n\n## Open questions we're working through\n\n- **Latency targets that hold under load.** The 50ms first-frame target is achievable in isolation. Whether it holds when there's a heavy SSH session, a big background build, and an active model streaming a response is the real question.\n- **Discoverability of the `p` prefix.** Users who don't know it's there will never use it. Users who do know will love it. The middle ground is a one-time hint, maybe? We don't want to be the terminal that runs a setup tutorial.\n- **Remote-backend safety.** When the model host falls back to a remote API, the prompt content leaves the machine. That's expected for the user who configured a remote backend, but it's the kind of thing that should be visible (a small indicator in the prompt UI) so it's never an accident.\n- **The right unit for plugins.** Should plugins extend the agent subsystem (new model backends, new prompt patterns)? Or extend the terminal itself (new render modes, new keybindings)? Currently both, which is probably one too many surfaces for a stable plugin contract."},{"slug":"cdrbrowser-architecture","title":"CDRbrowser: A Clean-Slate Browser Built and Maintained by AI","date":"2026-05-16","summary":"Not a Chromium fork. Not a Firefox fork. A Rust web engine paired with a native AI synthesis plane, with the codebase itself maintained by AI.","tags":["cdrbrowser","browser","rust","agents","synthesis"],"author":"Coastal Digital Research","unlisted":false,"content_md":"There are two kinds of \"AI browser\" projects right now. There are wrappers around Chromium that add a side panel for a chatbot. And there are scraping agents that drive a headless browser. Both treat the browser as a fixed substrate with AI bolted on. CDRbrowser asks a different question: what does a browser look like if the engine itself is greenfield, the AI features are native to the architecture instead of bolted on, and the codebase is maintained by AI agents using a WPT-gated CI pipeline?\n\n## Purpose\n\nCDRbrowser is a 100% AI-built and AI-maintained web browser. The engine is clean-slate Rust, not a Chromium or Firefox derivative. The AI synthesis plane is a first-class architectural component, not a feature panel. The browser ships video summaries, audio briefings, source collation, and agentic browsing as native capabilities of the runtime, rather than as features that wrap third-party APIs.\n\nThe \"AI-maintained\" framing matters too. The browser's codebase is intended to evolve through AI-authored PRs gated by the Web Platform Tests (WPT) suite in CI. Humans review, but the day-to-day work of fixing parser quirks and adding standards conformance is done by agents.\n\n## Architecture\n\nCDRbrowser splits cleanly into a web engine and an AI synthesis plane, with capability-based sandboxing as the security spine.\n\n```mermaid\nflowchart TB\n    subgraph Engine[\"Web Engine (Rust)\"]\n        Parser[HTML5 Parser]\n        CSS[CSS Layout]\n        Comp[wgpu Compositor]\n        JS[QuickJS++ JS Engine]\n        Net[HTTP/2/3 Stack]\n    end\n\n    subgraph AI[\"AI Synthesis Plane\"]\n        MCP[MCP/A2A Orchestrator]\n        Video[Video Summary Pipeline]\n        Audio[Audio Briefing Pipeline]\n        Source[Source Collation]\n        Memory[Memory Fabric]\n        AgentBrowse[Agentic Browsing]\n    end\n\n    subgraph Security[\"Security Layer\"]\n        Caps[Capability-based Sandboxing]\n        WASM[Per-site WASM Containers]\n    end\n\n    subgraph CI[\"Self-maintenance\"]\n        Bot[AI PR Author]\n        WPT[Web Platform Tests Gate]\n    end\n\n    Engine <--> Security\n    AI <--> Security\n    Engine <--> AI\n    Bot --> WPT\n    WPT -.->|gate| Engine\n    WPT -.->|gate| AI\n```\n\nThe web engine is straightforward as Rust browser engines go. HTML5 parsing, CSS layout, wgpu for composition, QuickJS++ for JavaScript, HTTP/2 and HTTP/3 for the network stack. The work is in the implementation, not the architectural novelty.\n\nThe AI synthesis plane is where CDRbrowser is different. The MCP/A2A orchestrator coordinates between the engine and a set of multimodal pipelines: video summaries pulled from page content, audio briefings generated for the visually impaired or the busy, source collation that traces claims back to references, and a memory fabric that remembers what you've already read so the synthesis layer can avoid repeating itself. Agentic browsing is the umbrella for \"the browser navigates on your behalf with a specific goal,\" which works because the AI plane shares the engine's view of the page rather than scraping HTML from outside.\n\nSecurity is capability-based throughout. Per-site WASM containers limit blast radius when a site does something hostile. Capabilities are scoped, not blanket; a site that asks for clipboard access doesn't get filesystem access by default.\n\nSelf-maintenance is the moonshot. The codebase has an AI PR author that proposes changes (parser bug fixes, conformance improvements, refactors) and gates them behind the Web Platform Tests suite. A change that breaks WPT doesn't merge. A change that improves WPT score is a candidate for review.\n\n## Design decisions we made on purpose\n\n**Clean-slate engine.** Forking Chromium would have been easier in the short term. It would also mean inheriting decades of accreted code that doesn't match how we want the AI plane to integrate. Greenfield Rust is the right substrate for AI-native browser features.\n\n**AI as architectural plane, not feature panel.** Chatbot side panels are afterthoughts. CDRbrowser's AI features have hooks into the engine itself: the renderer can ask the synthesis plane what to summarize, the synthesis plane can ask the renderer for layout structure. This isn't doable when AI is bolted on through an extension API.\n\n**WPT-gated CI.** A browser that breaks the web is worse than no browser. The Web Platform Tests are the closest thing to objective truth about whether a change broke standards. Every AI-authored PR runs against WPT and a regression on score is a blocking failure.\n\n**Capability-based sandboxing.** \"All or nothing\" permissions are how browsers got into the position where every site has access to too much. Capability-based scoping is the alternative: a site gets exactly the access it needs and nothing else, requested per-action rather than at install time.\n\n## Integration with other CDR projects\n\nCDRbrowser is a sibling to the rest of CDR work, not a building block for it. The integration story is mostly the other direction: CDRbrowser uses CDR infrastructure rather than the other way around.\n\n- [**CDRcache**](/blog/cdrcache-architecture) memoizes the synthesis plane's outputs. The \"video summary for URL X with model Y at quality Z\" call is exactly the kind of repeated, deterministic-given-inputs work that benefits from hash-addressed caching.\n- [**Orchestack**](/blog/orchestack-architecture) can host the AI synthesis plane's heavier workloads. Local pipelines for short summaries; Orchestack-routed agents for multi-step research that exceeds the browser's local budget.\n- [**TopoLI**](/blog/topoli-architecture) is a candidate for the source collation pipeline. Pruned token embeddings are exactly the kind of retrieval primitive that makes \"find related claims across pages\" tractable at browser-side scale.\n- [**CDRdistill**](/blog/cdrdistill-architecture)-trained smaller models could power the local synthesis pipelines, especially the ones that need to run inside a per-site WASM container with no network access.\n\n## Status\n\nEarly. Source at github.com/CoastalDigitalResearch/CDRbrowser.\n\nWhat's targeted for an initial usable build:\n\n- HTML5 parsing and CSS layout correct enough to render most modern sites\n- wgpu compositor with hardware-accelerated rendering\n- QuickJS++ JS engine integrated\n- HTTP/2 and HTTP/3 stack\n- The MCP/A2A orchestrator scaffold for the AI synthesis plane\n- A first video summary pipeline\n- Capability-based sandboxing scaffolding\n- AI PR author with WPT scoring in CI\n\nWhat's not yet in scope:\n\n- Full Web Platform Tests parity. The goal is \"viable for daily use against common sites,\" not \"drop-in replacement for Chrome.\"\n- DRM-protected content. Not a near-term focus.\n- A web store of third-party extensions. The plugin model will look more like capability-scoped WASM containers than the traditional extension API.\n\n## Open questions we're working through\n\n- **The viable WPT score threshold.** Above what level of WPT pass rate does CDRbrowser become useful for daily browsing? 70%? 85%? The honest answer depends on which 15% you're missing.\n- **The split between local synthesis and remote synthesis.** Some AI features only make sense with a real model behind them. How much of the synthesis plane has to be local-only, and how much can route remotely with the user's consent? The capability model gives the user fine-grained control; the defaults are the question.\n- **JS engine choices.** QuickJS++ is a sensible starting point. V8 is faster but huge. SpiderMonkey is solid but complicates licensing. As the engine matures, the JS layer is one of the places that may need to change.\n- **Self-maintenance scope.** AI-authored PRs for parser bug fixes is one thing. AI-authored PRs for security-sensitive changes is another. Where exactly do we draw the line? We'd rather draw it conservatively early and relax it as the AI author's track record builds."},{"slug":"topoli-architecture","title":"TopoLI: Topological Pruning for ColBERT-Style Retrieval","date":"2026-05-15","summary":"ColBERT stores documents as bags of token embeddings. Most of those tokens don't matter for retrieval. TopoLI uses persistent homology to find the ones that do.","tags":["topoli","retrieval","colbert","topological-data-analysis","research"],"author":"Coastal Digital Research","unlisted":false,"content_md":"ColBERT-style late-interaction models are the best-performing dense retrievers we have, but they pay for that quality in storage. A single document becomes a bag of token-level embeddings, dozens or hundreds per document, all of which have to be stored and scanned at query time. The standard answer is to compress (PQ, OPQ) or to prune somewhat-arbitrarily by frequency. TopoLI proposes a better question: which tokens are actually load-bearing for retrieval, and can we use topology to find them?\n\n## Purpose\n\nTopoLI (Topological Late Interaction) is a ColBERTv2 variant that uses persistent homology from topological data analysis to identify and keep only the tokens that contribute structural information to retrieval, pruning the rest. The result is fewer stored token embeddings per document with minimal retrieval-quality loss.\n\nThe intuition: in the embedding space of a document's tokens, some tokens are bridge points that connect distinct semantic clusters, and some tokens form boundary cycles around those clusters. Both are structurally informative. Tokens in the dense interior of a cluster are largely redundant. Persistent homology gives us a principled way to identify the structural tokens.\n\n## Architecture\n\nTopoLI is a drop-in modification to a standard ColBERT pipeline. Encoding and scoring happen as usual; an extra topological-pruning step sits between document encoding and indexing.\n\n```mermaid\nflowchart LR\n    subgraph Indexing[\"Indexing Path\"]\n        DocIn[Document]\n        Encoder[ColBERTv2 Encoder]\n        Pruner[Topological Pruner]\n        Index[(Index)]\n    end\n\n    subgraph Query[\"Query Path\"]\n        QIn[Query]\n        QEnc[ColBERTv2 Encoder]\n        MaxSim[MaxSim Scoring]\n        Results[Ranked Results]\n    end\n\n    DocIn --> Encoder\n    Encoder --> Pruner\n    Pruner --> Index\n    QIn --> QEnc\n    QEnc --> MaxSim\n    Index --> MaxSim\n    MaxSim --> Results\n\n    subgraph Pruning[\"Topological Pruner Detail\"]\n        Embs[Token Embeddings]\n        PH[Persistent Homology]\n        Score[Token Importance Score]\n        Keep[Keep Top-K Structural]\n    end\n```\n\nThe topological pruner takes the full set of token embeddings for a document and runs persistent homology over them. The output is a per-token structural importance score. The pruner then keeps the top-K most structurally significant tokens (bridges between clusters, points on topological cycles) and discards the rest.\n\nAt query time, scoring is unchanged: MaxSim between query tokens and the (now-smaller) document token bag, summed across query tokens. Because the kept tokens are the structurally important ones, retrieval quality drops far less than uniform random pruning would suggest.\n\nTopoLI ships three preset configurations:\n\n- `baseline_colbertv2()` keeps all tokens. The reference comparison.\n- `topo_aggressive()` prunes about 70% of tokens. Storage drops by roughly 3x.\n- `hybrid_topo_idf()` combines topological scoring with IDF weighting for tokens that are rare in the corpus and informative for retrieval.\n\n## Design decisions we made on purpose\n\n**Drop-in compatibility with ColBERTv2.** The query-time path doesn't change. Any system that already speaks ColBERT scoring can use a TopoLI-pruned index without code changes beyond the index format.\n\n**Topology, not just frequency.** IDF-style pruning is fast and obvious; it keeps rare tokens. The problem is that \"rare\" and \"structurally important\" aren't the same thing. Topology measures actual structural role in the embedding space of the specific document, not just statistical rarity in the corpus.\n\n**Configurable aggressiveness.** Pruning is a tradeoff. The presets cover the range from \"no pruning\" to \"aggressive\" to \"topo plus IDF combined.\" Real deployments will fall somewhere in this space depending on how much storage they're willing to trade for retrieval quality.\n\n**Apache-2.0 and reproducible.** The repo includes the full pipeline and benchmarks. Anyone can reproduce the storage-vs-quality tradeoff on their own corpus.\n\n## Integration with other CDR projects\n\nTopoLI is research-grade and connects to the more application-facing CDR work through retrieval pipelines.\n\n- [**CDRcache**](/blog/cdrcache-architecture) memoizes the persistent homology computations, which are the expensive part of indexing. Re-indexing a document corpus that mostly hasn't changed becomes substantially faster.\n- [**Orchestack**](/blog/orchestack-architecture)'s Model Router can dispatch retrieval queries to a TopoLI-backed index when the agent task involves retrieval over a corpus that has been prepared this way. The router knows which indexes are available and matches them to the query.\n- [**CDRbrowser**](/blog/cdrbrowser-architecture)'s source collation pipeline is a natural consumer. \"Find related claims across pages\" is a retrieval problem; TopoLI keeps the index small enough to ship inside the browser's resource budget.\n- [**CDRdistill**](/blog/cdrdistill-architecture) is a sister research project. Both ask \"what's the minimal representation that preserves the property we care about?\" CDRdistill asks it of model weights; TopoLI asks it of document embeddings. The mental models are similar.\n\n## Status\n\nActive research. Source at github.com/CoastalDigitalResearch/TopoLI.\n\nWhat's built:\n\n- The full TopoLI pipeline with topological pruner integrated into ColBERTv2 indexing\n- Three preset configurations (baseline, aggressive, hybrid)\n- A clean Python API with `execute_pipeline()` entry point\n- Test suite under `topoli/tests/` runnable via `uv run pytest`\n- Ruff lint and strict mypy enforced\n\nWhat's not yet in scope:\n\n- Production-scale benchmarks against MS MARCO and BEIR. The pipeline works; the published numbers come next.\n- Approximate persistent homology computation. The exact algorithm is correct but doesn't scale linearly to multi-million-document corpora yet.\n- A direct comparison against PQ/OPQ-based ColBERT compression. We expect TopoLI to win on quality at high compression ratios; we don't have the head-to-head numbers yet.\n\n## Open questions we're working through\n\n- **What's the right K?** Keeping the top-K most structural tokens is the lever. Too few and you lose retrieval quality; too many and you barely beat baseline. The optimal K likely depends on document length distribution; we don't have a clean heuristic yet.\n- **Persistent homology at scale.** The exact computation is expensive. Approximations (Vietoris-Rips with subsampling, sparse Rips) exist but introduce their own quality questions. Where on that frontier should TopoLI sit?\n- **The hybrid configuration's weighting.** Combining topological importance with IDF works in our tests; the relative weights are tuned by hand right now. A learned combination is probably better.\n- **Generalization to other late-interaction models.** ColPali, ColBERTv2.5, and other variants all share the bag-of-token-embeddings shape. The topological pruning idea should transfer, but we haven't yet validated."},{"slug":"fpre-architecture","title":"fpre: A First Principles Reasoning Engine","date":"2026-05-14","summary":"Pure LLM reasoning hits a ceiling on problems that need real symbolic structure. fpre is a tight loop between a model that proposes and a typed engine that verifies.","tags":["fpre","reasoning","rust","hybrid-systems","research"],"author":"Coastal Digital Research","unlisted":false,"content_md":"LLMs are remarkably good at intuiting answers and remarkably bad at the kinds of structured problems where every step has to be correct or the whole chain is wrong. Math proofs, dependency resolution, type checking, certain kinds of planning. The standard answer is \"scale the model\" and it works, sort of, up to a point. fpre takes a different bet: a smaller model paired with a typed reasoning engine that verifies each step is usually more reliable than a bigger model trying to bluff its way through.\n\n## Purpose\n\nfpre (First Principles Reasoning Engine) is a hybrid reasoning system. A neural component proposes candidate steps. A typed engine of primitives checks them. The loop continues until the problem is solved or the engine declares the proposed step inconsistent. The point is to push the structural correctness burden off the model and onto a system that's actually good at structure.\n\nThe architecture is split. The core is in Rust, with the engine, primitive set, scoring, and synthesis layers. The orchestration and benchmark layers are in Python, with bindings via PyO3 and maturin. The split matches the reality: the typed engine wants to be fast and rigorous, the orchestration wants to be flexible.\n\n## Architecture\n\n```mermaid\nflowchart TB\n    subgraph Python[\"Python Orchestration\"]\n        Orch[Orchestrator]\n        Config[Config]\n        CLI[CLI]\n        Bench[Benchmarks: ScienceQA, SciFact, Cladder]\n    end\n\n    subgraph Bindings[\"Bindings\"]\n        PyO3[PyO3 / maturin]\n    end\n\n    subgraph Rust[\"Rust Core (fpre-core)\"]\n        Engine[Reasoning Engine]\n        Prims[Typed Primitives]\n        Scoring[Step Scoring]\n        Synth[Synthesis]\n        Types[Type System]\n    end\n\n    subgraph Loop[\"Reasoning Loop\"]\n        Propose[Model Proposes Step]\n        Verify[Engine Verifies]\n        Score[Score Outcome]\n        Decide{Done?}\n    end\n\n    Orch --> PyO3\n    PyO3 --> Engine\n    Engine --> Prims\n    Engine --> Synth\n    Synth --> Types\n    Scoring -.->|feedback| Engine\n    Propose --> Verify\n    Verify --> Score\n    Score --> Decide\n    Decide -->|no| Propose\n    Decide -->|yes| Out[Solution]\n    Bench --> Orch\n```\n\nThe reasoning loop is straightforward. A neural model (configurable, from local 7B up to a remote large model) proposes a candidate next step toward solving the problem. The engine takes that step, checks it against the typed primitives, and either accepts it or rejects it with a reason. The scoring layer records both the model's confidence and the engine's verdict. If the step is accepted, the engine updates the problem state; if rejected, the model gets the rejection reason and proposes again. The loop continues until the problem is solved, the budget is exhausted, or the engine declares the trajectory unproductive.\n\nThe typed primitives are the heart of the engine. They're the operations the engine knows how to verify: type-checked function applications, validated set operations, range-bounded arithmetic, structurally correct logical inferences. The model can propose anything; the engine accepts only proposals that reduce to a valid primitive composition.\n\nSynthesis is where the engine assembles primitive compositions into higher-level steps. The model can suggest \"apply X to Y\"; synthesis figures out whether that suggestion reduces to a valid sequence of primitives. This is what lets the model speak in natural language while the engine still gets to verify in formal terms.\n\nBenchmarks include ScienceQA (multimodal science reasoning), SciFact (scientific fact verification), and Cladder (causal reasoning). These cover the range of problem types where hybrid neural-plus-symbolic systems tend to outperform either component alone.\n\n## Design decisions we made on purpose\n\n**Rust core, Python orchestration.** The engine has to be fast and correct. Rust gives us both. The orchestration and benchmarks want to be flexible and integrate with the broader Python ML ecosystem. Python wins there. PyO3 plus maturin makes the boundary clean.\n\n**Typed primitives, not free-form logic.** The engine doesn't try to verify arbitrary mathematical claims. It verifies compositions of a fixed, well-defined primitive set. This is both more tractable and more honest: the engine knows what it can check, and it refuses to bluff about the rest.\n\n**The model is a proposer, not an oracle.** Treating the model as an unreliable but creative source of next-step hypotheses is more productive than treating it as an authority. The engine is the authority. The model is the search.\n\n**Benchmark-driven development.** The three benchmarks aren't just for the eventual paper. They're the development feedback loop. A change that improves Cladder but breaks ScienceQA is suspect, and so on.\n\n## Integration with other CDR projects\n\nfpre is research-grade and connects to the rest of CDR through the agent reasoning surfaces.\n\n- [**rlm-linux**](/blog/rlm-linux-architecture)'s Conductor is a natural consumer. Workflow planning often requires structured reasoning (dependency resolution, version compatibility, ordering constraints) that fpre is built for. The Conductor proposes; fpre verifies.\n- [**Orchestack**](/blog/orchestack-architecture)'s Model Router could route reasoning-heavy agent tasks through fpre when the task warrants it. Most tasks don't need formal verification; some do.\n- [**CDRcache**](/blog/cdrcache-architecture) caches engine-verified step sequences. Reasoning over the same problem starting state with the same set of primitives reaches the same conclusion, every time. The cache makes that conclusion instant on the second pass.\n- [**CDRdistill**](/blog/cdrdistill-architecture) is research-adjacent. fpre asks \"what does a small model plus structure beat?\" CDRdistill asks \"what does a small grounded model beat?\" Different angles on the same question of how much you can do without scaling up.\n\n## Status\n\nActive research. Source at github.com/CoastalDigitalResearch/fpre.\n\nWhat's built:\n\n- Rust workspace with `fpre-core` (engine, primitives, scoring, synthesis, types) and `fpre-python` (PyO3 bindings)\n- Python orchestrator layer with config and CLI\n- Benchmark runners for ScienceQA, SciFact, and Cladder\n- Test suite covering the engine, orchestrator, synthesis, memory, and logical primitives\n\nWhat's not yet in scope:\n\n- Published benchmark numbers. The pipeline runs; the head-to-head against pure-LLM baselines comes next.\n- A formal verification of the typed primitive set itself. Currently the primitives are tested but not proved correct.\n- A GRPO or DPO fine-tune of the proposer model using engine-verdict signal. The reward signal exists in the scoring layer; the training pass against it doesn't yet.\n- Multi-modal primitives. ScienceQA has image components that the current primitive set doesn't handle.\n\n## Open questions we're working through\n\n- **The right primitive set.** Every primitive added expands what the engine can verify; every primitive added also adds surface area for bugs. Where's the sweet spot? Probably \"the smallest set that covers the target benchmark domain,\" but that's circular.\n- **Proposer-engine bandwidth.** The proposer sends natural-language steps; the engine has to parse them into primitive compositions. That parse step is itself error-prone. A more structured intermediate language would be cleaner but harder for the proposer to generate.\n- **Engine rejection as training signal.** When the engine rejects a proposed step, that's a strong signal the model should learn from. Getting that signal back into the proposer (online RL? batched fine-tune? in-context examples?) is an open design choice.\n- **What fpre is not.** It's not a theorem prover. It's not Lean or Coq. The typed primitive system is much weaker than either of those. The trade is that the model side can be much more capable than what a traditional theorem prover can interface with. Whether that trade is worth it for the target benchmark domains is the research question."},{"slug":"cdrmem-architecture","title":"CDRmem: Memory Access Gates and Fast-Weight Memory for Language Models","date":"2026-05-13","summary":"Less about storing vectors and more about how a model accesses what it remembers. Memory gates, fast-weight modules, and deterministic gating.","tags":["cdrmem","memory","architecture","language-models","research"],"author":"Coastal Digital Research","unlisted":false,"content_md":"When people say \"memory for language models\" they usually mean one of two things: a vector database the model can query, or a longer context window. Both are real, and neither is quite the same as how a model actually remembers things internally. CDRmem is the research project that asks a third question: what are the architectural memory patterns that should exist inside the model itself, with proper access gates and auditability?\n\n## Purpose\n\nCDRmem is research and implementation of architectural memory patterns for language models. The components include taxonomy-routed retrieval (deciding which memory to consult based on what's being asked), fast-weight episodic memory modules (short-horizon memory that can be written and overwritten without retraining), and deterministic memory gating (predictable, auditable rules for when and how memory is accessed).\n\nThe framing matters. Most memory work treats memory as an external substrate: \"the model is the inference engine, the memory is over there.\" CDRmem treats memory as architectural: gates and access patterns built into the model's computation, not retrofitted around it.\n\n## Architecture\n\nCDRmem composes three pieces: a taxonomy router that decides which memory to consult, fast-weight modules that hold short-horizon episodic context, and deterministic gates that govern when reads and writes happen.\n\n```mermaid\nflowchart TB\n    subgraph Input[\"Input\"]\n        Query[Query / Token Stream]\n    end\n\n    subgraph Router[\"Taxonomy Router\"]\n        Classify[Classify Memory Need]\n        Select[Select Memory Module]\n    end\n\n    subgraph Memory[\"Memory Modules\"]\n        FastEpi[Fast-Weight Episodic]\n        Param[Parametric Memory]\n        Retr[Retrieval Memory]\n        Cache[Activation Cache]\n    end\n\n    subgraph Gates[\"Deterministic Gates\"]\n        ReadGate[Read Gate]\n        WriteGate[Write Gate]\n        Audit[Audit Log]\n    end\n\n    subgraph Output[\"Model Forward Pass\"]\n        Forward[Inference]\n    end\n\n    Query --> Classify\n    Classify --> Select\n    Select --> ReadGate\n    ReadGate --> FastEpi\n    ReadGate --> Param\n    ReadGate --> Retr\n    ReadGate --> Cache\n    FastEpi --> Forward\n    Param --> Forward\n    Retr --> Forward\n    Cache --> Forward\n    Forward --> WriteGate\n    WriteGate --> FastEpi\n    WriteGate --> Cache\n    ReadGate --> Audit\n    WriteGate --> Audit\n```\n\nThe taxonomy router classifies the model's current memory need. Some queries need parametric memory (what the model learned during training). Some need retrieval memory (external knowledge bases). Some need fast-weight episodic memory (this session's recent context). Some need activation cache (intermediate computations from the current input). The router decides which to consult.\n\nFast-weight modules are the interesting research component. They're parameter sets that can be updated during inference on a short horizon, without backprop and without affecting the base model weights. A model can write a fact into a fast-weight module mid-conversation and read it back a few turns later. The module is wiped between sessions or on an explicit schedule. This is closer to how human working memory feels than either context windows or RAG.\n\nDeterministic gates govern access. Reads from each memory module are gated by explicit rules; writes are gated by separate rules. Every read and every write goes through an audit log. This is the part that makes memory access inspectable rather than mysterious.\n\n## Design decisions we made on purpose\n\n**Memory as architectural plane, not external substrate.** Putting memory inside the model's computational graph rather than alongside it changes what's possible. Fast-weight modules wouldn't be coherent as a separate service; they only work because they participate in the forward pass.\n\n**Taxonomy routing.** Different memory types have different cost and latency characteristics. Parametric memory is free at inference; retrieval memory is expensive; fast-weight is cheap but small; activation cache is free but volatile. The router matches the need to the right type rather than always defaulting to the heaviest option.\n\n**Deterministic gating.** Gates are rules, not heuristics. \"If the current token's classification is X and the context's recency is Y, read from module Z.\" This makes memory access predictable, debuggable, and auditable. Stochastic gates would be more flexible and less safe.\n\n**Audit by default.** Every read and every write is logged with enough detail to reconstruct the access pattern. This is essential for both research (what's the model actually doing?) and any future deployment (can we trust what it remembered?).\n\n## Integration with other CDR projects\n\nCDRmem is research that feeds into production agent memory through several integration paths.\n\n- **CDRnext** (private; the successor to both CDRmem and CDRdistill consolidating their research into a deployable architecture) is the direct downstream consumer. The memory gates and fast-weight modules from CDRmem are what we want plugged into the actual model architecture in CDRnext.\n- [**Orchestack**](/blog/orchestack-architecture)'s tiered memory system (hot, warm, cold, archive) is the storage side of the same problem. CDRmem is about how the model accesses memory; Orchestack's tiers are about where that memory physically lives. They're complementary.\n- [**CDRcache**](/blog/cdrcache-architecture) caches at the agent step level. CDRmem operates at the model-internal level. The two don't overlap, but a model using CDRmem-style memory gates produces more stable per-step outputs, which makes CDRcache more effective downstream.\n- [**fpre**](/blog/fpre-architecture)'s engine could use fast-weight modules as a working-memory substrate for in-progress proof states. That's speculative right now.\n\n## Status\n\nActive research. Source at github.com/CoastalDigitalResearch/CDRmem.\n\nWhat's built:\n\n- Architectural sketches and proof-of-concept implementations of the three memory components\n- The deterministic gate framework with audit logging\n- Taxonomy router with hand-built classification rules\n- Test harness for evaluating memory hit rates against synthetic conversation patterns\n\nWhat's not yet in scope:\n\n- Integration with a real production-scale model. Current experiments are on smaller research scales.\n- Learned classification for the taxonomy router. The hand-built rules are a starting point; a learned router is the obvious next step.\n- Cross-session memory persistence. Fast-weight modules currently wipe between sessions by design; some applications would want longer-horizon persistence with appropriate gating.\n\n## Open questions we're working through\n\n- **Fast-weight stability.** Fast-weight updates during inference can interfere with the base model's behavior in subtle ways. How small can the update be while still being useful? What kinds of writes are safe? We have some empirical answers; we don't have a clean theoretical bound.\n- **Gate complexity.** Simple gates are predictable but limited; complex gates can model more nuanced access patterns but lose the auditability advantage. The sweet spot is unclear and probably depends on the deployment.\n- **Taxonomy boundaries.** \"What kind of memory does this query need?\" sounds clean but isn't. Many queries genuinely need multiple memory types simultaneously. The router needs to handle that gracefully, which complicates its design.\n- **Relationship to RAG.** A retrieval-augmented system is essentially \"all memory access goes through one type.\" CDRmem says \"different access for different needs.\" Whether the additional complexity buys you enough on real workloads is the empirical question."},{"slug":"cdrdistill-architecture","title":"CDRdistill: H-Neuron Detection for Hallucination-Aware Distillation","date":"2026-05-12","summary":"Treating hallucination as something locatable in the network, not a black-box behavior. The result: smaller models that are measurably more grounded.","tags":["cdrdistill","distillation","hallucination","research"],"author":"Coastal Digital Research","unlisted":false,"content_md":"Distillation gives you a smaller model. Hallucination gives you an unreliable model. Most distillation work treats the second problem as a downstream concern: distill first, then bolt on RAG or fact-checking to clean up the lies. CDRdistill takes a more invasive bet: hallucination is not uniformly distributed in the model, and the components most responsible for it can be detected and treated during distillation itself.\n\n## Purpose\n\nCDRdistill is a hybrid distillation framework that uses H-subspace (hallucination neuron) detection to guide knowledge distillation. The output is smaller models that are measurably more grounded and less prone to hallucination than the same architecture distilled without H-aware guidance.\n\nThe premise is that hallucination correlates with specific subspaces of the model's activation patterns, not with the model as a whole. If you can identify those subspaces in the teacher model, you can preferentially preserve or penalize the corresponding behaviors during distillation, producing a student that inherits the teacher's useful behavior without inheriting as much of its tendency to make things up.\n\n## Architecture\n\nCDRdistill is a modification to a standard teacher-student distillation pipeline. The detection pass identifies H-neurons in the teacher. The distillation pass uses that information to adjust the loss function targets for the student.\n\n```mermaid\nflowchart TB\n    subgraph Teacher[\"Teacher Model (large, frozen)\"]\n        TForward[Forward Pass]\n        TActs[Activation Patterns]\n    end\n\n    subgraph Detection[\"H-Neuron Detection\"]\n        Probe[Hallucination Probe Set]\n        Score[Per-Neuron H-Score]\n        Mask[H-Subspace Mask]\n    end\n\n    subgraph Distill[\"Distillation\"]\n        Student[Student Model]\n        SLoss[Standard Distillation Loss]\n        HLoss[H-Aware Loss Term]\n        Combined[Combined Loss]\n    end\n\n    subgraph Eval[\"Evaluation\"]\n        Grounded[Groundedness Benchmarks]\n        Quality[Quality Benchmarks]\n    end\n\n    Teacher --> TForward\n    TForward --> TActs\n    TActs --> Probe\n    Probe --> Score\n    Score --> Mask\n    Mask --> HLoss\n    TForward --> SLoss\n    Student --> SLoss\n    Student --> HLoss\n    SLoss --> Combined\n    HLoss --> Combined\n    Combined --> Student\n    Student --> Grounded\n    Student --> Quality\n```\n\nThe detection pass uses a curated hallucination probe set: input/output pairs where the teacher's response is verifiably true or verifiably false. The activation patterns on hallucinating responses are compared against the activation patterns on grounded responses. Neurons whose activation strongly correlates with hallucinating outputs get a high H-score. The set of high-H-score neurons forms the H-subspace mask.\n\nDuring distillation, the standard loss (match the teacher's outputs) is augmented with an H-aware term that penalizes the student for producing activation patterns aligned with the teacher's H-subspace. The combined loss pushes the student to mimic the teacher in general while diverging from the teacher specifically where the teacher is most likely to hallucinate.\n\nEvaluation runs on two axes. Groundedness benchmarks measure how often the student produces factually correct answers on verifiable questions. Quality benchmarks measure that we haven't lost general capability in the process. A successful CDRdistill run shows the student stronger on groundedness without significant loss on quality.\n\n## Design decisions we made on purpose\n\n**Hallucination as a locatable phenomenon.** The whole framework rests on this premise. If hallucination were uniformly distributed across the model, there would be nothing to detect and nothing to selectively suppress. Empirically, it isn't uniform: certain neurons activate disproportionately on hallucinating outputs. We're working in that space.\n\n**Distillation-time intervention, not post-hoc filtering.** Adding a fact-check pass after the model speaks is reactive. Building groundedness into the student during training is proactive and doesn't require runtime overhead.\n\n**Probe-set-driven detection.** The H-subspace is defined empirically by what the probe set marks as hallucination. This means CDRdistill is only as good as the probe set, which is a real limitation. It also means the framework is auditable: someone can inspect the probe set and challenge the labeling.\n\n**Combined loss, not hard replacement.** The H-aware term is added to the standard distillation loss with a weighting factor, not used in place of it. The student is meant to be a useful smaller version of the teacher, not a different model that happens to hallucinate less.\n\n## Integration with other CDR projects\n\nCDRdistill is research that produces concrete artifacts (distilled, more grounded models) that the rest of CDR can use.\n\n- **CDRnext** (private) consolidates the CDRdistill and CDRmem research into a deployable architecture. The H-aware distillation technique is one of the components that informs how CDRnext approaches groundedness.\n- [**CDRmem**](/blog/cdrmem-architecture) is sister research. CDRmem is about how a model accesses memory; CDRdistill is about how that model gets to be both small and grounded. Both feed into the same broader goal: smaller, more reliable models.\n- [**Orchestack**](/blog/orchestack-architecture)'s Model Router benefits when more grounded smaller models are available. The router prefers local models when data sensitivity is high; a more grounded local model means the router can choose local more often without sacrificing quality.\n- [**fpre**](/blog/fpre-architecture)'s proposer model is a candidate target for CDRdistill. The proposer doesn't need to be a 70B; it needs to be reliable on the kinds of step proposals the engine cares about. A CDRdistill-trained proposer would be smaller and more grounded.\n- [**CDRcache**](/blog/cdrcache-architecture) memoizes the expensive parts of distillation runs. Re-running an H-detection pass against the same teacher and probe set is wasted work; the cache catches that.\n\n## Status\n\nActive research. Source at github.com/CoastalDigitalResearch/CDRdistill.\n\nWhat's built:\n\n- The H-neuron detection pipeline using probe-set activation analysis\n- The hybrid loss function combining standard distillation and H-aware penalty\n- A probe set covering common hallucination patterns (factual, citation, numerical)\n- Evaluation harness running groundedness and quality benchmarks\n\nWhat's not yet in scope:\n\n- Published numbers on a large teacher. Current experiments are on smaller teacher-student pairs; the scaling behavior to flagship-size teachers is the next question.\n- A learned probe set. The current probe set is hand-curated; expanding it through systematic generation would improve coverage but introduces its own labeling problems.\n- Integration with techniques that target hallucination through different mechanisms (retrieval augmentation, chain-of-thought verification). These are complementary; the right combination isn't obvious.\n\n## Open questions we're working through\n\n- **H-subspace stability.** Are the H-neurons consistent across training checkpoints of the same model? Across different fine-tunes? Across model families? The framework works best if the answer is \"yes, mostly.\" Empirically it seems to hold for similar models; we don't have clean cross-family data yet.\n- **Probe-set bias.** The detection inherits whatever blind spots are in the probe set. A probe set that misses an entire category of hallucination won't mark the responsible neurons. How to construct a probe set that's actually representative is an open problem and probably worth its own writeup.\n- **Compute cost.** Detection adds a pass over a curated dataset; the H-aware loss term adds compute per training step. Combined, CDRdistill is meaningfully more expensive than standard distillation. The cost-vs-groundedness trade is sensitive to model size and data scale.\n- **Relationship to RLHF and DPO.** Both of these methods can also suppress hallucination, with different mechanics and different costs. Does CDRdistill stack with them? Conflict with them? The interaction matrix isn't well understood yet."}]