GGL Docs

Jello Architecture

12 min read

Requirements

  • Vibe coding platform hosted web application
  • Mobile and desktop. No code and IDE workflows.
  • Chat integration Discord + Signal
  • Single pane for all subscriptions including harness specific subs like claude.
  • Mult-modal
    • Images
    • TTS
    • Video (chop into frames for the agent with TS)
    • PDF
  • Multi-user isolation
  • shareable sessions
  • Keep secrets unaccessible from an agent.
  • Backend uses GoLang - Embeds frontend
  • Frontend uses next.js and tailwind CSS - SPA
  • Backend/frontend app runs statelessly
  • k8s first deployment

Development phases

Phase 1

A thin vertical slice, not a feature. Prove that a chat message can spin up a Coder workspace, reach an agent inside it over agentapi, and stream a response back — for one user, one conversation, one harness. Everything else waits on this working.

Scope:

  • Go backend that embeds the built Next.js frontend, served statelessly.
  • Minimal Postgres schema: just conversation + workspace records.
  • Coder workspace lifecycle (create/start/stop) wired through the agentapi control-channel port-forward — the newest, riskiest integration, so it gets validated first.
  • One coding harness wired end-to-end (e.g. Claude via agentapi).
  • Auth: dev-mode/stubbed. Credentials: simplest viable path (agent holds its own key) — the Envoy masked-credential proxy is deferred, not skipped.
  • TODO: Outlining testing approach CRITICAL early on so we can drive dev with testing.

Explicitly out: Discord/Signal/GH-bot integrations, UI Styling, multi-modal input, session sharing, PR tracking, real OIDC/Coder-identity mapping, credential proxy.

Success criteria: send a message, watch a workspace spin up, get a streamed response, and confirm the conversation survives a backend restart (proves statelessness). If that loop holds, every later phase is additive, not foundational.

Full plan path: /persistent/workspaces/ggl_core_infra/src/2_app/services/jello/plans/phase-1-core-skeleton.md

Phase 2

the credential/security layer — replaces phase 1’s “agent holds its own key” shortcut with real isolation. Your notes already have the most design detail on this piece, and it’s the one hard requirement (“secrets inaccessible from the agent”) that’s expensive to retrofit once more phases build on top of it.

What’s being built:

  • Envoy egress gateway in front of LiteLLM, GitHub, and MCP servers — one origin host per service under *.aigateway.internal…
  • Per-conversation masked tokens (format-shaped so client-side validation still passes); the workspace never holds a real credential. Fallback to generic shaped uuid4 token.
  • ext_authz: Envoy asks Jello “is masked token T valid on route R?”, Jello resolves the real credential from Postgres, Envoy re-originates TLS upstream with it injected. No MITM, no private CA in the workspace.
  • Rotation/revocation = a Postgres row update, live on the next request, no agent restart.
  • Git stays out of this entirely — SSH via coder gitssh, not header injection.
  • Blocks: GitHub PR tracking, real MCP tool use, and any externally-reachable surface (Discord/Signal/GH bot) — all raise the cost of a credential leak, so none should ship before this exists.

Still deferred: real multi-user OIDC/Coder-identity mapping (phase 3 candidate — phase 1’s dev-mode user is fine here), multi-harness support, sharing.

Success criteria: phase 1’s harness reaches LiteLLM + GitHub MCP only through the gateway; rotating a key in Postgres takes effect with no agent restart; the workspace has zero usable real credentials in env or on disk.

Full plan path: /persistent/workspaces/ggl_core_infra/src/2_app/services/jello/plans/phase-2-credential-proxy.md

Phase 3

Real multi-user identity, replacing phase 1’s dev-mode/stubbed single user. Almost every “multi-user” requirement (isolation, shareable sessions, per-user conversation ownership) is unimplementable until this exists, so it’s the next blocking piece rather than a nice-to-have.

What’s being built:

  • Generic OIDC login (the old jello already proved this pattern — port the approach, not the code, from the deprecated TS packages/api/src/auth/{oidc,cookies,guards,session}.ts).
  • Resolve the open design question: is Coder the identity provider for Jello, or does Jello register its own OIDC client and map identities to Coder separately?
  • Wire the conversation owner field for real; scope conversation listing/access per authenticated user instead of the phase-1 single-tenant assumption.
  • Implement sharedWith: a list of {user, expiry} on each conversation, with presets (4hr / 1 day / 7 days / forever), enforced server-side and auto-revoked on expiry.
  • Decide and implement how coding agents authenticate to Jello’s own native tools (MCP) — scoped so an agent-facing token can’t be replayed to reach APIs beyond that one conversation’s entitlements.

Blocks: sharing UI, per-user conversation isolation, any multi-tenant deployment, and linking external chat identities (Discord/Signal/GH bot need a real Jello user to attach to).

Blocked by: phase 1 (needs the backend skeleton + conversation model to attach auth middleware to). Independent of phase 2, though both eventually meet at “how does an agent’s tool call authenticate.”

Still deferred: Discord/Signal/GH-bot integrations themselves, multi-modal input, full PR-tracking UI.

Success criteria: two real users log in via OIDC and each sees only their own conversations; a conversation shared for a fixed duration is accessible to the second user until expiry, then automatically inaccessible; an agent’s native-tool/MCP calls are confirmed scoped to their own conversation and can’t reach another user’s data or the sharer’s full account access.

Full plan path: /persistent/workspaces/ggl_core_infra/src/2_app/services/jello/plans/phase-3-identity-and-sharing.md

Phase 4

Additional client surfaces — Discord, Signal, and a GitHub bot/App — as alternate front-ends to the same conversation API the web client uses. No new backend concepts; this is adapters plus identity linking on top of phases 1–3.

What’s being built:

  • Account linking: a one-time code flow that ties a Discord user ID / Signal number / GitHub username to an existing Jello OIDC identity (phase 3) — unlinked senders are rejected, not silently given a conversation.
  • Discord adapter: bot via the Discord API/gateway, DM or channel thread maps 1:1 to a Jello conversation; streams by editing/following-up messages the same way the web UI streams.
  • Signal adapter: no real “bot” concept here — needs a phone-number-registered client (signal-cli or equivalent), which is architecturally the odd one out of the three and probably the long pole of this phase.
  • GitHub bot: a GitHub App responding to @mentions on issues/PRs via webhook (not persistent connection); comment-thread maps to a conversation; likely posts once rather than streaming, or edits a “thinking…” comment to the final result.
  • All three reuse phase 2’s credential proxy and phase 3’s auth as-is — bots are just another authenticated caller, not a new trust boundary.
  • Scope to explicitly-authorized guilds/numbers/repos rather than open registration.

Blocks: little downstream — this is the last piece of “single pane for all subscriptions,” not a dependency for other phases.

Blocked by: phase 3 (needs a real identity to link external accounts to).

Still deferred: multi-modal handling (images/PDF/video arriving over these surfaces still need phase-5 processing), multi-harness selection, PR-tracking UI.

Success criteria: a linked Discord user DMs the bot and gets a live conversation with streamed responses; same for Signal; a GitHub @mention on a PR creates/continues a conversation and posts the result back as a comment; an unlinked identity is rejected.

Full plan path: TBD

Jello agent Lifecycle

Creation

  1. User sets a workspace configuration
    • Define template and template parameters.
  2. User starts a conversations by sending a message.
  3. On first message a conversation is created and a workspace spawned.
    1. DB should hold a model of conversation
      • ID
      • workspace configuration
        • Presets available similar to claude code cloud.
      • Workspace ID
      • owner
      • sharedwith
        • List of
          • IDs this session is shared with
          • duration that ID should have access (preset UI options 4hr, 1 day, 7 days, forever)
      • archived - bool
      • timestamps
        • created
        • last modified
        • stopped (workspace stop time)
      • git prs
        • List of git prs (this will be passed back by the model through tool calls)
        • This will be used by the UI as a part of conversation state tracking.
          • Thinking, Waiting for user, PR Open, PR closed
          • Note: may contain multiple PRs if it does show the lowest state (eg. open)

Stopping

  1. If a workspace stops it should be started again when a new user message is sent.
  2. As a workspace starts the build logs for the workspace should be streamed to the jello UI so the user can monitor for progress and errors.
  3. A conversation may be archived by the user, this should mark the conversation DB record as archived and be hidden by default from the user. When a conversation is archived the backing workspace should be deleted.
    • Need a way to denote un-pushed changes. If a git repo is present check if theres’ unstaged or unpushed changes. If so warn the user before archiving or deleting.
      • These outstanding changes should be captured using the same mechanism as the git diff tab.

Failure

If a workspace fails to start bubble that error with the already streamed build log up to the user.

Resuming

  1. If an archived conversation is unarchived, and then a new message is sent to the conversation a new workspace should be spun up

Jello git integration

  • How does jello connect to github.com and the local git repo
    • local git repo
      • Needs to see diffs from origin/main
      • Needs to see unpushed changes
    • github
      • Track pull requests - Jello native tool allowing the agent to add PRs to it’s tracked list

Jello and Coder auth and identity

  • Jello and Coder identity should be closely mapped
  • Can coder be used as an identity provider for jello?
  • Can jello use the REST API with a users OIDC identity to leverage the required APIs?
  • How do coding agents auth to Jello for native tools?
    • MCP? - Auto auth for the user? - Ensure the MCP auth can’t use user intended APIs.
  • Can coder re-use MCP token for REST API interactions?

Jello ⇄ agent connectivity

1. Jello → agents (control channel, via coder agentapi)

  • Every coder workspace runs a coder agent that holds a persistent outbound connection to the coder server. Nothing in the workspace ever listens on the network — no 0.0.0.0, no exposed ports. All inbound operations ride over that existing outbound connection.

  • Jello authenticates to coder via its existing service token (the CF-Access service token / coder API key routes/coder.ts already uses) and drives workspace lifecycle through coder REST (start/stop/delete, already exercised today).

  • To reach an agent inside a workspace, Jello uses coder port-forward <workspace> --tcp <local>:<remote> per conversation. It creates a local TCP socket in the Jello pod that tunnels over the coder agent’s agentapi connection to 127.0.0.1:<remote> in the workspace. The port-forward resolves the workspace’s agent ID automatically; the agent-api inside binds loopback only.

2. Agents → outbound (data path, through an Envoy-like egress proxy)

  • Jello operates an Envoy egress gateway with one per-service origin host under a gateway domain: litellm.aigateway.internal.galaxygridlabs.com, mcp-github.aigateway.internal.galaxygridlabs.com, mcp-mem0.…, mcp-chat-notify.…, github.aigateway.…. Served by a wildcard cert *.aigateway.internal.galaxygridlabs.com signed by the ggl CA, which workspaces already trust (NODE_EXTRA_CA_CERTS).

  • Workspace config — Jello writes models.json/MCP catalog for pi; the template writes claude’s native config (ANTHROPIC_BASE_URL, .mcp.json per-server url, env) — points every credential-bearing service at its proxy host with a masked per-conversation token (format-shaped per service: sk-ant-…, ghp_…, sk-… so client-side validation passes). The proxy is agent-agnostic: it doesn’t know whether pi or claude is behind it.

    • ANTHROPIC_BASE_URL - Might break native claude integrations (anthropic expect you to use their SaaS for things like “remote-control” or other advanced features if so claude token may need to be exposed in the environment and that’s an acceptable risk.)

  • No MITM. Envoy terminates TLS at its own cert (client → Envoy), then re-originates a fresh TLS connection to the real upstream (Envoy → upstream), injecting the real credential on the upstream leg. There is no private root CA in the workspace, nothing impersonates upstream hostnames, nothing breaks cert pinning. It’s explicit proxying by config, not transparent interception.

  • ext_authz (per request): Envoy asks Jello “masked token T on route R?”; Jello returns the real credential (MCP OAuth token, LiteLLM key, GitHub token) from Postgres. Mid-session updates are table updates — key rotation, MCP OAuth refresh, server revocation, tightened ALLOWED_MODELS — effective on the next request, no agent restart.

  • 1:1 host → upstream means no path rewriting: /v1/messages, /v1/models, /mcp/, SSE all pass through untouched; only the credential header is swapped per route (Authorization → x-api-key for LiteLLM, etc.). If an upstream validates the Host header, a per-route host_rewrite to the real hostname fixes it. Streaming/WS/SSE pass through natively. The plan gate gains a server-side twin: ext_authz can deny mutating MCP calls or non-allowed models at the network layer, audited.

  • git is not coverable by a header-injecting proxy (the credential lives in the URL inside TLS) — the template ships git-credential-jello, a per-operation short-lived credential helper, the same pattern coder uses for external auth.

    • Skip this git should use ssh credentials which coder gitssh already resolves no need to intercept these.

3. The hostname story: what github.aigateway… vs api.github.com select

The hostname an agent dials is pure config, and it selects the credential mode for that workspace:

ModeAgent dialsCredentialsWorkspace holdsUse case
Gateway (default)github.aigateway.internal.galaxygridlabs.com (per service)Injected by Envoy per request from Jello’s PostgresMasked tokens only — useless outside the proxypi- and claude-hosted workspaces; zero-credential policy
Directapi.github.com (real upstreams)Workspace’s own real credentialsReal tokens in workspace configPersonal/trusted workspaces, BYO-token dev boxes, offline envs
Hybridper-service choiceMixedMixede.g. MCP via gateway, git via credential helper, LiteLLM on a personal key

Key point: the OAuth identity is bound to the OAuth app (the client-id Jello registered), not the hostname the agent used.

The user authorizes on real github.com through Jello’s flow (/api/v1/mcp/oauth/callback); the token lands in Postgres and the proxy injects it regardless of which hostname the agent dialed. So switching a workspace between github.aigateway… and api.github.com changes only who injects credentials — the gateway vs the workspace’s own config — never the token’s validity.

(Note the GitHub MCP server calls GitHub REST backend-to-backend; only the MCP leg traverses our proxy, which is where the credential swap happens.)

The two hostname sets therefore let one fleet carry both tenants: workspace configurations that must not see credentials (managed, masked tokens, everything through *.aigateway…) and configurations that opt into holding their own (direct upstreams), mixed per service as needed — with the gateway enforcing allow/deny, rotation, refresh, and audit for everything it fronts.

References

https://github.com/coder/agentapi https://gateway.envoyproxy.io/docs/tasks/security/credential-injection/

Graph View

Backlinks