BrainOS

BrainOS

Persistent, modular AI research system.

The normative architectural source of truth is indexed in specification/README.md; operational explanations are indexed in docs/README.md.

Infrastructure

The current two-resident-model decision, live measurements, Muse Glimmer / Qwen3.6 compatibility gate, activation procedure, and rollback are documented in docs/GPU_AND_MODEL_PLAN_2026-08-11.md. The 2026-07-27 plan is retained as historical benchmark and hardware-selection context.

Deployment snapshot (verified 2026-08-16): PostgreSQL, Redis, SearXNG, API, scheduler, autonomy worker, and Matrix are running; Matrix /sync returns HTTP 200; Telegram is disabled and its profile is absent. Muse Glimmer 30B and Qwen3.8-27B are installed, assigned, and simultaneously resident at 122,880 tokens as of 2026-08-16 (raised from 32,768; see "Real parallelism" below) — as two derived Ollama models, muse-glimmer-coding and qwen3.8-coding, each with a PARAMETER num_gpu baked in so the larger context fits alongside the other resident instead of evicting it. The coding, lease-recovery, host-repair, paper-reassessment, and dashboard-push timers are installed and armed; duplicate Claude triage is disabled. Every brainos-*.timer install script now also runs scripts/ensure-linger.sh (loginctl enable-linger), so these systemctl --user timers survive logout instead of existing only while a session is open — found 2026-08-15 after the paper timer's 30-minute schedule turned out to have actual gaps of 26h, 25h, 11h, and 7h across four days, invisible to systemctl --user list-timers because running that command itself requires being logged in. The host backup timer was not installed at that snapshot and remains a current TODO until live verification says otherwise. Volatile queue, publication, quota, and paper counts belong on the live dashboard at https://simb.one/, not in this README. Exact dated engineering evidence is recorded in TODO.md.

Implemented capabilities

Current state, in one paragraph: BrainOS runs continuously, unattended, on this host today as a computer-science research system. Its approved portfolio objective is rigorous, reproducible, paper-ready work for reputable IT conferences—not generic activity across unrelated social domains. A systemd timer polls roughly once a minute and advances coding jobs in "active mode," using exactly two locally routed generative models (currently Muse Glimmer 30B and Qwen3.8-27B, via Ollama, as the -coding-suffixed derived tags — see the deployment snapshot above) to find a research gap, implement methods and credible baselines, run controlled evaluations, preserve machine-readable results, and close the next scientific evidence gap. Both are live-verified co-resident at 122,880 tokens and both have executed disposable Codex OSS edit/test/finish sessions. Role rows seed one fully symmetric resident pool: every role, including coder/coding_model/code_reviewer/review_model, round-robins across both residents whenever both satisfy that role's capability requirements — role assignment picks a fallback/tiebreak, not an exclusive owner (see model_registry.capable_role_model_pool). On 2026-08-16 a real capability gap briefly collapsed coding to one lane: coding/code_review additionally require structured_output evidence, and only qwen3.8-coding:27b had it, so muse-glimmer-coding:30b-q4_K_M dropped out of that pool specifically (every other role, which doesn't require structured_output, kept round-robining across both the whole time). That was a capability-probe gap, not a design choice, and it closed itself automatically once probed evidence caught up — reconfirmed live 2026-08-17 11:00: both residents report structured_output: true, and brainos-coding.timer's own poll output (worker_models) has listed both since. The two-lane path needs no code change to "reactivate" because it was never actually gated by anything other than that evidence. Both installed models report completion, vision, tools, and thinking, so each can perform every local generative role. Every one of those generative calls runs through pinned Codex OSS in YOLO mode. BrainOS supplies the role and relevant state; Codex owns inspection, commands, editing, compaction, and web search. No BrainOS agent, command executor, controller tool protocol, or production raw Ollama chat path remains. Busy-state tracking directs work to the idle peer and balances queued calls when both are occupied. Twice a day a separate timer runs "sleep mode": persisted-state maintenance (memory consolidation, stalled-project restructuring, stale-lease recovery), then an independent review by Claude Code CLI (sonnet, with Codex fallback) that inspects the whole system, fixes critical bugs it finds directly — edit, run the quality gate, commit — and steers the portfolio away from weak, repetitive, non-IT, or non-publishable work toward its largest novelty, method, baseline, experiment, statistics, or reproducibility bottleneck. The same hosted review response owns the bounded BrainOS improvement plan; a local model no longer reinterprets architecture findings. A separate hosted repair lane runs no more often than every six hours, with at most three hosted CLI launches in a pass. A provider-wide owner-day ledger caps BrainOS's Claude launches at eight and Codex launches at twelve by default, reserving two calls for architecture review and three for paper stages before triage or repair can consume capacity. Claude leads and Codex may fall back for both ordinary research repair and BrainOS self-improvement. The local models run inside Codex OSS with the filesystem boundary of their host service or container. Codex uses its maintained workspace/shell/edit/web tools; BrainOS supplies role-relevant durable state in the prompt or job bundle. See Host coding and review agents below and the autonomous purpose loop for the full mechanics. The rest of this section describes how each capability layer was added, oldest first.

BrainOS is a typed modular monolith with an async FastAPI API and Typer CLI, PostgreSQL/pgvector persistence, Redis execution locking, an Ollama adapter, episodic/semantic/procedural memory, project/goal/task management, approval and audit records, and a bounded supervisor cycle.

The supervisor processes at most one ready task per invocation. It validates structured director output, enforces typed-tool policy, and records every transition. Registered class-2 local actions may run automatically; class-3 external effects require human approval and class-4 actions are prohibited. Shell and workspace execution is not tier-restricted: local models and hosted agents get the same capability, scoped to the job workspace.

Run 2 adds controlled web and academic research, immutable source snapshots and citation evidence, RSS/Atom and PDF ingestion, a provider-neutral persistent model registry and deterministic router, multimodal artifact/vision routing, typed workspace tools, a hardened Docker Python sandbox, restartable sleep maintenance, model discovery and benchmarks, strongest-model strategic review, and validated tuning-dataset export.

Local setup

Prerequisites: Docker with Compose, uv, and an existing Ollama service bound to the host with nomic-embed-text and both configured generative role models available. BrainOS discovers and invokes Ollama models. Root-owned Ollama startup parameters remain owner operations. Model installation is also owner- only; an agent may delete an installed model, but only an exact tag the owner names in that request and only after confirming no role still points at it (CLAUDE.md, "Allowed scope", carve-out added 2026-08-16).

cd /opt/brain
make setup

Edit .env and replace the local PostgreSQL password in both POSTGRES_PASSWORD and the host-side BRAINOS_DATABASE_URL. Compose overrides the application connection hosts internally. Never commit .env.

Start PostgreSQL and Redis, migrate, then run the API locally:

docker compose up -d postgres redis
make migrate
make run

The local API listens only on 127.0.0.1:8000. In another terminal:

make health
uv run brain health

To run the API in Compose instead (also localhost-only):

docker compose up -d --build postgres redis api
curl --fail --silent http://127.0.0.1:18000/health

Database and CLI operations

Apply migrations:

make migrate

Create a project and retain the printed UUID:

uv run brain project-create "Research project" --description "Initial scope"

Create a task with that project UUID, retain its UUID, and move it from proposed to ready through the audited transition endpoint:

uv run brain task-create PROJECT_UUID "Investigate a claim" --description "Collect evidence"
curl --fail --silent -X POST http://127.0.0.1:8000/tasks/TASK_UUID/transitions \
  -H 'content-type: application/json' \
  -d '{"state":"ready","actor":"owner"}'
uv run brain supervisor-cycle

The manual HTTP equivalent is:

curl --fail --silent -X POST http://127.0.0.1:8000/supervisor/cycles

Interactive API documentation is at http://127.0.0.1:8000/docs.

Quality checks

make format
make lint
make typecheck
make test

./scripts/install-git-hooks.sh   # enforce this gate on every commit, rebuild
                                  # containers automatically when brain/src changes

Docker builds copy the working tree, not git HEAD, so a change can end up running in production without ever being committed — this happened once in the now-removed native agent's context-compaction code, which ran uncommitted and broken in production for an unknown stretch. Build application images with ./scripts/docker-build.sh (used automatically by the post-commit hook, and by hand for a manual rebuild), not a bare docker compose build: it bakes the exact git commit and a brain/src dirty-file count into the image, readable at GET /health's build field and logged at every service's startup, and the same check is included in the twice-daily review's data snapshot with an explicit instruction to commit or revert anything found dirty. The post-commit hook also warns immediately if brain/src is still dirty right after a commit.

Run the opt-in PostgreSQL/pgvector integration test after exporting a URL that is reachable from the host:

export BRAINOS_TEST_DATABASE_URL='postgresql+psycopg://brain:YOUR_PASSWORD@127.0.0.1:15432/brain'
uv run pytest -m integration -q

Configuration

All application variables use the BRAINOS_ prefix. Important values are BRAINOS_DATABASE_URL, BRAINOS_REDIS_URL, BRAINOS_OLLAMA_URL, BRAINOS_CHAT_MODEL, BRAINOS_EMBEDDING_MODEL, BRAINOS_OLLAMA_CHAT_KEEP_ALIVE_SECONDS, BRAINOS_OLLAMA_EMBEDDING_KEEP_ALIVE_SECONDS, BRAINOS_LOG_LEVEL, BRAINOS_SUPERVISOR_MAX_ACTIONS, and BRAINOS_SUPERVISOR_MAX_RETRIES.

On this host the two generative models use chat keep-alive -1 (resident until explicitly unloaded) and embeddings use 0 (unload immediately). The root-owned Ollama service must allow three loaded slots: two generative models plus the transient embedding model. This does not authorize a third generative model; measured VRAM permits only two.

General web search additionally requires BRAINOS_RESEARCH_PROVIDER and BRAINOS_RESEARCH_BASE_URL; credentials belong only in the untracked .env. Supported JSON adapters cover Brave, Tavily, Serper, Bing-compatible and SearXNG services. Academic adapters support arXiv, Crossref, OpenAlex, Semantic Scholar and PubMed metadata without paid test dependencies.

The schema fixes embedding columns at 768 dimensions for the configured nomic-embed-text model. Every Ollama embedding response is checked before it can be inserted; changing models therefore requires an explicit dimensional migration and matching BRAINOS_EMBEDDING_DIMENSIONS value.

Run 2 commands

# Cited general web research (after provider configuration)
uv run brain research "What evidence addresses this question?"

# Academic metadata and local PDF ingestion
uv run brain academic-search "retrieval augmented generation" --provider crossref
uv run brain pdf-ingest /tmp/brainos-artifacts/paper.pdf --render-directory /tmp/brainos-artifacts/pages

# Registry and deterministic routing
uv run brain models --refresh
uv run brain model-route vision

# Build and use the network-disabled, non-root sandbox
make sandbox-build
printf 'print(6 * 7)\n' >/tmp/example.py
uv run brain python-run /tmp/example.py

# One-shot workflows
uv run brain sleep-run manual-2026-07-26
uv run brain model-discover
uv run brain benchmark MODEL_UUID
uv run brain strategic-review
uv run brain tuning-export /tmp/brainos-artifacts/tuning.jsonl

The corresponding APIs are POST /research/runs, POST /research/academic, POST /research/pdf, GET /models, POST /models/refresh/ollama, POST /models/route, and POST /sleep/runs. See specification/RESEARCH.md, specification/MODELS.md, and specification/OPERATIONS_RUN2.md.

Run 3 adds owner-approved missions, durable pause/emergency controls, a checkpointed autonomous worker, a persistent timezone-aware scheduler with dispatch/retry/recovery for every schedule, environment observations, an optional owner-only Telegram or Matrix long-polling daemons (commands and ordinary text), a local dashboard, verified backups, and user-systemd/Compose deployment.

Autonomous discovery generates multiple evidence-backed computer-science research opportunities, ranks them using BrainOS-owned scientific value, novelty, feasibility, and learning factors, and automatically starts one bounded evidence task when its threshold is met. Each opportunity must name a literature gap or falsifiable hypothesis, contribution, public data or benchmark, credible baselines, metrics, validity threats, reproducibility artifacts, and a first experiment. Generic apps, reports, directories, and toolkits are not contributions, and unit tests are not scientific results.

Publication is now a closed feedback loop rather than a once-daily prose attempt. The paper timer reassesses changed candidates every 30 minutes by default; unchanged source fingerprints cost no hosted calls. A negative area- chair or repair-review verdict becomes a durable, deduplicated publication gap. One candidate is focused by default, and its exact missing experiment, baseline, evidence, or reproducibility work is preferred by the Cortex until changed evidence resolves the verdict; pivots replace weak questions and stop verdicts retire them. The dashboard shows mission areas, eligible anchors, run outcomes, active/deferred gaps, hosted calls, and publication-ready papers.

# Core scheduled workflows (see below for installers)
./scripts/run-brain.sh brain coding-cycle          # active mode: drain both resident-model lanes
./scripts/run-brain.sh brain sleep-review-cycle    # maintenance + independent host-agent review
./scripts/run-brain.sh brain host-coding-repair-cycle # exhausted/self-improvement jobs
./scripts/run-brain.sh brain improvement-cycle     # host-agent review pass
./scripts/run-brain.sh brain problem-scan          # decompose the mission tree further
./scripts/run-brain.sh brain paper-writing-cycle   # readiness, draft, repair/review, render

# Everyday operational visibility
./scripts/run-brain.sh brain status                # task/coding-job/queue snapshot
./scripts/run-brain.sh brain doctor                # environment and dependency checks
./scripts/run-brain.sh brain autonomy              # inspect/control the autonomous worker
./scripts/run-brain.sh brain scheduler             # inspect/control the persistent scheduler
./scripts/run-brain.sh brain service               # manage installed systemd units
./scripts/run-brain.sh brain backup                # verified backup operations
./scripts/run-brain.sh brain workspace-cleanup     # preview reconstructible cache cleanup
./scripts/run-brain.sh brain workspace-cleanup --apply # preserve source/results/Git history
./scripts/run-brain.sh brain work-submit / work-retry # owner-originated coding jobs
./scripts/run-brain.sh brain mission --adopt-default --owner owner # adopt mission
./scripts/run-brain.sh brain mission / project-create / task-create # manual seeding

Run ./scripts/run-brain.sh brain --help for the complete, current command list; the ones above are the ones most relevant to understanding how the system actually runs day to day, not a full reference.

POSTGRES_* is the production credential source of truth. Compose and scripts/run-brain.sh construct the application URL without printing it; this keeps API, workers, migrations, CLI, and the initialized volume consistent.

See the autonomous purpose loop, operations, recovery, security, architecture, and policies.

Matrix owner interface

Matrix is the deployed owner transport on this host. It uses outbound HTTPS /sync long polling and opens no inbound port. Only the configured owner user ID in the configured private room is accepted, and the bot ignores its own events. Telegram remains implemented for compatibility but is disabled and its Compose profile is not started. Free-text replies use a 220-token generation ceiling and a 1,200-character transport ceiling. Markdown is normalized to plain text, and unsolicited fenced/code-shaped output is withheld unless the owner explicitly asks for code.

See MATRIX_SETUP.md for the complete setup and test procedure.

Use /dashboard for the combined live snapshot. Additional structured commands include /workers, /events, /errors, /resources, /network, /proxy, /memory, /tools, /repositories, /improvement, and /needs-user. Ordinary chat uses durable owner conversation history and answers the question directly. Operational questions and explicit status commands receive a bounded runtime snapshot; stale or unavailable operational data is reported explicitly. Status answers about a specific task or job walk that task's problem-tree ancestry (title/statement/rationale at every level up to the mission) so "what are you doing and why" is answered from real stored planner output, never invented at chat time.

External status dashboard (static, pushed over SSH)

Separate from the Matrix /dashboard chat command above: dashboard.py's DashboardService renders a static HTML snapshot (problem-tree counts and top active modules, recent coding jobs with errors, each job's workspace top-level contents, and README/TODO/ARCHITECTURE rendered from Markdown) and pushes it via rsync over outbound SSH to an owner-controlled external webserver, refreshed by brainos-dashboard-push.timer every 20 minutes. This exists because api_host is hard-enforced to loopback (Settings.secure_configuration) and the institute firewall cannot be changed, so nothing on this host can be reached directly from outside it - the dashboard is pushed out instead of served, the same outbound-only shape Matrix already uses.

The canonical public dashboard and paper site is directly at https://simb.one/. It does not require BrainOS, a /brainOS path, or any other dashboard host in the URL. The configured SSH destination may still use the server-side directory /home/daniel/brainOS/; that filesystem path is an implementation detail and must not be presented as part of the public URL.

Configure via BRAINOS_DASHBOARD_PUSH_* in .env (see .env.example); BRAINOS_DASHBOARD_PUSH_ENABLED=false (the default) makes the CLI command and timer a no-op. Institute traffic must go through the mandatory HTTP(S) proxy (webproxy.berlin.ptb.de:8080), which also tunnels arbitrary CONNECT targets including raw SSH — BRAINOS_DASHBOARD_PUSH_SSH_PROXY_COMMAND carries that as an nc -X connect -x host:port %h %p ProxyCommand. Install or remove the timer with scripts/install-dashboard-push-timer.sh / scripts/rollback-dashboard-push-timer.sh; run it once by hand with brain render-and-push-dashboard. See TODO.md's "Dashboard delivery to an owner-controlled webserver over SSH" for the live proxy-tunneling investigation this is built on and its known limitations (no auth on the pushed pages, uses the owner's general SSH key rather than a scoped one).

Host coding and review agents

Two different tiers do implementation work. They differ in cost and capability, not in what CLAUDE.md permits: execution capability is the same for both, while rule 11 still requires approval for external side effects. The no-root and Ollama/firewall/SSH restrictions also bind both alike.

Active mode (the ~1-minute brainos-coding.timer loop, coding-cycle) uses locally hosted resident models exclusively through Codex OSS in YOLO mode, confined to the job's own disposable or shared-project workspace. It admits up to two jobs at a time — one per resident capable of the coding task category — through separate model and database-session lanes, while a database exclusion prevents two lanes from writing the same workspace. As of 2026-08-16 that capability check also requires structured_output evidence, not just tool_use, so coder/code_reviewer briefly resolved to a single lane (qwen3.8-coding:27b) while muse-glimmer-coding:30b-q4_K_M still read structured_output: false; the two-lane code path itself never changed and needed no fix to reactivate, only fresh evidence. Measured live that evening: this single-lane bottleneck was not theoretical - 18 real experiment-implementation tasks (not scaffolding; titles like "Implement fixed atol/rtol baseline plus one candidate localization method, with fault injections") sat queued behind it at once, each CodingJob correctly queued with its Task correctly proposed (coding-owned tasks are deliberately excluded from the generic Supervisor's own eligibility query, so proposed there is expected, not stuck) - the real constraint was purely one active lane. Reconfirmed live 2026-08-17 11:00: a fresh capability probe gives muse-glimmer-coding structured_output: true, and both models have appeared together in worker_models on ordinary brainos-coding.timer polls (and running concurrently in ps) since — the second lane is back without any code change, exactly as designed. It explicitly excludes any job whose requested_by starts with improvement-cycle: — a local model never edits BrainOS's own source, in active mode or otherwise. A local reviewer model independently checks each job before it can be marked complete; a job that exhausts its retry budget (coding_max_attempts) becomes failed and waits for the next tier. That review sees a real pytest run, not the implementer's self-report — and, because problem-tree jobs share one workspace per project, it sees that run attributed: the same suite is executed once before the job starts, so the reviewer is told which failures are newly introduced and which were already there. Only newly introduced ones are grounds for rejection. The faster autonomy/Cortex loop applies cross-pipeline backpressure before creating more coding work. Two distinct pending workspaces supply the two local-model lanes; multiple sequential jobs in one shared Git tree count as one runnable lane. If the Cortex cannot find a second workspace, BRAINOS_PROBLEM_TREE_MAX_PENDING_CODING_JOBS is the hard queue ceiling (default six), while the unmaterialized problem tree remains the durable backlog.

Sleep mode (brainos-review.timer, twice daily) is where a frontier hosted agent CLI runs—Claude Code first, with Codex fallback inside the same cycle. The timer runs exactly twice daily and a durable owner-timezone limit rejects manual/catch-up attempts beyond two architecture-improvement cycles per day. The successful hosted response owns both assessment and plan, with no separate local planning call. This tier gets real write, edit, and shell access to /opt/brain — the same trust an engineer doing this work by hand would have — so it can fix a critical bug it finds directly (edit, run make format && make lint && make typecheck && make test, commit). It also inspects every shared project workspace for accumulated duplicate/divergent files (a historically observed implementation failure mode) and is authorized to consolidate that directly, in that workspace's own git repo. It never deploys, promotes a model, spends money, or initiates external communication without approval. The architecture review call uses Settings.claude_review_model—sonnet in both repository defaults and this host's untracked .env. The former live opus override was removed after the hosted-call audit showed that architecture review, repair, triage, and paper writing could collectively launch Claude up to nineteen times per day. Each of the two daily cycles accepts one structured review result; its response includes the improvement plan, so no local model subsequently plans BrainOS architecture. Its reasoning depth is now configurable (Settings.claude_review_effort, added 2026-08-15, threading the CLI's --effort argument — Claude's invocation never set one at all before, unlike Codex's codex_reasoning_effort) but deliberately left at "medium", the CLI's own default and hence a no-op today: codex_reasoning_effort carries the matching warning from the other direction, having been raised to "high" once and reverted 2026-08-03 after it exhausted the account's budget, on a lane that runs indefinitely. Its snapshot also now includes publication_focus — each active publication gap's disposition, age, prior continue-verdict count on that scientific module, gap rows created in the last 48h, and its work module's completed/live descendant counts — the same treatment research_output already got for whether a manuscript has rendered at all, so a reviewer no longer has to reconstruct a stalled candidate's gap-churn history from scratch to notice it. Hosted repair is a distinct lane, configured by BRAINOS_CODING_PROVIDER and its fallback. Its six-hour cycle (repair_stuck_jobs, at most 3 hosted calls per cycle, the same "implement" purpose active-mode local retries never reach) also uses sonnet (Settings.claude_coding_model). Both used to be opus: repair moved off it 2026-08-02, after one nominally "twice-daily" cycle placing up to 4 opus calls contributed to exhausting the account's rolling session limit overnight (an unrelated bug, see TODO.md, had inflated how many jobs needed repair that night); review itself moved off opus to sonnet on 2026-08-03, for the account's overall token budget rather than any one incident. The 2026-08-14 quota audit removed the stale live override.

Triage (TriageService in triage.py, provider-agnostic) runs through Codex four times daily (brainos-codex-triage.timer). The duplicate Claude triage timer may remain installed, but its lane is disabled by default; when explicitly enabled it uses sonnet and shares the global Claude launch budget. The twice-daily review pass is separate. Both triage implementations are deliberately narrow, single-agent passes, not a fallback list — the point is a specific agent's own pass at a specific time, not "whichever answers first" — and all timers are offset from each other so no two ever compete for the same host-agent CLI quota at once. Neither touches BrainOS's own architecture or source; that remains confined to the review and dedicated self-improvement repair lanes. Each pass sanity-checks the live problem tree (reporting, not directly editing, any module that is simply incoherent or depends on a resource BrainOS can genuinely never obtain — recruiting real people, for instance) and repairs real bugs, including broken or fabricated-looking external links and citations in workspace documentation, inside coding-job workspaces under /opt/brain/workspaces/. A module that is not fundamentally impossible but is blocked on one specific, nameable resource only the owner can provide (a paid API key, a payment processor account, hosting, licensed data, ...) is never abandoned — it is marked blocked (Module.blocked_reason), the owner is notified via NotificationService naming exactly what is needed, and it stays available to resume once provided, while the tree moves on to a genuinely different branch instead of idling on siblings that hit the same wall. It also reads a per-branch completion/activity summary and may nudge priority_score on specific modules (tree_rebalancing) when a small number of branches have absorbed nearly all of next_actionable's greedy selection while others sit undeveloped for no structural reason — including, since 2026-08-01, when a branch has simply drifted from the current mission's actual wording, not just when the tree is mechanically imbalanced. Two further, newer capabilities give the tree's own shape and lifespan real judgement rather than only ~10 fixed top-level branches that only ever end in abandonment: it can add a new, already-scored top-level branch directly (ProblemTreeService.add_root_branch, bypassing decompose()'s own LLM call since triage's candidate already exists — bounded to 3 per run) when real, mission-aligned territory is unexplored, and it can conclude a branch that has produced genuine value and has diminishing returns left (ProblemTreeService.conclude_branch — a new concluded module status, distinct from both completed and abandoned) by abandoning its other live descendants and giving it exactly one final task: a report or paper synthesizing what it accomplished, with the branch itself only becoming concluded once that finishes. Each run's own prompt input is kept deliberately tight: a short, fixed grounding line (TriageService. _mission_text, the mission statement, one query, not a general system essay) rather than nothing, and a workspace listing pre-sorted by last-modified time and capped (MAX_WORKSPACES) rather than an unranked directory dump the agent would otherwise have to explore blind with its own bounded tool-call budget. It runs in Codex's --dangerously-bypass-approvals-and-sandbox mode, or Claude Code's always-on bypassPermissions. This is now true of every Codex and Claude launch, not only triage; Codex also bypasses hook-trust prompts. The systemd writable-path allowlist and credential-scrubbed environment are the external sandbox. These CLI flags do not bypass BrainOS approval for external effects.

Triage is now explicitly a scientific-program steering pass. It sees recent area-chair rejections and their missing-work lists, treats feasible work with no defensible CS contribution as superseded, and asks the local loop for the next experiment or artifact that changes paper readiness. superseded differs from both abandoned and deletion: the work and evidence remain auditable, but the branch no longer competes for compute. Approving a materially new mission retargets the root and applies the same transition to unfinished old-mission modules, cancellable tasks/jobs, and active opportunities, preventing a mission row from changing while the actual queue silently continues the old objective.

Scientific publication (PaperWritingService in paper_writing.py, Claude with hosted Codex fallback, brainos-paper.timer, change-gated every 30 minutes) is a third and separate lane — neither the review lane above nor either triage lane writes papers, and this lane never touches BrainOS's own architecture or source. It replaces relying on paper_pipeline.py's older gate (a branch reaching Module.status == "concluded", which live data showed had fired zero times across roughly five thousand modules — see purpose.py's _research_output_health) with anchoring directly on a current-mission depth-2 problem-tree module (depth-3 when a depth-2 subtree exceeds MAX_ANCHOR_SUBTREE_DESCENDANTS). A preflight requires at least eight usable evidence items and a completed program, inventories actual result artifacts, and independently reruns each distinct workspace's discoverable pytest suite once. Content hashes keep timestamp-only output churn from invalidating a cached rejection. The first hosted stage acts as a senior area chair: it may reject the work or persist a complete tier-2 study plan with the large problem, motivation, gap, claims, baselines, datasets/workloads, metrics, statistical treatment, validity threats, exact rerun commands, evidence map, outline, and target venues. Claude leads each stage and Codex retries the same schema after provider failure. A passing plan permits one drafting stage and one repair/review stage—three successful stages maximum. Provider attempts are separately audited. All stages and the source fingerprint are durable, so an outage resumes rather than restarts and an unchanged weak result consumes no more calls. A rejection now creates a durable PublicationGap: continue creates one focused problem-tree branch for the missing evidence or experiment, pivot retires the weak framing and creates the named falsifiable replacement, and stop supersedes the direction. The default single focus slot boosts the gap and its ancestors ahead of unrelated breadth until the evidence fingerprint changes; further gaps wait in a durable deferred queue. The final reviewer has full shell/write access throughout the workspace tree while BrainOS source remains read-only. It must attempt feasible repairs to the paper, programs, analyses, and artifacts, verify them, apply its revisions, and judge the repaired paper. Optional future strengthening and honest limitations do not block a sound bounded contribution. Deterministic validation rejects unsupported or altered citations, short or long drafts, leaked orchestration details, failed scientific review, and PDFs outside the actual 15-20 page range. Only then is publication_ready set; the dashboard hides every legacy or rejected draft.

Because this is the only scheduled lane that runs on the host rather than in a container, it also owns rendering for the other two. coding_workspace_root is a host path that the scheduler and worker containers do not mount, so a manuscript drafted there is saved as render_deferred and rendered on this lane's next run. Frequent reassessment is inexpensive because unchanged fingerprints consume no hosted call. Nothing is lost, but only quality-gated papers are downloadable.

The agent subprocess receives a strict environment allowlist. As of 2026-08-17 (owner instruction, given after the tradeoff was raised explicitly) that allowlist includes real database credentials (BRAINOS_DATABASE_URL, POSTGRES_*) for every tier, hosted and local alike, so any agent can read and write BrainOS's own durable memory itself (brain memory-record, or a direct query) instead of a bounded job's findings being reachable only through its own summary field. Every owner-interface/chat token (BRAINOS_MATRIX_ACCESS_TOKEN etc.) remains excluded regardless of tier — that stays a separate concern from database access.

Every local generative role now uses the selected Muse/Qwen model through Codex OSS (codex exec --oss --local-provider ollama). Codex receives the same 32K working-window limit and runs with --dangerously-bypass-approvals-and-sandbox plus hook-trust bypass. The coding user service's writable-path allowlist is the external sandbox, so the process never asks for execution permission but remains non-root and OS-confined. Tasks target a tested result within 30 minutes. Codex OSS and one verification-guided corrective pass share a two-hour hung-process safety ceiling. A timeout kills the complete CLI process group, preventing an orphan from continuing to write; the old 45-minute wall stopped a live, GPU-active job and was removed. Live verification on 2026-08-12 found Muse actively editing files and running pytest through Codex OSS. Qwen launched via the same Codex OSS path but its first strict disposable smokes returned before all requested changes/artifacts/test evidence existed. BrainOS now rejects that result. Acceptance requires real workspace changes, reported test commands, and a Codex-created completion marker that is removed before persistence. BrainOS repeats the full task with exact verification feedback once. If both framework passes fail, the job follows its normal bounded retry path; no BrainOS-owned agent or command executor bypasses Codex. Claude/Codex hosted agents remain scheduled, bounded review/triage/paper/exhausted-repair lanes; they are not inserted into ordinary active-mode retries.

Live repeat verification after hardening produced one complete Qwen Codex OSS pass followed by one pass that edited and tested successfully but again stopped before its requested artifacts. The pure Qwen/Codex wire path is therefore not claimed repeat-reliable. The production path is fail-closed: incomplete results are rejected and retried through Codex OSS rather than accepted or handed to a second local framework.

Root-caused 2026-08-16. --output-schema validates the model's final message against the supplied schema and silently drops it from stdout if it fails — it does not error, retry, or fall back to raw text; Codex simply prints nothing. Reproduced live against a scratch repo: qwen3.8:27b completed a trivial task correctly (edited the file, verified it) but answered with a markdown "Result" block instead of bare JSON, and the correct answer sat in the CLI's own transcript while coding.py received an empty string. 48h of production coding_jobs logs showed every single implementation attempt landing on muse-glimmer instead of the assigned qwen3.8:27b (idle failover always preferred whichever resident wasn't currently busy, and qwen3.8:27b was tied up serving five other roles), and every one of those attempts failed the identical way. Two independent fixes landed the same day: agent_cli.py's CodexAgent.run() now also captures --output-last-message (the raw final message, written regardless of schema validation) and salvages a JSON object out of it via first_json_object whenever stdout comes back blank — the same embedded-JSON-salvage pattern Reviewer.review()/Director.plan() already used for direct Ollama calls, just never applied to the Codex-OSS coding path before; and TASK_REQUIREMENTS["coding"/"code_review"] now also requires a structured_output capability, recorded false for both qwen3.8:27b and muse-glimmer:30b-q4_K_M (evidence: this reproduction) so idle failover can no longer hand schema-critical work to a resident already falsified in production. Full incident writeup in the 35cf38c/3b38225 commit messages, per this repo's convention of putting the detailed narrative in git log rather than here.

Concentrated context for Codex OSS

More prompt text is not treated as more knowledge — and as of 2026-08-17 (owner instruction, following a same-day live investigation), that principle was pushed further than "concentrated": the twice-daily review prompt and the active-mode coding-lane prompt no longer carry any pre-digested context bundle at all. Both real bugs found that day were found by ignoring BrainOS's own curated context and verifying the live system directly - a publication_steering field silently null across a module pivot, and a contradiction counter that reads zero only because its detector needs exact string equality. A curated bundle the agent is told to trust is exactly the failure mode that let both survive unnoticed. Every agent, hosted or local, also now has real database credentials in its environment (see "Host coding and review agents" above for the environment-allowlist change), specifically so it doesn't need a bundle assembled for it - it can query the database itself.

This makes the task-relevant durable subset accessible while preserving attention and KV cache for the task itself. Raise context beyond 32K only when measured agent traces show relevant retrieved material is still being truncated.

Implementation streams progress: each tool use is persisted with a heartbeat, so /work JOB_UUID shows the current step while a long run is still going, and a stalled job raises a notification (once per attempt, deduplicated) instead of failing silently — coding_stall_seconds, raised from 15 to 45 minutes 2026-08-16 after it paged the owner for jobs that were merely slow, not dead. This is a warning threshold only, unrelated to the 45-minute hard kill-wall mentioned above that was removed 2026-08-12 for the opposite reason (it stopped genuinely live work); nothing here kills a running job early. Failed attempts are requeued with backoff and the previous failure is fed into the retry prompt. Token usage and cost are recorded per job.

Real parallelism, not just separate schedules

A dedicated host-coding-repair lane is the sole automated hosted-repair path; the twice-daily review no longer adds hidden repair passes between its timer ticks. It runs concurrently with the active-mode local coding-cycle — see CodingCycleService._repairable_job_query. The host-coding-repair-cycle timer uses an owner-selected six-hour minimum interval. Hosted-agent quota is scarce even when local GPU time is not, so the installer no longer launches an immediate repair call. Each repair pass admits at most three hosted CLI invocations total (BRAINOS_HOST_REPAIR_MAX_AGENT_CALLS_PER_CYCLE=3), counting both primary and fallback attempts; three jobs can therefore never silently become six calls. Both admission queries exclude a candidate whose workspace another job is already running in (_workspace_not_already_running_clause, a self-join on workspace_path, no separate lock table), so two lanes running at once can never end up as two concurrent writers in the same shared project git working tree. Both also exclude a candidate whose source problem-tree module is already abandoned/superseded/blocked/concluded (_source_module_not_retired_clause, added 2026-08-15): neither query had filtered on module status before, and nothing retires a queued job when its branch dies — recover_stale only ever looks at jobs already running — while both lanes order oldest-first, so dead work sorted first. Measured live before the fix: 11 immediately selectable queued jobs from 2026-08-05..09, all under abandoned modules, all ahead of that day's live research work, and 195 of the repair lane's 265 candidate rows (74%) belonged to dead branches, competing for three admissions a pass with the self-improvement queue that lane exists to drain.

Local model concurrency has a real, measured hardware ceiling on this host: two big models — currently qwen3.8-coding:27b and muse-glimmer-coding:30b-q4_K_M, the model identities behind these role names have changed more than once, check brain model-role-list rather than trusting a tag named here — run genuinely concurrently, verified by timing simultaneous requests (reconfirmed 2026-08-16: 26.7s+15.1s sequential vs. 26.1s wall concurrent, i.e. the slower model's own solo time, not the sum). A third concurrent model load while both are already resident caused an eviction and a stall rather than clean queuing, not just extra latency — this is about a third loaded model, not OLLAMA_NUM_PARALLEL, which only bounds how many concurrent requests one loaded model serves and has no bearing on whether two different resident models run together (they do). OllamaClient therefore tracks a per-model busy flag in Redis (visible across the separate OS processes that each talk to Ollama — the coding cycle, the review/repair cycles, and the Matrix chat daemon are never the same process), and owner chat can use the other capable resident when its selected model is busy. Codex OSS requests are covered by the same busy marker even though the CLI talks to Ollama directly. Every capable resident shares every generative role by cross-process round robin, with the two coding lanes pinned one per resident so a selection/reservation race never hands both lanes the same worker. As of 2026-08-16 the coding/code_review task categories additionally gate on structured_output evidence (see the dated note earlier in this section); that gate briefly left those two roles resolving to qwen3.8-coding:27b alone while muse-glimmer-coding still read structured_output: false, reconfirmed true 2026-08-17 and back to two lanes since — the pool computation itself was never role-exclusive, only temporarily short one qualifying resident. owner_chat/fast_model are the one deliberate, permanent exception: pinned to the same tag as coder so only one Qwen instance is ever resident (a third loaded model evicts one of the other two — see below), not because owner chat is incapable of sharing. Every other role (planner, researcher, summariser, critic, ...) shares across both residents as before. A third dedicated chatbot is neither needed nor part of the target role map, and the model roster is exactly these two generative residents plus nomic-embed-text — every non-current tag (qwen3.5:9b, qwen3.6:27b-q4_K_M, qwen3:32b, the Devstral GGUF) was deleted with all role assignments stripped first (2026-08-16).

The "third model load evicts" finding turned out to generalize: a bigger context request for an already-2-resident model evicts the other one too, live-confirmed 2026-08-16 — requesting 65,536 or 131,072 tokens for one resident dropped both GPUs to 0MB used, not a graceful resize, with reload times ranging 47s to 240s under real concurrent load. Root cause: Ollama's scheduler doesn't try to pack a larger context alongside the other resident even when total VRAM says it should fit. Fixed with num_gpu (the model's own transformer-layer count, unrelated to host CPU count): holding back a handful of a model's layers from GPU — they compute on CPU/RAM instead, of which this host has 251GB/48 cores mostly idle — frees enough VRAM for the other resident to stay loaded. Binary-searched live at 122,880 context: qwen3.8:27b (65 layers total) needs num_gpu<=~55 to coexist with muse-glimmer resident; muse-glimmer:30b-q4_K_M (52 layers) coexists even fully GPU-loaded. Verified both -coding variants loaded simultaneously at full 122,880 context with ~10GB VRAM still free, correct output (17*24=408, unprompted) confirming the CPU-offloaded layers don't corrupt results, at 15-18 tok/s — roughly 30% of full-GPU speed, but on a lane with up to a 2-hour budget that is still ~105,000 generated tokens per attempt, more than a single exploration-only pass used before this fix. Codex CLI has no per-request num_gpu passthrough (checked its full config surface), so this is baked into two derived Ollama models instead of passed at call time — qwen3.8-coding:27b (num_gpu=52) and muse-glimmer-coding:30b-q4_K_M (num_gpu=50), each built with ollama create from a Modelfile referencing the existing weight blobs (no extra disk cost) — and every role now points at the -coding tag instead of the bare model name. codex_oss_num_ctx moved from 32,768 to 122,880 to match (see that setting's docstring for the full 88,649-cumulative-token measurement that motivated it).

The exact upgrade, compatibility gate, historical smoke, role map, residency settings, and rollback are documented in docs/GPU_AND_MODEL_PLAN_2026-08-11.md (reconciled 2026-08-16). Codex OSS owns model interaction and tool execution, so BrainOS no longer normalizes model-specific textual function-call formats or carries a parallel read/create/edit/run loop. Actual activation passed ollama show capability confirmation and the live disposable smoke on 2026-08-12; a model card alone would not have been sufficient evidence.

./scripts/install-coding-worker.sh       # active-mode local implementation loop
./scripts/install-review-timer.sh        # twice-daily independent review
./scripts/install-codex-triage-timer.sh  # four-times-daily tree/workspace triage
./scripts/install-claude-triage-timer.sh # optional duplicate triage transport; disabled
                                          # by default to preserve Claude capacity
./scripts/install-host-repair-timer.sh   # host-agent job repair, no more often than every
                                          # six hours; this is its sole automated owner
./scripts/install-paper-timer.sh         # fingerprint-gated publication reassessment;
                                          # default every 30 minutes
./scripts/install-lease-recovery-timer.sh # stale coding-lease recovery, every 3 minutes,
                                          # independent of the coding-cycle timer itself

Every installer above also runs scripts/ensure-linger.sh first, which is not a standalone timer — it enables systemctl --user's systemd lingering for the owning user (loginctl enable-linger, no privilege required) so these brainos-*.timer units keep running across logout instead of existing only while a login session happens to be open.

A single active publication-focus candidate turned out to be a monopoly, not a concentration: pinning one branch's whole ancestor chain to top priority let it win every future selection against every sibling until its gap resolved, and it starved the coding lanes of independent (differently workspaced) work in the same stroke — measured live 2026-08-14/15, one branch held the sole focus slot for ~24h across 6 "continue" verdicts. BRAINOS_PAPER_FOCUS_MAX_CANDIDATES is now 3, and PublicationSteeringService.CONTINUE_MAX_ATTEMPTS (3) forces a branch that keeps failing readiness for the same reason to pivot/stop instead of re-asking forever. Separately, CodingCycleService._run_once_bounded adds a coding_job_hard_ceiling_seconds (3h) backstop around each lane's job: every external call inside a job is already individually wall-clock-bounded, but one measured job still ran ~11h past its own 2h implementation ceiling, apparently hung after implementation succeeded, and blocked its lane's admission the whole time; on timeout the job is simply abandoned mid-flight for the ordinary stale-lease recovery path to reclaim.

The real per-process worst case is the sum of two separate budgets, not either one alone — coding_cycle_drain_seconds (115 min) only bounds how long coding-cycle keeps admitting new jobs, checked between jobs, never while one is running; a job admitted right at that boundary can then still run the full coding_job_hard_ceiling_seconds (3h) before the backstop above fires. Worst case is therefore ~4h55m, not 115 minutes — nothing states that combined figure anywhere else, which made a real, otherwise unremarkable 2026-08-16 case (a job legitimately still inside its 3h allowance, at 3h15m elapsed) look like a hang until traced through both settings. It wasn't one: the job was slow because it had landed on muse-glimmer under the pre-fix role assignment (see the dated structured_output note earlier in this section) and was killed manually to unstick the lane under the corrected code rather than wait out the remaining ~1h40m of its own legitimate allowance.

The 3-way cap above stopped one candidate monopolizing the focus slot; it did not stop that candidate resetting itself, found 2026-08-15 after five days with zero rendered manuscripts despite correct decomposition into exactly the concrete experiments six review cycles kept demanding. PublicationSteeringService matched a continue verdict to its live gap only when the area chair's next_research_question was byte-identical (modulo whitespace/case) to the previous verdict's — the area chair is an LLM, not a deterministic string generator, so it restated two real questions eight ways across one 22-hour window, and two consecutive gaps differed by exactly one character in 523 (an em-dash where the next run wrote an ASCII hyphen). Each restatement minted a new PublicationGap, whose _resolve_replaced_gaps superseded the previous work module and cancelled its never-attempted coding jobs — the ONNX fixed-tolerance-baseline and fault-injection-dataset modules were superseded 93 minutes after creation with every job still at attempt_count 0. _questions_are_equivalent now compares content-word overlap instead of exact prose (0.6 threshold, calibrated against that incident's own data: rewordings scored 0.92–1.00, the genuinely different prior question 0.25–0.27), and ProblemTreeService.supersede_branch now shields a branch whose coding jobs have never been attempted for supersede_unattempted_grace_hours (12h, bounded so a genuinely starved queue can still be retired; an owner-approved mission change can still opt out with protect_unattempted=False).

Once that direct cause was fixed, a second one surfaced inside the same focus branch: next_actionable's only tiebreak beyond the publication-focus tag itself was raw priority_score, and score() rewards cheap and certain work — exactly literature/taxonomy/provenance scaffolding, never the experiment a focus branch exists to produce. The candidate's literature child scored 0.6175 with 7 live children while all nine implementation children (the pinned runtime substrate, the baseline, the fault-injection dataset) scored 0.43–0.57 with zero children each, so next_actionable for the entire tree returned a citation-extraction leaf three levels inside the literature branch. requires_implementation now breaks the tie between the focus tag and raw priority, scoped to exactly the node whose children are being ordered when that node itself carries the tag — deliberately not a global scoring change, since literature-before-implementation is the correct default for the rest of the tree.

Neither bug would have been caught by the twice-daily review that exists specifically to catch this failure class, for an unrelated third reason: ImprovementCycleService._activate_safe_tasks capped new self-improvement tasks against a count of every task the "BrainOS continuous improvement" project had ever held, completed and failed included, so the cap only ever grew — 20 completed + 15 failed + 12 proposed + 2 blocked + 1 running was exactly 50/50 against task_limit_per_project, and both of 2026-08-15's review cycles, including the one covering this exact outage, completed successfully and returned "portfolio_limit_reached", discarding whatever either found. Now excludes COMPLETED/FAILED/CANCELLED from the count.

Verified backups are scheduled only by the host-user brainos-backup.timer installed through make service-install; the matching PostgreSQL scheduler row is forced disabled so the non-root container does not race the host timer or fail on /opt/brain/backups. Compose-only operators must schedule the documented host CLI backup command separately — this host is Compose-only and has not, so as of 2026-08-08 there is no verified backup at all. Giving backup exactly one owner is correct; assuming that owner exists was not. The daily owner summary now reports the age of the newest verified backup and names a missing one as a fault (RuntimeDispatcher._backup_freshness, bounded by BRAINOS_BACKUP_MAX_AGE_DAYS, default 2), so a schedule with no installed owner cannot stay silent again.

See the owner coding workflow, the autonomous purpose loop, and CLAUDE.md, which holds the provider-neutral engineering contract for every agent working in this repository and defines exactly which tier "the AI" refers to in each rule.

Enterprise HTTPS proxies

BrainOS can add organization-managed CA roots to every Python HTTPS client without replacing the normal trust store. Place PEM certificates or bundles in the Git-ignored certs/ directory, or set BRAINOS_EXTRA_CA_CERT to a PEM path inside that read-only mount, then recreate the application containers. With no extra certificates configured, startup is a no-op and normal public PKI trust is unchanged.

See the enterprise proxy deployment guide and implementation report. TLS verification must never be disabled.

Production start

cd /opt/brain
cp -n .env.example .env
# Set POSTGRES_PASSWORD and the owner-only BRAINOS_MATRIX_* values in .env.
# Leave BRAINOS_TELEGRAM_ENABLED=false.
make setup
docker compose up -d postgres redis searxng
make migrate
./scripts/run-brain.sh brain mission --adopt-default --owner owner
./scripts/run-brain.sh brain scheduler --initialize
./scripts/docker-build.sh api scheduler worker matrix
docker compose --profile matrix up -d api scheduler worker matrix
curl --fail --silent http://127.0.0.1:18000/ready

If the package index is temporarily unreachable but a previously verified BrainOS image is present, use the documented offline overlay procedure in docs/OPERATIONS.md; it replaces application source only and explicitly reports optional dependency degradation.