BrainOS TODO

A living list of what's deliberately deferred and what's still open, as of 2026-08-16. This file used to accumulate a full narrative for every fixed incident — it grew to ~1,500 lines, most of it a changelog duplicating what git log already records in detail (this repo's commit messages are written as full incident/design writeups on purpose). Trimmed back to what's still actionable; the detailed history for anything marked "fixed" below is in git history, not here. Keep it this way: when something here gets fixed, delete the line instead of turning it into a paragraph.

Still to do

Repeat-verify Codex OSS's hardened completion path. It runs prompt-free YOLO with an execution-first contract, rejects actionless/untested results, requires a tool-created completion marker, and gets one corrective pass under the same deadline. There is no native BrainOS fallback: incomplete work fails closed into the bounded job retry path. Repeatability across models/tasks is still not fully characterized.
Measure publication yield under the CS-only mission, the 3-way focus cap (raised from 1 on 2026-08-15 — see README's "Real parallelism" section), and the same day's gap-churn/scoring/admission fixes (item 11 above). Observe whether several concurrent focus candidates and a candidate that can no longer reset itself actually increases how many programs reach a tier-2 readiness pass, which area-chair missing-work categories recur, elapsed compute-to-accepted-paper, and whether hosted steering changes the next local experiment. Run a blinded venue-specific review over every publication_ready paper; the 8–20 page and citation gates are necessary, not proof of publishability.
Finish and observe outcome-memory backfill. Automatic planning cycles continue the idempotent backfill from the first bounded batch. Monitor duplicate-reuse audit events, false-positive lexical matches, and whether cross-project repetition actually falls before increasing the comparison window.
Audit the historical failed/blocked coding backlog deliberately. recover_jobs_blocked_by_fixed_workspace_snapshot (2026-08-15) now auto-retries the one specific, already-diagnosed failure shape (a job's own review running against production instead of its snapshot); 35 jobs are still sitting blocked for other reasons as of today. Don't blindly requeue: distinguish obsolete requests, failures from superseded review policy, genuine external-resource blockers, duplicate work, and jobs worth one new attempt.
Add a small real-Postgres integration gate. The unit suite still cannot catch transaction-lifecycle and async ORM failures such as the historical MissingGreenlet incident. Cover mutate/commit/reload/ serialize, notification deduplication, and concurrent admission against throwaway PostgreSQL.
Make degraded scheduled results operationally distinct from success. A benchmark or research stage returning status=degraded can still sit inside a completed job run. Propagate degraded status into dashboard/alerts.
Install and verify a real backup owner. This host's active user timers still do not include brainos-backup.timer (confirmed still true 2026-08-15: systemctl --user list-timers lists coding, lease-recovery, dashboard-push, host-repair, paper, codex-triage, and review, but not backup). backups/runtime/ holds exactly one snapshot, from the original 2026-07-26 build — three weeks stale and the only recovery point that exists. Install the timer or document an equivalent external backup, then verify restore, not just archive creation.
Attest context consumption, not only availability. Every local generative role runs through Codex OSS with role-relevant durable state assembled into its prompt/workspace bundle. Bundle audits prove availability; add bounded evidence showing which supplied records actually influenced a result, without incentivizing indiscriminate context.
Root-cause the ~11h coding-job hang (2026-08-14/15). A job ran ~11h past its own 2h codex_oss_implementation_timeout_seconds ceiling — _implement()'s Codex CLI calls are individually wall-clock-bounded with a hard SIGKILL, so the overrun must be somewhere in _finalize() (baseline tests / independent pytest run / reviewer LLM call), none of which showed an obvious unbounded loop on inspection. A coding_job_hard_ceiling_seconds (3h) backstop now bounds the damage (CodingCycleService._run_once_bounded), but the actual hang mechanism is still unknown — worth root-causing properly rather than just bounding it, in case it recurs in a shape the backstop doesn't catch. A separate 2026-08-16 case (a process alive 3h15m) turned out to be a legitimately slow job inside the documented ~4h55m worst case (115min admission budget
this 3h ceiling — see README's "Real parallelism" section), not a repeat of this one; don't conflate the two next time something looks like this.
Remove the stale Ollama systemd drop-in. /etc/systemd/system/ollama.service.d/20-brainos-concurrency.conf still sets OLLAMA_MAX_LOADED_MODELS=2; brainos.conf (loaded after it alphabetically) sets =3 and wins today, so behavior is correct, but the superseded file is confusing leftover config. Root-owned — prepare the removal for the owner to apply per CLAUDE.md rule 9.
Observe whether the 2026-08-15 publication-lane fixes actually reach a rendered paper. Five fixes landed the same day (gap-identity churn, unattempted-work supersede guard, dead-module admission filter, literature-vs-experiment ordering, improvement-cycle task-cap) and each was individually verified against live data, but none has yet been watched carrying one candidate all the way through. Watch research_output.latest_manuscript_created_at move off 2026-08-10 and publication_focus.active_gaps[].work_module.completed_descendants climb instead of resetting to 0 on the next gap.
Decide whether to raise claude_review_effort above "medium". Added 2026-08-15 (the review's hosted Claude invocation never had an --effort argument at all before, unlike Codex's own codex_reasoning_effort), left at the CLI's own default deliberately: codex_reasoning_effort was raised to "high" once and reverted 2026-08-03 after it exhausted the account's budget, on a lane that also runs indefinitely. Watch a few review cycles' actual duration/cost at "medium" — now visibly deeper thanks to item 11's publication_focus addition — before considering "high".
Tidy the now-permanently-inert queued-job backlog (cosmetic). ~24 coding_jobs rows still show status='queued'/blocked under retired modules; the 2026-08-15 _source_module_not_retired_clause admission filter (see README's "Real parallelism" section) makes them permanently unselectable already, so this is bookkeeping only, not a functional issue.
Deliberately deferred: swap nomic-embed-text for qwen3-embedding (2026-08-16). Researched and scoped, not done: qwen3-embedding:8b (Ollama library, 4.7GB, #1 MTEB multilingual) fits alongside the qwen3.8/muse-glimmer pair with ~7.7GB VRAM to spare, but was deferred because episode/semantic-memory write volume has collapsed to ~1/day (peaked at 185/day on 2026-08-10) and nothing observed this session traced back to retrieval quality — the actual bottlenecks were the coding lane and the research pipeline, both unrelated to embeddings. Revisit once episode volume recovers and grows enough that retrieval quality plausibly matters again. If it happens: OllamaClient.embed() needs batching and a dimensions param (Matryoshka truncation to keep vector(768), or widen the column to the model's native size instead), plus a resumable migration script for the ~5,400 existing vectors (rough estimate: sub-hour run, can execute concurrently with normal coding/review work since it's an independent model instance). Also not worth pinning it resident (ollama_embedding_keep_alive_seconds) even if the swap happens later — call volume doesn't justify permanently reserving VRAM the two generative models can otherwise use.
RESOLVED 2026-08-17, answer was no. 0 of 14 tracked overnight experiment-implementation coding jobs completed; 8 permanently failed (coding_max_attempts exhausted), the rest queued/blocked. All measured failures shared one error, ValueError: Codex OSS left the workspace unchanged — not the context/model change misbehaving, but see item 16: the real limiter that night was the single coding lane, and the context raise landed the same evening as the structured_output gap that caused it. research_runs/paper_publication_runs were correspondingly stale (last rows 2026-08-15). Re-checked through 2026-08-17 16:35: the same project workspace has produced only 2 complete vs. 6 failed coding jobs since 2026-08-16 18:00 — better than zero, but still a low success rate with a distinct root cause fixed the same session (see the json_salvage.first_json_object fix, commit f2c96ce: a brace inside a JSON string value derailed the salvage parser used by, among others, the coding reviewer — confirmed live on job 58859c15's failure). Whether that fix meaningfully raises the success rate needs another live check after it's had a few hours to run, not assumed from the fix landing.
RESOLVED 2026-08-17. muse-glimmer-coding:30b-q4_K_M now reports structured_output: true (probe validated 2026-08-17 11:00, alongside qwen3.8-coding:27b), so both residents pass the coding/code_review capability gate again. No model_roles reassignment was needed or applied — role is that table's primary key (exactly one row per role), and the runtime pool was never sourced from having multiple rows there: model_registry.capable_role_model_pool computes the pool dynamically from every capability-qualified resident, using the assigned row only as a fallback/tiebreak. Confirmed live: brainos-coding.timer's poll output (worker_models) has listed both models together since, and both have been observed running concurrently in ps (one per resident, per the existing one-lane-per-resident design). The original item's proposed remedy ("reassign coder/coding_model/code_reviewer/review_model to include it") described a mechanism this table doesn't have; leaving this note so a future reader doesn't go looking for a second model_roles row that was never the right fix.
Route the 5 orphaned proposed tasks that have no CodingJob at all (found 2026-08-16 while investigating why muse-glimmer sat idle: 22 tasks were proposed, 17 correctly own a queued CodingJob and are just waiting on the single coding lane, but 5 — "Establish Ground Truth for the Vacuous-Tests Branch", "Conclusion canary", "Build Provider-Wide Hosted-Call Quota Ledger", "Mount Workspace Root in Containers", and one more — have neither a coding job nor supervisor eligibility (proposed isn't ready), so nothing will ever pick them up. Distinguish genuinely stale/superseded ones from ones that just need re-routing.
TriageService._prompt (triage.py, ~250 lines) still passes a large pre-digested bundle (tree, branch_summary, workspaces, publication_feedback as curated JSON) to a hosted agent with real tool access. Identified 2026-08-17 during an audit of every local/hosted LLM call site after the same pattern was found and fixed in the twice-daily review prompt (HostAgentReviewProvider._prompt) and the active-mode coding prompt (CodingCycleService._prompt) — not yet rewritten here, flagged instead of left undocumented. Unlike the review prompt's issue (an accreted checklist of specific past-incident field names), a real chunk of this prompt is genuine policy/semantics the output schema can't convey on its own (what modules_to_abandon vs. modules_to_supersede vs. modules_needing_resources actually mean, resource-blocked-vs-impossible criteria, tree-balance/mission-alignment judgment) — that part should stay. The strippable part is specifically the four data-dump arguments; the fix is "go query the tree/workspaces yourself" the same way the review prompt now does, not a wholesale rewrite. paper_writing.py's three stage prompts were checked too and judged lower priority — they already explicitly tell the agent to re-inspect the real workspaces with Read/Glob/Grep rather than trust their EVIDENCE_BUNDLE outright, which is most of what this fix is about. Director.plan/ProblemTreeService.decompose/owner chat are a different case on purpose (see README.md's "Concentrated context for Codex OSS" section) — non-agentic structured-output calls or truthfulness-critical live chat, not candidates for the same treatment.

Known open threads (not yet actionable; watch for recurrence)

Any caller of ollama.chat()/chat_raw() that doesn't pass model= explicitly silently rides Settings.chat_model's bare default instead of the role it's supposed to use — found twice already (Director.plan(), Reviewer.review()), both fixed. Worth an audit for a third instance before it's found live again.
A self-generated project branch can narrate a real-world step it has no actual ability to perform (e.g. "conducted usability testing with diverse learners") and nothing distinguishes the resulting fabricated-but- self-consistent numbers from a genuine finding. Not a bug in any single component — a real gap: nothing currently recognizes "this task requires real human/external input BrainOS cannot obtain" and blocks it, reframes it as simulated, or routes it to the owner. Needs a decision, not just a fix.
Approving a task moves it AWAITING_APPROVAL → READY, which re-plans from scratch rather than resuming the specific approved action. Less urgent since class 2 no longer routes through approval at all — only class 3 (external) still does, and for that there's still no owner affordance for "I did this by hand, mark it done."
Director.plan() has twice produced a plan exceeding supervisor_max_actions (fails cleanly, retried/re-planned — not a crash or data loss) without being root-caused. Worth a closer look if it keeps recurring.
An "idle in transaction" Postgres session from the research pipeline once blocked schema changes and a live job's heartbeat as a side effect of a pending DDL lock queue. Cleared once; the underlying transaction-not-closed path in the research pipeline was never identified. Watch for recurrence.
MemoryConsolidationService.consolidate()'s LLM-driven semantic distillation hadn't been observed producing a real consolidated memory in production as of its introduction; worth confirming it's actually firing now, not just eligible to.
2026-08-17's coding-lane fixes are not yet verified against a full day of production runs. Two distinct real bugs behind "Codex OSS left the workspace unchanged" were found and fixed the same day (json_salvage.first_json_object not being string-aware; Codex's own --search tool crashing under --oss --local-provider ollama), plus the review/coding prompts were rewritten to drop pre-digested context and every agent tier was given real database credentials. Check the actual coding_jobs success rate over the next full day before assuming any of this "worked" — this repo's own history (see the 2026-08-16/17 threads above) has more than once shown a fix that was real but insufficient, or a config change that looked safe and wasn't.
Job ebdfa175-0ca6-4b8e-a43f-822d647da197 (independent review of the real ONNX fault-injection experiment at experiment_2026_08_17/ in project workspace 1c692942-..., commit 93847b3 there) and problem-tree module 86ddccac (the hand-linked node wiring that job into the paper pipeline's evidence scope, status=active pending that job's verdict) were both still in flight as of this note. Check whether the job reached complete and whether paper-writing-cycle's next pass actually picked up the evidence once it did — this closes the loop the whole 2026-08-17 session was chasing, but wasn't confirmed closed before the session ended.

Deferred by design (explicit decision, not yet started)

Sleep-phase model tuning (LoRA/QLoRA, "fast methods", smaller models)

Owner decision, 2026-08-01: pushed to the far future, deliberately not scoped as near-term work. ImprovementCycleService._tuning_readiness() only ever reports "automatic_training": False — training examples accumulate (TrainingExample, tuning-export CLI command) but nothing ever launches a training job, creates an adapter, merges one, or retires one. This host's two Titan RTX GPUs (48GiB total) are already fully committed at inference time to keeping two resident models warm; there is no VRAM headroom left to also run a training job without evicting one of them, and training is far more VRAM-hungry per parameter than inference even with LoRA/QLoRA. Revisit only if a second, separate machine is ever dedicated to training. If picked up, it still needs: a reviewed, reproducible LoRA/QLoRA recipe; a bounded, cheap "does this look worth it" check before spending GPU time; and a full adapter lifecycle (create, benchmark, promote/discard, retire).

Implemented but currently dormant

Domain-expert model routing

Built 2026-07-30. The planner judges a domain (e.g. "medicine", "law", "physics") per decomposition candidate in the same structured-output call that already judges requires_implementation, stored on Module.domain. Routing (model_registry.resolve_domain_model, CodingCycleService._role_model) tries a "role:domain"-keyed model_roles row first, falling back to the plain generalist role whenever none exists or is installed. Wired into the coding pipeline's implementer role only, not the generic Supervisor/Director path or research synthesis. This is a routing mechanism only — it never installs a model, and with no domain-expert models actually installed, every domain-qualified lookup falls through to the generalist role today.

Open architecture questions (need a decision, not just code)

Sleep mode is still mostly diagnostic, for everything except memory. Memory actually gets reorganized (MemoryConsolidationService); everything else SleepService reports (stalled projects, unresolved contradictions, unmeasurable goals) is read-only findings with no automatic action. Decide how much of the rest becomes automatic (reversible only, per PERMISSIONS.md class 1) vs. stays a recommendation for the twice-daily external review to act on — which now actually can act on it directly.
Problem-tree restructuring is no longer failure-count-only. restructure_stalled still reopens a branch mechanically once every child has technically failed, but TriageService (Codex, 4x/day) now separately judges branch quality and mission-alignment directly and can grow or wind down the tree independent of technical failure. The vision's "new evidence invalidates the current plan" trigger still doesn't exist as its own thing, but is largely subsumed by that mission-alignment judgment now happening on every triage pass.
Scoring weights are fixed constants (ProblemTreeService.score: impact 0.32, probability 0.23, knowledge 0.12, confidence 0.18, resources −0.08, effort −0.07 — corrected 2026-08-15; this entry had drifted from the actual code). Reasonable starting weights, not tuned against outcomes. A natural sleep-mode candidate once enough modules complete or get abandoned: does the actual completion/abandonment rate correlate with the scores that were assigned?
score() has no concept of evidence vs. support work. Found 2026-08-15: under an active publication focus, cheap-and-certain literature/ taxonomy/provenance-manifest work structurally outscores the actual experiments a focus branch exists to produce, because requires_implementation was never a scoring input. Fixed narrowly — next_actionable now breaks ties on requires_implementation for exactly the node whose children are being ordered when that node itself carries the publication-focus tag — not as a change to score() itself, since literature-before-implementation is the right default everywhere else in the tree. Decide whether this should become a first-class, more general scoring dimension (e.g. a small bonus tied to requires_implementation any time a branch is under focus of any kind, not just publication) rather than a one-off tiebreak scoped to a single call site.