Architecture Overview¶
little-loops is a Claude Code plugin providing development workflow automation with issue management, code quality commands, and parallel processing capabilities.
Related Documentation: - Command Reference - All slash commands with usage - API Reference - Detailed class and method documentation - Troubleshooting - Common issues and solutions - README - Installation and quick start
System Components¶
The system consists of three main layers:
- Command Layer - Slash commands, skills, and agents for Claude Code
- Automation Layer - Python CLI tools for batch processing
- Configuration Layer - JSON-based project configuration
High-Level Architecture¶
flowchart TB
subgraph "Claude Code Plugin"
CMD[Commands<br/>29 slash commands]
AGT[Agents<br/>9 specialized agents]
SKL[Skills<br/>41 composable skills]
end
subgraph "Configuration"
CFG[ll-config.json]
SCHEMA[config-schema.json]
TPL[little_loops/templates/*.json]
end
subgraph "Python Automation"
CLI[cli/<br/>Entry points]
AUTO[issue_manager.py<br/>Sequential processing]
PARALLEL[parallel/<br/>Parallel processing]
end
subgraph "Issue Storage"
ISSUES[.issues/<br/>bugs/, features/, enhancements/, epics/]
end
CFG --> CMD
TPL --> CFG
SCHEMA -.->|validates| CFG
CFG --> CLI
CMD --> AGT
CMD --> SKL
CLI --> AUTO
CLI --> PARALLEL
AUTO --> ISSUES
PARALLEL --> ISSUES
Directory Structure¶
little-loops/
├── .claude-plugin/
│ └── plugin.json # Plugin manifest
├── commands/ # 29 slash command templates
│ ├── help.md
│ ├── check-code.md
│ ├── run-tests.md
│ ├── scan-codebase.md
│ ├── normalize-issues.md
│ └── ...
├── agents/ # 9 specialized agents
│ ├── codebase-analyzer.md
│ ├── codebase-locator.md
│ ├── codebase-pattern-finder.md
│ ├── consistency-checker.md
│ ├── loop-specialist.md
│ ├── plugin-config-auditor.md
│ ├── prompt-optimizer.md
│ ├── web-search-researcher.md
│ └── workflow-pattern-analyzer.md
├── hooks/ # Lifecycle hooks and validation scripts
│ ├── hooks.json # Hook configuration
│ ├── prompts/
│ │ └── continuation-prompt-template.md # Handoff prompt template
│ ├── adapters/ # Host-specific adapters → little_loops.hooks dispatcher
│ │ ├── claude-code/
│ │ │ ├── post-tool-use.sh
│ │ │ ├── pre-tool-use.sh
│ │ │ ├── precompact-handoff.sh
│ │ │ ├── precompact.sh
│ │ │ ├── session-end.sh
│ │ │ └── session-start.sh
│ │ ├── opencode/ # OpenCode TS plugin adapter (Bun runtime)
│ │ │ ├── index.ts # Plugin: session.created → session_start, session.compacted → pre_compact
│ │ │ ├── package.json
│ │ │ ├── tsconfig.json
│ │ │ └── README.md
│ │ └── codex/ # Codex CLI bash adapter — scripts and hooks.json moved in-package (FEAT-2274/BUG-2275)
│ │ └── README.md
│ └── scripts/ # Hook scripts
│ ├── check-duplicate-issue-id.sh
│ ├── check-duplicate-issue-id-post.sh
│ ├── context-monitor.sh
│ ├── precompact-state.sh # Legacy shell handler; replaced by adapters/claude-code/precompact.sh
│ ├── scratch-pad-redirect.sh
│ ├── scratch-cleanup.sh
│ ├── session-cleanup.sh
│ ├── session-start.sh # Legacy shell handler; replaced by adapters/claude-code/session-start.sh
│ ├── user-prompt-check.sh
│ └── lib/
│ └── common.sh # Shared shell functions
├── skills/ # 41 skill definitions + 29 command bridges — force-included into the wheel at
│ # little_loops/skills/ for ll-mcp's prompts surface (BUG-3177);
│ # stays physically here (host-plugin glue, FEAT-2274/BUG-938)
│ #
│ # Representative sample below — run `ls skills/` for the full
│ # set. Each skill is a directory with SKILL.md plus optional
│ # companion files (SKILL.md is capped at 500 lines; overflow
│ # extracts to a companion — ENH-494).
│ ├── capture-issue/ # Proactive
│ │ ├── SKILL.md
│ │ └── templates.md
│ ├── create-loop/ # User-invoked
│ │ ├── SKILL.md
│ │ ├── loop-types.md
│ │ ├── reference.md
│ │ └── templates.md
│ ├── manage-issue/ # User-invoked
│ │ ├── SKILL.md
│ │ └── templates.md
│ ├── confidence-check/ # Proactive
│ │ ├── SKILL.md
│ │ └── rubric.md
│ ├── ... # ~60 more, one directory per skill
│ │
│ │ # `ll-`-prefixed skills are Codex bridges: thin wrappers
│ │ # mirroring an existing skill onto a host that discovers
│ │ # skills differently — not separate capabilities.
│ ├── ll-capture-issue/ # Codex bridge → capture-issue
│ │ └── SKILL.md
│ └── ll-go-no-go/ # Codex bridge → go-no-go
│ └── SKILL.md
└── scripts/ # Python package
└── little_loops/
├── __init__.py
├── cli/ # CLI entrypoints (package)
│ ├── __init__.py
│ ├── harness.py # ll-harness one-shot runner evaluation CLI
│ ├── auto.py
│ ├── create_extension.py # ll-create-extension scaffold CLI
│ ├── parallel.py
│ ├── messages.py
│ ├── session.py # ll-session: search/recent/backfill/path the unified session store
│ ├── history_context.py # ll-history-context: render Historical Context block for an issue
│ ├── sync.py
│ ├── docs.py
│ ├── history.py
│ ├── deps.py # ll-deps entry point
│ ├── output.py # Shared CLI output utilities (colors, terminal width)
│ ├── sprint/
│ │ ├── __init__.py # Entry point (main_sprint) + argparse
│ │ ├── _helpers.py # Shared utilities (cli/sprint's own — unrelated to cli/loop)
│ │ ├── create.py # create subcommand
│ │ ├── edit.py # edit subcommand
│ │ ├── manage.py # delete, analyze subcommands
│ │ ├── run.py # run subcommand
│ │ └── show.py # list, show subcommands
│ ├── issues/
│ │ ├── __init__.py # Entry point (main_issues) + argparse
│ │ ├── list_cmd.py # list subcommand
│ │ ├── next_id.py # next-id subcommand
│ │ ├── count_cmd.py # count subcommand
│ │ ├── search.py # search subcommand
│ │ ├── sequence.py # sequence subcommand
│ │ ├── impact_effort.py # impact-effort subcommand
│ │ ├── show.py # show subcommand
│ │ ├── refine_status.py # refine-status subcommand
│ │ ├── append_log.py # append-log subcommand
│ │ ├── anchor_sweep.py # anchor-sweep subcommand (CLI wrapper)
│ │ ├── fingerprint.py # fingerprint subcommand (CLI wrapper)
│ │ └── epic_progress.py # epic-progress subcommand
│ ├── loop/
│ │ ├── __init__.py # Entry point (main_loop) + argparse
│ │ ├── run.py # run subcommand
│ │ ├── runner.py # background/foreground run orchestration, dry-run plan
│ │ ├── signals.py # SIGINT/SIGTERM/SIGWINCH handling
│ │ ├── feed.py # diagram/pinned-pane rendering + StateFeedRenderer
│ │ ├── header.py # artifact/path header helpers
│ │ ├── summary.py # run-completion summary printing (usage/A-B/cross-host)
│ │ ├── queue.py # process-backed run queue
│ │ ├── config_cmds.py # validate, install
│ │ ├── lifecycle.py # status, stop, resume
│ │ ├── info.py # list, history, show
│ │ └── testing.py # ll-loop test/simulate subcommand utilities
│ └── logs.py # ll-logs: discover/extract/sequences/stats/tail/dead-skills/scan-failures subcommands + index generation
├── cli_args.py # Argument parsing
├── config.py # Configuration loading
├── state.py # State persistence
├── logger.py # Logging utilities
├── logo.py # CLI logo display
├── frontmatter.py # YAML frontmatter parsing
├── decisions.py # Decisions and rules log data layer (FEAT-1891)
├── decisions_sync.py # Decisions sync and session start integration (FEAT-1895)
├── learning_tests.py # Learning test registry (CRUD for .ll/learning-tests/)
├── doc_counts.py # Documentation count utilities
├── link_checker.py # Link validation
├── issue_manager.py # Sequential automation
├── issue_parser.py # Issue file parsing
├── issue_discovery/ # Issue discovery and deduplication (package)
│ ├── __init__.py # Re-exports public API
│ ├── matching.py # Types and text similarity helpers
│ ├── extraction.py # Git history analysis and regression detection
│ └── search.py # Issue file search and discovery functions
├── issue_lifecycle.py # Issue lifecycle operations
├── issue_progress.py # Epic progress aggregation
├── issue_history/ # Issue history and statistics (package)
├── git_operations.py # Git utilities
├── work_verification.py # Verification helpers
├── text_utils.py # Text processing utilities
├── pii.py # PII detection and redaction utilities
├── subprocess_utils.py # Subprocess handling
├── host_runner.py # Host CLI abstraction (HostRunner Protocol + ClaudeCodeRunner + CodexRunner + GeminiRunner + OmpRunner + KimiRunner + QwenRunner + OpenCodeRunner + PiRunner)
├── sprint.py # Sprint definition and management
├── sync.py # GitHub Issues sync
├── goals_parser.py # Goals file parsing
├── dependency_graph.py # Dependency graph construction
├── dependency_mapper/ # Cross-issue dependency discovery (sub-package)
│ ├── __init__.py # Re-exports for backwards compatibility
│ ├── models.py # Data models (DependencyProposal, FixResult, etc.)
│ ├── analysis.py # Conflict scoring and dependency analysis
│ ├── formatting.py # Report and graph formatting
│ └── operations.py # File mutation operations (apply/fix)
├── issues/ # Issue utility sub-package (ENH-1300)
│ ├── __init__.py # Package init
│ ├── anchors.py # resolve_anchor(): language-agnostic backwards scan
│ └── anchor_sweep.py # sweep_issues(): two-phase scan-and-rewrite
├── session_log.py # Session log linking for issues
├── file_utils.py # Shared file I/O utilities (atomic writes)
├── user_messages.py # User message extraction
├── workflow_sequence/ # Workflow analysis (ll-workflows, sub-package)
│ ├── __init__.py # Re-exports: analyze_workflows, models
│ ├── analysis.py # Core analysis: boundaries, entity clustering
│ ├── models.py # Data models (Workflow, SessionLink, etc.)
│ └── io.py # YAML/JSON input-output helpers
├── fsm/ # FSM loop execution engine
│ ├── __init__.py
│ ├── schema.py # Loop schema definitions
│ ├── fsm-loop-schema.json # JSON Schema for loop files
│ ├── compilers.py # YAML to FSM compilation
│ ├── concurrency.py # Concurrent loop execution
│ ├── evaluators.py # Condition evaluation
│ ├── executor.py # Loop execution
│ ├── interpolation.py # Variable interpolation
│ ├── validation/ # Schema + meta-loop lint rules (ENH-2774 split of former flat validation.py)
│ │ ├── __init__.py # Public API re-exports
│ │ ├── _base.py # ValidationSeverity/ValidationError + cross-rule helpers
│ │ ├── structural_rules.py # Structural/static loop-shape checks
│ │ ├── reachability.py # capture-reachability, static loop-ref, session-mode-eval checks
│ │ ├── shell_safety.py # MR-7/MR-9/MR-11 bash-escaping rules
│ │ └── evaluator_rules.py, meta_rules.py # MR-1..MR-6, MR-8/MR-10/MR-12/MR-13 + haiku-gen
│ ├── persistence.py # State persistence
│ ├── signal_detector.py # Output signal detection
│ ├── handoff_handler.py # Session handoff handling
│ └── rate_limit_circuit.py # Shared cross-worktree 429 circuit breaker
├── extension.py # Extension protocol, loader, and reference implementation
├── testing.py # Offline LLTestBus test harness for extension development
├── output_parsing.py # Shared output parsing (ll-auto, ll-parallel)
├── output_cleaner.py # Anti-event + duplicate-window tool/log pre-filter (FEAT-2470)
├── output/ # Stop-sequence / prefill JSON output helpers (FEAT-2470)
│ ├── __init__.py
│ └── parse.py # extract_between_tags, parse_prefilled_json
├── parallel/
│ ├── __init__.py
│ ├── orchestrator.py
│ ├── worker_pool.py
│ ├── merge_coordinator.py
│ ├── priority_queue.py
│ ├── git_lock.py
│ ├── file_hints.py # File hint extraction
│ ├── overlap_detector.py # File overlap detection
│ ├── types.py
│ └── tasks/
│ ├── README.md
│ ├── lint-all.yaml
│ ├── test-suite.yaml
│ ├── build-assets.yaml
│ └── health-check.yaml
├── assets/ # Package data: CLI assets
│ └── ll-cli-logo.txt
├── templates/ # Package data: project-type configs and section templates
│ ├── python-generic.json
│ ├── javascript.json
│ ├── typescript.json
│ ├── go.json
│ ├── rust.json
│ ├── java-maven.json
│ ├── java-gradle.json
│ ├── dotnet.json
│ ├── bug-sections.json
│ ├── feat-sections.json
│ ├── enh-sections.json
│ ├── ll-goals-template.md
│ ├── design-tokens/ # Built-in accessible default palette
│ ├── extension/ # Extension scaffold templates (.tmpl)
│ └── generic.json
└── hooks/ # (package hook modules + in-package prompt/adapter data)
├── prompts/
│ └── optimize-prompt-hook.md # Package data: prompt optimization template
└── adapters/
├── codex/
│ ├── hooks.json # Package data: Codex adapter hooks template
│ ├── session-start.sh # SessionStart → session_start (sets LL_HOOK_HOST=codex)
│ ├── pre-compact.sh # PreCompact → pre_compact (sets LL_HOOK_HOST=codex)
│ ├── prompt-submit.sh # UserPromptSubmit → user_prompt_submit (sets LL_HOOK_HOST=codex)
│ └── post-tool-use.sh # PostToolUse → post_tool_use (sets LL_HOOK_HOST=codex)
└── kimi/
├── hooks.toml # Package data: Kimi adapter hooks template (managed [[hooks]] block)
└── *.sh # 8 shims: SessionStart/PreCompact/UserPromptSubmit/PreToolUse/PostToolUse/SessionEnd/SubagentStart/SubagentStop (set LL_HOOK_HOST=kimi-code)
Orchestration Layers¶
The three orchestration CLIs (ll-auto, ll-sprint, ll-parallel) are
organized as a layered architecture (EPIC-1867). The full decomposition
rationale lives in
docs/research/ll-orchestrator-decomposition-plan-v0.2.md;
this section summarizes the layers and their status.
flowchart TB
subgraph L3["Layer 3 — ll-parallel (canonical parallel substrate, kept as Python)"]
PAR[ParallelOrchestrator + WorkerPool + MergeCoordinator]
end
subgraph L2["Layer 2 — ll-sprint (wave driver + shim; FSM planned: FEAT-1899)"]
SPRINT[DependencyGraph waves + contention refinement]
end
subgraph L1["Layer 1 — ll-auto (FSM + shim; planned: FEAT-2000/2001/2002)"]
AUTO[Per-issue lifecycle: ready → manage → verify]
end
subgraph L0["Layer 0 — shared core (library + CLI subcommands; FEAT-1901)"]
CORE[issue scan/parse, state, config, host_runner, worktree_utils]
end
SPRINT -- "multi-issue waves" --> PAR
AUTO --> CORE
SPRINT --> CORE
PAR --> CORE
| Layer | Tool | Role | Status |
|---|---|---|---|
| 0 | (library) | Shared orchestration core exposed as internal library + CLI subcommands | In progress (FEAT-1901) |
| 1 | ll-auto |
Per-issue sequential lifecycle; target: FSM loop (loops/ll-auto.yaml) + thin CLI shim |
Python today; FSM migration planned (FEAT-2000/2001/2002) |
| 2 | ll-sprint |
Dependency-aware wave planning + execution; target: FSM wave driver + shim | Python today; FSM wave driver planned (FEAT-1899) |
| 3 | ll-parallel |
Canonical parallel substrate — worker pool, git worktrees, merge coordination | Kept as Python permanently — no FSM equivalent |
Layer 3 is normative: ll-parallel is the canonical parallel substrate for
the entire toolkit. The FSM engine has no concurrency primitive, so there is no
FSM replacement for the worker pool / worktree / MergeCoordinator machinery —
it stays Python. ll-sprint multi-issue waves already delegate to
ParallelOrchestrator today, and the planned Layer-2 FSM wave driver will
continue to shell out to it for multi-issue waves. Anything that needs
concurrent issue processing builds on ll-parallel rather than reimplementing
parallelism.
Sequential Mode (ll-auto)¶
The sequential mode processes issues one at a time in priority order.
sequenceDiagram
participant User
participant CLI as ll-auto
participant Manager as AutoManager
participant Claude as Claude CLI
participant Git
User->>CLI: ll-auto --max-issues 5
CLI->>Manager: Initialize with config
loop For each issue (priority order)
Manager->>Manager: Find highest priority issue
Note over Manager,Claude: Phase 1: Validation
Manager->>Manager: expand_skill("ready-issue") → prompt string
Manager->>Claude: expanded prompt (or /ll:ready-issue fallback)
Claude-->>Manager: READY / NOT_READY / CLOSE
alt READY
Note over Manager,Claude: Phase 2: Implementation
Manager->>Claude: /ll:manage-issue type action id
Claude->>Git: Make changes
Claude->>Git: Create commit
Claude-->>Manager: Success
Note over Manager,Git: Phase 3: Verification
Manager->>Git: Update issue status: done
Manager->>Manager: Verify completion
else NOT_READY
Manager->>Manager: Mark failed, skip
else CLOSE
Manager->>Git: Update issue status: done (closed)
end
Manager->>Manager: Save state
end
Manager-->>User: Summary report
Sequential Mode Components¶
| Component | File | Purpose |
|---|---|---|
AutoManager |
issue_manager.py |
Main orchestration loop |
IssueParser |
issue_parser.py |
Parse issue files |
StateManager |
state.py |
Persist state for resume |
Logger |
logger.py |
Colorized console output |
Parallel Mode (ll-parallel)¶
The parallel mode uses git worktrees to process multiple issues concurrently.
Per-EPIC integration branches (FEAT-2339). By default each worker forks
from and merges back to parallel.base_branch. When
parallel.epic_branches.enabled is true (config, or --epic-branches for a
single run), the WorkerPool resolves each issue's nearest ancestor EPIC and
routes all children of that EPIC onto one shared epic/<EPIC-ID>-<slug>
integration branch — both as fork point and merge target — via
WorkerResult.epic_branch. That integration branch forks from
parallel.base_branch by default, but an EPIC may declare a base_branch:
(alias target_branch:) frontmatter field to fork from a different ref; if a
declared base does not resolve locally or on remote, ll-sprint dispatch
hard-stops rather than degrading dependent children to a false partial
(FEAT-2652). The MergeCoordinator merges the EPIC branch back
to the base branch once the EPIC's last child completes
(epic_branches.merge_to_base_on_complete), optionally opening a PR
(epic_branches.open_pr). Standalone (parentless) issues keep the per-worker
branch behavior unchanged. When epic_branches.verify_before_merge is true,
that merge/PR-open is gated on a scratch-worktree run of test_cmd/lint_cmd
against the EPIC branch tip; a failure blocks it, leaves the branch open for
retry, and is surfaced in the run summary (ENH-2603).
epic_branches also has an FSM-loop-side consumer outside this WorkerPool
path (ENH-2601): auto-refine-and-implement/sprint-refine-and-implement
read parallel.epic_branches.enabled/.prefix to create (not check out) the
epic/<EPIC-ID>-<slug> branch when scope resolves to an EPIC-NNN id, then
run a post-implementation test_cmd/lint_cmd verify pass folded into
summary.json. After each successful delegate pass, recheck_set
re-resolves the EPIC's descendant set (transitive parent:-chain walk,
ENH-2615) and cycles newly-decomposed children back through delegate —
whose per-entry worktree attach re-attaches the same epic branch — so mid-run
decomposition work also lands on the integration branch instead of bypassing
it. A delegate pass that fails routes through delegate_failed instead
(ENH-3366): it reads ${captured.delegate.terminated_by} to tell autodev's
genuine failed terminal (recorded, then routed to finalize) from a
budget-exhaustion or signal class (still routed to recheck_set, so mid-drain
residuals are still folded back for re-dispatch) — on_success/on_failure
no longer collapse into the same route. Once all the EPIC's children are done, a merge_epic_branch
state merges (or, per epic_branches.open_pr, opens a PR for) the branch back to
base_branch, honoring merge_to_base_on_complete/verify_before_merge the same
way the WorkerPool path above does (BUG-2614) — both paths share the same
stateless free functions in little_loops.worktree_utils
(verify_epic_branch_before_merge/merge_epic_branch_to_base/open_pr_for_epic_branch).
Both call sites forward project.src_dir to verify_epic_branch_before_merge, which
prepends the scratch worktree's source dir onto PYTHONPATH before running
test_cmd/lint_cmd — so branch-only modules resolve to the worktree rather than the
editable-install .pth's main checkout, which would otherwise false-fail collection
for any EPIC branch that adds a new module (BUG-2629).
See LOOPS_REFERENCE.md § auto-refine-and-implement.
flowchart TB
subgraph Orchestrator["ParallelOrchestrator"]
ORCH[Main Controller]
QUEUE[IssuePriorityQueue]
STATE[OrchestratorState]
end
subgraph Workers["Worker Pool"]
POOL[WorkerPool]
W1[Worker 1]
W2[Worker 2]
WN[Worker N]
end
subgraph Merge["Merge Coordinator"]
MCOORD[MergeCoordinator]
MQUEUE[Merge Queue]
end
subgraph Worktrees["Git Worktrees"]
WT1[".worktrees/worker-1/"]
WT2[".worktrees/worker-2/"]
WTN[".worktrees/worker-N/"]
end
ORCH --> QUEUE
ORCH --> STATE
ORCH --> POOL
POOL --> W1
POOL --> W2
POOL --> WN
W1 --> WT1
W2 --> WT2
WN --> WTN
W1 --> MCOORD
W2 --> MCOORD
WN --> MCOORD
MCOORD --> MQUEUE
Parallel Processing Flow¶
sequenceDiagram
participant Orch as Orchestrator
participant Queue as PriorityQueue
participant Pool as WorkerPool
participant W1 as Worker 1
participant W2 as Worker 2
participant Merge as MergeCoordinator
participant Git
Note over Orch,Queue: Setup Phase
Orch->>Queue: Scan and queue issues
Note over Orch,Pool: Processing Phase
Orch->>Pool: Start workers
par Worker 1
Pool->>W1: Process BUG-001
W1->>Git: Create worktree + branch
W1->>W1: Run ready-issue
W1->>W1: Run manage-issue
W1->>Git: Commit in worktree
W1-->>Pool: WorkerResult
and Worker 2
Pool->>W2: Process BUG-002
W2->>Git: Create worktree + branch
W2->>W2: Run ready-issue
W2->>W2: Run manage-issue
W2->>Git: Commit in worktree
W2-->>Pool: WorkerResult
end
Note over Pool,Merge: Merge Phase (Sequential)
Pool->>Merge: Queue BUG-001 result
Merge->>Git: Merge branch to main
Merge-->>Orch: Merge complete
Pool->>Merge: Queue BUG-002 result
Merge->>Git: Merge branch to main
Merge-->>Orch: Merge complete
Note over Orch,Git: Cleanup Phase
Orch->>Git: Remove worktrees
Orch->>Git: Delete branches
Parallel Mode Components¶
| Component | File | Purpose |
|---|---|---|
ParallelOrchestrator |
orchestrator.py |
Coordinate all components |
IssuePriorityQueue |
priority_queue.py |
Priority-based issue ordering |
WorkerPool |
worker_pool.py |
Thread pool with worktrees |
MergeCoordinator |
merge_coordinator.py |
Sequential merge queue |
Extension Architecture & Event Flow¶
little-loops includes an extension architecture built on a structured event bus. Extensions implement the LLExtension protocol and receive LLEvent notifications from core subsystems. Topic-based filtering lets extensions subscribe only to the event namespaces they care about.
Components¶
| Component | File | Purpose |
|---|---|---|
LLEvent |
events.py |
Structured event dataclass (type, timestamp, payload) |
EventBus |
events.py |
Multi-observer dispatcher with pluggable Transport sinks (defined in transport.py; SQLiteTransport in session_store/writers.py): JsonlTransport, UnixSocketTransport, OTelTransport, WebhookTransport, SQLiteTransport |
LLExtension |
extension.py |
Runtime-checkable protocol for event consumers |
ExtensionLoader |
extension.py |
Discovers extensions from config paths and entry points |
InterceptorExtension |
extension.py |
Protocol for plugins providing before_route/after_route hooks; stored in FSMExecutor._interceptors |
ActionProviderExtension |
extension.py |
Protocol for plugins providing custom ActionRunner instances; populated into FSMExecutor._contributed_actions |
EvaluatorProviderExtension |
extension.py |
Protocol for plugins providing custom evaluator callables; populated into FSMExecutor._contributed_evaluators |
LLHookIntentExtension |
extension.py |
Protocol for plugins contributing hook intent handlers (provided_hook_intents()); detected via hasattr() in wire_extensions, merged into _HOOK_INTENT_REGISTRY in hooks/__init__.py |
ReferenceInterceptorExtension |
extensions/reference_interceptor.py |
Passthrough reference implementation of InterceptorExtension; copy-paste starting point for custom interceptors |
Event Emitters¶
The EventBus is wired into the following subsystems, which emit events at key lifecycle points:
| Subsystem | File | Events Emitted |
|---|---|---|
| FSM Executor | fsm/executor.py |
loop_start, state_enter, action_start, action_complete, loop_complete |
| StateManager | state.py |
State persistence events (save, load, mark completed/failed) |
| Issue Lifecycle | issue_lifecycle.py |
Issue status transitions (move, close, defer, skip, undefer) — emits issue.completed, issue.closed, issue.deferred, issue.skipped (from skip_issue()), issue.started (from undefer_issue()), issue.failure_captured |
| Parallel Orchestrator | parallel/orchestrator.py |
Worker start/complete, merge events, issue.closed (worker-completion close, sequential-merge close, and the frontmatter-only lifecycle-completion path, ENH-2783) |
Extensions are wired to the EventBus at CLI entry points via wire_extensions(), so they receive events from all subsystems during a run:
| CLI Entry Point | File | Extensions Wired | Transports Wired |
|---|---|---|---|
ll-loop run |
cli/loop/run.py |
Yes — EventBus + FSMExecutor registry wired (interceptors, contributed actions/evaluators populated) | Yes — wire_transports() after extensions; executor.close_transports() runs in finally before lock release |
ll-loop resume |
cli/loop/lifecycle.py |
Yes — EventBus + FSMExecutor registry wired | Yes — wire_transports() after extensions; executor.close_transports() runs in finally so transports flush on exit/exception |
ll-loop monitor |
cli/loop/lifecycle.py |
No — read-only attach: does not instantiate PersistentExecutor or subscribe to EventBus; reads <instance-id>.events.jsonl from disk and forwards events to StateFeedRenderer. Ctrl-C detaches without signaling the loop process. |
No |
ll-parallel |
cli/parallel.py |
Yes — EventBus only (no FSMExecutor wiring) | Yes — wire_transports() after extensions; teardown runs in ParallelOrchestrator._cleanup() via event_bus.close_transports(). Also wires SQLiteTransport directly (unconditionally, unless events.transports already lists "sqlite") so issue.closed events are live-written regardless of config (ENH-2783) |
ll-sprint |
cli/sprint/run.py |
Yes — EventBus only (no FSMExecutor wiring for parallel branch) | Yes — per-wave wire_transports() after extensions; teardown delegated to per-wave ParallelOrchestrator._cleanup(). Also wires SQLiteTransport directly for both the multi-issue wave bus and a dedicated bus constructed for the single-issue/contention-subwave branch (ENH-2783) |
ll-auto |
cli/auto.py |
No — EventBus is internal to AutoManager |
Yes — AutoManager.__init__() wires SQLiteTransport(db_path) directly; does not call wire_transports() |
The transport layer fans events out additively: every event emitted on the EventBus is delivered to every registered observer and every registered transport. Built-in transports: JsonlTransport (durable file log; selected via events.transports: ["jsonl"]), UnixSocketTransport (real-time AF_UNIX streaming for local TUIs and dashboards; selected via events.transports: ["socket"], requires POSIX), OTelTransport (OpenTelemetry OTLP exporter; selected via events.transports: ["otel"], requires pip install 'little-loops[otel]'), WebhookTransport (batched HTTP POST to a remote endpoint; selected via events.transports: ["webhook"], requires pip install 'little-loops[webhooks]'), and SQLiteTransport (writes events to the per-project .ll/history.db unified session store; selected via events.transports: ["sqlite"], queryable via ll-session). Note: AutoManager.__init__() wires SQLiteTransport directly (not via the config-driven events.transports path), so ll-auto records issue lifecycle events without requiring "sqlite" in the project config. As of ENH-2783, ll-parallel's and ll-sprint's CLI entry points do the same for their EventBus instances, so issue.closed events from those orchestration paths are also recorded without requiring "sqlite" in the project config.
UnixSocketTransport — initial state seeding: When a new client connects to events.sock, the transport immediately sends state_change events for all currently running loops (read from .loops/.running/*.state.json) before the client enters the regular event stream. This means a dashboard or TUI that connects mid-run receives the current FSM state of every active loop without waiting for the next state transition. Clients that connect before any loop is running receive no seed events (the event stream is empty until a loop starts).
UnixSocketTransport — concurrent-producer path claiming (BUG-3324): Construction never unlinks the configured path blind. It probes first (_probe_socket_path, a connect-and-classify that mutates nothing) and only unlinks when the path is reclaimable — a regular file or a bound-but-dead socket. If the probe finds a live listener, the new producer claims a {stem}-{pid}{suffix} sibling path instead (_claim_socket_path), so a second concurrent producer never evicts a first. A probe transiently triggers the live producer's on_connect seed callback (a real accept-and-close), which is self-healing and does not affect delivery to already-attached consumers. close() mirrors this: it only unlinks the path if the inode it stat'd at bind time still matches, so a producer that has been reclaimed mid-close (its up-to-10s client drain window) never deletes the reclaiming producer's socket.
SseBridge — out-of-process SSE consumer (FEAT-3323): ll-artifact serve is a separate, long-lived process that consumes UnixSocketTransport's existing socket output rather than adding a sixth transport or a new emit path — SseBridge is never a Transport and is never held by an EventBus. It globs the configured socket's directory for every live producer (the un-suffixed configured path plus its BUG-3324 pid-suffixed siblings), connects to each as an ordinary client (_fan_in_producer_sockets/_read_producer_socket, one reader thread per producer merging into one bounded queue), and relays the merged stream as Server-Sent Events to browser clients over a loopback-only, per-start-token-prefixed ThreadingHTTPServer — reusing LocalBridgeTransport's (ENH-3351) _SSEClient/_sse_encode/Host-guard/SSE-write-loop pieces, extracted into module-level helpers shared by both bridges. Because it is a separate process, a bridge that crashes or stops reading is indistinguishable, from a producer's side, from an ordinary slow or disconnected socket client — already handled by the existing _record_drop/_record_rejection accounting, with no EventBus.emit change required. UnixSocketTransport.send() stamps producer_pid = os.getpid() onto a copy of every event (never mutating the caller's dict) so a fanned-in multi-producer stream can be demultiplexed; the bridge itself owns SSE-client seeding (dropping the socket-side state_change seed and rebuilding it per connect from list_running_loops) rather than relaying N raw copies of it. See EVENT-SCHEMA.md § state_change and CONFIGURATION.md § events.bridge for the full contract.
OTel mapping: Each loop run becomes a trace. loop_start opens the root span; state_enter opens a child span (closing the prior state span); action_start/action_complete bracket a grandchild span; loop_complete closes all open spans and sets the trace status. Span events are recorded for evaluate, route, retry_exhausted, handoff_detected, handoff_spawned, and action_output on the innermost open span. loop_resume starts a new root span (new trace). Sub-loop events (depth > 0) are no-ops with a single per-session warning.
Webhook batching: WebhookTransport.send() enqueues non-blocking; a daemon thread POSTs accumulated events as a JSON array on each batch_ms tick. Failed POSTs retry with exponential backoff (up to 3 times, 0.5s–8s); after exhaustion the batch is dropped with a warning. close() does one final flush before joining the thread. New transports plug in through the same Transport protocol without changes to EventBus or the CLI wiring.
history.db schema versions: SQLiteTransport applies incremental PRAGMA user_version migrations on open. Each version adds tables or views without dropping prior ones.
| Version | Object | Purpose |
|---|---|---|
| v1 | tool_events, file_events, issue_events, correction_events |
Core event tables — tool calls, file reads/writes, issue lifecycle, user corrections |
| v2 | message_events |
User and assistant message text for FTS search |
| v3 | FTS5 index on message_events |
BM25 full-text search (ll-session search --fts) |
| v4 | sessions |
One row per Claude Code session; indexed by session_id for ll-session path resolution (ENH-1710) |
| v5 | issue_sessions VIEW |
Joins issue_events to message_events via overlapping timestamps; enables ll-history sessions <ID> and ll-session recent --issue <ID> (ENH-1711) |
| v6 | last_backfill_ts meta key |
Enables incremental JSONL backfill at session start; session_start hook records the last-run timestamp so only newly-modified JSONL files are processed on subsequent starts (ENH-1830) |
| v7 | skill_events |
Records /ll: skill invocations at dispatch time via the user_prompt_submit hook; enables ll-session recent --kind skill and FTS search with kind='skill' (ENH-1833) |
| v8 | cli_events |
Records ll- CLI invocations via cli_event_context() in session_store/writers.py; enables ll-session recent --kind cli (ENH-1848) |
| v9 | idx_corrections_dedup |
Unique index on user_corrections(session_id, content) enabling idempotent INSERT OR IGNORE during correction mining; backfill() and backfill_incremental() call mine_corrections_from_messages() to retroactively populate corrections from message_events (ENH-1904) |
| v10 | summary_nodes, summary_spans |
LCM-style hierarchical summary DAG (FEAT-1712): summary_nodes stores three-level LCM Algorithm 3 summaries (normal LLM → aggressive bullet-point LLM → deterministic truncation) as leaf and condensed nodes over message_events blocks; summary_spans links each node back to its source messages for lossless drill-down. Enables ll-session grep, ll-session expand, and ll-session describe. Compaction is opt-in via history.compaction.enabled in ll-config.json. |
| v11 | assistant_messages |
Stores concatenated text blocks from assistant responses so the SFT pipeline can read conversation turn-pairs from the database instead of re-parsing JSONL (ENH-1942). Includes tool_use_count for filter predicates and idx_assistant_messages_dedup for idempotent backfill. |
| v12 | summary_nodes.level, idx_summary_nodes_cross_dedup |
Adds level INTEGER DEFAULT 0 column to summary_nodes for N-level DAG traversal (0 = leaf/per-session condensed, 1+ = cross-session condensed, max = root) and a cross-session dedup index idx_summary_nodes_cross_dedup on (level, ts_start, ts_end) WHERE kind='condensed' AND session_id IS NULL (ENH-1953). |
| v13 | correction_retirements |
Records addressed correction clusters (topic fingerprint + optional rule ID) so detect_recurring_feedback() excludes already-ruled topics from future runs; unique index on topic_fingerprint for idempotent inserts (ENH-2046). |
| v14 | issue_snapshots |
Stores full issue content (title, priority, body, frontmatter) at key lifecycle transitions (captured, done, cancelled) so completed issue context is queryable from the DB even after the source .md file is moved or deleted. FTS via search_index with kind="snapshot" (ENH-2151). |
| v15 | skill_events.exit_code/success/duration_ms |
Completion-side columns on skill_events, written by skill_event_context() (the skill-host analogue of cli_event_context()); dispatch-only rows keep NULL. Enables ll-session skill-stats per-skill success-rate rollups (ENH-2460). |
| v16 | issue_events.session_id, idx_issue_events_session_id, rebuilt issue_sessions VIEW |
Authoritative session linkage captured at transition time by SQLiteTransport from the issue.* event payload; the timestamp-overlap heuristic is preserved as the deprecated legacy_issue_sessions_ts_overlap VIEW and the issue_sessions VIEW now prefers exact session_id joins, falling back to the legacy inference only for issues with no authoritative rows (ENH-2462). |
| v17 | commit_events |
Ground-truth record of what shipped: (ts, commit_sha UNIQUE, parent_sha, message, author, branch, issue_id, files_json). Written live by record_commit_event() (post-commit hook hooks/scripts/record-commit-post-commit → little_loops.hooks.post_commit) and retroactively by ll-session backfill walking git log --all; issue_id inferred from Closes/Fixes/Issue: references and branch naming. Enables ll-session recent --kind commit and FTS with kind="commit" (ENH-2458). |
| v18 | test_run_events |
Persisted pytest run results (pass/fail/error/skip counts, duration, failing node IDs, env label, HEAD sha, branch, command) written best-effort by the little_loops.pytest_history_plugin pytest11 plugin via record_test_run_event(); opt out with PYTEST_DISABLE_PLUGIN_LL_HISTORY=1. Enables ll-session recent --kind test_run and FTS with kind="test_run" (ENH-2459). |
| v19 | raw_events |
Verbatim-JSONL-line source of truth for the JSONL-derived cache tables (tool_events, message_events, assistant_messages, skill_events, sessions): (ts, session_id, host, source_path, line_no, event_type, raw_line, parsed_json, compacted, summary_node_id), unique on (source_path, line_no). ll-session backfill now ingests JSONL lines here only; ll-session rebuild wipes and re-derives the cache tables (plus user_corrections, summary_nodes/summary_spans, and the corresponding search_index rows) by replaying raw_events. ll-session compact [--and-prune] sweeps rows past analytics.retention.raw_event_max_age_days into per-session kind='retention' summary nodes and marks them compacted=1; ll-session prune now deletes only raw_events rows already marked compacted=1 (previously it deleted tool_events/cli_events/file_events/message_events directly and never touched search_index, leaving stale FTS rows — fixed by rebuild() always re-populating search_index from current state). The three legacy watermarks (last_backfill_ts, last_backfill_ts_assistant_messages, last_backfill_ts_skill_events) collapse to a single last_raw_event_ts meta key; a new last_rebuild_version key gates the SessionStart hook's opt-in-on-migration --rebuild pass. Issue/loop/commit/cli/file/test_run tables are outside this table's scope and keep their existing direct-write paths (ENH-2581). |
| v20 | usage_events |
Real LLM token counts the API returned (input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens) plus a derived cost_usd (via pricing.estimate_cost_usd, NULL for unpriced models) and a forward-compat nullable state column: (ts, session_id, model, state, input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens, cost_usd). Derived from raw_events by _backfill_usage_events() — one row per assistant turn, parsing message.usage on type == "assistant" transcript records. state is always NULL on parser-written rows (the transcript carries no FSM-state boundary); reserved for a future live per-state writer. Enables ll-session recent --kind usage, FTS with kind="usage", and history_reader.recent_usage_events()/aggregate_usage() (ENH-2461). |
| v21 | usage_events.invocation_id/provider_vendor |
OTel attribution addenda mapping live usage to gen_ai.invocation.id and gen_ai.provider.vendor; parser-derived rows remain nullable (FEAT-2478). |
| v22 | orchestration_runs |
Per-issue outcomes from ll-auto, ll-parallel, and ll-sprint: one UPSERTed row per (run_id, issue_id) with driver, final status, duration, failure reason, wave label, PR URL, timestamps, and git context. Producers use one opaque UUID per top-level invocation and suppress DB failures. Enables ll-session recent --kind orchestration_run, FTS, export, and typed history-reader rollups (ENH-2492). |
| v23 | loop_runs |
One row per completed FSM loop run — final state, iteration count, terminator, error, nullable evaluator score and diagnostics-artifact path, plus git context. A producer-side sibling of orchestration_runs: written best-effort by FSMExecutor._finish() via record_loop_run_summary(), keyed by the archive-time run_id so it JOINs to .loops/.history/<run_id>-<loop_name>/. Idempotent via INSERT OR IGNORE on run_id (a resumed-then-completed run contributes one row). Outside raw_events's rebuild scope, like loop_events. Enables ll-session recent --kind loop_run, FTS, export, and typed history-reader rollups via recent_loop_runs()/find_loop_run()/aggregate_loop_runs() (ENH-2463). |
| v24 | tool_events.agent_type, idx_tool_events_agent |
Nullable discriminator column on tool_events, populated for tool_name="Task" rows from tool_input.subagent_type (with a leading ll: plugin prefix stripped so built-in and plugin agent names group together); NULL for all other tools and pre-migration rows. Written live by post_tool_use.handle() and retroactively by _backfill_tool_events(), both inside the existing best-effort/contextlib.suppress write path. FTS-indexed (kind="tool") so ll-session search --fts "<agent>" surfaces spawns. Enables history_reader.agent_usage() (per-agent invocation counts) and recent_tool_events(agent_type=...) (ENH-2497). |
| v25 | tool_events.mcp_server/mcp_tool/mcp_outcome/latency_ms, idx_tool_events_mcp_server, idx_tool_events_mcp_outcome |
Four nullable columns on tool_events breaking out MCP tool calls: mcp_server/mcp_tool parsed from the mcp__<server>__<tool> tool_name prefix (populated by both the live write and _backfill_tool_events()); mcp_outcome (success/error) read from tool_response.isError (live write only — the backfill JSONL path has no access to the paired tool_result envelope); latency_ms from tool_call.started_at/completed_at when the host payload carries them (live write only, currently always NULL since Claude Code's PostToolUse payload doesn't yet supply these fields). Enables history_reader.mcp_server_usage()/mcp_failure_rate() and recent_tool_events(mcp_server=..., mcp_tool=..., mcp_outcome=...) (ENH-2511). |
| v26 | learning_test_events |
Mirror of the Learning Test Registry (.ll/learning-tests/*.md, owned by little_loops.learning_tests) into the DB: (ts, record_id UNIQUE, target, status, assertions_json, date, raw_output_path), keyed on record_id (the slugified target — the registry's own file-stem identity, not an issue ID). Written best-effort by record_learning_test_event() from ll-learning-tests prove/mark-stale/orphans --mark-stale (UPSERT semantics — a re-prove overwrites status/assertions_json/date rather than duplicating), and reconciled from disk by _backfill_learning_test_events() for out-of-band file edits (INSERT OR IGNORE, a companion for files never routed through the CLI). A file/external-source mirror like orchestration_runs/loop_runs, outside raw_events's rebuild scope. Enables ll-session recent --kind learning_test, ll-session search --fts ... --kind learning_test, and history_reader.recent_learning_tests()/find_learning_test() (ENH-2466). |
| v27 | session_lifecycle_events |
Session-lifecycle/handoff transitions: (ts, session_id, event, detail JSON, head_sha, branch), no CHECK constraint on event so ENH-2509's worktree_* discriminators can share the table. Written best-effort by record_session_lifecycle_event() from three producers — context-monitor.sh's first 80%-threshold crossing per pressure episode (handoff_needed, bash shell-out with \|\| true), pre_compact.handle() after state persistence (compaction), and sweep_stale_refs.handle() once per invocation including zero findings (stale_ref_sweep). First-write-only — no historical backfill. Enables ll-session recent --kind session_lifecycle, FTS, export, and history_reader.recent_lifecycle_events()/handoff_frequency() (ENH-2495). |
| v28 | subagent_runs |
Subagent (Task/Agent) spawn tree: (ts, parent_session_id, agent_id, agent_type, agent_transcript_path, started_at, ended_at, status, head_sha, branch), UNIQUE(parent_session_id, agent_id) — agent_id is spawn-local (scoped to its parent session per the SubagentStart/SubagentStop documented payload), not a sessions.session_id, so a subagent's transcript is a nested file (<parent-transcript-dir>/subagents/agent-<id>.jsonl), never a joinable top-level session row. Written by the new SubagentStart/SubagentStop lifecycle hooks (subagent_start.handle()/subagent_stop.handle()) via record_subagent_run_start()/record_subagent_run_stop() — start is INSERT OR IGNORE (idempotent replay), stop is an UPDATE matching on the composite key. _backfill_subagent_runs() seeds historical rows from nested subagents/*.jsonl transcripts (all backfilled rows land as status="completed" — a persisted transcript implies the spawn finished; backfill cannot reconstruct running/failed/timeout after the fact). Enables ll-session recent --kind subagent_run, FTS, export, and history_reader.subagent_tree()/subagent_retries()/subagent_budget() (ENH-2505). |
| v29 | usage_events.run_id, idx_usage_events_run_id |
Nullable TEXT join key on usage_events, added via plain ALTER TABLE (no FK — usage_events stays an independent table joined at the application/query level per ARCHITECTURE-145). Schema-only slice decomposed from ENH-2721 (ENH-2723); populated on new rows by the live per-invocation writer at loop-run finish (record_usage_event(), ENH-2724). Historical rows stay NULL until backfilled (ENH-2725). Consumed by history_reader.waste_attribution() (ENH-2722, EPIC-2456), which equi-joins usage_events.run_id = loop_runs.run_id to split per-loop token spend into wasted-vs-productive by terminal outcome; surfaced via ll-ctx-stats' "Waste" section. |
| v30 | hook_events |
Per-fire hook execution telemetry: (ts, session_id, event_name, matcher, script, exit_code, duration_ms, stderr_preview, head_sha, branch). Live-write-only — the Claude Code host does not emit hook execution results into the transcript JSONL, so there is no raw_events source and no _backfill_hook_events; excluded from rebuild()'s _REBUILD_TABLES (a wipe would be unrecoverable). Written by hook_event_context(), wrapped once around the handler(event) call inside main_hooks() (hooks/__init__.py) so every Python-dispatched intent is covered without per-handler edits; Stop (bash-only, never reaching the Python dispatcher) is covered by the hooks/scripts/record-hook-event.sh shim instead; SessionEnd has no registered hooks (BUG-3363 re-homed scratch-cleanup.sh to SessionStart). Gated on analytics.enabled + analytics.capture.hooks (default true). Enables ll-session recent --kind hook_event and history_reader.recent_hook_events()/hook_failure_rate()/hook_latency_p95() (ENH-2506). |
| v31 | harness_events |
ll-harness / eval outcome telemetry: (ts, runner, target, exit_code, semantic_verdict, semantic_passed, timed_out, duration_ms, head_sha, branch, parent_id, semantic_prompt, semantic_confidence, semantic_reason, semantic_evidence, semantic_model). parent_id links DSL per-task rows to their parent harness run (ENH-2740). Live-write-only, like hook_events — no raw_events source, excluded from rebuild()'s _REBUILD_TABLES. This migration lands the table + record_harness_event() recorder only; nothing calls the recorder yet (ENH-2740 wires the ll-harness producer, ENH-2741 adds the history_reader read API and ll-session recent --kind harness CLI surface) (ENH-2739). |
| v32 | prompt_opt_events |
Prompt-optimization offer/outcome telemetry: (ts, session_id, mode, offered, bypass_reason, raw_len, optimized_len, optimized_text, accepted). user_prompt_submit.py::handle() writes the offer row live at hook-fire time via record_prompt_opt_event() (mode, offered, bypass_reason across all ten bypass branches, raw_len), gated on analytics.enabled like the sibling record_correction/record_skill_event calls. Like hook_events/harness_events, the offer decision can't be reconstructed retroactively with historical-config confidence, so the table is excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS — a wipe-and-replay would destroy live rows. Unlike those two, it does get a non-destructive backfill pass: _backfill_prompt_opt() (called from rebuild() after the wipe-and-replay loop) UPDATEs still-unenriched offered=1 rows with optimized_len/optimized_text/accepted when the transcript's next assistant turn in the same session contains an ENHANCED: block (the confirm=true path in optimize-prompt-hook.md); confirm=false sessions have no recoverable replacement text and stay unenriched. Enables ll-session recent --kind prompt_opt, FTS, export, and history_reader.recent_prompt_opt_events()/prompt_opt_offer_rate() (ENH-2498). |
| v33 | verdict_events |
Verifier verdict outcome telemetry for the nine ll-action-bridged verifiers (ready-issue, confidence-check, go-no-go, tradeoff-review-issues, refine-issue, format-issue, verify-issues, prioritize-issues, align-issues): (ts, session_id, verdict_kind, target_kind, target_id, verdict, severity_counts JSON, findings_count, confidence, head_sha, branch). Written best-effort from cli/action.py::cmd_invoke() via record_verdict_event(). No skill currently emits a structured verdict at that call site, so verdict defaults to a coarse exit-code read (pass/fail); a VERDICT_JSON: {...} tagged line (output_parsing.extract_tagged_json()'s existing convention) overrides the coarse fields when a skill adopts it, without a further schema change. Live-write-only like hook_events/harness_events/prompt_opt_events — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS. Enables ll-session recent --kind verdict, FTS, export, and history_reader.recent_verdict_events()/verdict_pass_rate() (ENH-2504). |
| v34 | context_pressure_events |
Context-window pressure measurements: (ts, session_id, used_pct, used_tokens_est, threshold_crossed, crossed_level, head_sha, branch). Written best-effort by context-monitor.sh's record_context_pressure() (a shell-out mirroring record_handoff_needed()'s shape) after every sampled PostToolUse, sampled at most once per second per session — a new 50/75/80/90/100 pressure-level crossing always persists regardless of the cap. threshold_crossed/crossed_level are populated only on the row that first reaches a given level; the emitted-levels set and last-write epoch live in .ll/ll-context-state.json and reset on compaction alongside the existing threshold_crossed_at/breakdown fields. Live-write-only like hook_events/harness_events/prompt_opt_events/verdict_events — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS. Enables ll-session recent --kind context_pressure, FTS, export, history_reader.context_pressure_curve()/pressure_crossings()/pressure_summary(), and ll-ctx-stats's "Context pressure curve" rendering block (ENH-2507). |
| v35 | review_events |
Reviewer/audit outcome telemetry for the seven ll-action-bridged audits/reviews (review-epic, review-loop, audit-architecture, audit-claude-config, audit-docs, audit-loop-run, review-sprint): (ts, session_id, reviewer_skill, target_kind, target_id, severity_counts JSON, findings_count, findings_json_summary JSON, verdict, head_sha, branch). The third read-side signal alongside harness_events (executor, v31) and verdict_events (verifier, v33). Written best-effort from cli/action.py::cmd_invoke() via record_review_event(), following the same _VERIFIER_SKILLS/_record_verdict() pattern as v33 — a _REVIEWER_SKILLS frozenset gates a _record_review() helper. verdict defaults to a coarse exit-code read (pass/fail); a REVIEW_JSON: {...} tagged line (the same extract_tagged_json() convention as VERDICT_JSON) overrides the coarse fields — including verdict: "refused", which a pre-flight gate can't express via exit code alone (audit-loop-run's missing-run refusal). code-review/simplify are excluded — built-in Claude Code slash commands with no local scripts/little_loops/ entry point. Live-write-only like verdict_events — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS. Enables ll-session recent --kind review, FTS, export, and history_reader.recent_review_events()/review_velocity() (ENH-2512). |
| v36 | issue_events.issue_num, issue_snapshots.issue_num, rebuilt (issue_num, transition) dedup indexes |
Adds a stable numeric join key (issue_num INTEGER, trailing-digit extraction from issue_id) alongside the mutable issue_id TEXT display column, and replaces the old (issue_id, transition) unique index with (issue_num, transition) — deliberately type-blind, so an issue retyped mid-life (ENH-1234 -> FEAT-1234) keeps one continuous history instead of splitting across two issue_id values (ENH-2771). Trade-off: because the key is type-blind, two different issues that reuse the same bare number under different type prefixes also collide on (issue_num, transition) — the second issue's transition is discarded by INSERT OR IGNORE with no error. As of BUG-3006, every write site probes cursor.rowcount after the insert and logger.warnings when the suppressed row belongs to a different issue_id than the one just attempted, so a genuine number-reuse collision is now visible (though not auto-repaired); ll-history audit-issue-collisions reports every existing collision, classified as retype or number-reuse via each colliding id's on-disk file. |
| v38 | orchestration_runs.base_sha/base_dirty |
Two nullable columns recording the dequeue-time base state of a work item: the commit SHA the issue started from, and whether the tree had tracked modifications (git status --porcelain --untracked-files=no; an untracked scratch file does not count, since a base-state consumer reconstructs by checkout) at stamp time. Captured before anything mutates the tree or the issue file — in worker_pool._process_issue() before the worktree is created (ll-parallel, and worktree-mode ll-sprint waves transitively), and in process_issue_inplace() before Phase 1 (ll-auto, ll-sprint's sequential branch, and autodev.yaml transitively via implement_current's ll-auto --only shell-out). Persisted by an early record_orchestration_run(status="running", …) upsert at dequeue rather than at end-of-run, so the stamp is readable while the issue is still in flight; the existing terminal call upserts the outcome onto the same (run_id, issue_id) row, with base_sha/base_dirty/started_at COALESCEd so it cannot null them. NULL means unstamped — the orchestrator predates the stamp, opted out, or its git rev-parse failed — and history_reader.read_base_sha() returns None so consumers fall back to merge-base. Deliberately not on loop_runs: that table is one row per run with no issue dimension. Accepted consequence: a crashed run now leaves a permanent status='running' row where none existed before, slightly lowering aggregate_orchestration_runs' reported success rate (ENH-2866). |
| v44 | verdict_events.abstention_reason + verdict CHECK |
Lets an LLM-judged gate abstain instead of being forced into a pass/fail it cannot support (ENH-230). verdict is pinned to pass/fail/implement/cannot_judge — refused stays review_events-only (v35), where a producer actually emits it. abstention_reason is a closed four-tag enum (missing_artifacts, unparseable_criteria, evaluation_context_unavailable, circular_dependencies) pinned by a cross-column CHECK: NULL for every non-abstention verdict, required for cannot_judge. SQLite cannot add a CHECK to an existing column, so this rebuilds verdict_events (create/copy/drop/rename) — the first such rebuild in the schema; existing rows survive with abstention_reason NULL. Producer is skills/confidence-check/rubric.md's VERDICT_JSON trailer; readers bucket abstentions via verdict_pass_rate()'s cannot_judge_count and never coalesce a NULL findings_count to 0. |
| v45 | advisor_consults |
Advisor consult telemetry: (ts, session_id, task_key, signal, advisor_host, advisor_model, main_model, floor_status, outcome, latency_ms, input_tokens, output_tokens, confidence, verdict_body). One row per consult_for_trigger() invocation (advisor.py) — issued, every skipped_reason (disabled/trigger_not_allowed/budget_exhausted/not_configured/floor_violation/failed/timeout), written best-effort via write_advisor_consult(). verdict_body stays NULL unless the advisor.store_verdict_body opt-in is set; token columns stay NULL until a host surfaces usage. Live-write-only, like verdict_events/context_pressure_events/review_events — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS. Enables ll-session recent --kind advisor_consult, FTS, export, and history_reader.query_advisor_consults()/consult_stats() (FEAT-3300). This is the standalone persistence half of FEAT-3040 — no ll-ctx-stats report section ships with it, matching the v33/v34/v35 precedent (FEAT-3301 wires reporting later). |
| v46 | research_triage_events |
Live research-triage skip-rate telemetry: (ts, session_id, issue_id, axis, covered, reason, refined_at, evidence). One row per axis per ll-issues research-triage invocation (three rows share one ts + issue_id, written in a single transaction by write_research_triage()), gated on analytics.capture.cli_commands explicitly (cli_event_context's own gate was dead at v46 time — no caller passed it config; ll-history/ll-session wire config= since ENH-3449, but ll-issues still does not). axis and reason are both closed sets enforced by a SQL CHECK, mirroring verdict_events.verdict. reason is NULL when covered, else one of no_qualified_refs/below_threshold/missing_symbol/stale/program_design_unmet/unreadable. refined_at is nullable — NULL means a first-refine invocation with no prior Session Log entry to stale-check against; the read side conditions the headline rate on refined_at IS NOT NULL to isolate the re-refine population ENH-2990 exists to measure, and excludes program_design_unmet rows from the rate while reporting their count separately. Replaces ENH-2971's corpus-sweep proxy (which scored each issue's current state, not the state at each historical refine) with a measurement of real invocations. Live-write-only, like advisor_consults — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS; rows are not indexed into search_index. Enables ll-session recent --kind research_triage and history_reader.research_triage_stats() (ENH-2990). |
| v47 | credential_scope_events |
After-the-fact audit of the credential scope a run was granted (ENH-3204): (ts, run_id, state, scopes_json, var_names_json). One row per declaring-state dispatch, written by FSMExecutor from write_credential_scope() immediately before the spawn — scopes_json is the declared StateConfig.scopes names, var_names_json is resolve_scopes()'s resolved env-var names (both JSON arrays; names only, never values — a record that could leak a credential value is worse than no record). No row is written for an undeclared state (state.scopes is None). Live-write-only, like advisor_consults/research_triage_events — excluded from rebuild()'s _REBUILD_TABLES/_REBUILD_SEARCH_KINDS; rows are not indexed into search_index. Enables ll-session recent --kind credential_scope. |
| v48 | orchestration_runs.ll_version / loop_runs.ll_version |
Little-loops-version stamp (FEAT-3404): ALTER TABLE ... ADD COLUMN ll_version TEXT on both tables. The writers default it from the installed little_loops.__version__ when the caller passes none, so no orchestrator call site changes. On orchestration_runs it is write-once via COALESCE(ll_version, excluded.ll_version) in the UPSERT — mirroring the base_sha/base_dirty precedent (v38) — so a terminal upsert issued after a mid-run pip install -e upgrade cannot overwrite the version recorded at dequeue. On loop_runs the write is a plain INSERT OR IGNORE with no merge clause needed. NULL means unstamped: the row predates this column. Exposed through the typed reader path (recent_orchestration_runs()/aggregate_orchestration_runs()/recent_loop_runs()/aggregate_loop_runs()/find_loop_run() and the OrchestrationRun/LoopRun dataclasses), not only via raw SQL. |
| v49 | harness_events run-model columns + harness_admissions |
Schema foundation for the harness run model (ENH-3406), first of three issues decomposed from ENH-3397. harness_events gains five nullable columns: cell_key (opaque TEXT identity — target/task/subject — canonical encoding owned by ENH-3407), repetition (stamped index per cell), attempt_kind (repetition/infra_retry, CHECK-enforced), continuations (reserved, unpopulated until a future producer exists), and superseded_by (self-referential attempt id, mirroring the parent_id precedent). A new append-only harness_admissions table (id, ts, attempt_id, superseded_id, reason CHECK-enforced to timeout/host_crash/harness_error/network) audits when and why an infra retry was admitted — INSERT-only, like hook_events/commit_events. A partial UNIQUE index on (cell_key, repetition) WHERE attempt_kind = 'repetition' is the DB-level anti-p-hacking guard: two repetition rows can never share an index for one cell, surfacing a concurrent MAX+1 race as IntegrityError instead of a silent duplicate sample. Schema-only — no existing CLI command's behavior changes; ENH-3407 (writers) and ENH-3408 (read-path counting) build on this. Enables ll-session recent --kind harness_admission. |
Schema migration runs automatically; no manual ll-session backfill is needed for new tables. The issue_sessions VIEW requires captured_at populated on issue_events rows, which ll-session backfill seeds from on-disk sources for pre-v4 databases. As of ENH-1830, session_start automatically triggers an incremental backfill in a background thread, so new interactive session data is indexed without manual intervention.
Extension Loading¶
Extensions are loaded via two mechanisms:
1. Config paths: "extensions": ["my_package:MyExtension"] in ll-config.json
2. Entry points: importlib.metadata discovery under the little_loops.extensions group
Topic-Based Event Filtering¶
Extensions can declare an event_filter class attribute to subscribe only to specific event namespaces, using fnmatch glob patterns matched against the event's "event" key:
class MyExtension:
event_filter = "fsm.*" # only FSM lifecycle events
# event_filter = ["fsm.*", "issue.*"] # multiple namespaces
# event_filter = None # all events (default)
def on_event(self, event: LLEvent) -> None:
...
wire_extensions() forwards event_filter to bus.register(). If the attribute is absent or None, the extension receives all events.
See API Reference — Extension API for full protocol, loader, and wire_extensions() documentation.
History DB: Producer→Consumer Flow¶
.ll/history.db is the per-project event history store — a SQLite database populated by hook writers and queryable in milliseconds without re-parsing JSONL or markdown. It provides agent context (user corrections, related file edits, prior issue work) to skills like refine-issue, ready-issue, and confidence-check without the overhead of full-log scanning. A multi-repo workspace can declare its topology in an ll-workspace.yaml manifest, discovered via little_loops.workspace.discover_workspace_members() (FEAT-3409), so a consumer can aggregate several projects' .ll/history.db stores instead of reading just one — aggregate_history_dbs() for quality analysis (FEAT-3410) and its sibling aggregate_workspace_activity() for per-repo/workspace-total loop and issue-lifecycle activity counts over a since/until window (FEAT-3445).
Write Path¶
sequenceDiagram
participant SS as session_start
participant PTU as post_tool_use
participant UPS as user_prompt_submit
participant EB as EventBus
participant ST as SQLiteTransport
participant DB as history.db
SS->>DB: ensure_db() — bootstrap schema (v1–v34)
SS-->>DB: backfill_incremental() ingests JSONL into raw_events (background thread; --rebuild only when SCHEMA_VERSION > last_rebuild_version)
PTU->>DB: tool_events / file_events (direct write, analytics.enabled)
UPS->>DB: user_corrections / skill_events via record_correction() / record_skill_event()
EB->>ST: emit(IssueEvent | LoopEvent)
ST->>DB: INSERT INTO issue_events / loop_events
Read Path¶
flowchart TB
DB[history.db]
HR[history_reader/]
DB --> HR
HR --> HC["ll-history-context CLI<br/>find_user_corrections + recent_file_events<br/>→ ## Historical Context block"]
HR --> LS["ll-session CLI<br/>search + related_issue_events<br/>+ sessions_for_issue"]
HR --> SK["Skills<br/>refine-issue / ready-issue / confidence-check<br/>/ create-sprint / scope-epic / manage-issue / review-epic"]
HR --> SS2["session_start hook<br/>project_digest → render_project_context<br/>→ <project_context> block (ENH-1907)"]
Components¶
| Component | File | Role |
|---|---|---|
ensure_db() |
session_store/schema.py |
Bootstrap schema (v1–v34 migrations) at session start |
backfill_incremental() |
session_store/lifecycle.py |
Background JSONL → DB seed thread |
compact_session() |
session_store/lifecycle.py |
LCM-style compaction: groups message_events into blocks and creates summary_nodes/summary_spans; opt-in via history.compaction.enabled (FEAT-1712). After per-session passes, cross-session recursive condensation (ENH-1954) groups condensed nodes level-by-level into a multi-level DAG terminating at a single project-root summary node (session_id=NULL, level=max); gated by history.compaction.cross_session_enabled. Once the session's message total crosses the 7,500-token soft threshold, _maybe_soft_threshold_summary() fires a background thread that bounds its input via compaction.instant.evict_sink_and_window() (always-on, structural, no LLM cost) and produces a 6-section (compaction.instant.summarize_6_section()) summary, updating the existing per-session condensed node in place — no schema change (FEAT-2598). |
compaction.instant / compaction.result |
compaction/instant.py, compaction/result.py |
StreamingLLM-style sink+window eviction, Letta-style sliding-window selection, and the CompactResult dataclass wrapper over summary_nodes rows. Manually triggerable via ll-compact-session (FEAT-2598). |
compression.heuristic |
compression/heuristic.py |
Token-cost layer (EPIC-2456 Tier 3, FEAT-2675). Zero-dependency heuristic prompt compressor hooked into FSMExecutor._run_action(): three extractive passes (drop stale tool results, dedupe stable system blocks, tail-truncate assistant turns) behind a window-relative trigger (trigger_pct * context_window vs trigger_tokens, lower wins). Adapts the eviction/boundary logic proven in compaction/instant.py but operates on the live FSM prompt. Runs after the ENH-2486 prompt_size_guard measurement and only for prompt-mode actions; project-configured via compression.*. The LLMLingua-gated benchmark comparator is FEAT-2676. |
cache_marking_oracle.decide_cache_marking() |
cache_marking_oracle.py |
Cache-marking cost oracle (EPIC-2456 F1, FEAT-2673). Decides whether a stable prompt block is safe to mark cache_control: ephemeral via two gates: a per-model cacheable-prefix token minimum (1024 Sonnet / 4096 Opus), and a reuse-stability signal from prompts.fragment_store.FragmentStore (FEAT-2671) — a block is marked only after its content-hash key has already been observed once, avoiding the unamortized 1.25x write premium on never-reused blocks. |
PruningProfileConfig |
fsm/schema.py |
Automation-context static-prefix pruning (EPIC-2456, ENH-2714). Opt-in per-loop/per-state profile (default off) that sets LL_AUTOMATION=1 / LL_AUTOMATION_PROFILE=<name> in the child process env (host_runner.py build_streaming(..., automation=AutomationContext(profile=...)), ENH-3095); automation-aware hooks (session_start.py, history_context.py) check the signal and suppress their static-prefix output (config dump, project_context digest). A None profile is an active opt-out: it clears any inherited LL_AUTOMATION to "" (present-but-falsy) rather than passing the parent's value through, so a non-automation invocation never silently carries the signal (ENH-3081). suppress_catalog/suppress_claude_md are declarative-only forward-declarations — no runtime consumer reads them, so the catalog (~6.4K tokens) and CLAUDE.md (~7.7K tokens) still load in full; suppress_catalog only triggers an MR-12 validator WARN, and claude_md_suppression is unsupported on every host (the claude CLI has no flag to skip CLAUDE.md). Realized saving is the hook-output pruning alone (~1K tokens/invocation). See Loops Guide § Automation-Context Pruning. |
CLAUDE_CODE_DISABLE_BACKGROUND_TASKS |
host_runner.py |
Hard-disable tool-level background tasks in automation (FEAT-3078/FEAT-3060). Shares PruningProfileConfig's automation.profile is not None gate (both fields live on the same AutomationContext, ENH-3095) but is sourced from a distinct, global origin — orchestration.disable_background_tasks config (default false), not a per-loop/per-state pruning_profile:. When enabled, ClaudeCodeRunner.build_streaming() injects CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1, preventing a Claude Code child from silently discarding completed work via Bash run_in_background: true or an Agent/Task-tool spawn left to its background-by-default behavior (BUG-3209) whose result the parent never retrieves. Claude-Code-only; the other seven runners accept and ignore the field. automation.profile is None (or automation is None) explicitly neutralizes the var to "" rather than omitting it, same leak-prevention pattern as LL_AUTOMATION above. |
SQLiteTransport.send() |
session_store/writers.py |
Routes issue.* / loop.* events to DB |
EventBus.emit() |
events.py |
Dispatches events to registered transports |
post_tool_use hook |
hooks/post_tool_use.py |
Writes tool_events / file_events per call |
user_prompt_submit hook |
hooks/user_prompt_submit.py |
Writes user_corrections / skill_events via is_correction() heuristic |
context-monitor.sh |
hooks/scripts/context-monitor.sh |
Writes a handoff_needed session_lifecycle_events row on the first 80%-threshold crossing per pressure episode (bash shell-out with \|\| true, ENH-2495) |
pre_compact hook |
hooks/pre_compact.py |
Writes a compaction session_lifecycle_events row after .ll/ll-precompact-state.json persists (ENH-2495) |
sweep_stale_refs hook |
hooks/sweep_stale_refs.py |
Writes one stale_ref_sweep session_lifecycle_events row per invocation, including zero findings (ENH-2495) |
cli_event_context() |
session_store/writers.py |
Context manager that records ll- CLI entry-point invocations to cli_events (ENH-1849). Honors LL_HISTORY_DB env var for path override. LL_ANALYTICS_CAPTURE=0 (kill switch) and, when the caller passes config=, a present-and-false analytics.enabled or a cli_commands glob exclusion suppress the row before the db is ever resolved (ENH-3449). |
history_reader/ |
history_reader/__init__.py |
Public read API package (ENH-2775 split from the former flat history_reader.py): ~65 query functions across 13 concern submodules (search.py, sessions.py, usage.py, runs.py, context.py, subagents.py, hooks.py, harness.py, events.py, summary_dag.py, digest.py, formatting.py for ll_grep/ll_expand/ll_describe (FEAT-1712) and project_digest/render_project_context (ENH-1907)) plus a shared models.py (dataclasses) and _base.py (connection/row-mapping helpers) |
ll-history-context CLI |
cli/history_context.py |
Primary consumer: ## Historical Context block (issue mode) + project digest dry-run (--project) |
ll-session CLI |
cli/session.py |
Secondary consumer: search, issue events, sessions, grep/expand/describe (FEAT-1712) |
| Skills | commands/refine-issue.md etc. |
Call ll-history-context for agent context injection |
session_start hook |
hooks/session_start.py |
Ambient consumer: injects <project_context> block at session start (opt-in, ENH-1907) |
Decisions Log: .ll/decisions.yaml + .ll/decisions.d/¶
The decisions log is the per-project decisions and rules persistence layer, managed by ll-issues decisions subcommands and the decisions.py data layer. Storage is hybrid (BUG-2642): new entries are written as append-only per-entry fragments under .ll/decisions.d/<uuid4>.json (UUID4 ids, so concurrent EPIC-branch appends never collide), and a legacy .ll/decisions.yaml flat file may also exist. load_decisions() reads the union of both tiers; save_decisions() (compaction) folds every fragment into the flat file and deletes the fragment directory. A fresh install has only .ll/decisions.d/. It stores three entry types:
| Entry Type | Purpose |
|---|---|
rule |
Enforced policies (advisory or required); required rules surface in /ll:ready-issue validation |
decision |
Recorded architectural or process decisions; auto-generated from completed issues via ll-issues decisions generate |
exception |
One-time exceptions to existing rules; suppress false-positive violations in /ll:ready-issue and /ll:verify-issues |
coupling |
Wire-issue static layer: maps if_changed glob patterns to then_check audit targets; tier (hard/soft/fyi) controls how matches are injected into agent prompts; optional archetype groups rules into named bundles (e.g., add-cli-command) |
Opt-in: An absent decisions log (neither .ll/decisions.yaml nor .ll/decisions.d/) is never an error — all integrations gracefully skip when it is missing. Presence gates must accept either tier ([ -f .ll/decisions.yaml ] || [ -d .ll/decisions.d ]); gating on the flat file alone silently skips governance on never-compacted installs. Enable the feature by adding a decisions: block to .ll/ll-config.json.
Integrity transports: the schema and load semantics of both tiers (flat file
and .ll/decisions.d/*.json fragments) are gated by ll-verify-decisions
(ENH-2589) at three transport-layer integrations, listed in firing order:
- Claude Code
PreToolUsehook (ENH-2592,hooks/scripts/check-decisions-yaml.sh) — innermost belt; stagesWrite/Editcandidate content in a temp config root and runs the validator against it. Block (exit 2) fires before the host writes the file. - Git pre-commit hook (ENH-2590,
.pre-commit-config.yamlrepo: localblock) — runs the validator on staged changes; fails thegit commiton any caught exception. - Pytest CI belt (ENH-2591,
scripts/tests/test_decisions_yaml_gate.py) — wide-net belt coveringgit commit --no-verifyand non-hook edit paths; runs as part ofpython -m pytest scripts/tests/.
Key consumers: /ll:ready-issue (Decisions Gate), /ll:verify-issues (rule violation detection), /ll:format-issue (quality analysis), decisions_sync.py (active rules → .ll/ll.local.md sync), /ll:wire-issue Phase 3.5 (coupling entries → MUST_AUDIT injection into agent prompts).
Correction Detection Heuristic¶
is_correction() in session_store/writers.py decides whether a user message should be recorded as a user_corrections row. It applies three independent pattern sets in order:
- Prefix patterns (
_CORRECTION_RE) — Opening phrases like "no,", "wrong,", "actually,", "that's not", "you're wrong". - Phrase-internal patterns (
_PHRASE_RE) — Mid-sentence signals: "instead", "you missed", "should be" (guarded against false-positive affirmatives like "should be fine"), "wrong approach", "remember that", "always use", "never use", "from now on", "I meant … not". (ENH-1887) - Explicit escape hatch (
_REMEMBER_RE) — A message beginning with!rememberis always classified as a correction regardless of phrasing. (ENH-1887)
Any match across the three sets records the message as a correction. A fourth mechanism is available via the optional extra_patterns argument to is_correction(): user-configured raw regex phrases from analytics.capture.correction_patterns are compiled and evaluated as an additional search() pass. The three module-level pattern sets remain the built-in base and are never replaced. Consumers (refine-issue, ready-issue, confidence-check, go-no-go) retrieve these rows via ll-history-context to surface prior corrections as context before generating a response.
Graceful-Degradation Contract¶
_connect_readonly()returnsNoneon schema-version mismatch, file-not-found, or any open failure- All query functions (
find_user_corrections,recent_file_events,search,related_issue_events,sessions_for_issue) return[]when the connection isNone - All hook writers wrap DB calls in
contextlib.suppress(Exception)so a write failure never aborts a tool call SQLiteTransport.send()is a no-op whenself._conn is None
See also: Extension Architecture & Event Flow for the full schema-version table (v1–v34) and CLI transport-wiring table.
Queue DB (ll-queue)¶
.ll/queue.db is a sibling per-project SQLite database — distinct from .ll/history.db — backing ll-queue's persisted work-item queue (little_loops.queue_store, FEAT-2682). It copies session_store/schema.py's _configure_connection/_apply_migrations/ensure_db/connect shape (own _MIGRATIONS/SCHEMA_VERSION) rather than sharing code, matching every other sqlite consumer in this codebase.
| Version | Table | Purpose |
|---|---|---|
| v1 | queue_entries |
One row per queued work item: id, action (JSON ActionSpec), enqueued_at, priority (0=P0..5=P5), status, result (JSON, nullable). Ordered ORDER BY priority ASC, enqueued_at ASC — replicates QueuedIssue.__lt__'s tiered-then-FIFO comparator (parallel/types.py) without importing that class. |
| v2 | queue_entries |
Adds claimed_at (TEXT, nullable) and owner_pid (INTEGER, nullable) (FEAT-2930) — ownership tracking for the --watch long-lived drainer's stale-entry reclaim; both are NULL on a pending/done/failed entry. |
| v3 | queue_entries |
Adds attempt (INTEGER NOT NULL DEFAULT 0) and next_attempt_at (TEXT, nullable) (ENH-3416) — attempt budget and backoff. attempt increments inside claim_entry's UPDATE, so it survives a dead drainer with no extra bookkeeping; next_attempt_at gates claim_entry's eligibility filter (NULL or elapsed = claimable). |
add_entry/list_entries/get_entry/resolve_entry/remove_entry back ll-queue's add/list/status/remove commands (cli/queue.py). ll-queue run (FEAT-2683), the dequeue-and-execute worker loop, uses two distinct writes: claim_entry performs the pending -> running acquisition inside a BEGIN IMMEDIATE transaction so two concurrent drainers cannot both win the same entry (BUG-2929) — a lost claim advances to the next pending candidate rather than dispatching or breaking the drain loop, sleeping --poll-interval before retrying rather than busy-spinning (FEAT-2930) — and also stamps claimed_at/owner_pid and increments attempt in the same transaction, refusing a row whose next_attempt_at is still in the future (ENH-3416); update_entry_result performs the completion write once the caller already owns the entry, recording the real status/result after dispatch through run_action() (runner_spec.py) in priority/FIFO order, and nulls both ownership columns. This completion write, and its siblings schedule_retry/dead_letter_entry/cancel_entry (ENH-3416), are all guarded with AND status = 'running' (or IN ('pending', 'running') for cancel_entry): a rowcount of 0 means an operator cancel or a stale-entry reclaim beat the drainer to the row, and _drain_once re-reads the row and records its actual status rather than assuming its own write landed. RunnerType.LOOP entries are intercepted before reaching run_action() (which explicitly excludes that runner) and driven instead via a subprocess ll-loop run shell-out (FEAT-2906) — .ll/queue.db does carry FSM loop work, just not through run_action()'s dispatch path. ll-loop queue's PID-liveness JSON markers under .loops/.queue/*.json (cli/loop/queue.py) remain a separate, unchanged surface for FSM lock contention, preserved by FEAT-2684 as a compat shim rather than migrated into this store.
A failed dispatch is classified by _classify_dispatch (cli/queue.py, ENH-3416): an explicit timeout, or a LOOP entry whose own internal retry budget (fsm/executor.py) is already exhausted (error == "terminal failure"), is never retryable; otherwise issue_lifecycle.classify_failure's reason string is checked against the narrow QUEUE_RETRYABLE_REASONS allowlist (quota/rate-limit, network, API server error, infra teardown) rather than trusting its broad FailureType.TRANSIENT — a queue entry's stderr is arbitrary program output (a whole ll-loop run log, a pytest run), where the classifier's substrings ("timeout", "429") would otherwise fire on unrelated text. A retryable failure with budget remaining (attempt < QUEUE_MAX_ATTEMPTS = 5) is returned to pending with next_attempt_at = now + compute_backoff_s(attempt) (non-jittered doubling, 5, 10, 20, 40, 80... seconds, capped at QUEUE_BACKOFF_CEILING_S = 300); exhausted, it moves to terminal dead_letter. A non-retryable failure lands on failed on the first attempt, unchanged from pre-ENH-3416 behavior.
ll-queue run --watch (FEAT-2930) turns the one-shot drainer into a long-lived one: after draining, it sleep-polls for new entries instead of exiting, and on startup plus each idle poll it sweeps running entries whose owner_pid is dead back to pending (_reclaim_stale, cli/queue.py) — a SIGKILLed/OOM-killed/rebooted owner is the normal failure mode for a long-lived drainer rather than a rare one a human is present to witness. attempt is shared between transient dispatch failures and owner deaths (ENH-3416): once it reaches QUEUE_MAX_ATTEMPTS, _reclaim_stale dead-letters the entry instead of reclaiming it again, so a poison entry that reliably kills its own drainer is not re-dispatched forever; next_attempt_at is never touched by this path — an owner death is a slot to refill, not a backoff-eligible failure. The liveness check is a psutil identity check (not a bare os.kill(pid, 0)), mirroring cli/loop/queue.py's _verify_queue_pid_identity but parameterized for ll-queue's own process markers, so a recycled PID is never mistaken for the original owner. ll-queue requeue <id> [--force] is the manual escape hatch for a stranded running entry, and (ENH-3416) is also widened to revive a terminal dead_letter/failed/cancelled entry with a fresh attempt budget, preserving the prior result under result.previous. ll-queue cancel <id> [--reason TEXT] (ENH-3416) moves a pending/running entry to terminal cancelled without signaling an in-flight process — killing arbitrary runner subprocesses from a second CLI process is out of scope, so a cancel of a running entry is a status-only mark that the guarded completion write above then no-ops against. Shutdown is two-stage: a first SIGINT/SIGTERM lets the in-flight entry finish before the drainer exits; a second forwards SIGTERM to an in-flight LOOP child's process group (launched with start_new_session=True so the forward is a targeted os.killpg) and marks that entry cancelled with reason: "interrupted by operator" (routed through the same cancel_entry, so partial stdout/stderr is preserved).
Host Runner Layer¶
Sitting alongside the hook-intent layer is the host_runner abstraction
(scripts/little_loops/host_runner.py). Where hook intents normalize
incoming host events into the LLHookEvent envelope, the host runner
normalizes outgoing CLI invocations: every shell-out to a host CLI
(claude, codex, opencode, pi, gemini, omp, kimi) is built through a HostRunner
implementation rather than hard-coded argv. This makes the orchestration
layer host-agnostic and keeps host-specific argv shape out of call sites
like ll-auto, ll-parallel, ll-action, ll-loop, FSM evaluators, and
FSM handoff.
| Component | Purpose |
|---|---|
HostRunner (Protocol) |
Contract every runner satisfies — detect(), build_streaming(), build_blocking_json(), build_version_check(), build_detached() factories returning HostInvocation; describe_capabilities() returning CapabilityReport |
HostInvocation (frozen dataclass) |
Value object holding binary, args, env, capabilities, cleanup_paths, and env_allow — passed to subprocess.Popen/run; callers must unlink cleanup_paths after the subprocess completes |
HostCapabilities (frozen dataclass) |
Capability flags (streaming, permission_skip, agent_select, tool_allowlist, structured_output, workspace_sandboxed) describing what a host supports |
ClaudeCodeRunner |
Production runner for the claude CLI |
CodexRunner |
Production runner for the codex CLI; auto-detected when codex is on PATH |
GeminiRunner |
Production runner for the gemini CLI (Gemini CLI); auto-detected when gemini is on PATH (ENH-2185) |
OmpRunner |
Production runner for the oh-my-pi omp CLI; auto-detected when omp is on PATH (FEAT-1850) |
KimiRunner |
Production runner for the Kimi Code kimi CLI; auto-detected when kimi is on PATH (FEAT-2911 flag translation, FEAT-2914 wiring; thoughts/research/kimi-cli-surface.md) |
QwenRunner |
Production runner for Qwen Code's qwen CLI; auto-detected when qwen is on PATH (FEAT-3155 flag translation, ENH-3156 wiring; the second host with structured_output=True via inline --json-schema; thoughts/research/qwen-code-surface.md) |
OpenCodeRunner |
Stub for the opencode CLI (FEAT-1472 stub state) |
PiRunner |
Frozen stub for the vanilla pi-mono pi CLI (cancelled — ARCHITECTURE-050; superseded by OmpRunner) |
FakeHostRunner |
Test-only runner for the ll-fake-host console script — a real executable driven through the untouched subprocess.Popen spawn path so a scripted "directives" prompt exercises event ordering, abort, idle-timeout, and failed-start paths with no live host CLI and no model (FEAT-3454). Registered in _HOST_RUNNER_REGISTRY under TEST_ONLY_HOSTS, absent from _PROBE_ORDER; never appears in user-facing host lists. |
FakeMinimalHostRunner |
Second test-only runner, deliberately divergent from FakeHostRunner (subcommand-style ["run", prompt] argv, always-empty env, all-six-flags-False default capabilities) — proves the executor reads only the abstract HostRunner/HostInvocation surface rather than either fake's shape (ENH-3459). Registered in _HOST_RUNNER_REGISTRY under TEST_ONLY_HOSTS, absent from _PROBE_ORDER; never appears in user-facing host lists. |
resolve_host() |
Discovery entry point — honors LL_HOST_CLI / orchestration.host_cli overrides, then probes PATH for known host binaries |
HostNotConfigured |
Raised when no runner can be resolved — error includes LL_HOST_CLI remediation hint |
CapabilityNotSupported |
UserWarning subclass emitted when a caller requests a capability the active host lacks |
CapabilityReport (frozen dataclass) |
Structured preflight report returned by describe_capabilities() — holds host, binary, version, and capabilities; consumed by ll-doctor and ll-action. ll-doctor --json's payload is a superset of this dataclass — it also adds analytics_capture/issues keys sourced from BRConfig (ENH-2762), plus install-surface keys (entry_points, skills_commands, decisions_store, history_db, loop_validity, schema_drift, advisor, and full under --full) covering little-loops' own project state, none of which come from CapabilityReport itself (FEAT-2793/FEAT-2795/ENH-3242/FEAT-3122) |
CapabilityEntry (frozen dataclass) |
One capability's name and "full" / "partial" / "unsupported" status |
apply_host_cli_from_config() |
Reads orchestration.host_cli from BRConfig and exports it as LL_HOST_CLI before resolve_host() runs |
New host-CLI call sites MUST go through resolve_host() rather than
adding new "claude" literals. See
HOST_COMPATIBILITY.md
SDK/Batches Dispatch Path (orchestration.request_path)¶
The same host_runner.py module also hosts a second, structurally distinct
mechanism (FEAT-2673, FEAT-2710, FEAT-2716, EPIC-2456 F1): opt-in dispatch
straight through the anthropic SDK client instead of a host-CLI
subprocess. It does not implement HostRunner and produces no
HostInvocation — build_anthropic_request()/build_batch_request()
assemble request kwargs, and dispatch_anthropic_request()/
dispatch_batch_request()/poll_batch_result() perform the actual
anthropic.Anthropic().messages.create() / .messages.batches.* network
calls, normalizing responses into the same ActionResult shape the
CLI-subprocess path returns. FSMExecutor selects between the two
mechanisms per state via state.request_path or
orchestration_config.request_path ("cli" default, "sdk", or "batch");
a configured "sdk"/"batch" value automatically downgrades to "cli" (with
a one-shot warning) if anthropic is not importable, no credential is
resolvable via the SDK's auth chain — env key/token or the on-disk OAuth
profile from ant auth login (ENH-2737) — or the state's action invokes a
/ll: skill or declares tools: (BUG-2831): the SDK/Batches dispatch path
sends a bare, tool-less single-turn call with no host-CLI agentic tool
loop, so a skill invocation can only emit its intended actions as inert
text and silently no-op if left on this path — the downgrade applies
unconditionally, overriding even an explicit per-state request_path: sdk.
This keeps "sdk"/"batch" scoped to what the mechanism can actually
serve: pure text-in/text-out evaluator prompts. Either way,
a run never hard-fails on a host that only has the CLI available; see
API.md § little_loops.host_runner
for the dispatch function reference.
for the per-host orchestration matrix and
API Reference — little_loops.host_runner
for the full public surface.
Artifact Control Layer¶
Sitting alongside the Host Runner Layer above is the artifact control contract
(docs/reference/ARTIFACT_CONTROL_LEVELS.md). Where the Host Runner Layer
normalizes outgoing host-CLI invocations, this layer normalizes what a rendered
artifact (a dashboard, an MCP Apps interactive resource, ll-loop run --serve's
SSE-bridged view) may do when a user interacts with it, and which layer — the
host session or the FSM executor — owns the resulting state change. The FSM
executor is the sole routing authority (FSMExecutor.run(),
scripts/little_loops/fsm/executor.py); ll-loop run --serve's
LocalBridgeTransport (ENH-3351) is the first render target that routes an
interaction back into it, via FSMExecutor._drain_inbound() — record-and-emit
only, with no FSM guard/transition semantics yet. The contract names three levels — notify,
ask-to-run-prompt, host-owned — so that future render targets converge on one
re-entry vocabulary instead of each defining it implicitly. See
ARTIFACT_CONTROL_LEVELS.md for the full
per-level obligations table and prohibitions.
Class Relationships¶
classDiagram
class BRConfig {
+project: ProjectConfig
+issues: IssuesConfig
+automation: AutomationConfig
+parallel: ParallelAutomationConfig
+get_issue_dir(category) Path
+create_parallel_config() ParallelConfig
+to_dict() dict
}
class IssueParser {
+config: BRConfig
+parse_file(path) IssueInfo
}
class IssueInfo {
+path: Path
+issue_type: str
+priority: str
+issue_id: str
+title: str
+status: str
+priority_int: int
}
class AutoManager {
+config: BRConfig
+state_manager: StateManager
+event_bus: EventBus
+db_path: Path | None
+run() int
}
class StateManager {
+state_file: Path
+load() ProcessingState
+save()
+mark_completed(issue_id)
+mark_failed(issue_id, reason)
}
class ParallelOrchestrator {
+parallel_config: ParallelConfig
+br_config: BRConfig
+queue: IssuePriorityQueue
+worker_pool: WorkerPool
+merge_coordinator: MergeCoordinator
+run() int
}
class WorkerPool {
+parallel_config: ParallelConfig
+start()
+submit(issue) Future
+shutdown()
+cleanup_all_worktrees()
}
class MergeCoordinator {
+config: ParallelConfig
+start()
+queue_merge(result)
+shutdown()
}
BRConfig --> IssueParser
IssueParser --> IssueInfo
BRConfig --> AutoManager
AutoManager --> StateManager
BRConfig --> ParallelOrchestrator
ParallelOrchestrator --> WorkerPool
ParallelOrchestrator --> MergeCoordinator
ParallelOrchestrator --> IssuePriorityQueue
Configuration Flow¶
flowchart LR
subgraph Load["Load Phase"]
JSON[".ll/ll-config.json"]
INIT["BRConfig.__init__()"]
PARSE["_parse_config()"]
end
subgraph Objects["Config Objects"]
PC[ProjectConfig]
IC[IssuesConfig]
AC[AutomationConfig]
PAC[ParallelAutomationConfig]
end
subgraph Usage["Usage"]
CMD["Command Templates<br/>{{config.project.*}}"]
AUTO_CLI["ll-auto"]
PAR_CLI["ll-parallel"]
end
JSON --> INIT
INIT --> PARSE
PARSE --> PC
PARSE --> IC
PARSE --> AC
PARSE --> PAC
PC --> CMD
IC --> CMD
AC --> AUTO_CLI
PAC --> PAR_CLI
Design tokens (DesignTokensConfig) serve as a cross-cutting input to artifact-generating loops: ll-loop run and ll-loop resume pre-inject the resolved token set into the FSM initial context before the first state is entered.
Project-enriched artifacts. This is one instance of a broader principle: little-loops generators stamp resolved project context into their output at generation time rather than fetching it at runtime. Design tokens are the shipped example (render_as_css_vars injected into the loop context above). The pattern extends to other generators — e.g. the policy-router HTML builder (ll-artifact policy-builder, FEAT-2390) stamps the canonical predicate grammar (policy_rules.grammar_spec()) and the project's invokable skill/command catalog (cli/action.py:_load_skills()) into a self-contained .html as JSON <script> islands. This is deliberate: the enrichment is what makes a generated artifact useful in this project, so outputs are point-in-time snapshots tied to the project, not portable-generic templates. "Self-contained" describes a runtime property (no fetch, works over file://), not cross-project portability — regenerate to pick up project changes.
Turning a generated artifact into a reusable template (ll-artifact templatize, FEAT-3308; token lifting, ENH-3319). A one-off generated artifact can be templatized: a region map (hand-written via --regions, or LLM-discovered via discover_regions) identifies the spans that came from the source document, templatize splices those spans into Jinja2 [[= expr =]]/[[% for %]] stamp points, and derives data.json + data_schema from the extracted bytes. The result is built in a temp directory, verified against a byte-exact round trip (re-render must reproduce the original artifact bytes exactly), and only then promoted atomically into a .llat/ template directory — render later stamps that template against a different data.json with no LLM call.
Because a lifted design-token stamp point ([[= ll.theme_css =]]) is by definition not the literal hex value the round trip requires, a lift is opt-in and mutually exclusive with the byte-exact guarantee: with --lift-tokens off (the default), templatize only scans the spliced template body for baked color literals matching the project's resolved design tokens and writes unlifted-tokens.json alongside the template, never rewriting anything — the byte-exact round trip holds unconditionally. With --lift-tokens on, a matched literal in CSS-value position that resolves unambiguously to a single token name is rewritten to a var(--dotted-name) reference, and the template gains the [[= ll.theme_css =]] stamp point, a data-theme attribute, and manifest["theme"] = "design-tokens" — the rewrite and the stamp point land together or not at all, gated on five hard preconditions (a placeable stamp point, a root <html>, no disagreeing data-theme, active_theme in {light, dark} or a design_md source, and full var-name coverage in themed_css_vars(config) at lift time). In place of the byte-exact check, a lifted body is verified in three stages: the pre-lift round trip (unchanged), a span-tracked reversibility check (undoing the recorded lift and stamp spans reproduces the verified pre-lift body), and a runtime post-lift render check (the re-serialized template, re-rendered from disk, must declare a --x: custom property for every emitted var(--x) reference). Two limitations are accepted rather than hidden: a lifted .llat later re-rendered in a project with no design tokens configured gets empty :root {}/[data-theme=dark] {} blocks from themed_css_vars and renders colorless, since the declarations are resolved at render time against whatever project is rendering; and color-valued presentation attributes (inline SVG fill/stroke/stop-color, bgcolor, <meta name="theme-color">) are reported but never rewritten, since the CSS-context guard's scope test covers only <style> elements and style="..." attributes.
Issue Processing Lifecycle¶
stateDiagram-v2
[*] --> Discovered: /ll:scan-codebase
Discovered --> Prioritized: /ll:prioritize-issues
Prioritized --> Validating: /ll:ready-issue
Validating --> Ready: READY verdict
Validating --> NotReady: NOT_READY verdict
Validating --> ShouldClose: CLOSE verdict
Ready --> Deciding: decision_needed: true
Deciding --> Ready: /ll:decide-issue
Ready --> InProgress: /ll:manage-issue
InProgress --> Verifying: Implementation done
Verifying --> Completed: Tests pass
Verifying --> Failed: Tests fail
NotReady --> Discovered: Fix issue file
ShouldClose --> Completed: Set status: done
Failed --> Discovered: Create follow-up issue
Discovered --> Deferred: Defer issue
Deferred --> Discovered: Undefer issue
Completed --> [*]: status: done (stays in its type dir)
Deferred --> [*]: status: deferred (stays in its type dir)
Priority Queue Design¶
The priority queue separates P0 (critical) issues for sequential processing while allowing P1-P5 to be processed in parallel.
flowchart TB
subgraph Input["Issue Scanning"]
SCAN[Scan .issues/ directories]
end
subgraph Queue["IssuePriorityQueue"]
P0Q[P0 Queue<br/>Sequential]
PARQ[P1-P5 Queue<br/>Parallel]
end
subgraph Processing["Processing"]
SEQ[Sequential<br/>One at a time]
PAR[Parallel<br/>Up to max_workers]
end
SCAN --> P0Q
SCAN --> PARQ
P0Q --> SEQ
PARQ --> PAR
SEQ --> |Complete before| PAR
Rationale: P0 issues are critical and may have dependencies. Processing them sequentially ensures stability before parallel work begins.
Sprint Mode (ll-sprint)¶
Sprint execution uses dependency-aware wave-based scheduling. Issues are grouped into waves where each wave contains issues whose blockers have all completed.
flowchart TB
subgraph Build["Build Phase"]
LOAD[Load sprint issues]
INFO[Load IssueInfo objects]
GRAPH[Build DependencyGraph]
WAVES[Calculate execution waves]
end
subgraph Waves["Wave Execution"]
W1[Wave 1<br/>No blockers]
W2[Wave 2<br/>Blocked by Wave 1]
W3[Wave N<br/>Blocked by Wave N-1]
end
subgraph Parallel["ParallelOrchestrator"]
ORCH[Execute wave in parallel]
WORKERS[Workers process issues]
MERGE[Merge results]
end
LOAD --> INFO
INFO --> GRAPH
GRAPH --> WAVES
WAVES --> W1
W1 --> ORCH
ORCH --> WORKERS
WORKERS --> MERGE
MERGE --> W2
W2 --> ORCH
MERGE --> W3
Sprint Execution Flow¶
sequenceDiagram
participant User
participant CLI as ll-sprint
participant Manager as SprintManager
participant Graph as DependencyGraph
participant Orch as ParallelOrchestrator
User->>CLI: ll-sprint run sprint-1
CLI->>Manager: Load sprint
Manager-->>CLI: Sprint with issues
CLI->>Manager: load_issue_infos(issues)
Manager-->>CLI: List[IssueInfo]
CLI->>Graph: from_issues(issue_infos)
Graph-->>CLI: DependencyGraph
CLI->>Graph: get_execution_waves()
Graph-->>CLI: [[Wave1], [Wave2], ...]
loop For each wave
CLI->>CLI: Log wave issues
CLI->>Orch: Execute wave issues
Orch-->>CLI: Wave complete
end
CLI-->>User: Sprint complete
Wave Calculation Example¶
Given issues with dependencies:
- FEAT-001: No blockers
- BUG-001: No blockers
- FEAT-002: Blocked by FEAT-001
- FEAT-003: Blocked by FEAT-001
- FEAT-004: Blocked by FEAT-002, FEAT-003
The DependencyGraph.get_execution_waves() returns:
| Wave | Issues | Reason |
|---|---|---|
| 1 | FEAT-001, BUG-001 | No blockers |
| 2 | FEAT-002, FEAT-003 | FEAT-001 completed in Wave 1 |
| 3 | FEAT-004 | FEAT-002, FEAT-003 completed in Wave 2 |
Issues within each wave execute in parallel. Waves execute sequentially.
Dependency Discovery¶
The dependency_mapper module complements dependency_graph by discovering new dependency relationships:
- dependency_graph.py: Execution ordering from existing
Blocked Bydata - dependency_mapper/: Discovery of new relationships via file overlap + semantic conflict analysis (split into
models,analysis,formatting,operationssub-modules)
The /ll:map-dependencies skill uses dependency_mapper to analyze active issues, propose dependencies based on shared file references, validate existing dependency integrity (broken refs, missing backlinks, cycles), and write approved relationships to issue files.
Semantic Conflict Analysis¶
When two issues reference the same file, the mapper goes beyond simple file overlap to determine whether they actually conflict. It computes a conflict score (0.0–1.0) from three signals:
- Semantic target overlap (weight 0.5) — Extracts PascalCase component names, function references, and explicit scope mentions from issue content, then computes Jaccard similarity
- Section mention overlap (weight 0.3) — Detects UI region keywords (header, body, sidebar, footer, card, modal, form) and checks if both issues target the same region
- Modification type match (weight 0.2) — Classifies each issue as structural, infrastructure, or enhancement based on keyword matching
Score thresholds: - < 0.4: Parallel-safe — issues touch different sections of the same file and can run concurrently - >= 0.4: Dependency proposed — issues likely conflict and should be sequenced
Same-priority ordering: When two conflicting issues share the same priority, the mapper uses modification type to determine direction (structural → infrastructure → enhancement) rather than arbitrary ID ordering.
Key Design Decisions¶
Git Worktree Isolation¶
Each parallel worker operates in a separate git worktree:
.worktrees/
├── worker-1/ # ll-parallel worker (full repo copy)
│ ├── src/
│ ├── tests/
│ └── .claude/
├── worker-2/
├── worker-N/
└── <timestamp>-<loop-name>/ # ll-loop --worktree isolated run
Benefits: - No file conflicts between workers - Each worker has isolated branch - Clean rollback on failure
Trade-offs: - Disk space usage (full copy per worker) - Initial setup time for worktrees
Sequential Merging¶
Despite parallel issue processing, merges happen one at a time:
flowchart LR
W1[Worker 1<br/>Complete] --> MQ[Merge Queue]
W2[Worker 2<br/>Complete] --> MQ
W3[Worker 3<br/>Complete] --> MQ
MQ --> M1[Merge 1]
M1 --> M2[Merge 2]
M2 --> M3[Merge 3]
Rationale: Parallel merges would cause conflicts. Sequential merging with rebase-on-conflict ensures clean integration.
State Persistence¶
Both modes save state for resume capability:
| Mode | State File | Contents |
|---|---|---|
| Sequential | .auto-manage-state.json |
Current issue, completed list, failed list, timing |
| Parallel | .parallel-manage-state.json |
In-progress, completed, failed, pending merges |
Format:
{
"completed_issues": ["BUG-001", "BUG-002"],
"failed_issues": {"BUG-003": "Test failure"},
"attempted_issues": ["BUG-001", "BUG-002", "BUG-003"],
"timing": {
"BUG-001": {"ready": 30.5, "implement": 120.2, "verify": 5.1}
}
}
Merge Strategy¶
The merge coordinator is a sophisticated git operations state machine that handles:
1. Sequential merge queue (one at a time to avoid conflicts)
2. Automatic stash/unstash of local changes with smart exclusions
3. Adaptive pull strategy (tracks problematic commits, switches to merge on repeat)
4. Index recovery (detects and repairs corrupted git state)
5. Lifecycle file coordination (auto-commits pending moves)
6. Conflict retry with rebase (up to max_merge_retries times)
7. Circuit breaker (pauses after consecutive failures)
8. Untracked file backup and retry
See MERGE-COORDINATOR.md for comprehensive documentation.
Host Adapter Capability Map¶
adapters/capabilities.py (ENH-2873/ENH-2874) is a declarative per-host
capability map for ll-adapt's build-time artifact generation: one
HostCapabilityEntry per adapter host (codex, gemini, omp,
kimi-code, claude-code) in adapters/core.py's _EMITTER_MAP, replacing
knowledge that was previously scattered as conditional code across
codex.py/gemini.py/omp.py/kimi.py/claude_code.py. Each
entry's SubagentSupport field ("native" vs "none", ENH-2874) drives
whether core.py emits a real subagent file for a role or falls back to a
degraded inline-role file (0af1e555) when the target host can't spawn
subagents.
This is distinct from host_runner.HostCapabilities (Option B, decided
2026-07-28): that is a runtime invocation surface — "what can this host's
CLI do when it's running" (claude-code/opencode/pi) — while
capabilities.py is a build-time surface with a different host set
(codex/gemini/omp/kimi-code) and no inheritance relationship between the two.
See docs/reference/HOST_COMPATIBILITY.md for the parity matrix both sides
are checked against.
host_runner.RUNTIME_HOST_CAPABILITIES (ENH-3453) is the runtime half of the
same declarative discipline — one RuntimeHostEntry per host in
host_runner._HOST_RUNNER_REGISTRY (the eight real runners, a strict
superset of capabilities.py's host set — TEST_ONLY_HOSTS entries like
fake, FEAT-3454, source capabilities from their own constructor instead
and are exempt), read via load_runtime_capabilities() and
rendered as a CapabilityReport via render_capability_report(). Every
runner's capabilities class attribute and describe_capabilities() body
are sourced from this map rather than a per-subclass literal/method — adding
a host or correcting a capability flag is now a data change in
host_runner.py, not a new subclass method. ll-verify-host-map enforces
key parity between this map and the registry, plus flag/row consistency
(a report row cannot claim "full" for a flag that's False, or
"unsupported" for a flag that's True).
Context Monitor and Session Continuation¶
When context window limits approach, the system can automatically preserve work and spawn fresh sessions.
flowchart TB
subgraph Hook["PostToolUse Hook"]
ESTIMATE[Estimate context usage]
CHECK[Check threshold]
end
subgraph Handoff["Active Handoff Path"]
TRIGGER[Trigger /ll:handoff]
WRITE[Write continuation prompt]
SIGNAL[Output CONTEXT_HANDOFF signal]
end
subgraph CLI["CLI Detection"]
DETECT[Detect handoff signal]
READ[Read continuation prompt]
SPAWN[Spawn fresh session]
end
subgraph PassivePath["Passive Handoff Path (PreCompact)"]
PC_WRITE[precompact-handoff.sh writes continuation prompt]
COMPACT[Claude Code compacts context]
RESUME[/ll:resume re-injects context in current session]
end
ESTIMATE --> CHECK
CHECK -->|>= 80%| TRIGGER
TRIGGER --> WRITE
WRITE --> SIGNAL
SIGNAL --> DETECT
DETECT --> READ
READ --> SPAWN
SPAWN -->|Resume work| ESTIMATE
PC_WRITE --> COMPACT
COMPACT --> RESUME
RESUME -->|Work continues| ESTIMATE
Context Estimation: The hook uses a three-tier priority for token counts:
| Priority | Source | When Active |
|---|---|---|
| 1 (highest) | result_token_count in state file |
Non-zero; written by on_usage callback from stream-json result events — zero lag, authoritative |
| 2 | transcript_baseline_tokens |
use_transcript_baseline: true and transcript available — one-turn lag, API-exact |
| 3 (fallback) | Heuristic estimates | When both above are absent |
When result_token_count > 0 in .ll/ll-context-state.json, the context monitor uses it directly and skips heuristics entirely.
Heuristic estimates (fallback only):
| Tool | Estimation |
|---|---|
| Read | lines × 10 tokens |
| Grep | output_lines × 5 tokens |
| Bash | chars × 0.3 tokens |
| Task | 2000 tokens (summarized) |
| WebFetch | 1500 tokens |
| Other | 100 tokens base |
Continuation Flow:
- Hook triggers at 80% estimated context usage (configurable)
- Handoff command generates
.ll/ll-continue-prompt.mdwith session state - CLI tools (
ll-auto,ll-parallel) detectCONTEXT_HANDOFFsignal in output - Fresh session spawned with continuation prompt
- Work continues seamlessly from saved state
Configuration (enabled by default):
Files:
- hooks/prompts/continuation-prompt-template.md - Template for handoff prompts
- .ll/ll-context-state.json - Running context usage state
- .ll/ll-continue-prompt.md - Generated continuation prompt
- subprocess_utils.py - Handoff detection and continuation reading
Session Log Auto-Linking¶
When an issue file is written with status: done in its frontmatter, a PostToolUse hook automatically appends a Session Log entry. This ensures session logs are linked regardless of which path completed the issue.
Trigger: Any Write tool call whose file path is in .issues/ and whose frontmatter contains status: done.
Covered completion paths:
- manage-issue skill (Phase 5)
- ll-auto (sequential batch)
- ll-parallel (concurrent worktree)
- ll-sprint (dependency-ordered)
- Manual git mv during a Claude session
Implementation:
- Hook script: hooks/scripts/issue-completion-log.sh
- Uses little_loops.session_log.append_session_log_entry() with source hook:posttooluse-git-mv
- Session JSONL path is read directly from the transcript_path field in the PostToolUse stdin payload
- BUG-3424: append_session_log_entry() collapses any pre-existing duplicate ## Session Log headings (via merge_session_log_blocks()) before inserting, so a file left in a duplicate-heading state self-heals on its next hook-driven append.
Issue Auto-Commit¶
When issues.auto_commit: true is set in .ll/ll-config.json, a PostToolUse hook automatically commits issue file changes after every Write or Edit operation on a file in .issues/. The hook skips gracefully if any other changes are staged or unstaged in the working tree.
Trigger: Any Write or Edit tool call whose file path is in .issues/.
Implementation:
- Hook script: hooks/scripts/issue-auto-commit.sh
- Config flags: issues.auto_commit (bool, default false), issues.auto_commit_prefix (string, default "chore(issues)")
- Commit message format: <prefix>: <verb> <filename> where verb is add (new file) or update (existing file)
- Python handler: _maybe_auto_commit() in scripts/little_loops/hooks/post_tool_use.py
Session Event Capture¶
When session_capture.enabled: true is set in .ll/ll-config.json, a PostToolUse hook fires on every tool invocation and appends one structured JSON event record to .ll/ll-session-events.jsonl. This event log is the data source for FEAT-1264's PreCompact snapshot builder, which uses it to reconstruct a structured handoff context (pending tasks, net-modified files, unresolved errors) that is more accurate than the current git-diff-based approach.
Trigger: Any tool invocation (matcher: *).
Event types captured:
- file — Read, Write, Edit, Glob, Grep tool calls (subject = file path)
- task — TodoWrite, TaskCreate, TaskUpdate tool calls (subject = content/id, status = task status)
- git — Bash invocations containing git with exit 0 (op = git subcommand, subject = args)
- error — Bash invocations with non-zero exit code (op = "bash_error", subject = command, status = exit code)
Implementation:
- Hook script: hooks/scripts/session-capture.sh
- Output file: .ll/ll-session-events.jsonl (one compact JSON object per line)
- Config flag: session_capture.enabled (bool, default false)
- Failure-safe: all error paths exit 0; capture failures never block tool execution
Consumer: FEAT-1264 (precompact-handoff.sh) reads this log to build the structured handoff snapshot.
Session-Discovery Seam (Discovery/Read and Ingest, unified)¶
little_loops.session_store discovers, reads, and ingests host session logs through one seam as of ENH-3422 (phase 2 of the discovery/ingest unification ENH-3420 started):
- Discovery and reading:
sessions.py'sdetect_sessions()/iter_events()/list_workspaces(). Every registered host (Codex, Claude Code, opencode, pi, kimi-code, qwen, gemini, omp) goes through the one per-host parser dispatch (_PARSERS). Payload is host-native where no normalizer to Claude shape exists (claude-code,codex,kimi-code); where a host already ships a normalizer (qwen,gemini,omp), payload is that normalizer's own output, wrapped and host-stamped;opencode/piare Claude-shaped on disk and reuse the Claude per-line loop. - Ingest to
history.db:lifecycle.py's_backfill_raw_eventsconsumesiter_events(handle)for every host — the same dispatch discovery uses — instead of a second, per-hostHostLayoutbranch.HostLayout/host_layout_for()(writers.py) shrank to path metadata (glob,session_glob,projects_root,sessions_subdir, the ENH-3165 subagent fields); thenormalize/skip_at_ingest/normalize_filecallables it used to carry are gone.kimi-codehas a realHostLayoutentry (session_glob = "session_*/agents/main/wire.jsonl"), socli/backfill_worker.py/cli/session.py's glob sites (which still readsession_globdirectly rather than going throughiter_events()) find kimi wires too.
The public backfill wrappers (backfill_raw_events/backfill/backfill_incremental) still accept jsonl_files: list[Path] for callers with a project folder to glob, widening internally via sessions.py::handles_from_paths; they additionally accept handles: list[SessionHandle] directly for callers that can't glob a folder — Codex, which has no ~/.codex/projects/ tree and discovers via detect_sessions() instead. A DB ingested before ENH-3422 that still holds raw (pre-normalization) qwen rows replays correctly: writers.py::_iter_events's replay shim re-normalizes only rows shaped like raw qwen wire format (qwen.py::is_raw_qwen_record), leaving already-normalized rows (including every other host) untouched.
Context Efficiency¶
Efficiency metric: tokens-per-task, not tokens-per-request.
For ll-auto, ll-parallel, and ll-sprint, the correct optimization target is minimizing total tokens consumed per completed issue, not per individual turn. Over-aggressive compression that causes retries, re-reads, or error recovery is less efficient than a longer conversation that completes the task cleanly.
This principle is validated by published research on long-context LLM architectures (see docs/research/LCM-Lossless-Context-Management.md, Section 4.3): systems that aggressively chunk context introduce variance and error cascades, while systems that preserve working context through task completion achieve better reliability per token.
Implications for compression decisions:
- Compress at 80% context utilization (see auto_handoff_threshold in ### Context Monitor and Session Continuation, above), not earlier
- Prefer keeping relevant tool outputs in context over re-fetching when needed again
- A failed task that restarts from scratch costs more tokens than a task that completes in a longer conversation
Relationship to ENH-499: The inter-issue context checkpoint (implemented in ENH-499) applies this principle at issue boundaries — it triggers a structured summarization reset rather than re-running tool calls to reconstruct state.
- Skill pre-expansion (
skill_expander.expand_skill) eliminates theToolSearch → Skilldeferred-tool round-trip whenll-autospawns Claude subprocesses: the full skill/command Markdown is read, config placeholders substituted, and the resulting self-contained prompt string is passed directly. This removes one tool call from every Phase 1 and Phase 2 invocation.
Learning Test Registry¶
The Learning Test Registry is a persistent store of proven facts about external systems (APIs, SDKs, libraries) that the codebase or its agents depend on. It exists so that expensive exploration work — "how does the Anthropic streaming API actually shape its events?" — is captured once and reused indefinitely.
Lifecycle¶
The registry is populated by the /ll:explore-api skill, which walks the four-phase Feathers Learning Test loop:
flowchart LR
INGEST[Phase 1: Ingest<br/>check existing record<br/>read docs/source]
HYPOTHESIZE[Phase 2: Hypothesize<br/>3–7 falsifiable claims]
EXECUTE[Phase 3: Execute<br/>run proof script<br/>capture stdout/stderr]
REFINE[Phase 4: Refine<br/>classify pass/fail/untested<br/>write LearnTestRecord]
INGEST --> HYPOTHESIZE
HYPOTHESIZE --> EXECUTE
EXECUTE --> REFINE
REFINE -.-> INGEST
Phase 1 short-circuits if ll-learning-tests check "<target>" already returns a record — future agents skip rediscovery for free, which is the whole point.
Schema¶
Records are YAML-frontmatter Markdown files stored under .ll/learning-tests/<slug>.md. The LearnTestRecord dataclass (scripts/little_loops/learning_tests.py) has five fields:
| Field | Type | Notes |
|---|---|---|
target |
str |
Free-text human-readable name |
date |
str |
ISO date the record was written |
status |
Literal["proven", "refuted", "stale"] |
proven if any assertion passed; stale is set via mark-stale |
assertions |
list[Assertion] |
Each {claim: str, result: "pass"|"fail"|"untested"} |
raw_output_path |
str \| None |
Pointer to .ll/learning-tests/raw/<slug>.txt |
Slug derivation uses little_loops.issue_parser.slugify() (lowercase, strip non-word chars, collapse whitespace and hyphens), so "Anthropic SDK streaming" becomes anthropic-sdk-streaming.md.
Storage Layout¶
.ll/learning-tests/
├── <slug>.md # one LearnTestRecord per target
├── ...
└── raw/ # raw stdout/stderr captures from proof scripts
├── <slug>.txt
└── ...
The raw/ subdirectory is created on demand by /ll:explore-api — write_record() does not auto-create it. Files in raw/ are the unedited output of the proof script; they are evidence, not summaries.
Spike Plan Docs — Storage Layout¶
The /ll:spike skill (skills/spike/SKILL.md Phase 3) writes its plan doc to a
resolved artifact directory: ${context.run_dir} when running inside an FSM loop
(injected by scripts/little_loops/cli/loop/run.py, propagated to child contexts
by scripts/little_loops/fsm/executor.py), and the standardized .ll/spikes/
directory when invoked interactively (no run_dir exists). The skill body
mkdir -p .ll/spikes/ on demand — nothing pre-creates it. Plan docs are curated
evidence paired with the issue's committed ## Spike Results section, so
.ll/spikes/ is git-tracked via the !/.ll/ un-ignore (no .gitignore entry),
mirroring .ll/learning-tests/ and .ll/decisions.d/ (ENH-2655).
CLI Surface¶
ll-learning-tests (scripts/little_loops/cli/learning_tests.py) is intentionally narrow: it owns reads and stale-marking, but not writes.
| Subcommand | Purpose | Exit codes |
|---|---|---|
check "<target>" |
Print JSON record by target name | 0 if found, 1 if missing |
list |
Print JSON array of all records | always 0 |
mark-stale "<target>" |
Set status: stale on an existing record |
0 |
backfill-versions [--dry-run] |
Stamp proven_package/proven_version onto existing records (ENH-3125) |
0 |
There is no write/add subcommand. Record creation is owned by /ll:explore-api (and any future skill variants) so the prompt context — claims, reasoning, proof script — is captured alongside the result, not just the result alone. Skills emit the on-disk YAML directly via the Write tool to match the format that write_record() produces.
Record creation is still skill-owned; record enrichment is not (ENH-3125). After the skill writes the record, cmd_prove re-reads it and stamps proven_package/proven_version from importlib.metadata via update_frontmatter. That split is deliberate: the version is the one field that must be deterministic — an LLM-typed version could hallucinate a match and silently poison version-drift staleness toward "not stale". The skill template documents the two keys as optional, but nothing depends on the skill emitting them.
For automated bulk staleness detection across all records, use ll-loop run learning-tests-audit — a built-in FSM loop that compares record dates against PyPI/npm registry release timelines and batch-marks stale records. Once records are marked stale, run ll-loop run migrate-sdk-version to re-prove them: it iterates the stale queue, re-runs /ll:explore-api for each target, classifies each result as still-valid, needs-upgrade, or refuted, and produces a triage report. Together these two loops form the two-step registry maintenance workflow. See docs/guides/LOOPS_REFERENCE.md → API Adoption.
LearningTestsConfig Consumers¶
The LearningTestsConfig dataclass (scripts/little_loops/config/features.py) is consumed by three call sites within EPIC-2207's scope:
| Call Site | Issue | Config Field Read |
|---|---|---|
learning_tests_gate.py |
ENH-2208 | stale_after_days |
/ll:refine-issue / /ll:wire-issue skills |
ENH-2209 | learning_tests.enabled |
Sprint pre-flight (fsm/executor.py) |
ENH-2210 | learning_tests.enabled |
A future refactor of this config schema must update all three. See config-schema.json for the full LearningTestsConfig schema definition.
Data Flow Summary¶
flowchart TB
subgraph User["User Input"]
CMD_INPUT["ll-auto / ll-parallel"]
FLAGS["--max-issues, --workers, etc."]
end
subgraph Config["Configuration"]
LOAD["Load .ll/ll-config.json"]
MERGE_CFG["Merge with defaults"]
end
subgraph Discovery["Issue Discovery"]
SCAN["Scan .issues/*/"]
PARSE["Parse markdown files"]
SORT["Sort by priority"]
end
subgraph Processing["Processing"]
VALIDATE["Validate (ready-issue)"]
IMPLEMENT["Implement (manage-issue)"]
VERIFY["Verify (tests pass)"]
end
subgraph Completion["Completion"]
MOVE["Move to completed/"]
COMMIT["Git commit"]
REPORT["Summary report"]
end
CMD_INPUT --> LOAD
FLAGS --> LOAD
LOAD --> MERGE_CFG
MERGE_CFG --> SCAN
SCAN --> PARSE
PARSE --> SORT
SORT --> VALIDATE
VALIDATE --> IMPLEMENT
IMPLEMENT --> VERIFY
VERIFY --> MOVE
MOVE --> COMMIT
COMMIT --> REPORT