Deep Memory Assistant (DMA)

Three-tier memory architecture + tool-using 1.5B agent. The assistant speaks in a clean generic register, not a cloned persona.

Direction (2026-09-03): DMA is growing from a tool-using assistant into an OS-level agent for Linux — a local, fine-tuned "my Linux Aion" (cf. Microsoft's leaked Project Aion prototype: an agent-first OS shell where the assistant coordinates apps, files, and everyday tasks). On Linux that means launching apps, finding files across the whole system or a folder, disk and environment lookups, and plain answers — all grounded in the real filesystem, with every action confirmed by its real tool result. Data passes now add more prose, app-launching, and everyday file/system tasks; the user writes the prompts, the mechanical row-building is scripted (reground_rows.py, gen_*.py).

~/brain/
  references.txt        # Tier 2: index of `filepath | one-line description`
  memories/YYYY-MM-DD.txt  # Tier 3: chronological daily logs (greppable)
/tmp/working_memory.txt # Tier 1: short-term memory, at most a paragraph,
                        #         read+rewritten each cycle (task list, crossed off)

Tool schema

The agent can emit 7 tool payloads: read_file, write_file, append_file, grep_memories (pattern, file_pattern), execute_bash, web_search, and read_url. web_search (DuckDuckGo HTML, no API key) and read_url were added after the original spec; the file tools are sandboxed to $HOME + /tmp, and execute_bash is off unless you pass --allow-bash.

Files

File	Purpose
agent_loop.py	Tool-executing runtime. Parses the model's JSON payloads and dispatches the 7 tools. Feeds results back into the chat as role:"tool". Every failure path returns a graceful <error>/<cancelled> string (sandbox refusals, disabled bash, offline web, missing files) — the REPL never crashes.
cli.py	Polished terminal front-end for the agent (tool cards, colors, /mem, /steps).
collect_data.py	Interactive dataset collector. Chats an Ollama model, shows N candidates, lets you rewrite the assistant text, insert JSON tool calls with a keypress, and appends HF chat-template JSONL to dataset.jsonl. /chain mode captures real multi-step tool sequences.
collect_candidates.py	Batch "model proposes" half of reject sampling: runs prompt.txt prompts through the base model into raw candidates.jsonl (write/bash blocked) for curation.
eval_agent.py	Field-test harness. Runs held-out prompts through the loop and scores tool choice / valid JSON / sandbox paths / prose finish. Run before and after training.
validate_dataset.py	Linter for dataset.jsonl — flags malformed rows, fake paths, missing tool args, broken chains. System paths are allowed when the row demonstrates the sandbox refusing them.
gen_gap_rows.py	Idempotent generator for the gap-bank rows (prose, grep_memories, read_url, write_file, honest failures). Rows 1–18 follow your phrasing in gap_bank_ans.txt; 19–48 come from gap_bank.txt. Re-runnable.
gen_error_rows.py	Idempotent generator for the error-coverage rows: every runtime failure format (sandbox refusals, no ~/brain, bash disabled/cancelled, zero web results, no readable URL text). Re-runnable.
view_dataset.py	Scroll through dataset.jsonl rows rendered exactly like a cli.py session (analysis / tool card / result / answer). --list index, --quiet dump, jump by row number, s files improvement suggestions to dataset_suggestions.txt (auto-cleared when the row is improved).
apply_suggestions.py	Applies the filed dataset_suggestions.txt fixes plus the same look-then-act errors found dataset-wide (read before append, honest launch failures). --dry-run to preview.
gen_multi.py	Writes the 14 multi/01–14.txt conversation scripts and converts them into multi-turn dataset rows (the first multi-turn data). Re-runnable.
gen_skill_rows.py	13 single-turn skill rows: error-recovery chains, code-writing, task-cycles, tool-choice. --dry-run to preview.
gen_next_rows.py	Current generator: converts the approved next_bank.txt into 40 real-grounded rows (prose, read_url, code-writing, memory, multi-turn, honest failures) and patches drift-affected rows. Idempotent; materializes every /tmp path it references.
verify_rows.py	Replay harness: re-runs each row's read-only tool calls against the live machine and diffs vs the embedded results. --from N, --contains STR.
probe_weak_spots.py	Drives the base model through weak-skill probes with the real tool loop → probe_data.jsonl (raw) + probe_report.txt (readable).
train.py	Fine-tuning rig. QLoRA (bitsandbytes 4-bit) on NVIDIA GPUs, plain-LoRA fallback on CPU. Auto-detects device.
requirements.txt	Laptop deps (light: just requests — Ollama serves the model locally).
requirements-gpu.txt	Training deps for the GPU box / Colab T4 (pip install -r requirements-gpu.txt; torch ships with Colab).
DEPLOY.md	LoRA adapter → merge → GGUF → ollama create dma → run guide.
context.md	Full project-context snapshot (read this first if you lose context).
archive/idea.txt	Original project spec — superseded (persona cloning, 5-tool schema, laptop training all outdated). Kept for history.

Prompt-bank / data files: question_bank.md (categorized prompt bank), gap_bank.txt + gap_bank_ans.txt (gap-targeting prompts + your paraphrases), multi.txt + multi/01..14.txt (multi-turn conversation scripts, now converted to rows), prompt.txt (feed for collect_candidates.py), dataset.jsonl (the training data), candidates.jsonl (raw candidate transcripts), example_rows.jsonl (reference row shapes), dataset_suggestions.txt (open improvement suggestions filed from view_dataset.py; entries disappear once the row content changes), probe_data.jsonl + probe_report.txt (31 base-model weak-skill probes), probe_followup.jsonl (11-probe pre-training baseline on the new patterns).

1. Collect the dataset

pip install -r requirements.txt
ollama pull qwen2.5:1.5b-instruct
python collect_data.py                       # or: python collect_data.py llama3.2:1b

• Pick a candidate [1..n], hit r to regenerate, s to skip.
• At the save prompt: t opens an interactive tool-call builder (pick a tool, fill its args, it emits valid JSON), e rewrites it in a $EDITOR, n keeps it.
• /stats shows rows so far, /reset clears history.
• /chain toggles chain mode: the collector actually runs the model's tool calls on your host, feeds real results back, and saves the whole multi-step sequence as one row — teaching the model to react to what the filesystem returns. Ideal for "look-then-write-then-report" tasks.
• Assistant output is written in a clean, generic agent voice (no persona/clone).
• Each line saved to dataset.jsonl. Single tool / voice turns: {"messages": [{"role":"user",...},{"role":"assistant",...}]} Chain turns interleave the tool role: {"messages": [{"role":"user",...},{"role":"assistant",...}, {"role":"tool",...},{"role":"assistant",...},{"role":"tool",...},{"role":"assistant",...}]}

Target: ~1000 curated turns mixing plain answers, single tool calls, and multi-step chains (e.g. write a task list to /tmp/working_memory.txt, execute, then rewrite it with the finished item crossed off). Currently at 283 rows (27 prose / 256 tool chains), 0 errors / 0 warnings — grounded on the real filesystem (09-03 regrounding, 09-10 next-bank pass + drift patches; see gen_next_rows.py and context.md §6).

Workflow split: you write the prompts (in your own voice), the assistant side (think-aloud → tool JSON → result → report) is the mechanical part.

2026-09-03: the old gen_*.py row generators are superseded (their rows were corrected in place by reground_rows.py; re-running them re-adds fiction). New rows come from the current prompt banks only.

2. Run the agent loop (test your tool schema before training)

python agent_loop.py
python agent_loop.py --allow-bash --model llama3.2:1b

• The loop watches /tmp/working_memory.txt and truncates it when it exceeds ~4 KB.
• File tools are sandboxed to your home directory + /tmp; system paths (/etc, /usr, /var, ...) are refused.
• execute_bash is off by default; enable with --allow-bash (prompts each call).

Lint your growing dataset any time:

python validate_dataset.py            # checks dataset.jsonl

Flags broken JSON, fake/placeholder paths, missing tool args, tool calls without results, rows ending mid-chain, and dangerous bash commands.

3. Train

Hardware reality: this laptop (Ryzen 5500U, AMD iGPU, 5.6 GB RAM) has no CUDA — QLoRA needs an NVIDIA GPU. Either: a) run train.py on a free/cheap cloud GPU (Kaggle / Colab, GPU runtime), or b) run locally on CPU — plain LoRA will work but be very slow.

# On a GPU box (e.g. Colab T4) — install deps first:
pip install -r requirements-gpu.txt
python train.py --data dataset.jsonl --model Qwen/Qwen2.5-1.5B-Instruct --epochs 3

# Local CPU (slow):
python train.py --data dataset.jsonl --device cpu --epochs 1 --batch 2

A 1.5B QLoRA run on a Colab T4 finishes in minutes. Save the ./dma-<model>/ adapter before the session ends (see DEPLOY.md).

Outputs a trained LoRA adapter under ./dma-<model-short>/.

283 rows is enough for a real first pass — a few minutes on a T4. Expect behavior shifts on the well-covered skills (grep/read/bash chains, memory discipline, honest failures); multi-turn (20 rows), read_url (17) and code-writing (11) will still underfit — train, measure, then add data and retrain.

Measure with the baseline before you train: probe_followup.jsonl — 11 probes of exactly the new patterns against the base model (still fails 9/11). Re-run the same probes after training to see the delta.

Tips

• Keep dataset.jsonl rows consistent in voice — one clean, generic agent register. Mixed personas confuse the fine-tune.
• Use the tool-payload rows to teach the memory-tier discipline: working-memory task lists with [ ]/[x], references as path | description lines, and timestamped, greppable daily logs.
• Prefer multi-step chain rows for anything involving "look first, then act" (check for an existing file, then write/append, then report).