livewiki

A live, growing personal wiki. One memory for humans and agents.

Stop telling your agents "You are a great assistant." Start telling them "You are me."

What it is

• Files first. Your vault is plain markdown + frontmatter — Obsidian-compatible. Every index is derived and rebuildable. Delete .livewiki/ anytime; nothing is lost.

• Hybrid recall. bm25 keywords + local embeddings + knowledge-graph expansion. No LLM in the query path: reads are fast and free.

• Protected terms. A page can be private; so can a string. Put names in .livewiki/protected-terms.txt, set [privacy] protected_terms = true, and check and reindex tell you when one of them turns up in a page a reader can see — the source's real name pasted into a public draft. The list lives outside the vault proper on purpose: it is never indexed, never served, and never pushed, because it is a written record of everything you are protecting. Nothing that crosses a network ever names a term.

• Answers "what did I learn recently". Hits and pages carry created and modified — and created_from/modified_from, which say whether a date was written by the tool or is just the file's mtime. A restore or a git checkout moves an mtime without the note changing, so the difference is published rather than hidden. since and order=recent filter and sort on it.

• Refuses a parameter it does not know. ?tag=episode on an endpoint with no tag filter is a 422 naming what it does accept, not a 200 that answers as though the filter had been honoured.

• Shows you what a reader sees, from the terminal. livewiki recall marks private hits [private] so you know which rows they are before you share a screen; --as-reader excludes them exactly as the server does. livewiki check --list-private names the withheld pages instead of counting them, and keeps "you marked this" separate from "livewiki could not read your setting, so it withheld the page to be safe".

• Agents can retire a note; nothing can delete one. Folding ten episodes into a summary used to leave the ten behind forever — there was no delete, and emptying a page left a titled stub the graph still listed. POST /api/page/{id}/retire writes status: retired: the page leaves recall, the graph and the suggestion queue, the file stays where it is, and reviving it means deleting one line. There is no HTTP delete and there will not be one — unlinking a file on your disk is not something the lowest-trust surface gets to do.

• Two agents can share a page without one silently losing. A read carries an ETag; send it back as If-Match and a write that would clobber someone else's edit is refused with 412 instead of quietly winning. Opt-in — omit the header and nothing changes.

• A link can mean something. - contradicts: [[Beta]] in a note, or POST /api/link {"kind": "contradicts"}, and the graph gives the word back as relation — while kind stays wikilink, so you can still tell a link someone wrote from one livewiki inferred.

• Hand someone a copy of only what they may see. livewiki export ./bundle writes exactly the reader view — asserted against the real served API, not a second guess at it. Pages you marked private are omitted and counted; a page whose privacy value livewiki cannot read STOPS the export, because its author never chose to withhold it.

• Let your own agent read named private notes — deliberately. [mcp] grant = ["sources/*"] opens exactly that scope to an MCP client, and livewiki check says so with a live count. It is not called "local-only": livewiki is the MCP server and cannot see what model your client talks to, so it will not claim a check it cannot perform. There is no "everything" switch — writing the scope is the point.

• Agent-native. An MCP server exposes your memory to any agent runtime (Claude Code, OpenCode, Goose, …).

• Provider-agnostic. OCR/enrichment models are configurable: OpenAI, Anthropic, claude CLI, Ollama, vLLM, OpenRouter — your choice, per role.

• Privacy-tainted. privacy: local pages never leave your machine. Enforced structurally at the model seam and at every network read, not by discipline. local means this page never leaves this machine; on this machine it is yours — a reader on the machine itself needs no password. A tester read "private" as "nobody sees this without the password", opened her own vault on her own computer, saw her private note, and reported a leak. There was none — but the product had never said which of the two promises it makes.

Quick start

pip install -e .
livewiki init ~/vault
cd ~/vault && livewiki reindex --semantic
livewiki recall "that embedded clock bug"
livewiki check          # what is risky right now: where private notes live, who can read them
livewiki watch          # keep index hot on every save
livewiki enrich         # LLM entity edges + named category clusters ([models].enrich)
livewiki sync --push    # git the vault: commit, pull, reindex, publish

Multi-user is git. Put the vault in a repo, share a remote, and livewiki sync commits your edits, merges theirs, and reindexes only what changed. A merge conflict is left in the file for you — the indexer skips a page holding conflict markers and keeps the last good version, and nothing is pushed until it is resolved. Resolving is just editing the file, then syncing again.

Many machines, one vault. A vault is a directory and the index is derived from it, so there is no server to point at: every device runs its own livewiki over its own clone, with its own index, its own UI and its own MCP server, and they converge through livewiki sync. That is the answer to "do the vault and the app have to be on the same machine" — no, but the writer needs the vault on a filesystem it can write to. That is replication, not remote access.

Three things about that are worth knowing before you rely on it:

• .livewiki/ and .activity/ must not be committed, and the ignore line now arrives with the directory rather than with init. livewiki writes it wherever it creates one of those directories inside a git repo — so a folder you turned into a repo yourself, and never ran livewiki init on, is covered too. That was the gap: it is the vault shape where one git push put .livewiki/protected-terms.txt on a remote. Outside a git repo nothing is written, and a vault already covered by a parent repo's .gitignore gets no second file. What a line cannot do is untrack. If git already tracks either directory — a repo that predates the line, or an add -f — the ignore line changes nothing and the files keep being pushed, so sync and watch say so and name the git rm -r --cached that fixes it. For the protected-terms list specifically, sync --push refuses, with no override.
• The index is per-machine, so search quality can differ between your own devices. A laptop without sentence-transformers silently falls back to the bag-of-words hash while your desktop searches semantically. livewiki names the active embedder in reindex, recall and stats rather than letting you assume they match.
• Activity history is per-device and does not converge. The log under .activity/ never leaves the machine that wrote it, deliberately: a row says a page changed and when, which is a fact about that page, and once it is on a remote it cannot be taken back if the page later becomes private. So each device knows what happened on it, and there is no combined view.

Works offline with zero API keys out of the box — but read this before judging the search. Without sentence-transformers installed, embed = "auto" falls back to a bag-of-words hash. It is deterministic, offline and instant, and it is not semantic: it matches shared words, not shared meaning, so "pull-up resistor" will not find a page that says "pullup". livewiki says so now — reindex, recall and stats all name the active embedder, and the embedding leg is switched OFF rather than fused as noise, so a search that finds nothing tells you why instead of guessing. The fix is one command:

pip install "livewiki[embed-local]"        # local sentence-transformers, still offline
livewiki reindex ~/vault --force --semantic

embed = "auto" upgrades itself: once the extra is installed, the next reindex uses the real model with no config change. Measured on the ARM demo host (4 GB RAM): the install pulls ~6.2 GB and takes ~2.5 minutes, and a first embed peaks around 826 MB resident. If pip fails with "No space left on device" while df shows plenty free, it is unpacking into a tmpfs /tmp — set TMPDIR to a directory on disk.

--force is not optional here. Vectors are stored with the model that built them, and retrieval only compares against the active model's — so after swapping embedders a plain reindex leaves every existing vector unusable and the vector half of search silently returns nothing. livewiki stats will tell you: it reports usable vectors for the active embedder and marks them STALE when they belong to someone else.

(From a source checkout, pip install -e ".[embed-local]".)

Driving it from an agent

livewiki ui serves a JSON API alongside the graph. Two things are worth knowing before you write a client, because both have caught people out:

The ?token= in the printed URL is not a credential. It is how the token reaches the browser, which stores it and thereafter sends it as a header. The server reads headers only — a token in a query string leaks into proxy logs, browser history and Referer, so it is never accepted as one. Send it yourself:

curl -X POST http://127.0.0.1:8137/api/page \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"id": "notes/idea", "content": "A fresh thought."}'

Both headers are needed: without Content-Type: application/json, curl -d sends a form encoding and the body fails to parse. X-Livewiki-Token: $TOKEN works in place of the bearer header.

Whether reads need a token depends on the vault. If any page is marked privacy: local — or if any file has not been indexed yet, so the answer is unknown — livewiki ui implies --private and every read needs the token too. Pass --open to serve reads without one. The startup banner states which case you are in, so a client that starts getting 401s can be diagnosed in one read.

A page's display title comes from its frontmatter, not from a title field on the request — pass {"frontmatter": {"title": "A Fresh Thought"}}. With no frontmatter title the id's last segment is used.

The contract is at GET /openapi.json — every route, every request shape, every required field. Read it or generate a client from it rather than guessing field names. Write bodies are strict: an unknown or misspelled key is a 422 naming the key, never a silent partial write. So {"id": "x", "body": "..."} fails loudly instead of creating an empty page, and content must be present — "" is a legitimate empty page, but leaving the key out is a caller bug.

(/docs is deliberately off. FastAPI's Swagger page pulls its assets from a public CDN, which renders blank offline and puts a third-party origin in front of a local-first tool.)

For a stricter, schema-checked surface with the same guarantees, livewiki mcp speaks MCP over stdio.

Docs

• DESIGN.md — architecture and locked decisions
• SPEC.md — the vault format specification
• docs/integration.md — how another app reads from, or writes into, the shared vault running on mikulas

Development

python3 -m venv .venv && .venv/bin/pip install -e ".[ui,mcp,dev]"
.venv/bin/pytest

[dev] alone is not enough to run the suite, despite its name. It carries pytest, ruff, mypy and hypothesis — not the optional extras the tests exercise. Measured on a clean [dev]-only virtualenv, pytest did not run at all: it failed during COLLECTION on a missing fastapi and reported "1 skipped, 1 error". With [ui,mcp,dev]: 608 passed, 4 skipped. The four are honest skips for things a checkout cannot assume — three need feedparser (the [ingest] extra) and one needs vtracer. The browser smoke is a separate gate that pytest never collects at all; see tests/e2e/README.md.

Code-structure questions: use graphify (graphify . → queryable code graph). Dev tool only — never a livewiki dependency. SESSION_STATE.md stays brief because of it. Agents picking this up: read SESSION_STATE.md, then NEXT_SESSION.md for the single next move.

Status

v0.1 shipped: M1 core ✅ · M2 MCP server ✅ · M3 graph UI ✅ · M4 ingestion ✅. v0.2 shipped: enrich ✅ · persona ✅ · community summaries ✅ · staleness ✅ · contradictions ✅ · daily digest ✅. v0.3 shipped: UI writes ✅ · git-first sync ✅ · UI auth ✅. v0.4 shipped: UI --private read-gate ✅ · conflicts surfaced in reports ✅. v0.5 shipped: RSS slug-collision fix ✅ · contradiction semantic net ✅. v0.6 shipped: organic living graph (Phase 1) ✅ · auto-link tier (structural + opt-in LLM) ✅ · UI concurrency-race fix ✅. v0.7 shipped: organic graph interaction — group-drag, hover-focus, zoom-to- cluster (Phase 2) ✅ · the graph's first shader — breathing cells, hub nucleus (Phase 3 slice 1) ✅ · the living-tissue palette, WCAG AA gated (Phase 3 slice 2) ✅. v0.8 shipped — the first release driven by user testing, not the roadmap: a page can be found by its own name ✅ · search says which embedder it is running instead of passing a bag-of-words hash off as semantic ✅ · the HTTP write path rejects a malformed body instead of silently saving an empty page, and /openapi.json works ✅. v0.9 shipped: search stopped fusing a leg that is noise — with no semantic model installed the embedding leg is switched off rather than weighted equally with keywords ✅ · a search that finds nothing says why, in the terminal, the browser, the HTTP API and MCP ✅ · concurrent agent writes to a new page no longer fail as "human-owned", and pages land atomically ✅. v0.10 shipped — three of its four changes came from testers using the tool: every chunk is bounded, so a note with no blank lines is no longer one half-megabyte chunk with a 200-character snippet standing in for it ✅ · a page that mentions a topic in passing no longer outranks the page about it ✅ · results say why they are there — which of your words matched, and where — in plain English rather than algorithm names ✅ · a search can no longer be answered by the word "while": grammar words are no longer scored as subjects, and the share of searches for absent topics that correctly return nothing rose from 22% to 41% ✅ — on the default install; with an embedding model the meaning-based leg still answers, so that share differs. v0.10.1 shipped: [[wikilinks]] written in a different form from their target's name now resolve, so links stop becoming dangling ghosts and link expansion actually fires ✅ · a result reached at two hops says how far, instead of naming a page it is not joined to ✅ · stats reports how many links point at nothing ✅. v0.11 shipped: a search whose results contain none of your words now says so, instead of handing you confident-looking pages with nothing marking them ✅ · the strongest result is no longer the least explained — each matched word is named under the place it was actually found ✅ · /api/recall always reports which of the four result states you got, so an agent can tell "could not look" from "found none" ✅ · the "no embedding model" note stopped printing after every single search ✅. v0.12 shipped: a search names the words it could not find, instead of letting one ordinary word carry a whole result list ✅ · the score column is gone — it restated list position and read as a confidence rating, which livewiki deliberately does not have ✅ · a [[wikilink]] finds a page by its own heading ✅ · two missing pages with non-Latin titles stop counting as one ✅ · the "no embedding model" note stopped blaming itself for word-form misses ✅. v0.12.1 fixed: asking for a negative number of results quietly degraded the rest, and asking for zero returned nothing — the number bounds how much of your vault is examined, not just how many results come back ✅. v0.13 shipped: results reached only by following links now sit in their own section below the ones that actually matched, instead of padding the list a reader had to sort by hand ✅ · a search made only of very common words says so, instead of reporting the same "nothing matched" as a genuinely absent topic ✅ · /api/recall/detailed returns the hits AND everything the terminal prints in one typed body, so a long or non-Latin query stops losing its explanation to a header size limit ✅ · /api/recall and /api/page publish real schemas, so a generated client can see every field it will receive — including privacy ✅ · a rejected token is now distinguishable from a missing one without reading English ✅ · AI agents no longer receive pages you marked privacy: local ✅. A SEARCH tells them how many results were withheld; a direct read of one answers exactly as it would for a page that does not exist. That asymmetry is deliberate: a read that announced "withheld" would confirm the page exists to anyone who guessed its name, so the counts carry that duty and the direct read stays silent. v0.14 shipped (an index built by an older version re-chunks itself on the next run; a vault you have never indexed still needs livewiki reindex): code inside a fence is no longer read as headings or wikilinks, so a note documenting a shell script stops growing phantom pages in your graph ✅ · a page with many section headings is findable by all of them, not just the first few ✅ · the evidence line tells a page's title apart from a heading inside it ✅ · a broken link says whether two pages claim the same heading or nothing claims it at all ✅ · a result found by meaning says so instead of saying nothing ✅. v0.15 shipped: the "reached by following links" section now appears on installs with an embedding model, where it had never once fired — measured on the live demo, ten queries went from 0 context every time to a real split ✅. Known open: on installs with a real embedding model a search that finds nothing still returns results rather than none — it now tells you they contain none of your words, but it cannot tell a nonsense query from a good one phrased in your own words, and no statistic was found that can; a # comment inside a code fence is still read as a heading; and two livewiki processes on one vault can still lose an appended note. See the CHANGELOG.

Full history: CHANGELOG.md.