Deploy via archive (prod has no .git); one-time compose down/up for pinned subnet 172.28.0.0/16. Verified: worker secrets promotion, v11 CV E2E (enhanced:true, 0.92), admin sessions IP tracking. Gotcha added: hairpin NAT — on-box verification must use 127.0.0.1 ports. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .agent_states | ||
| .claude | ||
| .forgejo/workflows | ||
| .github/workflows | ||
| .opencode/agents | ||
| backend | ||
| docker | ||
| docs | ||
| frontend | ||
| scripts | ||
| .agents.local.md | ||
| .dockerignore | ||
| .env.example | ||
| .env.production | ||
| .gitignore | ||
| .pre-commit-config.yaml | ||
| .project_rules.md | ||
| .wip-manifest.json | ||
| AGENTS.md | ||
| check_db.py | ||
| CLAUDE.md | ||
| DEPLOYMENT.md | ||
| future_improvements.md | ||
| GIT_WORKFLOW.md | ||
| gotchas.md | ||
| package-lock.json | ||
| platforms.md | ||
| RCA_TASK_STALL.md | ||
| README.md | ||
| requirements.md | ||
| run_local.sh | ||
| SESSION_STATE.md | ||
| start_backend.sh | ||
Job Hunter Platform
Multi-user job-hunting automation platform. Deployed on Raspberry Pi 5 (aarch64) behind Caddy with automatic TLS.
Architecture Overview
Secrets & Configuration — Two Sources
The system uses two separate mechanisms for configuration, intentionally:
| Source | Mechanism | What goes there | Priority |
|---|---|---|---|
docker/secrets/*.txt |
Docker secrets — mounted into containers at /run/secrets/ |
API keys, HMAC key — anything secret | Highest (runtime) |
docker-compose.yml environment: section |
Plain env vars — visible in docker inspect |
LLM_PROVIDERS, NODE_ENV, URLs, port config — non-secret config |
Fallback |
Why two sources?
- Docker secrets are never exposed in
docker inspect, logs, or compose dumps. They're securely mounted as tmpfs files. - Regular env vars are fine for non-sensitive config like
LLM_PROVIDERS=minimax,openrouterorNODE_ENV=production. .env.productionat the project root is a template/documentation only — the system never reads it directly at runtime. It exists so you know what to set.
At startup, the backend scans /run/secrets/ and promotes every file into an environment variable (backend/src/api/main.py:_validate_env). So a file named minimax_api_key becomes MINIMAX_API_KEY in the application. This means all code reads from os.environ uniformly, regardless of whether the value came from a Docker secret or an env var.
HMAC_SECRET_KEY — What It Does
The root signing key for the entire platform (backend/src/api/auth.py, backend/src/api/admin/auth.py):
| Use | Where | What it signs |
|---|---|---|
| Session code signing | backend/src/api/auth.py:104 |
HMAC-SHA256 of each user's session code. Prevents session forgery. |
| Admin JWT secret | backend/src/api/admin/auth.py:21 |
Signs all admin JWT tokens (if ADMIN_JWT_SECRET not set separately). |
If you lose HMAC_SECRET_KEY, all user sessions and admin tokens become invalid.
Admin JWT — How It Works
Generated when an admin logs in via POST /api/v1/admin/auth/login:
def create_admin_token(admin_id, username, role):
# Returns a JWT with: sub, username, role, iat, exp
# Signed with HS256 (HMAC-SHA256) using ADMIN_JWT_SECRET
Stored client-side only — the admin browser keeps it in localStorage.
Sent on every admin API request as:
Authorization: Bearer <token>
Validated by the require_admin FastAPI dependency on every protected route.
Expiration: 24 hours by default, configurable via ADMIN_JWT_EXPIRY_HOURS env var.
Multiple admins: Yes — create additional accounts via POST /api/v1/admin/accounts (requires existing admin auth) or direct DB insert into the admin_accounts table. Each admin gets their own token on login.
LLM Provider Chain
Set via LLM_PROVIDERS=minimax,openrouter,ollama (comma-separated, in priority order).
If the first provider fails (timeout, 5xx, network error), the chain auto-falls to the next.
Each provider reads {PREFIX}_API_KEY, {PREFIX}_BASE_URL, {PREFIX}_MODEL:
| Provider in chain | Env prefix | Default model |
|---|---|---|
minimax |
MINIMAX_* |
MiniMax-Text-01 |
openai |
OPENAI_* |
gpt-4o-mini |
openrouter |
OPENROUTER_* |
anthropic/claude-3-haiku |
ollama |
OLLAMA_* |
llama3.2 |
llamacpp |
LLAMACPP_* |
default |
vllm |
VLLM_* |
default |
Example — use MiniMax as primary, fall back to OpenRouter's GPT-4o:
LLM_PROVIDERS=minimax,openrouter
MINIMAX_MODEL=MiniMax-Text-01
OPENROUTER_MODEL=openai/gpt-4o
Edit these in docker-compose.yml environment: section or in .env.production (for reference).
Prerequisites (Target Machine)
- Docker Engine 20.10+ & Docker Compose v2
- Caddy (or another reverse proxy for TLS) — only if using the standard architecture
gitto clone the repo- API keys for each provider in your chain
Quick Start — First Deployment
1. Clone & Prepare Secrets
git clone ssh://git@<your-git-server>:2222/namcho/job-hunter-public.git
cd job-hunter-public
mkdir -p docker/secrets
Create secret files (one per file, no trailing newline):
python3 -c "import secrets; print(secrets.token_hex(32))" > docker/secrets/hmac_secret_key.txt
python3 -c "import secrets; print(secrets.token_urlsafe(32))" > docker/secrets/cleanup_api_key.txt
echo -n "your-minimax-api-key" > docker/secrets/minimax_api_key.txt
echo -n "your-openrouter-api-key" > docker/secrets/openrouter_api_key.txt
2. Configure Provider Models (if not using defaults)
Edit docker-compose.yml and set models under the backend environment: section:
- OPENROUTER_MODEL=openai/gpt-4o
- MINIMAX_MODEL=MiniMax-Text-01
3. Build & Deploy
LLM_PROVIDERS=minimax,openrouter docker compose -f docker/docker-compose.yml build
LLM_PROVIDERS=minimax,openrouter docker compose -f docker/docker-compose.yml up -d
4. Configure Reverse Proxy (if using Caddy)
Add to /etc/caddy/Caddyfile:
jh.yourdomain.com {
reverse_proxy localhost:8307
}
sudo caddy reload --config /etc/caddy/Caddyfile
5. Bootstrap Admin Account
docker compose -f docker/docker-compose.yml exec backend python -c "
from src.db_utils import get_db_context
from src.api.admin.auth import hash_admin_password, generate_default_password
import asyncio, uuid
async def create_admin():
async with get_db_context() as db:
pwd = generate_default_password()
hashed = hash_admin_password(pwd)
await db.execute(
'INSERT INTO admin_accounts (id, username, password_hash, role, created_at, is_active) VALUES (?, ?, ?, ?, datetime(\"now\"), 1)',
(str(uuid.uuid4()), 'admin', hashed, 'superadmin')
)
await db.commit()
print(f'Admin user: admin')
print(f'Password: {pwd}')
asyncio.run(create_admin())
"
Save the generated password — you won't see it again.
6. Verify
| Check | URL / Method |
|---|---|
| API health | GET https://jh.yourdomain.com/health |
| Prometheus metrics | GET https://jh.yourdomain.com/metrics |
| Admin login | https://jh.yourdomain.com/admin/login |
| Health dashboard | https://jh.yourdomain.com/admin/health |
| Rate limit header | Check X-RateLimit-Remaining on any API response |
Migration to Another Server
Step 1 — Database
The SQLite database lives in a Docker volume named sqlite_data. Find it on the source server:
docker volume inspect job-hunter_sqlite_data # note the Mountpoint path
# Usually: /var/lib/docker/volumes/job-hunter_sqlite_data/_data/job_hunter.db
Copy the .db file to the target server:
scp user@old-server:/path/to/job_hunter.db ./job_hunter.db
On the target, before starting the stack, restore:
docker volume create job-hunter_sqlite_data
# Stop stack, copy db into volume, restart
Alternative (if backup container was running): check docker/backups/ for timestamped .db files:
ls docker/backups/
cp docker/backups/job_hunter_20250101_120000.db ./job_hunter.db
Step 2 — Secrets
Copy the docker/secrets/ directory exactly as-is:
rsync -av docker/secrets/ user@new-server:~/job-hunter-public/docker/secrets/
If HMAC_SECRET_KEY changes, all existing sessions and admin tokens are invalidated.
Step 3 — Images
Either rebuild on the target:
cd ~/job-hunter-public
docker compose -f docker/docker-compose.yml build
Or save/load the images:
# Source
docker save docker-backend docker-frontend docker-worker | gzip > images.tar.gz
scp images.tar.gz user@new-server:~/
# Target
docker load < images.tar.gz
Step 4 — Full checklist after migration
- Secrets match source (especially
HMAC_SECRET_KEY) - Database file copied and in place
- Images built or loaded
- Caddy config updated for the new host
LLM_PROVIDERSmatches original- Provider API keys are valid
- Admin can log in with existing credentials
/healthreturns{"status": "ok"}/metricsreturns valid Prometheus output
From-Scratch Setup (New Hardware)
1. OS + Docker
# Debian/Ubuntu / Raspberry Pi OS
sudo apt update && sudo apt install -y docker.io docker-compose-v2 git
sudo usermod -aG docker $USER
newgrp docker
2. Reverse Proxy (Caddy)
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install caddy
3. Clone + Secrets + Deploy
Follow Quick Start steps 1–6 above.
4. Optional: Local LLM Fallback (Ollama)
docker run -d --restart unless-stopped --name ollama -p 11434:11434 ollama/ollama:latest
docker exec ollama ollama pull llama3.2 # or whatever model you want
The provider chain will automatically detect Ollama at host.docker.internal:11434/v1.
File Reference
| File | Purpose |
|---|---|
docker/docker-compose.yml |
Service definitions, networks, volumes |
docker/Dockerfile.backend |
FastAPI backend image |
docker/Dockerfile.frontend |
React SPA + nginx image |
docker/Dockerfile.worker |
Celery worker image |
docker/nginx-frontend.conf |
Frontend container nginx config (proxies /api/ to backend) |
docker/secrets/*.txt |
Docker secrets (API keys, HMAC key) |
.env.production |
Template / documentation for all env vars and providers |
backend/src/services/llm_service.py |
LLM provider chain logic |
backend/src/api/admin/auth.py |
Admin JWT + password auth |
backend/src/api/auth.py |
User session codes + HMAC signing |
backend/src/api/main.py |
App startup, env validation, Docker secrets loader |