# Repository Guidelines

## Project Structure & Module Organization
- Core pipeline lives in `src/`, with stage logic in `src/generator`, `src/verifier`, `src/reward`, `src/question`, retrieval helpers in `src/retrieval`, and the CLI entrypoint at `src/pipeline/pipeline_cli.js`.
- Prompts are in `prompts/`; tweak these before changing stage behaviour.
- Tests sit in `tests/` (Vitest), with sample seeds in `test_samples/`; pipeline outputs write to `gold/`.
- Config baselines (models, limits) are in `configs/pipeline.json`; run scripts live at the repo root (`run.sh`, `try_prompt.sh`).
- Cached intermediates (questions/gens/verifications/rewards) live in `data/cache/*.jsonl`; set `PIPELINE_CACHE_DIR` to redirect.
- Random walk over chunks: set `PIPELINE_RANDOM_WALK=1` (or `PIPELINE_CHUNK_ORDER=random`) to shuffle chunk order using crypto randomness.

## Build, Test, and Development Commands
- `npm install` – install dependencies.
- `npm run pipeline -- --limit 20 --verbose` – run the default pipeline using static seeds.
- `PIPELINE_SEED_MODE=question-first npm run pipeline -- --limit 20 --verbose` – enable question-first seeding.
- Random-walk mode: `PIPELINE_RANDOM_WALK=1 QUESTION_MAX_PER_CHUNK=3 npm run pipeline -- --limit 3 --chunk-limit 10` shuffles chunks, caps questions per chunk at 3, processes at most 3 questions overall, and samples up to 10 chunks.
- `npm test` – run all unit tests (mocked by default).
- `REAL_ES=1 npm test` – exercise retrieval against a live Elasticsearch + embedding endpoint.
- Red/green pathway: use `*_PROVIDER=mock` plus JSONL chunk source to dry-run (green) without models; switch to real providers for red runs and the cache will skip already-completed stages.
- Verifier contract: models return JSON `{"REASONING": [...], "SCORE": <number|\"PASS\"|\"FAIL\">}`; SCORE >=0.5 or PASS → accepted. Prompt must remain unchanged; parsing is tolerant of the PASS/FAIL token format.
- Generator output/logging: verbose runs show parsed `thought`, raw provider `thinking`, answer, confidence, evidence, limitations, and raw response (pretty-printed if JSON). Gold stores `answer`, `thought`, `raw`, `confidence`, `evidence`, `limitations`, `thinking`.

## Coding Style & Naming Conventions
- ECMAScript modules (`type: "module"`); prefer `.mjs` for shared code.
- Two-space indentation, single quotes unless template strings add clarity, and keep functions small and pure where possible (CLI glue stays in `pipeline_cli.js`).
- Use descriptive, lower_snake or camelCase for variables; exported helpers use camelCase.
- Keep prompts and stage logic separate; place reusable utilities in `src/pipeline/util.mjs`.
- Deterministic IDs: chunks are hashed from content+source; questions/gens/rewards are keyed in JSONL caches so reruns can skip already-processed work.

## Testing Guidelines
- Vitest is the test runner; add new tests under `tests/` with `.test.mjs` suffix.
- Mirror stage names in test files (e.g., `generator_core.test.mjs`), and include both happy-path and malformed-input cases.
- For retrieval, default mocks cover most cases; only opt into `REAL_ES=1` when you have a running distill-rag stack.
- Aim to keep tests deterministic—mock providers and network calls unless explicitly validating integrations.

## Commit & Pull Request Guidelines
- Follow the existing history: short, present-tense summaries (e.g., `add generator test`, `maintain chunk ordering`); include scoped prefixes only when they improve clarity.
- Keep commits focused (one concern each) and ensure `npm test` passes before pushing.
- PRs should state the goal, main changes, test evidence, and any required `.env` or config updates; include sample command/output paths when relevant (e.g., `gold/pipeline_gold.jsonl`).
- Link issues when applicable and note any provider/model assumptions or external services needed for validation.

## Configuration & Environment Tips
- Runtime configuration comes from `.env` (ES node, embedding endpoint, provider selections, stage models); avoid committing secrets.
- When changing model or provider choices, update `configs/pipeline.json` if you want a sharable default, and document overrides in your PR description.