Configuration

All configuration lives in one .groundtruth.yml at the root of the repository being reviewed. Every setting has a working default, so the file is optional.

The file is safe to commit#

Not as a convention — as an enforced rule. config.py refuses to load a config containing anything key-shaped (an sk- prefix, or a long opaque token) and fails loudly, naming the offending field. A file that structurally cannot hold a secret is one nobody has to remember not to commit.

Keys come from environment variables only: ANTHROPIC_API_KEY, OPENAI_API_KEY and so on, matching whichever provider your model names.

A complete file#

# .groundtruth.yml
model: anthropic/claude-sonnet-5   # or openai/gpt-4o, ollama/qwen2.5-coder, ...
max_cost_per_run: 1.00             # whole-run ceiling, in USD
min_confidence: 0.7                # the bar a finding must clear to post
max_inline_comments: 10            # cap on comments per review
context_token_budget: 25000        # tokens of context assembled per review
max_diff_tokens_per_call: 6000     # a bigger file is reviewed in hunk groups
summary: true                      # one cheap call to group the findings
dimensions: [correctness, security, conventions]

# Optional: a cheaper model for the checking stages. Unset means every
# stage uses `model` above.
# verify_model: anthropic/claude-haiku-4-5-20251001
# summary_model: anthropic/claude-haiku-4-5-20251001

# Optional: sampling settings, per model. See "Sampling settings" below.
# model_params: {temperature: 1, top_p: 1, max_tokens: 16384}
# verify_model_params: {temperature: 0.1, max_tokens: 2048}

# Endpoints are NOT set here -- see "Endpoints" below.

Every setting#

Making it quieter#

If the reviewer is saying too much, in the order worth trying:

• Raise min_confidence to 0.8 or 0.85. This is the strongest lever: confidence is the reviewer's own score multiplied by the skeptic's, so raising the bar cuts anything either model hedged on.
• Lower max_inline_comments to 3 or 5. Findings are ranked before the cut, so you keep the most severe and most confident ones.
• Narrow dimensions to [correctness, security]. Convention findings are the most subjective and the most numerous.

Making it catch more#

If it is too quiet, first find out why — silence has two very different causes and they have opposite fixes. Run with --format text and read the drop line:

No findings survived the quality gate — 4 dropped (3 low confidence, 1 hallucination).

Findings dropped at low confidence means the reviewer proposed them and the gate rejected them: lower min_confidence. Findings that were never proposed at all do not appear in that line: that is a context or prompt problem, and lowering the bar will not help. Troubleshooting walks through both.

Splitting the model tiers#

Three stages call a model, and they do not need the same one. Stage 4 does the reasoning and sets the ceiling on review quality; stages 5 and 6 answer bounded questions on short prompts. A common setup:

model: anthropic/claude-sonnet-5
verify_model: anthropic/claude-haiku-4-5-20251001
summary_model: anthropic/claude-haiku-4-5-20251001

Both tiers default to whatever model is set to, so splitting them is something you opt into rather than something that happens to you on an upgrade.

Verify with a different provider#

The tiers do not have to come from the same company. Review with Claude and verify with GPT, or the other way round — each stage reads its own model string, and LiteLLM picks the provider and the key from its prefix:

# Claude proposes, GPT cross-examines
model: anthropic/claude-sonnet-5
verify_model: openai/gpt-4o-mini

# or the other way round
model: openai/gpt-4o
verify_model: anthropic/claude-haiku-4-5-20251001

Why this is worth doing: the skeptic pass exists to be a second opinion, and a second opinion from the same model is not much of one. Two runs of one model share the same training, the same blind spots and the same habits of mind, and a model asked to judge its own kind of output is inclined to agree with it. A different provider's model was trained differently, so it is less likely to share the specific mistake the first one made. The verifier is never told which model proposed the finding, so the cross-examination is independent in fact, not just in name.

This is reasoning about why independence helps, not a measurement that it does on your code. groundtruth eval --live is how you find out — it runs the recall cases against your real models, and --verify-model lets you swap only the verifier between runs:

# same provider for both
groundtruth eval --live --model anthropic/claude-sonnet-5   --verify-model anthropic/claude-haiku-4-5-20251001

# cross-provider verifier, same cases, same reviewer
groundtruth eval --live --model anthropic/claude-sonnet-5   --verify-model openai/gpt-4o-mini

Compare the catch rate and the false positives. These are real, billed calls, and a handful of cases will not settle the question on their own — but it is a measurement rather than an argument.

Sampling settings#

Each model can have its own temperature, top_p and max_tokens. Most setups never need them — the defaults are a low temperature, so the same diff gets the same review, and 4096 output tokens. The usual reason to change them is a reasoning model: many expect temperature: 1, and they spend part of max_tokens thinking before they answer, so a budget sized for a plain model can run out halfway through the JSON.

model: nvidia_nim/moonshotai/kimi-k3
model_params: {temperature: 1, top_p: 1, max_tokens: 16384}

verify_model: nvidia_nim/z-ai/glm-5.3-flash
verify_model_params: {temperature: 0.1, max_tokens: 2048}

Settings belong to the model they were tuned for. A stage inherits them only when it runs the same model — so if the verifier names a different model, it does not pick up the reviewer's temperature: 1.

• stream is not supported. The review has to read the whole reply to check its JSON, so a streamed reply cannot be used, and in CI nobody is watching the tokens arrive. The file refuses to load if it is set.
• Only these three settings are accepted, within bounds — temperature 0–2, top_p 0–1, max_tokens 1–65,536. This file can be edited by the pull request under review, so an open list would let it pass api_base off as a setting, and an unbounded max_tokens would let it make every call on your key enormous.

Endpoints#

By default each model is called at its provider's own endpoint. To send a stage somewhere else — NVIDIA's API catalog, Azure, a model on your own GPUs, or your org's LiteLLM proxy — set its endpoint in the environment or on the command line:

Flags beat the environment. An endpoint follows the model it serves: when the summary inherits the verifier's model it inherits the verifier's endpoint too, so a model is never sent to another provider's server.

Example: NVIDIA's API catalog

export NVIDIA_NIM_API_KEY=...
export GROUNDTRUTH_BASE_URL=https://integrate.api.nvidia.com/v1

groundtruth review --base main --model nvidia_nim/moonshotai/kimi-k3

Use the model id shown on the model's page at build.nvidia.com, with nvidia_nim/ in front of the whole id. NVIDIA's ids already carry their vendor — openai/gpt-oss-20b, moonshotai/kimi-k3 — and LiteLLM reads the first segment as the provider, so openai/gpt-oss-20b on its own would be sent as an OpenAI call. The right name is nvidia_nim/openai/gpt-oss-20b, and Groundtruth refuses the other spelling before making any call. Note the key too: NVIDIA's endpoint reads NVIDIA_NIM_API_KEY, not OPENAI_API_KEY, even though it speaks the OpenAI protocol.

Example: Claude reviews, a model on NVIDIA verifies

export ANTHROPIC_API_KEY=...
export NVIDIA_NIM_API_KEY=...

groundtruth review --base main   --model anthropic/claude-sonnet-5   --verify-base-url https://integrate.api.nvidia.com/v1

with verify_model: nvidia_nim/<model> in .groundtruth.yml. The review goes to Anthropic, the verification to NVIDIA, and the summary follows the verifier.

Flags that override the file#