How-to· 1 min
Making AI analysis reproducible enough to report
The largest source of variance is not the model. It is that your terms are undefined.
AI Tools Hub Editorial · published · updated
Why it varies
- Generated SQL differs between runs in joins and filters
- Terms like "last month" or "active" get reinterpreted each time
- Null and duplicate handling is unstated
- The model version moved
Fix in this order
| Order | Action | Effect |
|---|---|---|
| 1 | Document definitions and include them in the prompt | Largest |
| 2 | Save and reuse verified SQL | Large |
| 3 | Cross-check results by another route | Moderate |
| 4 | Pin the model version | Small, but helps explainability |
What to document
- Period boundaries and timezone
- Rows excluded: test data, internal accounts, cancellations
- Formulas for the headline metrics
- Join keys and where duplication occurs
Turn SQL into an asset
Once a generated query is verified, name it and save it. Run the saved one next month; use the model only for new questions. Standing numbers stop moving and become explainable.
FAQ
Where do the definitions live?
In the tool's system prompt or context settings if it has them, otherwise pasted at the top of each session. Tedious, and the single highest-impact step.