--- name: statistics-basics-checks description: 15 rules from the Noesa course "Statistics, understood". For students, professionals, founders, PMs, and AI-assisted builders who see charts, reports, experiments, or forecasts and need to judge whether the conclusions follow. Everyday arithmetic assumed; no statistics or code. --- # Statistics, understood — the rules Use with: Claude Code or Claude (save as a skill), Cursor (save under .cursor/rules as .mdc), ChatGPT or any other assistant (paste the text below into custom instructions or a project's instructions). 15 rules, taken from the course at https://noesa.leafsoft.online/c/statistics-basics Each heading is one thing the course teaches. Most are checks to run on your own output before presenting it as done; a few are background you are expected to have. 9 also name a mistake models make by default, under "Watch for". Apply these to the thing you are producing — the type, the schema, the query, the copy — not only to how you explain it. Where a rule names a field, a format or an identifier, that name belongs in the output. ## Read the claim before the number Separate a numerical claim into its measure, comparison, population, period, and conclusion. ## Choose the right average Choose between mean and median when an extreme value distorts what “typical” means. **Watch for:** If AI is asked for “the average wait,” why might it return a correct but unhelpful number? “Average” is ambiguous and the decision context is absent. AI may default to the common mean calculation, so you must specify whether you need total impact, typical experience, or visibility into extremes. ## Notice the denominator Convert a count into a rate and identify the denominator needed for a fair comparison. **Watch for:** Why might AI turn “50 bookings versus 40” into an upbeat conclusion? The counts are present while the opportunity totals may be omitted or buried. AI can continue the obvious growth pattern, so you must supply or demand the denominator before accepting the narrative. ## Separate percent from percentage points Calculate and explain the difference between relative percent change and percentage-point change. ## See the spread, not only center Compare two distributions that share a center but differ in spread. ## Test whether the sample represents reality Identify how a sample was selected and explain whether it represents the population in a claim. **Watch for:** Why might AI praise a large survey without noticing that it sampled the wrong people? Sample size is visible and familiar, while the selection path is often absent from the data. AI may default to “larger is stronger,” so you must provide or investigate who could enter and who was excluded. ## Separate correlation from causation Distinguish correlation from causation and propose plausible alternative explanations. **Watch for:** Why might AI turn a strong relationship into a causal product recommendation? Patterns are present in the data, while history, selection, and rival explanations may be absent. AI often completes a persuasive narrative from the visible association, so you must challenge direction and confounders. ## Use probability without pretending certainty Interpret an observed success rate as evidence about uncertainty rather than a guarantee about the next event. ## Respect the base rate Combine prevalence, sensitivity, and false-positive rate to interpret an alert. **Watch for:** Why might AI repeat the detector's 95% figure as the chance that an alert is correct? The impressive sensitivity is explicit, while prevalence and false positives may require reconstruction. AI may follow the common accuracy framing, so you must rebuild the population counts before acting on an alert. ## Read confidence as a range Explain why an estimate from a sample needs an uncertainty range and compare wider with narrower ranges. **Watch for:** Why might AI present 55.00% with a confident recommendation despite limited data? Calculation produces exact-looking digits, but decision thresholds, sampling quality, and acceptable uncertainty come from context. AI may supply false precision, so you must ask for a range and test the decision across it. ## Judge an experiment fairly Inspect an experiment's assignment, comparison, outcome, and stopping rule before accepting its result. **Watch for:** Why might AI recommend shipping as soon as the variant number exceeds control? The outcome counts are visible, while assignment quality, stopping rules, concurrent changes, and product guardrails live outside the table. AI may default to “higher wins,” so you must audit the experiment design before the result. ## Expect regression toward the mean Recognize regression toward the mean and avoid crediting an intervention for a likely return from an extreme result. ## Catch a misleading chart Inspect axes, intervals, omitted values, and visual area before trusting a chart's apparent magnitude. ## Treat forecasts as ranges Turn a point forecast into scenarios with explicit assumptions and uncertainty. **Watch for:** Why might AI produce a polished five-year line without showing a useful range? Extending a supplied growth rate is mechanically complete, while business constraints and alternative futures require missing context. AI may default to one smooth path, so you must demand assumptions, scenarios, and decision thresholds. ## Audit an AI-generated analysis Audit an AI-generated analysis for claim strength, fair comparison, uncertainty, and decision consequences. **Watch for:** An AI writes a smooth, confident analysis. Why is that most dangerous at the moment someone signs it off? AI can connect visible numbers into a persuasive story while lacking selection history, decision thresholds, costs, and rival explanations. Fluency hides those absences, so audit evidence and consequences rather than tone.