How accurate is this?
Every number here is measured, recomputed on request, and explained. If something looks wrong, the record's source link is the fastest check, and the contact address at the bottom is the second.
Data reliability
Checks failingEach night an automated audit reconciles published totals and checks the source-to-record chain, source health, official revision dates, the active extraction evaluation, and the latest stored snapshot from each source. A failed check stays visible here and makes the deep system health check fail; no person can mark it passed.
Last run Nov 5, 2026.
| Automated check | Status | Observed | Expected |
|---|---|---|---|
| published provenance Published incidents with an incomplete source, snapshot, extraction, prompt, model, score, or publication chain. | pass | 0 | 0 |
| council provenance Published council actions with an incomplete source, snapshot, extraction, prompt, or model chain. | pass | 0 | 0 |
| official provenance Official statistics with an incomplete source-item or snapshot chain. | pass | 0 | 0 |
| snapshot integrity Latest stored source snapshots missing from private storage or not matching their recorded SHA-256 digest. | pass | 0/7 | 0 failed latest-source snapshots |
| extraction backlog Enabled-source items still fetched or extracting after the processing window. | pass | 0 | 0 older than 6 hours |
| source health Enabled ingestion sources marked degraded or failing. | pass | 0 | 0 |
| official revision dates Official values missing a revision key, date basis, or an available publisher report date. | pass | 0 | 0 |
| official content horizon Official datasets with stored values whose latest period is older than their source-specific publication allowance. | pass | 3/3 within horizon | latest stored period within the configured horizon for every official dataset |
| rollup total The city-month all-type rollups must reconcile to the published incident population. | pass | 269/269 | rollup total equals published incidents |
| eval current The latest police extraction evaluation must describe the active prompt and model. | fail | extract-police-v4 / gpt-5-mini | extract-police-v4 / gpt-5-mini, run within 35 days |
| eval precision Incident-detection precision on the labelled police fixture set. | fail | 0.8108108108108109 | >= 0.9 |
| eval recall Incident-detection recall on the labelled police fixture set. | fail | 0.7792207792207793 | >= 0.8 |
| eval pii Labelled identifier matches that escaped both automated privacy checks. | fail | 2 | 0 |
Extraction accuracy
93 real police releases, hand-labelled, run through the live model (gpt-5-mini, prompt extract-police-v4) on Sep 10, 2026. Detection is whether the model found the same incidents a person did; field accuracy is whether each detail matched.
| Field | Accuracy on matched incidents |
|---|---|
| type | 92% |
| occurred on | 95% |
| occurred precision | 100% |
| street | 100% |
| block | 100% |
| area | 100% |
| target kind | 80% |
| stage | 98% |
| charges | 80% |
| outcome | 100% |
| estimated damage cad | 98% |
| is follow up | 88% |
Against the Police Services Board
The Board's monthly reports count every incident the police recorded. We count only incidents that appeared in a public release. The gap is the point: this is a record of what was made public, calibrated against the official total, not a substitute for it. Twelve months to 2025-07, the latest the Board has reported; Board figures on file for 12 of those months.
| Category (Board metric) | On the ledger | Board total | Share |
|---|---|---|---|
| Mischief / vandalism to property (property.mischief) | 0 | 451 | 0.0% |
| Break and enter — business (property.break_enter_business) | 0 | 101 | 0.0% |
| Break and enter — other (property.break_enter_residence) | 1 | 283 | 0.4% |
| Theft under $5,000 (incl. shoplifting) (property.theft) | 1 | 1,762 | 0.1% |
| Theft from motor vehicle (property.theft_from_motor_vehicle) | 0 | 355 | 0.0% |
| Assault (public setting) (violence.assault) | 6 | 782 | 0.8% |
| Robbery (violence.robbery) | 1 | 63 | 1.6% |
What the rules did
No person reviews or edits a record's content; deterministic rules publish or discard, and "held back" means discarded. The operator can remove a record or send it back through the rules, and every such action is logged. Every rule that fires is logged with what it did. The rules.
| Rule | Result | Records |
|---|---|---|
| Privacy check fired (free-text detail removed) | published | 222 |
| Street not matched (published without a block) | published | 76 |
| Date implausible (publication date used) | published | 30 |
| Model confidence below the flag threshold | published | 28 |
| Privacy check fired (free-text detail removed) | held back | 7 |
| Unusual number of incidents in one release | published | 6 |
| Model confidence below the flag threshold | held back | 3 |
Sources
| Source | Status | Last read |
|---|---|---|
| Sault Ste. Marie Police Service — media releases | ok | Nov 6, 2026 |
| Wayback Machine — historical saultpolice.ca releases | ok | — |
| SSM Police Services Board — monthly reports (PDF) | ok | Nov 5, 2026 |
| City of Sault Ste. Marie — council (eSCRIBE portal) | ok | Nov 5, 2026 |
| Ontario Court of Justice — courthouse-level criminal statistics | ok | Oct 14, 2026 |
| StatCan 35-10-0180-01 — incident-based crime, SSM Police Service | ok | Oct 14, 2026 |
| SIU news releases (filtered to SSM) | ok | Sep 8, 2026 |
| SooToday — City Police Beat | off | — |
Partner feeds
Active-feed participation is withheld until three organisations participate. A block shows partner rows only once three partners have rows there; rows below that threshold are not counted publicly. The partner program.
No partner rows yet.
Corrections
Corrections and takedowns to date: 1. Report a wrong record or privacy concern. Privacy concerns alert the operator immediately; removals are logged and reach licensees through the corrections feed. We answer within five Ontario business days.
Names are never published, so a correction concerns the record's facts, not a person. A record's source link points to the source's own publication; what that document says is theirs to correct.