# Crossing-risk research release — 2026-09-06

Analysis mode: **retrospective_revision_date_asof_inventory**. This is a retrospective benchmark, not a demonstrated operational forecasting system.

## Fixed evaluation design

Fit years: [2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021]; calibration fit: [2022]; calibration selection: [2023]; final test: [2024, 2025].
All six models share the same rows. Hyperparameters and the primary HGB-history versus HGB-exposure contrast are fixed before this release run.
The primary ranking metric is scikit-learn average precision, not trapezoidal area under a precision-recall curve.
Calibration methods are chosen by selection-year Brier score; test labels never choose a calibration method.
Ranking and capacity allocation use raw scores with ascending crossing-ID tie-breaking within each year.

## Results

Test sample: 251,028 crossing-years, 126,447 crossings, and 3,498 positive crossing-years (1.39%).

| Model | AP | Brier | Brier skill vs training prevalence | Top 10% capture |
| --- | ---: | ---: | ---: | ---: |
| Training prevalence | 0.0139 | 0.01374 | 0.000 | 10.0% |
| Exposure logistic | 0.0487 | 0.01350 | 0.017 | 37.8% |
| History-only logistic | 0.0729 | 0.01330 | 0.032 | 38.2% |
| Infrastructure + exposure logistic | 0.0541 | 0.01346 | 0.020 | 37.9% |
| Infrastructure + exposure HGB | 0.0766 | 0.01329 | 0.033 | 45.1% |
| Infrastructure + exposure + history HGB | 0.1012 | 0.01312 | 0.045 | 48.8% |

Adding prior history to HGB changes AP from 0.0766 to 0.1012: difference +0.0246, 95% crossing-cluster interval [+0.0167, +0.0316].
Its top-10% positive-year capture is 48.8%; the difference from HGB without history is +3.7% [95% interval +2.3%, +4.9%].
The selected HGB-history probability calibration is platt. Its Brier skill relative to a constant training-prevalence predictor is 0.045; positive skill indicates improvement.

## Limits and interpretation

- The outcome is a crossing-year with at least one reported incident. Capture does not count every incident and does not estimate incidents prevented.
- Incident-history windows are updated annually. A 2025 row may therefore use 2024 incidents if those years are in the release; this is annual updating, not a single fixed-origin two-year forecast. Historical report-availability vintages remain unverified.
- The 2009 start of incident retrieval provides the full five-year prewindow for 2014. Recency means years since the last observed incident since 2009; it is left-censored and is not an all-time last-incident measure.
- Read data_manifest.json and the data-audit artifacts for source-vintage and eligibility limitations. Revision-date as-of reconstruction can exclude later revisions but cannot establish what was actually available at a historical forecast date; source receipt vintages remain unverified.
- Legacy exploration already examined 2022–2023. Any new release using those years is a revised evaluation, not a fresh untouched holdout. If 2024–2025 are used, those years were not in the legacy panel.
- The 200 bootstrap replicates resample whole crossings and keep their test years together. Intervals condition on these fitted models; they exclude refitting, temporal shocks, source uncertainty, and spatial dependence across crossings. Six models and secondary contrasts are descriptive; no multiplicity-adjusted claims are made.
- Fixed ID tie-breaking ensures reproducibility, but arbitrary ties have no substantive priority meaning. Calibration can create ties; it is used for probabilities only.
- This release does not establish causal effects, geographic transfer, deployment readiness, or that a more complex model is worth its operational cost.

## Reproduction

The run manifest records arguments, split counts, source/code/output hashes, package versions, runtime, and calibration choices. Predictions retain crossing IDs, years, outcomes, raw scores, and selected calibrated probabilities. Bootstrap replicates and calibration-selection scores are saved alongside the figures. A completed output directory is not overwritten; choose another output directory for a reproduction.