# New York journalist data pack — methodology

## What this pack is

A per-state downloadable pack for journalists and researchers: every New York
dashboard's headline figures with vintages and citations.
Generated by `fleet_pack.py` from the dashboard files in the build tree.

## Files

- `newyork-headline-figures.csv` — one row per headline figure (393
  rows from 84 of 84 New York dashboards).
- `newyork-coverage.csv` — one row per New York dashboard (84
  rows): figures extracted, or a recorded missing reason.

- `newyork-methodology.md` — this note (regenerated with every pack run).

## Headline-figure schema

| Column | Meaning |
| --- | --- |
| jurisdiction | Jurisdiction name as shown on the dashboard (h1, " Pulse" suffix removed). |
| dashboard_slug | Dashboard file name without `.html`. |
| dashboard_path | Build-relative path of the dashboard file. |
| dashboard_url | Live extensionless URL of the dashboard. |
| section_heading | Headline/at-a-glance heading text, verbatim. |
| fiscal_year | Fiscal-year label extracted from the heading, if present. |
| figure_label | Figure label, verbatim from the dashboard. |
| figure_value | Figure value, verbatim from the dashboard (money, rates, and counts kept as printed, e.g. "13.5 mills"). |
| figure_note | Figure note from the dashboard, if any. |
| basis_status | Basis/status chip from the dashboard (stat pattern only), if any. |
| source_citation | Best available source citation for the section (see provenance rules). |
| citation_basis | Which level the citation came from (section source line, section intro, page Sources section, or dashboard page only). |

## Provenance rules

1. Every figure value and label is copied verbatim from the dashboard file.
   The generator never guesses, estimates, substitutes, or sums figures.
2. Citations prefer, in order: the headline section's own source line, the
   section intro paragraph, the page's Sources section (truncated to 500
   characters in the CSV; see the dashboard for the full text), and finally
   the dashboard page itself. The `citation_basis` column states which level
   was used, so a reader can judge each row.
3. Rankings values are copied verbatim from the rankings page, including
   "—" where the published table shows no component value. "—" means the
   published table has no value there, not zero.
4. Figures in this pack are as-published on the dashboards. The generator
   does not re-verify them against their source documents; re-verification
   is the dashboard build's job, and the `source_citation` column is what
   lets a reader check any figure against its document.

## Missing figures — recorded, never filled

A dashboard with no extractable headline section is listed in
`newyork-coverage.csv` with status `missing` and a reason, and contributes
no rows to the figures CSV. Reasons used:

- `no headline/at-a-glance section` — the dashboard uses an older template
  with detailed fund tables but no single headline block.
- `section present but no extractable figures` — a headline heading exists
  but none of the five markup patterns matched.

In this run, 0 of 84 New York dashboards are
missing (0 for the first reason, 0 for
the second; see coverage.csv for the per-dashboard split).
Filling those gaps would require mining detailed fund tables and choosing
which fund-table row counts as "the headline" — an editorial choice the
generator must not make silently. That is a v2 decision for the site owner,
with per-template rules and a second verification pass.

## Regeneration design (how this re-runs per deploy)

1. After the build tree is final for a deploy, run the generator against
   that tree: `python3 fleet_pack.py --build-root <build/> --out-dir <fleet/>`.
2. The generator is deterministic: same tree in, same pack out. It writes
   all files fresh every run; no pack file is ever hand-edited.
3. Diff the new pack against the previous pack and report: dashboards added,
   dashboards newly missing, figures changed, figures added or removed.
   Changed figures are a review flag, not an auto-accept: a changed headline
   figure means the dashboard changed, and the dashboard's own verification
   must have covered it.
4. The pack is published as a unit with the deploy it was generated from;
   the methodology note records the generation date below.

## Citation request

If you use this data, please cite Hometown Numbers (hometownnumbers.com) and
the source document named in the `source_citation` column for each figure you
use. Figures are published as shown on the dashboards; please check any
figure against its cited source document before republication.

## Limits

- Headline sections only (see missing-figures section above).
- No API, no per-figure history, no change feed — those are later items.
- No livability-rankings page exists for New York in this build, so this pack carries no rankings CSV. A rankings file appears automatically once the state publishes a rankings page.

- Generation date: 2026-10-27