# callbase — Data Contract (v1) > The **stable interface** between callbase and every downstream project. This is what clients are > allowed to depend on. Anything not described here (the `raw/` zone, internal staging, file counts) > is an implementation detail that may change. Breaking changes bump `schema_version` and are > announced via the manifest. > > **Read §0 first.** Parts of this document are live and verified; parts have never been implemented. > The difference matters, and is not otherwise obvious from the prose. > > Base URL placeholder below: `https://data.quartercall.com` (a public R2/GCS bucket on a custom > domain). Companion: `collect/DESIGN.md`. --- ## 0. Status — what is promised, and what is still design intent **Not every clause below has met live data.** This contract was drafted before the collector existed, so it is part verified promise and part intention. Distinguishing the two is the point of this section: a clause that has never been implemented is a *sketch of* an interface, not an interface. Clauses here that have already failed on contact with the real service: | Clause | As specified | Live reality | Resolution | |---|---|---|---| | §2 manifest | assumed `curated/`, `index/submissions.parquet`, `catalog/` all exist | none of them did | unbuilt fields **omitted**, never written as null; `quarters: []` means "nothing published" | | §3 `account_value` type | `DECIMAL(20,2)` | some codes hold text, dates or `n/a` | NULL in `accounts`, the text in `curated/accounts_text` (D1c); the Dropdowns turned out to be `0`/`1`, numeric (§3, read back 2026-09-22) | | §3 `account_value` scale | `DECIMAL(20,2)` | 15 codes hold ratios; `A998 Net Worth Ratio` = `0.1472` | widened to `DECIMAL(24,8)` | The last one is the instructive case. `DECIMAL(20,2)` would render `A998` as `0.15` — and `A998` is the figure that *determines* a credit union's net-worth classification, so that is not a rounding error, it is a wrong answer. Nothing in the specification was obviously wrong; it was simply written by someone reasoning about what call report data ought to look like. The same lesson arrived from outside, too: **NCUA's own published OpenAPI** declares the discovery and universe endpoints as returning bare JSON arrays, while the live service returns a `{"CreditUnion":[…]}` envelope. A client generated from that schema deserializes to zero rows — silently, and indistinguishably from "nothing new". Interfaces written ahead of payloads are wrong in both directions. ### Status of each section | Section | Status | |---|---| | §1 `index/manifest.json` | **live** — published every run | | §1 `curated/` | **live** — first published 2026-08-24 (16 cycles); **30 cycles, 2019Q1–2026Q2, since 2026-09-20**, readable at `https://data.quartercall.com/curated/`. The series begins at 2019Q1 by decision (`collect/docs/PROGRESS.md` §1) | | §1 `catalog/` | designed, not built | | §1 `external/` | **live** — `external/fred/`, written by `connect fred` and put by `connect upload` (`collect/connect/`), weekly by `connect-fred.yml`. First upload 2026-10-15 (run 38689371758, `last_run` 2026-10-15T05:54:56Z): `sources.json` read back, 36 series, none failed; every series file matches the catalogue's `file_sha256` and `row_count`, 138,631 rows, 1943-01-01 to 2026-10-13 [measured 2026-10-15, meta spike `2026-10-14-fred-series-ids@31924ca`]. Not in `index/manifest.json` (PC5) | | §1 source catalogue declarations and leased publication | **live for FRED** — post-#209 owner run [38787016488](https://github.com/QuarterCall/platform/actions/runs/38787016488) on merged `3fe467b`, 2026-10-15. Ordinary URLs read back after the 300-second cache window: all 36 files match catalogue hashes and row counts (138,631 rows); all 35 unchanged observation hashes retain `fetched_at` and exact file bytes; the lease is released [measured 2026-10-15, platform spike `2026-10-15-platform-209-live-acceptance@0eee07a`]. Cross-source races and interrupted dataset publication have synthetic tests; they were not exercised by this single-source live run | | §1 `external/fhfa/` | **live** — owner run [38794963814](https://github.com/QuarterCall/platform/actions/runs/38794963814) on merged `a51f93f`, 2026-10-15. Ordinary public URLs read back after the 300-second cache window: raw source and panel hashes verified, 7,242 actual rows, 50 states plus DC, 1991Q1–2026Q2; canonical content digest and DuckDB v1.5.6 / Parquet V1 / Snappy receipt verified; all three catalogues agree on one artifact and one event. FRED's source entry, catalogue and all 36 files (138,631 rows) remain unchanged; FHFA lease released [measured 2026-10-15, platform spike `2026-10-15-fhfa-live-acceptance@4bf955a`]. Sequential publication is verified; cross-source races and interrupted publication remain synthetic acceptance. Engine state join and PC5 manifest integration remain separate work | | §2 core manifest fields (`schema_version`, `generated_at`, `collector_run_id`, `freshness`) | **live** | | §2 `quarters[]` counts (`quarter`, `cycle_date`, `status`, `submission_count`, `cu_count`) | **live** — populated from the collector's holdings | | §2 `quarters[].status` = `dormant`, `quarters[].dormant_since` | **live 2026-09-20 — published and read back.** Set by `callbase curate` from the catalog on every run: 18 quarters `open`, 14 `dormant` (2018Q3–2021Q4), each dormant one with a `dormant_since` and no open one with it | | §2 `quarters[].updated_at` | **redefined 2026-09-20; merged 2026-09-21 (#133); not yet live** — when the quarter's files were last built, set by `callbase curate` and left alone by a sweep. The first scheduled run carrying it (2026-09-22) lost its runner before `reconcile` finished, so the live manifest still shows every quarter re-stamped by the last sweep **[read 2026-09-22: one distinct value across 30 quarters]**. Live at the next successful `curate` | | §3 `ingested_at`, and when a cycle's files are rewritten | **changed 2026-09-20, live the same day — read back:** a run with nothing new left all 90 cycle objects with the `ETag` and `Last-Modified` they had. `callbase curate` used to rebuild every cycle every night, so `ingested_at` said when the last run was and every object's `Last-Modified` moved nightly. It now leaves a cycle alone unless its filings, the collector's code or its outputs changed, so an untouched cycle keeps its bytes, its `ETag` and its `ingested_at`. Once a week every cycle is rebuilt as an audit: `accounts` and `accounts_text` then get a new `ingested_at` and `ETag` with identical rows, while `profiles` and the index, which carry no run clock, stay byte-identical | | §2 `quarters[].accounts_path`, `accounts_text_path`, `profiles_path` | **live** on all 30 quarters **[read 2026-09-22]**. `accounts_path` is still the discriminator: its presence, not a non-empty `quarters`, is what means "readable" | | §2 `index` | **live** — `{submissions_path, first_seen_from, rows}` **[read 2026-09-22: 147,476 rows, `first_seen_from` 2026-09-05T02:01:41Z]** | | §2 `latest_bulk_cycle`, `catalog` | **unverified** — omitted until real | | §2 `quarters[].sealed`, `quarters[].authorized` | **removed 2026-09-20.** Both were published as `false` on every quarter. `sealed` restated `status`, and promised an immutability no quarter can have while amendments reach cycles of any age; `status` now carries stability alone, as `open` or `dormant`. `authorized` was untrue of every cycle whose bulk file had shipped, and is one frontier, not a per-quarter fact: `quarter <= freshness.latest_bulk_cycle`, once that is tracked. Neither was ever marked live here, so `schema_version` stays `1.0` | | §2 `inventory` | **live** — operational; `universe.record_count` is every CU record NCUA holds, **not** a denominator | | §2 `quorum_policies`, `quarters[].quorum` | **unverified** — added 2026-09-26 at `schema_version` 1.1 (`collect/docs/design/filing-quorum.md` §12). Written by `callbase curate`; checked on every PR against the fixture store, not the live domain: `contracts/data.quartercall.com/docs/tests/test_fixture_world.py` holds the two launch policies, the object on 2026Q2 and not on 2026Q1, and each verdict recomputed from the published integers. Not yet read back from `https://data.quartercall.com` | | §3 `index/quorum/quarter=YYYYQn/data.parquet` | **unverified** — added 2026-09-26 with `quarters[].quorum`, which points at it. Its columns are held to what the collector writes by `test_fixture_schema.py` on every PR; not yet read back from the live domain | | §3 `curated/accounts` | **live** — **165,669,629 rows across 30 cycles [measured 2026-09-21]**; schema verified by reading the published files back, and held to what the collector writes by `contracts/data.quartercall.com/docs/tests/test_fixture_schema.py` on every PR | | §3 `index/submissions.parquet` | **live** — written by `callbase curate` from the catalog every run (E-001); **[read 2026-09-22]** 147,476 rows, `status` `current` 146,742 / `superseded` 734, the §4 recipe returns rows from the live domain. `first_seen` is NULL for the filings held before the catalog stamped them (`index.first_seen_from` says where stamps begin); `validation_date` and `content_hash` are filled for those by `callbase reconcile` from the archived bodies (2026-09-13), so they are NULL only for a body that could not be read | | §3 `curated/accounts_text/` | **live** — every value that did not parse as a number, as filed, per quarter, for the winning filing (D1c); **[measured 2026-09-21]** 413,265 rows, 30 cycles, five codes (§3's note, read back 2026-09-22) | | §3 `cuid` on `curated/accounts` and `index/submissions.parquet` | **live 2026-09-20 — published, read back, and listed in §3's schema tables** (*Which key*) — NCUA's entity id, the key that survives a charter conversion (`stage-2-curated-zone.md` D11, D14; agreed 2026-09-19). A trailing nullable `INTEGER`, so nothing positional moved. **Read back 2026-09-20:** present on 148,014 of 148,016 index rows — NULL only for the two filings whose payload has no `Profile` block; 5,480 entity ids over 5,586 charters; no `cuid` on two current rows in a cycle; no charter with two. One institution is one `cuid` across a conversion: NORTHERN is 5163 as charter 68696 through 2026Q1 and as 24989 from 2026Q2 | | §3 `curated/profiles/` | **live 2026-09-20 — published, read back, and specified in §3** — the payload's `Profile` block, one row per winning filing: eight identity and observation columns from the catalog, then NCUA's 179 fields typed as its OpenAPI document declares them (`stage-2-curated-zone.md` D10, D10b). **Read back 2026-09-20:** 147,280 rows, 32 cycles, 187 columns; identity agrees with `index/submissions.parquet` on every row; `observed_at` is never NULL (123,551 rows from `first_seen`, 23,729 from the archive object's write time); no `cu_number`, `cuid` or `rssd_number` appears twice in a cycle. The three things a reader must know, which §3 now says in full: **`cu_number` is the filing's, and `profile_charter_number` is a signal, never a key** — they differ on 1,676 rows, which are 110 charter conversions; `assets_prior_cycle` and `members_prior_cycle` are the *previous* cycle's except on 2026Q2, the newest cycle when measured, where they have been the cycle's own **[measured 2026-09-22]** — for assets read `A010`; and **before 2022Q4 those two and `peer_group` are NULL** (78,530 of 78,530 rows), because NCUA serves a placeholder there. It does not cover `curated/accounts`: **two** winning filings have no `Profile` block (charter 21782 at 2020Q3, charter 566 at 2026Q2), `quarters[].profiles_missing` says so per cycle, so join LEFT. `credit_union_type_id` is the cycle's, not today's: it moved for 99 of 99 institutions that changed charter, which is what makes `IN ('FCU','FISCU')` the federally-insured filter across a conversion (2026Q1: 2,672 FCU, 1,578 FISCU, 86 NFICU). **Measured 2026-09-20, per field, in `collect/reference/callbase/profile-fields.csv`:** 136 of the 179 fields carry the cycle's value. **Two do not: `profile_charter_number` and `credit_union_name` are what NCUA held when callbase fetched the filing, on every cycle's row** — of 305 charters that renamed between 2019Q1 and 2026Q1, the 2019Q1 row carries the 2026 name for 303. A name read from this table is the credit union's name *now*, and a rename cannot be seen here. 34 fields never differ between cycles and are **unverified**: stable, or as-of-fetch | | §4 recipes | **run 2026-09-20** against `https://data.quartercall.com` — every recipe executed as written. The recipes §4 carried before that had never been run and did not work: they globbed over HTTPS, which DuckDB cannot do. `catalog/init.sql` is still not built | | §4 query recipes | **live and credential-free** — `read_parquet` over `https://data.quartercall.com/...` was run against the published files, including partition pruning. The `submissions` recipe's path exists (the index is live); the `catalog/` recipe's does not. The block's "32 files" predates the series floor: the manifest lists 30 | | §1 public read URL | **live** — `https://data.quartercall.com` on `callbase-prod`, published as `base_url`. CORS is on **[measured 2026-09-21]**. Any `User-Agent` reads it — `curl`, `wget`, `python-requests`, none at all — since a WAF skip rule for the host **[measured 2026-09-22]**; `collect/OPERATIONS.md` §3.1 | | §5 semantics | mixed; see the notes on individual bullets. *Deduped & newest-wins* is **enforced and verified**: zero duplicate `(quarter, cu_number, account_code)` keys and no quarter with two versions of one credit union, **across all 30 cycles [measured 2026-09-22, 165,669,629 rows]** | | §2 digests (`quarters[].accounts_sha256`, `accounts_bytes` and their text and profiles kin, `index.submissions_sha256`, `submissions_bytes`, `quarters[].quorum.detail_sha256`, `detail_bytes`) | **unverified** — added 2026-10-28 at `schema_version` 1.2 (*Fields added 2026-10-28*). Written by `callbase curate` from each staged file before its put; checked on every PR against the fixture store, where every digest matches its file. Not yet read back from `https://data.quartercall.com` | | §2 `freshness.last_curate_run`, `last_curate_run_id`, `inventory.catalog_at`, `quarters[].counts_at` | **unverified** — added 2026-10-28 with the digests. Written by the collector; not yet read back from the live domain | | §6 versioning | **in force since 2026-08-24** — see below. First minor bump, to `1.1`, made 2026-09-26 and effective from the first manifest published with it. Second, to `1.2`, made 2026-10-28, from the first manifest `callbase curate` publishes with every path digested | **The rule going forward:** no clause is promoted from *unverified* to *promised* until it has been implemented **and** checked against a real payload. Two of the three failures above would have been caught by a single afternoon with 30 live reports, which is exactly how they eventually were. That rule is also what the §6 row means. The corrections in the table above cost nothing because nothing was published; the first published curated file is the moment that stops being true, and from then on a retype is a major version bump rather than an edit to this document. > **That moment has passed.** `curated/accounts` was first published on **2026-08-24**, 16 cycles and > 5,431,189 rows; it has been 30 cycles since 2026-09-20. The schema in §3 is now a promise rather than a proposal: changing a column's type, > name or nullability is a major version bump, and the freedom this document had to correct itself in > place — which it used three times, for `DECIMAL(20,2)`, for the nine non-numeric codes, and for the > ratio scale — is spent. --- ## 1. Layout ``` {base}/ index/ manifest.json # START HERE — describes everything submissions.parquet # master index: one row per submission held, all quarters quorum/quarter=YYYYQn/data.parquet # filing quorum: one row per expected filer (§3) curated/ accounts/quarter=YYYYQn/data.parquet # the facts (long/EAV), partitioned by quarter accounts_text/quarter=YYYYQn/data.parquet # every value that is not a number, as filed (D1c) profiles/quarter=YYYYQn/data.parquet # the credit-union dimension: NCUA's Profile form, one row per filing submissions/quarter=YYYYQn/data.parquet # per-quarter header rows — not built catalog/ init.sql # ready-made DuckDB views over the above catalog.duckdb # (optional) same views, pre-attached external/ sources.json # one entry per source: last_run, failed[], catalogue, catalogue_kind fred/ series.parquet # the catalogue, one row per series: FRED's metadata, source, licence, source_url, fetched_at, sha256, row_count, file_sha256 series=/data.parquet # date DATE, value DOUBLE (NULL for FRED's ".") fhfa/ # live; state purchase-only HPI (§0) datasets.parquet # one current artifact per dataset; commit point artifacts.parquet # all retained artifacts publications.parquet # ordered events: each time an artifact became current source/dataset=hpi_po_state/sha256=/hpi_po_state.txt artifact/dataset=hpi_po_state/sha256=/data.parquet ``` `external/` holds public series from other sources, each row attributed to its source, the URL it was fetched from (without a key), when, and the hash of what was fetched: it is source-attributed, not as filed. FRED's files are replaced in place when the source revises them. A FRED catalogue row's `sha256` is the hash of the observations as fetched; its `file_sha256`, of the exact bytes of the series' `data.parquet`. FHFA retains both raw bytes and panels at immutable paths; only its catalogues and source index entry are replaced. `sources.json` holds one entry per source, keyed by the name of its directory under `external/`. An entry names its catalogue (`catalogue`, a path relative to `external/`, inside that directory) and the catalogue's kind (`catalogue_kind`), which says how its rows name their files: | `catalogue_kind` | `catalogue` | The files its rows name | |---|---|---| | `fred-series-v1` | `fred/series.parquet`, one row per series | `series=/data.parquet`, its bytes `file_sha256`; `series_id` matches `^[A-Z0-9]+$`. FRED's entry also carries `series_count` | | `datasets-v1` | `/datasets.parquet`, one row per dataset naming its current artifact, beside `artifacts.parquet`, every artifact published | each row's `path` (bytes `file_sha256`) and `source_path` (bytes `source_sha256`), relative to `external//`; content-addressed, so never rewritten. FHFA is specified below; publication status is in §0 | A source's upload replaces its own entry and keeps every other as the bucket holds it, so an entry written before `catalogue` and `catalogue_kind` existed may lack them. Both fields are now live for FRED (§0); verification refuses an entry that lacks them or declares an unknown kind. ### FHFA state purchase-only HPI (`datasets-v1`) Dataset `hpi_po_state` reads `https://www.fhfa.gov/hpi/download/quarterly_datasets/hpi_po_state.txt`: traditional, purchase-only, quarterly, State. Both seasonal adjustments and the whole history from 1991Q1 are retained. Local source acceptance: 7,242 rows, 50 states and DC, 1991Q1–2026Q2; identical bytes on a second run changed no artifact [measured 2026-10-15, platform spike `2026-10-15-fhfa-hpi-connector@8e591d1`]. There is no stable panel URL: read `datasets.parquet`, then the exact `path` it names, never a glob. `sources.json`'s `fhfa` entry has `catalogue: fhfa/datasets.parquet`, `catalogue_kind: datasets-v1`, `dataset_count: 1`, `last_run` (successful source check), and `failed: []`. `datasets.parquet` columns, in order: | Column | Type | Meaning | |---|---|---| | `dataset_id` | VARCHAR | `hpi_po_state` | | `hpi_type`, `hpi_flavor`, `frequency`, `level` | VARCHAR | `traditional`, `purchase-only`, `quarterly`, `State` | | `units` | VARCHAR | `index, 1991Q1=100` | | `source`, `licence`, `notice` | VARCHAR | `FHFA`; public domain, HPI® trademark, cite “Source: FHFA HPI®”; “This product uses FHFA data but is neither endorsed nor certified by FHFA.” | | `source_url` | VARCHAR | The fixed source URL above | | `source_sha256`, `transform` | VARCHAR, INTEGER | Raw bytes' SHA-256 and positive connector transform version; together with `dataset_id`, the artifact identity | | `path`, `source_path` | VARCHAR | Panel and raw source paths, relative to `external/fhfa/`, matching the layout above | | `file_sha256` | VARCHAR | Exact panel bytes' SHA-256; integrity and publication pin | | `content_sha256` | VARCHAR | Canonical typed panel content's SHA-256, independent of Parquet encoding | | `writer_duckdb_version`, `writer_options` | VARCHAR | Actual DuckDB engine version (`SELECT version()`) and JSON object with explicit `compression` and `parquet_version` COPY options | | `row_count` | BIGINT | Panel row count | | `first_quarter`, `last_quarter` | VARCHAR | Observation coverage (`YYYYQn`), not vintage identifiers | | `fetched_at`, `published_at` | VARCHAR | UTC ISO-8601: source bytes first fetched; artifact selected as current by the prepared publication | `artifacts.parquet` holds `dataset_id` followed by the columns from `source_sha256` through `fetched_at`, one row per `(dataset_id, source_sha256, transform)`. `publications.parquet` holds `dataset_id`, `source_sha256`, `transform`, `published_at` in event order. A → B → A has two artifacts and three events. A source hash plus transform equal to the current artifact changes neither its timestamps nor its catalogues; only the successful source-check clock advances. The transform is bumped when identical source bytes would produce different panel schema or values, guarded by a canonical-content golden fixture. A writer upgrade or encoding change alone does not bump it. Existing artifacts retain their exact bytes and original writer receipt; unchanged catalogues retain their bytes too, including on seeded runners. New panels currently use Parquet V1 with Snappy. The uploader compares every inventory field with the current selection and preserves every field of previously published artifact records, including these source-specific receipts. `content_sha256` hashes UTF-8 JSON (no whitespace, no ASCII escaping), with this array structure: `["fhfa-content-v1", [[column, type], ...], [[value, ...], ...]]`. The schema and values use the panel's column order below; rows sort by `(state, quarter)`. Each non-null DOUBLE becomes its exact Python `float.hex()` string; strings remain strings and NULL becomes JSON `null`. The digest includes warning text and missingness. The connector verifies written and retained panels against it. `file_sha256` still identifies exact immutable bytes; a content digest never substitutes for that integrity check. Different source bytes may have identical typed content (for example decimal formatting changed): their source provenance and selection events remain distinct, with the shared content digest making that distinction visible. These fields were finalized before the first FHFA publication; there is no adapter for earlier local preparations. The panel is sorted and unique by `(state, quarter)`. Its columns are `state VARCHAR` (50 USPS states and DC), `quarter VARCHAR` (`YYYYQn`), `index_nsa DOUBLE`, `index_sa DOUBLE`, `warning VARCHAR`. Empty or whitespace-only index cells remain NULL; absent observations remain absent. Non-blank index cells must match `[0-9]+\.[0-9]+` exactly; precision is not restricted to two decimal places. The exact known warning is “* Note that this state has fewer than 15,000 transactions over the latest ten years”. No fill-forward or geographic interpolation occurs. The engine's planned SA join is labelled *headquarters state, not lending geography* and cites `file_sha256`; that consumer is not part of this connector. Unknown headers, states or warning text, duplicate keys, invalid years/quarters, non-positive or non-finite or non-decimal non-blank indexes, a missing or changed 1991Q1 = 100 baseline in either adjustment for any state, disappearing published keys, and retreating coverage refuse the run. The existing current dataset stays selected. Paths must match their dataset and hash, with no leading slash, traversal, backslash or scheme; they are validated before use. Publication order is source objects, panel objects, `artifacts.parquet`, `datasets.parquet`, `publications.parquet`, then the source index entry. The current catalogue only names complete objects. A retry first recovers any missing current event using the current catalogue's original `published_at`, then appends a new selection. Immutable objects are never overwritten. Mutable catalogues use the 300-second cache policy; source and panel objects may be cached indefinitely. Clients read `index/`, `curated/` and `external/`. **Do not** depend on `raw/` or any path not listed here. ## 2. `manifest.json` (the entry point) Small, cacheable, regenerated every run. Clients poll it to learn what exists and how fresh it is. ```json { "schema_version": "1.2", "generated_at": "2026-09-27T06:50:02Z", "collector_run_id": "2026-09-27T0649Z-a1b2", "base_url": "https://data.quartercall.com", "freshness": { "service_watermark": "2026-09-26", "latest_bulk_cycle": "2025Q4", "last_successful_run": "2026-09-27T06:50:02Z", "last_curate_run": "2026-09-27T06:50:02Z", "last_curate_run_id": "2026-09-27T0649Z-a1b2" }, "quarters": [ { "quarter": "2026Q2", "cycle_date": "2026-06-30", "status": "open", "submission_count": 4318, "cu_count": 4300, "counts_at": "2026-09-27T06:49:02Z", "accounts_path": "curated/accounts/quarter=2026Q2/", "accounts_sha256": "4eb80630638758eb6030ae3cff395b67baaea46c39dde6694316b301c13d0935", "accounts_bytes": 48211907, "quorum": { "measurement": "filing-quorum/2", "measured_at": "2026-09-27T06:49:37Z", "due_date": "2026-07-30", "expected_basis": "prior_cycle", "weights_from": "2026Q1", "expected": 4336, "filed": 4299, "filed_unweighted": 3, "filed_by_charter": 0, "outstanding_count": 37, "excused": 0, "assets_expected": 2412345678901, "assets_filed": 2404000000000, "largest_outstanding_assets": 7489000000, "coverage": { "by_count": 0.9914667896678966789667896679, "by_assets": 0.9965404299334073599670900767, "by_assets_ceiling": 1, "ceiling_basis": "excuses", "largest_outstanding_share": 0.0031044472877584391509124532 }, "verdicts": { "floor/1": { "status": "reached" }, "default/1": { "status": "reached" } }, "materiality": 0.0025, "outstanding_material": [ { "cuid": 5536, "cu_number": 5536, "name": "EXAMPLE CREDIT UNION", "last_known_assets": 7489000000, "share": 0.0031044472877584391509124532 } ], "detail_path": "index/quorum/quarter=2026Q2/data.parquet", "detail_sha256": "29f7fe418a1c889a2ade798cd04cc3a8add195dce5bc2922abc4edf6434ec23e", "detail_bytes": 412377 }, "updated_at": "2026-09-27T06:49:37Z" }, { "quarter": "2026Q1", "cycle_date": "2026-03-31", "status": "open", "submission_count": 4123, "cu_count": 4031, "counts_at": "2026-09-27T06:49:02Z", "accounts_path": "curated/accounts/quarter=2026Q1/", "accounts_sha256": "267dce97df8bbd4a7c8b39f39617c6c085a4242477c6e1b0725f02a5fae7e16b", "accounts_bytes": 47530112, "updated_at": "2026-06-26T13:32:10Z" }, { "quarter": "2022Q4", "cycle_date": "2022-12-31", "status": "dormant", "dormant_since": "2023-11-02", "submission_count": 4912, "cu_count": 4760, "counts_at": "2026-09-27T06:49:02Z", "accounts_path": "curated/accounts/quarter=2022Q4/", "accounts_sha256": "727fe9886802ebaa0928b31e0fb0134e9fe6fe95efb5ca1362a3772a2fb0dc78", "accounts_bytes": 52004418, "updated_at": "2026-06-26T13:32:10Z" } ], "index": { "submissions_path": "index/submissions.parquet", "submissions_sha256": "21e945ad5e3b81adf2ac82114f7e701ca858a81c9b678ea7d23f95927b71d424", "submissions_bytes": 6120994 }, "catalog": { "init_sql": "catalog/init.sql", "duckdb": "catalog/catalog.duckdb" }, "quorum_policies": { "floor/1": { "owner": "callbase", "measurement": "filing-quorum/2", "min_by_count_bp": 9000, "min_by_assets_bp": 9000, "max_outstanding_share_bp": 100, "rule": "reached when filed*10000 >= min_by_count_bp*expected and assets_filed*10000 >= min_by_assets_bp*assets_expected and largest_outstanding_assets*10000 <= max_outstanding_share_bp*assets_expected; indeterminate when expected_basis is partial; below otherwise", "approved": "2026-09-26", "evidence": "the owner's decision; spike 2026-09-26-floor-and-giants@81e60fa" }, "default/1": { "owner": "callseries", "measurement": "filing-quorum/2", "min_by_count_bp": 9800, "min_by_assets_bp": 9800, "max_outstanding_share_bp": 50, "rule": "reached when filed*10000 >= min_by_count_bp*expected and assets_filed*10000 >= min_by_assets_bp*assets_expected and largest_outstanding_assets*10000 <= max_outstanding_share_bp*assets_expected; indeterminate when expected_basis is partial; below otherwise", "approved": "2026-09-26", "evidence": "spike 2026-09-26-quorum-policy@9924aa0; spike 2026-09-26-floor-and-giants@81e60fa" } }, "inventory": { "catalog_at": "2026-09-27T06:49:02Z", "unattributed_filings": 0, "universe": { "as_of": "2026-06-25", "record_count": 23725 } } } ``` The 2026Q2 verdicts carry no `reached_at`: the quarter was already `reached` when first measured, and a time callbase did not observe is never written (*Fields added 2026-09-26* below). > **Bootstrapping state (current).** The omit-rather-than-null rule still governs: fields describing > unbuilt things are **omitted entirely rather than written as `null`**, because a manifest > advertising `index/submissions.parquet` before it exists would send every client to a 404. `index`, > `base_url` and `latest_bulk_cycle` are still absent for exactly that reason. Clients should test for > a key's presence rather than assume it. > > **`quarters: []` would mean nothing has been collected yet** — "no data available", not an error. > As of 2026-08-24 it is not empty: 16 cycles are published. > > ⚠️ **`accounts_path` is what means "readable", not a non-empty `quarters`.** A quarter entry appears > as soon as the collector *holds* filings for a cycle, which can precede the curated file for it — > that was the state for every cycle until 2026-08-24. Test for `accounts_path`; the distinction > outlives the bootstrap, because a newly collected cycle is held before it is built. > (`www/status/docs/status-page.md` §7 has the reasoning.) Field contract: - `schema_version` — semver-ish; a major bump = breaking change to schemas/layout. - `quarters[]` — **the series: every cycle from 2019Q1, newest first, and nothing older.** The archive holds part of two earlier cycles (2018Q3, 2018Q4) that NCUA will not serve in full; they are kept and are not listed here, not built into `curated/`, and not in `index/submissions.parquet`. Iterating `quarters[]` therefore never meets a cycle that cannot be completed. _(Until 2026-09-20 both were listed, with a few hundred filers each.)_ - `quarters[].status` — the manifest's one statement about **stability**: how likely a quarter's files are to move. - `open` — filings for the cycle are still arriving; its files may change between runs. - `dormant` — NCUA has validated no filing for the cycle in **more than 365 days**; its files change only if a late amendment does arrive, and that makes the quarter `open` again. 365 was measured, not chosen: over 30 cycles it is the point at which a quiet cycle stops being woken (once, against 7 times at 180 days), and filings have been seen up to 1,456 days after a cycle ends and not beyond (`collect/DESIGN.md` §9). Neither is a promise. NCUA accepts an amendment to a cycle of any age, so no quarter is ever final, and `dormant` describes what has been observed rather than what will happen. **A client must treat any `status` it does not recognise as `open`.** `status` says nothing about whether enough of the system has filed (that is `quarters[].quorum` — *Fields added 2026-09-26* below) or whether NCUA's scrub is complete (`freshness.latest_bulk_cycle`). - `quarters[].updated_at` — **when this quarter's files were last built.** It moves when the quarter was rebuilt — because a filing arrived, the collector's code changed, or the weekly rebuild ran — and at no other time; it is absent for a quarter that is held and not yet built. For `accounts` and `accounts_text` it is, to within a minute, the `ingested_at` on the quarter's rows. _Until 2026-09-20 every sweep re-stamped it for every quarter, so it said when a sweep last ran._ It says the files *may* have changed, not that they did: for what moved, query `first_seen` on the index (§4). - `quarters[].dormant_since` — present only on a `dormant` quarter: the date of the newest filing NCUA validated for the cycle, i.e. when activity stopped — not the later date on which callbase began calling it dormant, so it does not move if the threshold is retuned. - `freshness.latest_bulk_cycle` — the newest cycle whose official NCUA bulk ZIP has been released: the **authorization frontier**. A quarter at or before it has been through NCUA's data scrub; a later one holds **preliminary, pre-scrub web-service rows** that NCUA itself labels *non-authorized* — usable, but not yet the official record. One value answers it for every quarter, which is why there is no per-quarter flag. (See §5 "Preliminary vs authorized".) - `freshness.last_successful_run` — **liveness**: when a run last finished without faulting. A run that paused because the NCUA access window was shut still counts, because the machinery worked. A faulted run does not update it, so a broken collector goes visibly stale. - `quarters[].submission_count` / `cu_count` — filings held, and **distinct institutions** holding them. They differ by the number of amendments received: an amendment is a new `submission_id` for the same `(cu_number, quarter)`, so only `cu_count` answers "how many credit unions have filed". Populated from the collector's own holdings, so they count what is archived rather than what is published — see the `accounts_path` note above. - `inventory` — **operational, and about the collector rather than the data**. Optional; absent on manifests written before it existed. It carries `unattributed_filings` (filings held whose cycle is not yet known, so the per-quarter counts are short by exactly that much) and `universe` (`as_of` + `record_count`, how many credit union records NCUA's universe operation last returned). ⚠️ **`record_count` is not a count of active charters and is not a denominator.** The no-args universe call returns every credit union NCUA holds an id for, including charters that merged or liquidated years ago — ~23,700 against roughly 4,300 that actually file — and the payload carries no status field to filter on. Do not compute a coverage rate from it. The expected-filer denominator is `quarters[].quorum.expected` (*Fields added 2026-09-26* below). - `freshness.service_watermark` — **data freshness**: the newest filing update-date collected. Monotonic; never moves backwards. Distinct from `last_successful_run` on purpose — if NCUA were unreachable for a week the collector would keep looking alive while the data quietly aged, so a monitor that reads only `last_successful_run` cannot see that. Watch both. **Fields added 2026-09-04.** `index.submissions_path` is set only once `index/submissions.parquet` has landed (pointer-last, like `accounts_path`), with `index.first_seen_from` — the earliest `first_seen` the index carries; filings held before it have a NULL — and `index.rows`. `quarters[].accounts_text_path` is present only for a quarter with published non-numeric text. **Fields added 2026-09-20.** `quarters[].profiles_path` is set once `curated/profiles` has landed for the quarter (pointer-last), with `profile_rows` and **`profiles_missing`** — winning filings whose payload has no `Profile` block, which are in `curated/accounts` and not in `curated/profiles`. It is 0 for 30 of 32 cycles and 1 for two; it is why the join in §4 is a LEFT JOIN. **Fields added 2026-09-26: `quorum_policies`, `quarters[].quorum`** (`schema_version` 1.1, the first minor bump; `collect/docs/design/filing-quorum.md` §12). Filing quorum answers *has enough of the system filed this quarter to analyse it?* **Evidence first** (Q12): each quarter carries the counts and exact integer terms, the material outstanding filers and a per-filer table (`index/quorum`, §3), and **verdicts derived from that evidence** under named policies. A consumer with its own tolerance applies its own thresholds to the same evidence and names its own policy in its own output. - **The object is omitted, never null, for a quarter whose prior cycle is not held**: its expected set is unknown. Every other quarter carries one; in this series the first, 2019Q1, has none. - **The measurement, and its version** (Q18). `measurement` names it; `filing-quorum/2` means `expected` = the `cuid`s that filed Q−1 ∪ the `cuid`s that filed this quarter, weighted on both sides by Q−1 total assets (`A010`; the cycle named by `weights_from`), with excuses applied only to non-filers. So `expected = filed + outstanding_count`; `filed` includes `filed_unweighted`, the filers with no Q−1 filing, which count in `expected` and `filed` and weigh nothing; `excused` filers are in neither. **A filing with no `cuid`** (its payload has no `Profile` block) **takes the `cuid` its charter number filed under in Q−1**, and is then a filer like any other: counted in `filed`, weighted, and its assets the weight of Q+1. `filed_by_charter` counts them, and their detail rows say `cuid_basis = prior_cycle_charter` (§3). Only Q−1 is looked at. Where that charter number did not file in Q−1, where two `cuid`s filed under it, or where another filing this quarter already carries that `cuid`, the filing stays without one and is in neither count: `filed` plus those equals the quarter's `cu_count` as of the `callbase curate` run that measured it, and that run reports a difference rather than failing. A change to any of this is a new measurement version, never a silent shift, and a policy names the measurement it was calibrated on. `filing-quorum/1`, until 2026-10-02, left a filing with no `cuid` out of `filed` and left its `cuid` in `expected`: it listed as outstanding an institution whose filing `curated/accounts` held, in 2020Q3 and in 2026Q2 (`collect/docs/design/filing-quorum.md` §13). - **`measured_at` moves only when the evidence does** (Q17): when any field but `measured_at`, `due_date`, `verdicts` and `detail_path` changes. `due_date` is left out on purpose: it is a calendar reference, not a measurement, so a cited deadline override changes it without moving `measured_at`. It is not the run's time; diff on it. - **`quorum` is as of the last curate run, not live.** Each scheduled `callbase curate` run (daily, about 05:07 UTC) recomputes every quarter's object; a sweep carries it forward unchanged. A sweep between runs can raise `cu_count` and `submission_count` ahead of it, so the equality above holds as of that run, not at every read. - **`due_date`** is NCUA's filing deadline for the cycle: the 30th of the month after the cycle ends, **11:59:59 pm Eastern on that date** (NCUA's 5300 Call Report FAQs). It is a date, not a timestamp. An extension NCUA grants is a reviewed, cited override in the collector, never an edit of the rule. - **`quorum_policies`, at the root, is the registry** (Q13): every policy a `verdicts` entry is named after, with its `owner`, the `measurement` it applies to, three integer thresholds in basis points, the `rule` in words, the date `approved` and its `evidence`. **A policy is never edited; a change is a new name.** A new consumer's policy is a new entry, with no schema change. Launch entries: `floor/1` (owner `callbase`; 9000 / 9000 / 100) and `default/1` (owner `callseries`, the engine's gate for its free peer aggregates; 9800 / 9800 / 50). - **The rule is normative; the ratios are informative** (Q14). Under a policy a quarter is `reached` when all three terms hold: `filed × 10000 ≥ min_by_count_bp × expected`, `assets_filed × 10000 ≥ min_by_assets_bp × assets_expected`, and `largest_outstanding_assets × 10000 ≤ max_outstanding_share_bp × assets_expected`; `indeterminate` when `expected_basis` is `partial`; `below` otherwise. `assets_expected`, `assets_filed` and `largest_outstanding_assets` are whole dollars (0 for the last when nobody is outstanding). ⚠️ **Compute the rule in 64-bit integers or BigInt, never floating point.** Its products exceed 2^53 — `assets_filed × 10000` is about 2.4 × 10^16 — so a double, and so a plain JavaScript number, can compare them wrongly at the boundary. `coverage.by_count`, `by_assets` and `largest_outstanding_share` are the same quantities as decimals of up to 28 significant digits, for reading; a verdict is never recomputed from them. - **The largest outstanding filer is a term, not a list** (Q23). Asset coverage bounds the *total* missing, not *who*: at 90% by assets the largest credit union, about 7% of the system, could be absent. The third term says no single outstanding filer may hold more than `max_outstanding_share_bp` of Q−1 assets — at 100 bp, about the seven largest. A rule picks the same institutions each quarter by itself; a named list would need upkeep as membership turns over and a merged giant reappears under another `cuid`. - **`verdicts` maps policy name to `status`** ∈ `reached`, `below`, `indeterminate`, a pure function of this payload. **Consumers gate on `status`** (Q15); treat a value you do not recognise as not reached. `reached_at` and `below_since` are an **audit trail**, the one thing only callbase can keep, because it needs the previous run: `reached_at` is when a status of `reached` was first recorded, never removed; `below_since` is present while the status is `below` after a `reached_at` exists. A consumer that publishes something immutable records the `(policy, status, measured_at)` it used, so a later `below_since` tells its reader the basis moved without retracting anything. - **A `reached_at` is only as honest as its basis, and one without a basis is never written** (Q16). `reached_at_basis` is `observed`, stamped only when a run saw the status go from `below` to `reached`; it lags the true crossing by up to one run. After `indeterminate` no `reached_at` is invented: a `below → indeterminate → reached` quarter that had none still has none. A quarter already `reached` when first measured has no `reached_at` — not the launch date. `derived` is reserved for a time computed from each filing's earliest evidence and is not written today (§5). - **`partial` means `indeterminate`** (Q19). `expected_basis` is `partial` when Q−1's filers are fewer than 97% of Q−2's (a hollow prior cycle), and every verdict is then `indeterminate`, with `reached_at` and `below_since` left as they were. The terms are still published, so a reader can see why. A quarter with no Q−2 held is judged on Q−1 alone. - **`floor/1` is a hard stop for every consumer.** Whatever its own policy says, a consumer treats `status` other than `reached` under `floor/1` as "not viable for any analysis". A `reached → below` transition under it is alerted as an incident (an outage or a restatement); it does not withdraw the filings, which remain valid and published. - **`outstanding_material`** lists the outstanding filers at or above `materiality` (0.0025 of Q−1 assets), largest first, each with `cuid`, `cu_number`, `name` (omitted when unknown), `last_known_assets` (its Q−1 `A010`) and `share`. `coverage.largest_outstanding_share` is the largest outstanding filer's share, material or not. `coverage.by_assets_ceiling` is the highest `by_assets` can reach given the cessations known; `ceiling_basis` `excuses` says the only known cessations are the reviewed excuses, which are in neither term, so under `filing-quorum/2` it is 1. - **`detail_path` appears only after the detail file has landed** (pointer-last). If a detail write fails, a quarter whose previous `quorum` object had a `detail_path` keeps that object whole — counts, verdicts and `measured_at` — so the summary and its table never disagree; a quarter whose previous object had none is published without `detail_path`. The detail key is fixed, so between a detail write and the manifest write the file at `detail_path` can briefly hold newer rows than the object describes. A consumer that needs the two to agree compares the file's row counts by `outcome` with the object's `filed`, `outstanding_count` and `excused`; from `schema_version` 1.2 it compares the file with `detail_sha256` and `detail_bytes` (*Fields added 2026-10-28* below). **Fields added 2026-10-28: digests, catalog times and curate's liveness** (`schema_version` 1.2; meta `plans/2026-10-17-manifest-digests.md`, `plans/2026-10-17-curate-liveness.md`, `plans/2026-10-28-release-input-resilience.md` Q6, Q7). - **Every path names its object's bytes.** Beside each path the manifest carries the lower-case hex SHA-256 of the object's exact bytes, and how many bytes there are: `accounts_sha256` and `accounts_bytes` beside `accounts_path`, `accounts_text_sha256` and `accounts_text_bytes`, `profiles_sha256` and `profiles_bytes`, `index.submissions_sha256` and `index.submissions_bytes`, and `quarters[].quorum.detail_sha256` and `detail_bytes`. A quarter's path names its directory (`…/quarter=YYYYQn/`); the digest is of the one object in it, `data.parquet`. - **Why a client checks them.** An object's key is fixed, and `callbase curate` replaces a quarter's objects one after another before it publishes the manifest that describes them. Between those writes the object at a path can belong to a newer build than the manifest names. A body whose digest or length is not the manifest's is such a body: read the manifest again later, fetch again, and never mix it with files the manifest does describe. Before 1.2 nothing told a client that this had happened. - **Under 1.2 every path carries both.** A manifest is 1.2 only when every path it names has its digest and its byte count. Once callbase has published a 1.2 manifest it never publishes one without them: a write that would leave a path undigested is refused, and the published manifest stands. A 1.1 manifest may name a path with neither. A client may refuse a 1.2 manifest that names a path without its digest; the engine does. - **`freshness.last_curate_run` and `last_curate_run_id`** say when `callbase curate` last published everything it owed (every cycle built or carried, the index and every detail file), and which run that was. A sweep carries them forward unchanged, so they show what `last_successful_run` cannot: a sweep moves that whether or not curate runs. Absent until the first such run. - **`inventory.catalog_at` and `quarters[].counts_at`** say when the catalog behind `inventory`, and behind each quarter's `submission_count` and `cu_count`, was read. A sweep and `callbase curate` both publish the manifest, from catalogs read at different times; each keeps the later of its counts and the published ones, so an older reading never replaces a newer one. ## 3. Schemas ### `curated/accounts/quarter=YYYYQn/data.parquet` (facts, long/EAV) | column | type | notes | |---|---|---| | `quarter` | VARCHAR | `2026Q1` (also the Hive partition) | | `cycle_date` | DATE | quarter-end, `2026-03-31` | | `cu_number` | INTEGER | charter number (the CU key) | | `submission_id` | BIGINT | unique per filing | | `version` | SMALLINT | amendment version; **curated contains newest only** (see §5) | | `account_code` | VARCHAR | **raw NCUA code, exactly as filed** — no remapping/series logic | | `account_value` | DECIMAL(24,8) | money-exact **and** ratio-exact — see the note below | | `ingested_at` | TIMESTAMP | when callbase last **built this cycle's file** — one value for every row of a cycle, not a per-filing time. A cycle is rebuilt only when something it is built from changed (§0), so this moves when the cycle's content may have. For when a *filing* arrived, use `first_seen` on the index | | `cuid` | INTEGER | NCUA's entity id — **the key that survives a charter conversion**; see *Which key* below. NULL for the two filings whose payload has no `Profile` block | Rows are **sorted by `cu_number`** within a file (enables row-group pruning). `account_value` is absent rather than zero when a CU didn't report a code — **never assume 0**. A NULL has two possible causes, and they are not the same claim; see the note immediately below before reading one. > **A value that is not a number is published as text, not here.** `account_value` is `DECIMAL`, so > a value NCUA sends as text — a classification, a company name, a date, `n/a` — cannot be stored in > it. The row is still here, one per CU per filing like any other code, with **`account_value IS > NULL`**, and the text is in `curated/accounts_text` (below). **The rule is the value, not the code.** > callbase parses every value; what parses is a number here, what does not is a text row there. So > whether a code appears in `accounts_text` depends on what NCUA sends for it, which is not always > what its declared type suggests **[read back from the published data 2026-09-22, all 30 cycles]**: > > | code | NCUA type | what it holds | what NCUA sends | where it is | > |---|---|---|---|---| > | `A700` | Text | Net Worth Classification, if the CU is not new | `Well Capitalized` … (5 values) | text, every filing | > | `A701` | Text | Net Worth Classification, if the CU is a new credit union | (6 values) | text, every filing | > | `A876` | Text | Company providing share/deposit insurance coverage other than the NCUSIF | free text, 460 distinct | text (12,405 rows); 2 filers typed a number, which is here as a number | > | `ANW0001` | Dropdown | Date of adoption of ASC Topic 326 | a date or a fiscal-year label, 37 distinct | text (51,010 rows) | > | `A999` | Text | **RBNW requirement** — the NCUA-computed risk-based net-worth requirement, a percentage of assets. **Retired**: filed 2019Q1–2021Q4, replaced by the 2022Q1 risk-based-capital codes, and no longer in NCUA's current code list | `6.92` where it applied, the literal `n/a` where it did not | **both**: 6,744 numbers here, 56,366 `n/a` in text | > | `A566B` | Dropdown | Does the CU plan to add branches or expand facilities in the next 12 months | `0` / `1` | **here, as a number** (146,741 rows), never text | > | `A875` | Dropdown | Does the CU maintain share/deposit insurance coverage other than the NCUSIF | `0` / `1` | here, as a number, never text | > | `AAS0010` | Dropdown | CECL adoption indicator | `0` / `1` | here, as a number, never text | > | `ALR0001` | Dropdown | CCULR election (Complex Credit Union Leverage Ratio); from 2022Q1 | `0` / `1`, blank for most filers | here, as a number (13,187 rows), never text | > | `ALR0008` | Dropdown | CCULR grace-period indicator; from 2022Q1 | `0` / `1`, blank for most filers | here, as a number (7,822 rows), never text | > > Two corrections to what this note said before 2026-09-22. **The five `0`/`1` Dropdowns are numeric > here**, not NULL: NCUA's web service sends them as digits (verified in the raw bodies of 2021Q4 and > 2025Q4 filings), so `A566B = 1` means the CU answered yes. Read them as flags. **And a code can be > both**: `A999` is a figure for the credit unions the requirement applied to and `n/a` for the rest, > so a NULL `A999` here with an `n/a` row in `accounts_text` is *not applicable*, not *not reported*. > The earlier claim that declared type and observed kind coincide for every code (D1c: "0 of 1,105 > codes mixed") was measured on a code list that no longer carried `A999`. > > **A NULL means "published as text" only when `curated/accounts_text` has the row** — join to it > and look. The code does not decide it **[measured 2026-09-22, 2026Q1]**: a blank is NULL with no > text row, whatever the code, so `A876` has 488 NULLs with text and 3,848 without, and `ANW0001` > 3,636 and 700. `A700` and `A701` carry text on every filing. The five Dropdowns are numeric and > blank for most filers, so `ALR0001` and `ALR0008` carry thousands of NULLs, none of them text. A > NULL with a text row is *published as text*; a NULL without one is *not reported*. It is the > opposite reading from the one §5's *absent ≠ zero* rule invites only when the row exists. Measured > across 30 live reports the text rows are **0.28% of rows**. > > NCUA's current code list (`collect/reference/ncua/2026Q2/account-codes.csv`) declares **9 of > 1,112** codes non-numeric; retired codes such as `A999` are in earlier lists only > (`collect/reference/ncua/2020Q1/accountDescription_March2020.pdf`, p. 37). The list of codes that > appear in `accounts_text` therefore changes when NCUA adds, retires or re-encodes a code, and this > table is a read-back of the data, not a promise about the schema. > **Why scale 8, not 2.** This column was specified as `DECIMAL(20,2)`, which silently corrupts real > data. NCUA has two families of numeric account code: `Money`/`Positive Money`/`Integer` hold whole > dollars, but `Decimal`/`Positive Decimal` hold **ratios**. Measured against 30 live reports, 15 > codes carry more than two decimal places — every interest rate, and `A998 Net Worth Ratio`: > > ``` > A998 Net Worth Ratio 0.1472 -> DECIMAL(20,2) gives 0.15 > A523 Interest rate of New Vehicle Loans 0.067 -> DECIMAL(20,2) gives 0.07 > ``` > > `A998` is the figure that determines a credit union's net-worth classification, so rounding it does > not lose precision — it changes the answer. Scale 8 keeps money exact, keeps ratios well beyond the > precision NCUA actually reports, and absorbs the float artefacts NCUA emits (values arriving at > scale 17–18, e.g. `0.06700000000000001`, which collapse back to `0.067`). > > **Where scale 8 does round something real — `A998`, mostly across 2021** [measured 2026-09-20, every > cycle]. "Well beyond the precision NCUA actually reports" is true of what a credit union types and > not always of what NCUA computes. For 2021Q2–2021Q4, and tapering back to 2019, NCUA served > `A998 Net Worth Ratio` as a raw computed number — `0.05572056716666699` — on about one filing in > one; since 2022Q1 a handful of filings a cycle carry a computed rate the same way (`A568`, `A595A`, > `A595B`). These are published at 8 places, rounded half-to-even. **The most any value moves is > 5e-9** — the ninth decimal place of a ratio, 0.0000005 of a percentage point — no money value is > affected, and every digit NCUA sent is in `raw/`. `DECIMAL(24,8)` stands on that measurement. One > thing follows for a reader: **do not re-derive a net-worth classification by comparing `A998` with > a threshold. Read NCUA's own determination, `A700` and `A701`, from `curated/accounts_text`** — > which is the right source in any cycle, and the only one that cannot be moved by a ninth place. > `curate` counts these values every run, apart from the float noise, as *rounded beyond float noise*. ### `curated/profiles/quarter=YYYYQn/data.parquet` (the credit-union dimension) > **Live 2026-09-20; read back the same day** (§0). NCUA's *Credit Union Profile* is a separate form > from the 5300, filed with it and returned in the same payload. This is that form, **one row per > winning filing** — the same filing `curated/accounts` publishes — **sorted by `cu_number`**, 187 > columns: eight of callbase's, then all 179 of NCUA's. **callbase's columns — from its catalog, never from the Profile block:** | column | type | notes | |---|---|---| | `quarter` | VARCHAR | | | `cycle_date` | DATE | | | `cu_number` | INTEGER | the charter the filing was made under | | `cuid` | INTEGER | NCUA's entity id; see *Which key* below | | `submission_id` | BIGINT | joins to `index/submissions.parquet` | | `version` | SMALLINT | | | `observed_at` | TIMESTAMP | **when callbase fetched this filing.** Never NULL | | `observed_at_source` | VARCHAR | `first_seen` (stamped at acquisition) or `archived` (the archive object's write time, for a filing held before the stamp existed) | **NCUA's columns, 179.** Each is NCUA's field under a `snake_case` name — `CreditUnionTypeID` is `credit_union_type_id`, `Acct_892A` is `acct_892a` — typed **as NCUA's OpenAPI document declares it**: `INTEGER` (134) or `VARCHAR` (45). Declared type, not apparent type: a date, a `Yes`/`No` and a number in a string all stay the string NCUA sent. **The full list — NCUA's name, the column, its type and its temporal class — is `collect/reference/callbase/profile-fields.csv`.** The ones most read: | column | type | class | notes | |---|---|---|---| | `credit_union_name` | VARCHAR | `as_of_fetch` | name. **As-of-fetch: today's name on every cycle's row** — a rename cannot be seen here | | `credit_union_type_id` | VARCHAR | `cycle` | `FCU`, `FISCU` or `NFICU`. `IN ('FCU','FISCU')` is the federally insured population | | `state` | VARCHAR | `cycle` | two-letter state | | `charter_state_code` | VARCHAR | `cycle` | chartering state, spelled out | | `region` | VARCHAR | `cycle` | NCUA region | | `peer_group` | VARCHAR | `cycle` | NCUA asset-size peer group. **NULL before 2022Q4** | | `is_low_income` | VARCHAR | `cycle` | `Yes`/`No` — low-income designation | | `fom_type` | VARCHAR | `cycle` | field of membership type | | `date_chartered` | VARCHAR | `unproven` | `YYYY`, as NCUA sends it | | `date_insured` | VARCHAR | `unproven` | `M/D/YYYY`; `1/1/1900` is NCUA's "unknown" | | `rssd_number` | VARCHAR | `cycle` | Federal Reserve RSSD id — an external join key. A string, as declared | | `acct_pg0001` | VARCHAR | `cycle` | LEI, where the credit union has one (populated from 2023Q4) | | `acct_891` | VARCHAR | `cycle` | website | | `organizational_email` | VARCHAR | `cycle` | the institution's contact address | Five of NCUA's fields are renamed, each because its own name would mislead: | NCUA's field | column | why | |---|---|---| | `CharterNumber` | `profile_charter_number` | **a signal, never a key.** NCUA stamps it with the institution's charter *as of the fetch*, so on a filing made before a conversion it names a charter that did not yet exist. It differs from `cu_number` on 1,676 rows — 110 charter conversions | | `CreditUnionId` | `profile_credit_union_id` | the block's own copy of the entity id. Read `cuid` | | `CycleDate` | `profile_cycle_date` | the block's own copy, as NCUA's `M/D/YYYY` string. Read `cycle_date` | | `Assets` | `assets_prior_cycle` | **the previous cycle's total assets — except on 2026Q2, the newest cycle when measured, where it has been the cycle's own — whether a cycle's rows change once a newer one is published is not measured** [measured 2026-09-22: at 2026Q2 equal to its own `A010` for 4,299 of 4,299; at 2026Q1 to the prior cycle's for 4,320 of 4,336]. For a credit union's assets read `A010` from `curated/accounts`, always | | `NumberOfMembers` | `members_prior_cycle` | likewise, including the newest cycle. Read `A083` | **What each column's value is a value *of* — measured, not assumed** (2026-09-20, per field, in the reference file above). A row is keyed by cycle, and most of it is that cycle's; not all of it is: | class | columns | read it as | |---|---|---| | `cycle` | 136 | what was true for that cycle. Proven by cycles of one institution, fetched together, differing | | `as_of_fetch` | **2** — `profile_charter_number`, **`credit_union_name`** | **what NCUA held at `observed_at`**, whatever the row's cycle. Of 305 charters that renamed between 2019Q1 and 2026Q1, the 2019Q1 row carries the 2026 name for 303 | | `prior_cycle` | 2 | the cycle before — except on 2026Q2's rows, the newest cycle when measured (above) | | `every_cycle` | 3 | the filing's own dates | | `unproven` | 34 | never differs between cycles: stable, **or** as-of-fetch. Mostly flags that are `0` for nearly everybody. Do not build history on one | **Three rules a reader must know:** 1. **Before 2022Q4, `assets_prior_cycle`, `members_prior_cycle` and `peer_group` are NULL.** NCUA serves a placeholder there — 0, 0 and the smallest peer group, for every institution, in 78,530 of 78,530 rows — and a multi-billion credit union in the smallest peer group for fourteen cycles is a wrong answer, not a faithful one. The rule is keyed on the cycle, so a genuine zero from 2022Q4 on is published as filed. It is the one place this table withholds what NCUA sent; the bytes are in the raw zone. 2. **An empty string is NULL.** NCUA's `''` means "not reported", which is what NULL already means. Nothing else is normalised: `Yes` is `Yes`, `092021` is `092021`, and `1/1/1900` is what NCUA sent. 3. **It does not cover `curated/accounts`.** Two winning filings in the archive have no `Profile` block (charter 21782 at 2020Q3, charter 566 at 2026Q2). `quarters[].profiles_missing` says so per cycle. **Join LEFT**, and decide what a missing row means for your question. There is **no `ingested_at`**: `observed_at` is this table's time. With no run's clock in a row, a cycle's file is byte-identical from one run to the next when nothing in it changed — cacheable on `ETag`, which `curated/accounts` is not. #### Which key — `cu_number`, `cuid`, `rssd_number` **`cu_number` is a charter key, not an entity key.** A credit union that converts its charter is issued a new number, and its series under the old one simply stops: nothing errors, a $700M institution appears to cease. 99 institutions in the archive have done it; three have done it twice. | key | on | use it for | where it fails | |---|---|---|---| | `cu_number` | every table | one charter; joining to NCUA's own files, which are keyed on it | **a conversion mints a new one** | | **`cuid`** | `accounts`, `profiles`, the index | **one institution over time.** Equal across all 108 conversions measured; never on two current rows in a cycle; no charter has two | NULL for the two filings with no `Profile` block. NCUA's own id: it has been seen to change once (`SPIKE-002` §1.2) | | `rssd_number` | `profiles` | joining to Federal Reserve data | on `profiles` only, a string, and an external identifier with its own lifecycle | None is a perfect entity key, and callbase does not pretend one is (`stage-2-curated-zone.md` D14). **For a series over time, filter on `cuid`** — §4 has the recipe. ### `index/submissions.parquet` (master index, all quarters) > **Verified 2026-07-29** against 203,717 live filings across 14 years — full write-up in > `collect/docs/design/index-schema-verification.md`. `validation_date` was `DATE` and is now `TIMESTAMP`: > 80.4% of filings carry a non-midnight validation time, and truncating it would hide an amendment > that lands on the day an article was published. > > **`version` and `status` are derived by callbase, not reported by NCUA.** The service returns > exactly one filing per `(cu_number, cycle_date)` — the current one — and no endpoint accepts a > `submission_id`. An amendment is recognised as *"a pair we already hold, now carrying a different > `submission_id`"*, which is only possible because callbase kept the previous one. **An amendment > that occurred before callbase began collecting is undetectable**, and always will be. | column | type | notes | |---|---|---| | `quarter` | VARCHAR | | | `cycle_date` | DATE | | | `cu_number` | INTEGER | | | `submission_id` | BIGINT | | | `version` | SMALLINT | | | `validation_date` | TIMESTAMP | NCUA validation timestamp — **not** a date; 80% carry a real time | | `first_seen` | TIMESTAMP | when callbase first ingested this submission (use for "recent") | | `source` | VARCHAR | `web` (authoritative REST web service), `seed` (imported prior web-service capture), or `bulk` (backfill/cross-check, amendment-incomplete) | | `status` | VARCHAR | `current` or `superseded` — **derived by callbase**, see note below | | `content_hash` | VARCHAR | integrity / change detection | | `account_rows` | INTEGER | row count in curated for this filing | | `cuid` | INTEGER | NCUA's entity id, as on `curated/accounts`. On 148,014 of 148,016 rows | **Row grain (decided 2026-09-04): one row per submission held, superseded ones included** — `status` says which stands. The amendment history is what `raw/` uniquely holds, and this is the cheapest place to surface it (D9). `first_seen` is NULL for the filings the collector held before it stamped them — nothing in the archive records when callbase first held a filing — and `account_rows` is NULL for a filing the current build did not write. `validation_date` and `content_hash` are recoverable from the archived body and are filled for those filings by `callbase reconcile`. Never a synthetic value: a "recent" query sees what was recorded from `index.first_seen_from` on, and the manifest says so. `curated/submissions/quarter=.../data.parquet` is the same schema, partitioned per quarter — **not built**. ### `curated/accounts_text/quarter=YYYYQn/data.parquet` (every non-numeric value, as filed) | column | type | notes | |---|---|---| | `quarter` | VARCHAR | | | `cycle_date` | DATE | | | `cu_number` | INTEGER | | | `submission_id` | BIGINT | the winning filing only, like `curated/accounts` | | `account_code` | VARCHAR | a code whose value did not parse as a number — five today: `A700`, `A701`, `A876`, `ANW0001`, `A999` (the note in §3 above) | | `account_text` | VARCHAR | the value **exactly as filed**, trimmed — `Yes`, `Well Capitalized`, … | | `ingested_at` | TIMESTAMP | | One row per non-numeric value the winning filing carried; the matching `curated/accounts` row keeps its NULL. A quarter with no such values has no file, and no `accounts_text_path` in the manifest. This is D1's blueprint built as D1c, because a consumer arrived — the engine's `cculr_election`, which was then thought to read `ALR0001` as `Yes`/`No`. It does not: NCUA sends that code as `0`/`1`, which is numeric and in `curated/accounts` **[read back 2026-09-22]**; what the table carries is the five codes above. The routing is by observed value (`AccountValueParser`), so a code NCUA re-encodes moves between the two tables without a change here. ### `index/quorum/quarter=YYYYQn/data.parquet` (filing quorum, one row per expected filer) | column | type | notes | |---|---|---| | `quarter` | VARCHAR | `2026Q2` (also the Hive partition) | | `cuid` | INTEGER | **the key** — one row per `cuid`, so an institution that converted its charter is one row (*Which key* above) | | `cu_number` | INTEGER | for display: the charter it filed under this quarter, or, if it has not, the one it filed under in Q−1 | | `cu_name` | VARCHAR | for display, chosen like `cu_number` | | `outcome` | VARCHAR | `filed`, `outstanding` or `excused` | | `weight_assets` | DECIMAL(24,8) | its Q−1 total assets (`A010`), in dollars. **NULL means no prior-cycle filing**: a filer new since Q−1, counted in `filed_unweighted`. **0 means a Q−1 filing with no `A010` value** (or one of zero): it counts in `expected`, weighs nothing and is not `filed_unweighted` | | `weight_as_of` | VARCHAR | the quarter `weight_assets` is from — the manifest's `quorum.weights_from`; NULL where `weight_assets` is | | `expected_basis` | VARCHAR | `prior_cycle` or `partial`, as on the quarter's `quorum` | | `cuid_basis` | VARCHAR | where `cuid` came from: `profile`, the filing's own `Profile` block; or `prior_cycle_charter`, the `cuid` the charter number filed under in Q−1, for a filing with no `Profile` block (§2, `filing-quorum/2`). On an `outstanding` or `excused` row it is the Q−1 filing's | The rows behind a quarter's `quarters[].quorum` (§2, *Fields added 2026-09-26*), written only for a quarter that carries one, and pointed at by its `detail_path`. **The row count is `expected + excused`**: every expected filer, plus the excused filers, who are in neither term. By `outcome` the rows count `filed`, `outstanding_count` and `excused`; the sum of `weight_assets` over the `filed` and `outstanding` rows is `assets_expected` before it is rounded to whole dollars. The engine groups these rows by peer group to form `n` and `expected_n`. Keyed on `cuid`, never `cu_number`. The file is rewritten only when its rows change, and its key is fixed: §2's `detail_path` rule says what that means for reading it beside the manifest. ## 4. Querying from DuckDB (the supported recipes) > **Run 2026-09-20 against `https://data.quartercall.com`, DuckDB v1.5.** Every recipe in this block > was executed as written and returned the result beside it. The recipes this section carried until > then had never been run (§0), and **did not work**: they read `…/quarter=2026Q1/*.parquet` and > `…/accounts/**/*.parquet`, and DuckDB cannot glob over plain HTTPS — there is nothing to list. A > quarter is one object, `data.parquet`, so name it; and for several quarters take the list from the > manifest, which is what the manifest is for. **Zero-setup, public HTTPS, no credentials:** ```sql -- one cycle: name the file -> 4,782,608 rows SELECT count(*) FROM read_parquet( 'https://data.quartercall.com/curated/accounts/quarter=2026Q1/data.parquet'); -- every cycle: the manifest lists them -> 32 files SET VARIABLE base = 'https://data.quartercall.com/'; SET VARIABLE accounts = ( SELECT list(getvariable('base') || q.accounts_path || 'data.parquet') FROM (SELECT unnest(quarters) AS q FROM read_json(getvariable('base') || 'index/manifest.json')) WHERE q.accounts_path IS NOT NULL); -- accounts_path is what means "readable" (§2) -- one INSTITUTION's total assets over time. On cuid, not cu_number: this credit union converted -- its charter in 2026, and on cu_number = 68696 the series stops at 2026Q1 with no error (§3). SELECT quarter, cu_number, account_value AS total_assets FROM read_parquet(getvariable('accounts')) WHERE cuid = 5163 AND account_code = 'A010' ORDER BY cycle_date; -- … 2026Q1 68696 682,119,422 · 2026Q2 24989 698,197,125 -- recently filed (last 7 days) SELECT cu_number, cuid, submission_id, validation_date, first_seen FROM read_parquet('https://data.quartercall.com/index/submissions.parquet') WHERE first_seen >= now() - INTERVAL 7 DAY ORDER BY first_seen DESC; -- the dimension: total assets by charter type for a cycle. LEFT JOIN — profiles does not cover -- accounts (§3) — and keep the unmatched visible rather than dropping them in a WHERE. SELECT coalesce(p.credit_union_type_id, '(no profile row)') AS type, count(*) AS credit_unions, sum(a.account_value) AS total_assets FROM read_parquet('https://data.quartercall.com/curated/accounts/quarter=2026Q1/data.parquet') a LEFT JOIN read_parquet('https://data.quartercall.com/curated/profiles/quarter=2026Q1/data.parquet') p USING (cu_number) WHERE a.account_code = 'A010' GROUP BY 1 ORDER BY 3 DESC; -- FCU 2,672 $1,299.2B · FISCU 1,578 $1,184.9B · NFICU 86 $23.7B ``` **The federally insured population is `credit_union_type_id IN ('FCU', 'FISCU')`.** The 86 `NFICU` filers — state-chartered and privately insured, $23.7B at 2026Q1, about 1% of the system — are published here like any other filer. QuarterCall's own figures cover federally insured credit unions (owner, 2026-09-19), which is 4,250 filers and $2,484.0B at 2026Q1, and this filter is how to reproduce them. The type is the cycle's: it moved with the cycle for 99 of 99 institutions that changed charter. > **Corrected 2026-09-20.** This note first said the `NFICU` filers are "not in NCUA's public bulk > files". **They are** — all 86, as `CU_TYPE` 3, with the same $23,718,765,117 of total assets > **[measured: `call-report-data-2026-03.zip`, `FOICU.txt` and `FS220.txt`]**. What omits them is the > engine's legacy bulk build, which filters to `CU_TYPE` 1 and 2 unless told otherwise > (`build_db3.py`, `include_non_federally_insured=False`). The population was always a choice made > downstream of NCUA, and a total that includes `NFICU` differs from QuarterCall's for that reason, > not from NCUA's file. **Clean view names via the published catalog:** ```sql .read https://data.quartercall.com/catalog/init.sql -- defines: accounts, submissions, cycles SELECT count(*) FROM accounts WHERE quarter = '2026Q1'; ``` `catalog/init.sql` is **not built** (§0), and the sketch below cannot be built as drawn: it globs over HTTPS, which the note above explains does not work. When it is built it will list the files, from the manifest. Roughly: ```sql CREATE OR REPLACE VIEW accounts AS SELECT * FROM read_parquet( 'https://data.quartercall.com/curated/accounts/**/*.parquet', hive_partitioning => true); CREATE OR REPLACE VIEW submissions AS SELECT * FROM read_parquet('https://data.quartercall.com/index/submissions.parquet'); ``` ## 5. Semantics clients can rely on - **`cu_number` is a charter, `cuid` is the institution.** A charter conversion mints a new `cu_number`; `cuid` is equal across every conversion measured. For anything over time, filter on `cuid` (§3 *Which key*, §4). - **`curated/profiles` is one row per winning filing, keyed like `curated/accounts`, and most of it is the cycle's — but `credit_union_name` and `profile_charter_number` are as-of-fetch**, and three columns are NULL before 2022Q4 (§3). It does not cover `curated/accounts`: join LEFT. - **Deduped & newest-wins:** `curated/accounts` and `submissions` with `status='current'` contain exactly one version per `(cu_number, quarter)` — the latest amendment. callbase does the dedup; clients never resolve amendments themselves. (Full amendment history is available via `superseded` rows in the index.) - **Superseded versions exist only here.** NCUA replaces a filing when it is amended: a re-fetch returns whatever is current, and no endpoint accepts a `submission_id`, so a prior version cannot be retrieved from the service at any point. The `superseded` rows in this index, and the payloads in callbase's raw zone, are the **only** record that an earlier version existed. Correspondingly, callbase's amendment history begins when its collection began — earlier amendments are already unrecoverable. - **Amendment-complete, beyond what NCUA's bulk files offer:** because the web service is the backbone, callbase captures amendments to *any* prior cycle (which the published bulk files silently omit). When the same filing exists from multiple sources, **`web`/`seed` always supersedes `bulk`**. A `bulk`-sourced row means "no web-service capture of this filing (yet)." - **Atomic snapshots:** a client reading after any run sees a consistent dataset — never a partial write. Read `manifest.json` first if you need the matching file set. - **Dormant ≠ immutable:** a quarter with `status: "dormant"` has stopped changing and can be cached for a long time, but not for ever — a late amendment reopens it. Re-read the manifest to find out. - **Accuracy is NCUA's disclaimer, and it travels with the data.** callbase reproduces what the CUO Data Service returned; it does not validate or correct it. NCUA's Rules of Behavior state that the data comes from reliable sources but *"no guarantee is given as to data accuracy"*, that it has generally had only a *"first level review/validation but may still be considered raw data"*, and that **conclusions drawn from it are the user's own and are not to be attributed to NCUA**. Any onward publication should carry the same caveat.
One known failure mode is worth naming because it is invisible in a typed column: values meant as **basis points or percentages, filed as whole integers**. Ratio-typed codes — interest rates, `A998 Net Worth Ratio` — are where to expect it. Errors can be reported to `CUOnline@NCUA.gov`. - **Preliminary vs authorized:** current-cycle data collected from the web service *before NCUA's quarterly data scrub* is **preliminary** — NCUA labels such pre-scrub data *non-authorized*. callbase still publishes it and never presents it as the official record; once NCUA releases that cycle's bulk ZIP, `freshness.latest_bulk_cycle` advances to it. Don't headline the figures of a quarter later than `latest_bulk_cycle` as final — and until that field is tracked (§0), treat every quarter that way. - **When a filing was first certified — filing quorum's clock** (`collect/docs/design/filing-quorum.md` §12 Q20). **No `certified_at` is published.** NCUA's `Profile.CertificationDate`, in `curated/profiles` as `certification_date`, survives an amendment made after the cycle closes, but not a re-certification while it is open: in 2026Q2, 105 of 513 multi-version filers carry a later version's date **[measured 2026-09-26, spike `2026-09-26-quorum-policy@9924aa0`]**. So the earliest evidence of a filer's filing is `min(validation_date)` over its rows in `index/submissions.parquet`, superseded ones included — and for a filing amended before callbase held it, that is the amendment's. No `reached_at` is `derived` from it yet; every `reached_at` today is `observed` (§2). - **Raw codes only:** callbase stores account codes **as filed**. Series-break / account-code-change comparability is a *downstream* concern (see `analytics-projects`), deliberately not in this layer. - **Absent ≠ zero:** a missing `account_code` for a CU means "not reported," not 0. A **present row with a NULL `account_value`** means *published as text* when `curated/accounts_text` has the row, and *not reported* when it does not — a blank arrives as NULL on any code (§3). Neither ever means 0. ## 6. Versioning policy **`schema_version` 1.0 begins at the first published curated file.** Until then this document is partly a draft — §0 records three types that were specified before anyone had measured a payload and were corrected once one arrived, all under a stationary `1.0`. That was free because nothing was published and no client could have depended on it. **After the first publish it stops being free:** every subsequent change follows the rules below, and a correction of the same kind would cost a major bump. §0's table is the record of what was changed while it was still cheap, and it is retained rather than tidied away precisely so the `1.0` a client sees is not mistaken for a version that has been stable all along. - Additive changes (new optional column, new manifest field, **a new table such as `curated/accounts_text`**) keep `schema_version` major, bump minor. Nothing a client already reads changes, so nothing a client already wrote breaks. **First applied 2026-09-26**: `quorum_policies`, `quarters[].quorum` and `index/quorum` make it `1.1`, from the first manifest published with them. The additive fields and tables before them — `index`, `accounts_text`, `profiles`, `cuid`, among others — shipped under `1.0` without a bump; the changelog says so. A client that read `1.0` reads `1.1` unchanged. **Applied again 2026-10-28**: the digests, `freshness.last_curate_run`, `inventory.catalog_at` and `quarters[].counts_at` make it `1.2`. `1.2` also carries a rule `1.1` did not: every path names its digest and its byte count. A client that read `1.1` reads `1.2` unchanged. - Breaking changes (column removed/retyped, layout change) bump the **major** `schema_version` and are published side-by-side under a versioned prefix (e.g. `/v2/...`) for a migration window. - The current major version's paths above are stable for the life of that major version. - Clients should read `schema_version` from the manifest and refuse a **major** they do not recognise, rather than assume forward compatibility. Minors are safe to ignore.