countries-states-cities-database

Data versioning: what an id means and what a release may change

This page is the contract between the data and the people who store its ids. It applies from v5.0.0. It says what an id denotes, what each kind of release may change, how to pin a release, and how to migrate stored ids between releases.

What an id denotes

A key is a table plus an id, for example cities 1403 or states 1509. A key identifies one geographic entity, such as a town, a municipality or a province. It does not identify a fixed set of attributes.

Some records predate typing, and what they first denoted (a town, or the municipality around it) can no longer be established. These keys get an unresolved identity decision, and never redirect automatically. A later type, local type (type_local), level, parent_id or county_id is kept on such a key only when the evidence ties it to the entity the key already denoted; otherwise the field keeps its v4.0 value, or stays absent if it had none. An unproved change of Wikidata item keeps the v4.0 item; if evidence proves that old item wrong but proves no replacement, the key has no item. Facts that are proved stay; unproved assertions that would tie the key to another entity do not ship.

identity_changes.jsonl holds the decisions made in its release’s interval, so a key missing from a later file is not resolved by that. Before v5.0.0 is published, its cumulative identity file must also provide a complete reviewed baseline of every retained unresolved key, including legacy rows unchanged since v4.0. The reviewed input is migrations/unresolved_baseline.jsonl: each key binds its snapshot commit, full-row hash, reason, decision round, review and evidence. It does not fabricate a changed-row event. The generator validates the snapshot and includes it in the cumulative file as baseline: true (migration format 3). Keep that baseline immutable; later releases append decisions to the ordinary ledger. A guarded decision_only identity event permits a later review without a scalar edit. See the input and ordering rules.

Starting from that verified baseline, apply subsequent identity decisions in first-parent order and retain each key’s latest decision; reviewed removals retire keys. Absence from a later file does not resolve a key. Field admission does not itself replace an explicit unresolved identity decision. Strict territorial consumers exclude retained keys whose latest decision is unresolved. Inventory disclosure does not close a substantive publication hold or authorise an otherwise unproved transfer.

Data PRs carry their reviewed ledger events in the same PR: use a positive integer event_pr instead of event_commit. The generator requires GitHub’s verified merge_commit_sha association on the pinned ledger’s first-parent line and publishes that single-parent squash SHA; subjects are not proof. The Ledger migrations PR check uses --pending-pr N to validate those events against the merge preview before merge; the historical row, target, baseline and identity rules still apply. See how to add events in a data PR. Existing SHA-keyed events remain valid. Require that check in branch protection to block merges on ledger errors.

What each kind of release may change

Releases are numbered vMAJOR.MINOR.PATCH, for example v5.0.0 and then v5.1.0. Each published export of a release has its own tag, v5.0.0-export.1, v5.0.0-export.2 and so on, bound to the commit it was built from. Exports can hold different data. Changes between published snapshots within one major follow the regular-release rules below. The first published export of a new major may include the changes allowed only in a major.

Change Regular release (minor or patch) Major release
Correct a fact about the same entity (name, translation, coordinates, time zone, item, population) yes yes
Correct a type, level or parent under the published rules yes yes
Add records (new places, counties, postcodes) yes yes
Move a misfiled place to its proved state or country yes yes
Remove a key, or move it to another table no yes
Repair a key that came to denote another entity (the key retires with its first meaning; the other entity gets a fresh, never-used key, or an existing key already proved to denote it) no yes
Change a field’s meaning or nullability, an id namespace, a table’s scope, how countries, codes or letter case are written, or an incompatible schema or documented query rule no yes

The results of an unchanged query may change when facts about the same entity are corrected.

Between majors, existing keys stay lookupable. A record found to be a unit or invalid is classified or flagged only where the published model allows it; otherwise that change is held. Both keys of a duplicate stay, with the proof recorded. Removing a key or moving it to another table waits for the next major.

Transition from contract 5 to contract 6 (v6.0.0)

v6.0.0 ships the merged additions and factual corrections since v5.2.0 and one incompatible documented query change: settlement selection also excludes US municipality rows. Romanian comune and Polish gminy and legacy adm2 and adm3 rows stay in the examples: many of their seats have no row of their own yet, and some of the Polish rows are towns, so their review waits for v7. The 776 never-released New York territorial towns from #2170 change from town / town to municipality / town; functioning townships already use municipality / township or municipality / charter township. These keys continue to denote the same territorial units. The schema stays at version 5; contract version is 6. No key is removed, moved to another table or repurposed by this transition. All other query-rule changes and every removal wait for v7.

Consumers must pin their exact snapshot, refresh the data, and upgrade their settlement query together: add US to the country-specific municipality exclusion in both TYPE_FIELD.md examples. A type whitelist that keeps town is insufficient on the old master data. Use the reviewed NY ids if migrating that unpublished snapshot; do not retype every town / town row. Stored ids remain valid; no automatic redirect is required for these 776 rows. Do not sum territorial populations with those of their contained places. 1,478 released territorial-town rows keep their published type and remain in the filter results; see the US section and its per-id inventory before treating a result as a settlement. Apart from the United States, no country’s filter is changed by this release.

Attach and verify the migration archive at the release commit. The release wrapper’s contract-6 cumulative base is the newest published prior-major feature release, v5.2.0 (e39d7109); its contribution data equals the latest v5 download, v5.2.0-export.1 (source 9b255c45). Check from.sha, to.sha, counts and checksums in the manifest. Keep the older v4.0-to-v5 migration chain for consumers starting there; a direct v4.0-to-v6 validation does not replace that published chain. After the first v6 export, archives also cover each subsequent v6 snapshot step. The no-removals-between-majors rule above applies immediately after that first published v6 export, including any further v6.0.0 export. This bounded release creates no exception for deferred work or later incompatible query changes.

Pinning and migrating

Pin the exact export tag and its commit, or the commit of the snapshot your product actually ships, not “latest”. A package version or an API protocol version alone does not name a data snapshot. The API and the packages report the data version they serve; a live API client picks its data version as the API documentation describes.

Each release attaches migration files to its GitHub release, described in migrations/README.md. They cover countries, states, counties, cities and postcodes. They do not describe changes to the separate regions and subregions grouping tables.

To migrate stored ids, take a set whose release_manifest.json starts at your snapshot (from.sha) and ends at the one you move to (to.sha), and apply it; then do the same with the next release’s set. Each set describes the step from the previous published snapshot. v5.0.0 carries one: the cumulative set from v4.0. There is no step from a later v4 export: the source of v4.0-export.51 held postcodes whose ids the export assigned and synced back later, and every id must be known at both ends. If you use a later v4 export, load the v5 data fresh and use the cumulative set to map the ids you stored; do not patch that export with its added or changed files, which are relative to v4.0 (see migrations/README.md). From the second v5 export on, each release also carries the step from the previous v5 export.

Example: list the stored city ids that may be redirected to another record.

python3 -c "
import json
for line in open('removed_ids.jsonl', encoding='utf-8'):
    e = json.loads(line)
    if e['old_table'] != 'cities':
        continue
    for t in e['current_targets']:
        if t['automatic_redirect']:
            print(e['old_table'], e['old_id'], '->', t['table'], t['id'])
"

It prints the mappings; updating your stored ids is up to you.

Support

When v5.0.0 is published, the API keeps serving v4 data for six months, and v4’s downloads stay available. The release notes give the exact dates.

Where the rules come from

The rules for places, types and hierarchy are in ADMINISTRATIVE_STRUCTURE.md and TYPE_FIELD.md. The rules for disputed and special territories are in DISPUTED_AND_SPECIAL_TERRITORIES.md. Source credits are in .github/licenses/SOURCES.txt.