Task: Model the geo ip2country store

Table of Contents

This page documents a task in the Move the lookup artefact tables and settle the remaining DQ artefact models story. It captures the goal, current status, acceptance, and any notes or results.

1. Goal

The geo ip2country store and its staging table are generated from one geo model.

2. Status

Field Value
State DONE
Parent story Move the lookup artefact tables and settle the remaining DQ artefact models
Now Nothing.
Waiting on Nothing.
Next Nothing.
Last touched 2026-10-13

3. Acceptance

  • ores.geo has a model for ip2country that generates the store and the staging table.
  • The ip2country populate and the lookup function work as before, and pgTAP passes.

4. Plan

4.1. What is there today

Two hand-written files hold the whole feature. geo_ip2country_create.sql has the store, ores_geo_ip2country_tbl – ip_range int8range not null, tenant_id uuid not null, country_code text not null, no primary key, a GiST index on ip_range and a btree index on tenant_id – and the two functions ores_geo_inet_to_bigint_fn and ores_geo_ip2country_lookup_fn, which finds a country by ip_range @> address. dq_ip2country_artefact_create.sql has the staging table: dataset_id, tenant_id, range_start bigint, range_end bigint, country_code text, with a GiST index on the derived int8range(range_start, range_end + 1, '[)'). The publish function ores_dq_ip2country_publish_fn truncates the store and inserts the derived range. The artefact-type row names ip2country as the artefact type and geo_ip2country_tbl as its target. No pgTAP file covers either table.

ores.geo is a flat component – include/, src/, tests/, no api/core/service split – and it is not in the codegen component catalogue, so compass codegen regenerate --component geo refuses it before any model is read. Registering it is the first step either way.

4.2. What the codegen can and cannot express

The model can already state almost all of it. An * Artefact columns section carries range_start, range_end and country_code, and its key becomes the artefact's staging key. int8range passes through the templates as a column type: every column renders as {{name}} {{type}}, and the type check compares the model's type against the emitted one.

Two things are missing.

  1. No index method. Both index faces render create index ... on "tbl" (columns), and neither can say using gist. The entity's index table takes name, columns, unique, current_only, where_extra and nulls_not_distinct, and the loader reads exactly those; the artefact's index drawer already forwards any property and the template simply does not render one. A using column on the entity index table plus a using clause in both templates is small and reusable.
  2. A range column has no C++ type. A keyed entity gets a domain class, a repository and a protocol, and each needs a C++ type for every column. Nothing maps int8range, so an entity whose key or column is a range cannot render its client surface.

4.3. The decision

The second gap shapes the rest. Either

  • (A) let a model state that it has no client surface, as a profile that withdraws the C++ and TypeScript facets the way artefact-staging-only withdraws the entity's own create script, keeping the store's ip_range int8range column exactly as it is; or
  • (B) keep the range out of the model, storing it the way the staging table already does, as range_start and range_end bigints, and indexing the derived int8range(...) expression in both tables. The lookup predicate becomes that same expression, which PostgreSQL matches to the expression index, so containment lookups stay indexed.

Both keep the two functions working and both need the using change. (B) changes the store's columns and the lookup predicate; (A) leaves the SQL alone and adds a profile, which is more machinery but is also the shape every other SQL-only table will need.

5. Measurements

Every number this task claims about a tree, with the command that produced it and the commit it was measured at. A number without both is not a measurement, because a clean-up record describes a tree and trunk moves under a branch: a consumer count, a gate's component count and a suite's size all change with no code changing, and the reader of a bare number cannot tell which tree it came from. Where the raw output matters, put it in an evidence/ file beside the task and name that file in the Command column.

What Value Command Commit
Hand-written artefact tables left 0 the census loop in Model the hand-written tables that have a codegen shape f86f35ce60
Store columns identical to the hand-written table, plus primary key (ip_range) git diff on the create table body f86f35ce60
Staging table body byte-identical to the hand-written one git diff on the create table body f86f35ce60
Lookup, after compass db recreate 0.0.0.1 and 0.255.255.255 answer GB, 1.0.0.0 is past the half-open end, an address outside every range and an IPv6 address answer nothing compass sql against ores_geo_ip2country_lookup_fn f86f35ce60
Local gate, non-heavy 35 of 35 compass check --exclude-heavy fb84e8bbb3
ctest 93 of 93 compass check --only ctest this branch
pgTAP 52 files, all pass compass check --only pgtap this branch

6. Notes

6.1. The primary key's dependent test, and how it was settled

compass check --only ctest caught what the non-heavy gate cannot: it runs no C++ suite, so the primary key this task added reached a consumer the gate does not see. ores.database.tests failed four unit_of_work_tests.cpp cases with duplicate key value violates unique constraint "ores_geo_ip2country_tbl_pkey". The suite uses ores_geo_ip2country_tbl as a generic probe, inserting '[1,2)' under a fresh random country_code per case to exercise rollback and visibility; the hand-written table was keyless and accepted that.

Settled by keeping the key and moving the probe. The key is honest – a range that appears twice is meaningless – and the suite wants a table of the right shape rather than this table. Each probe now draws its range from the same random source as its code, so no case can collide with another however many are added. The invalid-range probe keeps its literal: it is meant to fail.

ctest is 93 of 93 afterwards, and pgTAP 52 files.

7. Test Scenarios

Manual QA scenarios (scaffolded via compass add test_scenario) that verify this task. Link new ones here as they're created; the scenario doc itself links back via its "Verifies task" field.

Scenario State Notes
     

8. PRs

PR Title
#3316 [geo] Model the ip2country store and generate its staging table

9. Review

Comment summary File Decision Notes
       

10. Result

ores_geo_ip2country_tbl and its DQ staging table both come from ores.geo.ip2country. The artefact table's body is byte-identical to the hand-written one; the store's columns, their order and both of its indexes are identical, and it gains a primary key on ip_range.

Acceptance: ores.geo has the model, and it generates both tables. The publish function is unchanged and still compiles against them; the lookup function and ores_geo_inet_to_bigint_fn moved to a file of their own, unchanged, and answer correctly against a recreated database.

Three things the work needed that the task did not name:

  • ores.geo was not in the codegen component catalogue at all, so no model of it could be read. It is flat, so the model states :no_subcomponent:.
  • The index faces rendered on tbl (columns) and could not say using gist, which is the only index that answers ip_range @> address. Both now take an access method; the entity index table gained a using column.
  • A range has no C++ type, so the store cannot have a client surface. The new SQL-only store profile withdraws one per technical space, which also takes the notify trigger: a per-row pg_notify over a table loaded in bulk is the publish path's whole cost.

The one shape change is the primary key. The current-state template asks a stored table for a key, and a range that appears twice is meaningless, so the key is the range. A dataset carrying the same range twice would now refuse at publish instead of storing it twice, and the ores.database.tests suite, which used the table as a keyless probe, now draws a fresh range per case.

Heavy phase: ctest 93 of 93 and pgTAP 52 files, against a database recreated from this branch.

The staging table's indexes follow the generated naming (dq_ip2country_artefact_*_idx) rather than the hand-written ip2country_artefact_*_idx, and it gains the key index the template emits. Its GiST index is kept as written.

Emacs 29.3 (Org mode 9.6.15)