Task: Model the geo ip2country store
Table of Contents
This page documents a task in the Move the lookup artefact tables and settle the remaining DQ artefact models story. It captures the goal, current status, acceptance, and any notes or results.
1. Goal
The geo ip2country store and its staging table are generated from one geo model.
2. Status
| Field | Value |
|---|---|
| State | DONE |
| Parent story | Move the lookup artefact tables and settle the remaining DQ artefact models |
| Now | Nothing. |
| Waiting on | Nothing. |
| Next | Nothing. |
| Last touched | 2026-10-13 |
3. Acceptance
- ores.geo has a model for ip2country that generates the store and the staging table.
- The ip2country populate and the lookup function work as before, and pgTAP passes.
4. Plan
4.1. What is there today
Two hand-written files hold the whole feature. geo_ip2country_create.sql has
the store, ores_geo_ip2country_tbl – ip_range int8range not null,
tenant_id uuid not null, country_code text not null, no primary key, a GiST
index on ip_range and a btree index on tenant_id – and the two functions
ores_geo_inet_to_bigint_fn and ores_geo_ip2country_lookup_fn, which finds a
country by ip_range @> address. dq_ip2country_artefact_create.sql has the
staging table: dataset_id, tenant_id, range_start bigint,
range_end bigint, country_code text, with a GiST index on the derived
int8range(range_start, range_end + 1, '[)'). The publish function
ores_dq_ip2country_publish_fn truncates the store and inserts the derived
range. The artefact-type row names ip2country as the artefact type and
geo_ip2country_tbl as its target. No pgTAP file covers either table.
ores.geo is a flat component – include/, src/, tests/, no
api/core/service split – and it is not in the codegen component catalogue,
so compass codegen regenerate --component geo refuses it before any model is
read. Registering it is the first step either way.
4.2. What the codegen can and cannot express
The model can already state almost all of it. An * Artefact columns section
carries range_start, range_end and country_code, and its key becomes the
artefact's staging key. int8range passes through the templates as a column
type: every column renders as {{name}} {{type}}, and the type check compares
the model's type against the emitted one.
Two things are missing.
- No index method. Both index faces render
create index ... on "tbl" (columns), and neither can sayusing gist. The entity's index table takesname,columns,unique,current_only,where_extraandnulls_not_distinct, and the loader reads exactly those; the artefact's index drawer already forwards any property and the template simply does not render one. Ausingcolumn on the entity index table plus ausingclause in both templates is small and reusable. - A range column has no C++ type. A keyed entity gets a domain class, a
repository and a protocol, and each needs a C++ type for every column.
Nothing maps
int8range, so an entity whose key or column is a range cannot render its client surface.
4.3. The decision
The second gap shapes the rest. Either
- (A) let a model state that it has no client surface, as a profile that
withdraws the C++ and TypeScript facets the way
artefact-staging-onlywithdraws the entity's own create script, keeping the store'sip_range int8rangecolumn exactly as it is; or - (B) keep the range out of the model, storing it the way the staging table
already does, as
range_startandrange_endbigints, and indexing the derivedint8range(...)expression in both tables. The lookup predicate becomes that same expression, which PostgreSQL matches to the expression index, so containment lookups stay indexed.
Both keep the two functions working and both need the using change. (B)
changes the store's columns and the lookup predicate; (A) leaves the SQL alone
and adds a profile, which is more machinery but is also the shape every other
SQL-only table will need.
5. Measurements
Every number this task claims about a tree, with the command that produced it
and the commit it was measured at. A number without both is not a measurement,
because a clean-up record describes a tree and trunk moves under a branch: a
consumer count, a gate's component count and a suite's size all change with no
code changing, and the reader of a bare number cannot tell which tree it came
from. Where the raw output matters, put it in an evidence/ file beside the
task and name that file in the Command column.
| What | Value | Command | Commit |
|---|---|---|---|
| Hand-written artefact tables left | 0 | the census loop in Model the hand-written tables that have a codegen shape | f86f35ce60 |
| Store columns | identical to the hand-written table, plus primary key (ip_range) |
git diff on the create table body |
f86f35ce60 |
| Staging table body | byte-identical to the hand-written one | git diff on the create table body |
f86f35ce60 |
Lookup, after compass db recreate |
0.0.0.1 and 0.255.255.255 answer GB, 1.0.0.0 is past the half-open end, an address outside every range and an IPv6 address answer nothing |
compass sql against ores_geo_ip2country_lookup_fn |
f86f35ce60 |
| Local gate, non-heavy | 35 of 35 | compass check --exclude-heavy |
fb84e8bbb3 |
| ctest | 93 of 93 | compass check --only ctest |
this branch |
| pgTAP | 52 files, all pass | compass check --only pgtap |
this branch |
6. Notes
6.1. The primary key's dependent test, and how it was settled
compass check --only ctest caught what the non-heavy gate cannot: it runs no
C++ suite, so the primary key this task added reached a consumer the gate does
not see. ores.database.tests failed four unit_of_work_tests.cpp cases with
duplicate key value violates unique constraint "ores_geo_ip2country_tbl_pkey".
The suite uses ores_geo_ip2country_tbl as a generic probe, inserting '[1,2)'
under a fresh random country_code per case to exercise rollback and
visibility; the hand-written table was keyless and accepted that.
Settled by keeping the key and moving the probe. The key is honest – a range that appears twice is meaningless – and the suite wants a table of the right shape rather than this table. Each probe now draws its range from the same random source as its code, so no case can collide with another however many are added. The invalid-range probe keeps its literal: it is meant to fail.
ctest is 93 of 93 afterwards, and pgTAP 52 files.
7. Test Scenarios
Manual QA scenarios (scaffolded via compass add test_scenario)
that verify this task. Link
new ones here as they're created; the scenario doc itself links back
via its "Verifies task" field.
| Scenario | State | Notes |
|---|---|---|
8. PRs
| PR | Title |
|---|---|
| #3316 | [geo] Model the ip2country store and generate its staging table |
9. Review
| Comment summary | File | Decision | Notes |
|---|---|---|---|
10. Result
ores_geo_ip2country_tbl and its DQ staging table both come from
ores.geo.ip2country. The artefact table's body is byte-identical to the
hand-written one; the store's columns, their order and both of its indexes are
identical, and it gains a primary key on ip_range.
Acceptance: ores.geo has the model, and it generates both tables. The publish
function is unchanged and still compiles against them; the lookup function and
ores_geo_inet_to_bigint_fn moved to a file of their own, unchanged, and answer
correctly against a recreated database.
Three things the work needed that the task did not name:
ores.geowas not in the codegen component catalogue at all, so no model of it could be read. It is flat, so the model states:no_subcomponent:.- The index faces rendered
on tbl (columns)and could not sayusing gist, which is the only index that answersip_range @> address. Both now take an access method; the entity index table gained ausingcolumn. - A range has no C++ type, so the store cannot have a client surface. The new
SQL-only store profile withdraws
one per technical space, which also takes the notify trigger: a per-row
pg_notifyover a table loaded in bulk is the publish path's whole cost.
The one shape change is the primary key. The current-state template asks a
stored table for a key, and a range that appears twice is meaningless, so the
key is the range. A dataset carrying the same range twice would now refuse at
publish instead of storing it twice, and the ores.database.tests suite, which
used the table as a keyless probe, now draws a fresh range per case.
Heavy phase: ctest 93 of 93 and pgTAP 52 files, against a database recreated from this branch.
The staging table's indexes follow the generated naming
(dq_ip2country_artefact_*_idx) rather than the hand-written
ip2country_artefact_*_idx, and it gains the key index the template emits. Its
GiST index is kept as written.