Part 3 of 7 · 6 min

A closed list of things that can happen to a value

A mapping is not just a pair of columns. It is a claim about what happens to a value in flight. Part 3 of the Proof Series: eight operations, a decode the platform computes, and the escape hatch that can never confirm itself.

Part 2 left Danubia's supplier master in a named state. One table pair confirmed by a delivery lead. Eighty-four columns open, each with ranked candidates waiting on the suggested tier. The afternoon of column work can begin.

Here is what that afternoon actually decides, and it is more than which column goes where. A supplier number can cross unchanged. A date crosses through a format change. A status code crosses through a translation table that maps the legacy codes to the target's domain. Every field mapping is a pair plus a transformation, and the transformation is where migrations quietly go wrong.

So before the first column is confirmed, one design question decides how much the eventual proof will be worth. What is a transformation allowed to be?

Eight verbs and no ninth

In most projects, the answer is free text. The mapping spreadsheet has a column called transformation logic. It holds whatever the analyst typed: convert as needed, see lookup tab, handle nulls per Radu's email. Every row is a small essay, and no machine can check an essay.

Our answer is a closed list. A value that crosses from source to target may undergo exactly eight operations. It copies unchanged, changes type, or converts units. It translates through a lookup, joins with another field, or splits into parts. It takes a default when empty. Or it passes through free-form logic.

The first seven are deterministic templates. Given the same input, they produce the same output, every time, and a machine can execute and verify them. The eighth is the escape hatch, and the whole section after next is about its cage.

The reason the list is closed is the reason this series exists. A transformation from a closed grammar can be compiled, run against real rows, and checked. The reconciliation in part 5 is derived from the confirmed map, and that derivation is only possible because every entry in the map is machine-readable. An essay in a spreadsheet cell can be right, but nothing can prove it. A closed verb can be wrong, and the tie-out will catch it.

The decode the platform computes

Now the column this series keeps returning to. Danubia's status column holds one character, eleven distinct values, and no data dictionary. The target field takes an enumerated domain with a validation rule. Somebody has to say that legacy code A means active and legacy code R means retired, eleven times, correctly.

The traditional method is archaeology. Somebody finds a 2009 screen printout, interviews the two people who remember the old order entry flow, and types a translation table by hand. The table is authored, which means it says whatever its author believed.

The platform computes its proposal instead. It profiles the values on both sides and aligns them by how they distribute. The code that covers most supplier rows on the legacy side pairs with the status that covers most suppliers in the target's world. The alignment continues down the frequency order. Out of that alignment comes a proposed lookup, pair by pair, sitting on the suggested tier like every other machine product in this series.

The boundary rides in the same breath, twice. The alignment works from a sample, so a rare code that never entered the sample gets no proposed pair rather than an invented one. And a computed proposal is still a proposal. A person confirms the decode pair by pair, and the confirmation is what the map carries forward. What the computation removes is not the human decision. It is the blank page, and the archaeology.

The escape hatch that cannot confirm itself

One of Danubia's columns defeats all seven templates. The legacy system stored a credit limit in a currency that no longer exists, with a divisor buried in an old configuration table. Getting it across takes real logic: arithmetic, a join, a condition. No closed verb covers it.

For that column there is the eighth operation: free-form logic, written out, attached to the mapping. The escape hatch exists because real estates contain cases no grammar anticipates, and a grammar that pretended otherwise would push those cases back into spreadsheets.

But the hatch is caged, permanently, by two rules. Free-form logic is born on the suggested tier and can never confirm itself. No threshold promotes it, no automation ratifies it, and only a person who read the logic can confirm it. And free-form logic never counts as deterministic, however simple it looks, because nobody can verify by name what somebody wrote by hand.

The same cage holds a small set of remediation operations: masking a value, regenerating it, carrying it frozen. Each of those is a design decision about data, and no name similarity can derive a design decision. They are authored by people only.

The reasoning deserves one plain sentence. The moment free-form logic can promote itself, the closed grammar is decoration, because every hard case would arrive pre-approved through the hatch.

The map at the end of the afternoon

Count what Danubia holds now, because the standing changes in this part.

Most of the eighty-four columns carry confirmed template operations: copies, casts, a handful of defaults. The status column carries a confirmed lookup, eleven pairs, each one a person's yes over a computed proposal. The credit limit column carries free-form logic that a reviewer read and confirmed by hand. Every confirmed entry is machine-readable, and part 5 will hold the whole map to account with it.

One column carries nothing. The legacy retention flag has no defensible target anywhere in the new system. The matcher ranked its candidates, and every one of them means something else. The temptation is to map it to the least-bad option and move on, because an unmapped column looks like unfinished work.

What the platform does with that column instead is the strongest idea in this series, and it gets the next part to itself.

Part 4: The column that gets no target. Why the matcher refuses to force a mapping, and why an unmapped column must be decided by a person rather than absorbed by a deadline.

Model your first domain today.

You send five documents, we model them, and the first cut comes back in days.

This site uses cookies

We use essential cookies for the site to function and analytics cookies (Google Analytics) to understand how you use it. Analytics cookies are only activated with your consent. We do not track you across other websites. Your data is stored in the EU and processed in accordance with GDPR. Read our Privacy Policy