Null is not a measurement

I added attribution to my product on 8 August: five columns on the profile recording where each new user came from. Source, medium, campaign, referrer, country. Written at signup, first touch only, never overwritten.

It had tests. It passed review. It shipped.

Ten days later I ran the obvious query for the first time:

select coalesce(signup_country, '?') as country,
       coalesce(signup_source, 'direct') as source,
       count(*) as signups
from profiles
group by 1, 2;

One row back. ?, direct, 12.

My first thought was that twelve users is not many and the data would fill in. That thought was wrong in a specific way I want to describe, because I think it is common.

Zero rows and "not yet" look identical

A wrong value announces itself. If the column had said US for a user in Kyiv, I would have gone looking within a minute.

An empty column says nothing at all, and the mind supplies the friendliest explanation available: the feature is new, traffic is small, give it time. That explanation is always available, it is often true, and it is unfalsifiable without going and checking. So the check does not happen.

What made me check was noticing that some of those twelve had signed up after the feature shipped. So I split the question:

select coalesce(u.raw_app_meta_data->>'provider', 'email')  as provider,
       count(*)                                             as users,
       count(*) filter (where p.created_at >= '2026-08-08') as after_it_shipped,
       count(*) filter (where p.signup_country is not null) as with_country
from auth.users u
join profiles p on p.id = u.id
group by 1;

Six users had gone through the code after it existed and produced nothing. That is not a sample-size problem. That is code that does not run.

Where it did not run

Attribution was written in the signup API route. Email signups POST to it. OAuth does not — it talks to the auth provider directly and comes back to a callback page, which knows nothing about any of this.

Three quarters of my signups are Google.

The comment explaining this was already in the callback file, written by me, weeks earlier:

OAuth sign-ups skip /api/auth/signup, so the Telegram alert is sent from here.

I had known about the fork. I had handled it for the notification. When I later added attribution, I added it to the route — the place where signup "obviously" happens — and did not think about the fork again, because nothing made me. The tests I wrote called the route. Of course they did; that was the code I had written.

A test suite covers the paths you thought of, and so does your memory of the architecture. The fork was documented four files away, in a comment I wrote myself, and it did not help.

The second bug, which was worse

Fixing the OAuth path, I went to check where the capture happens in the browser. captureAttribution() — the function that stores the referrer on arrival — ran on exactly two pages: the landing page and the register form.

Those are not where people arrive.

My entire content strategy points search traffic at role-specific question pages and blog articles. Somebody who finds an article in Google lands there, reads it, and clicks through. captureAttribution() never ran, because those pages are server components with no place to call it. By the time they reached the register form, the only referrer available was our own domain — and the function discards internal referrers, correctly, because they are not sources.

So the visitor was recorded as direct.

That is worse than null, and it is worth being precise about why. Null is missing data, and missing data mostly gets ignored. direct is an answer. It says: this person typed your domain in, nobody sent them. Stack up enough of those and the report says search brings you nobody — which is the exact opposite of the truth, and it is the kind of conclusion that gets acted on. You do not audit a number that answers your question.

Both bugs produced silence. One produced a lie.

The fixes

The OAuth path now posts to a small route from the callback. Not awaited before the redirect — knowing where somebody came from is worth nothing next to letting them in.

The condition lives in the write, because the callback runs on every sign-in, not only the first:

await supabaseAdmin
  .from('profiles')
  .update(values)
  .eq('id', user.id)
  .is('signup_source', null)     // first touch wins
  .is('signup_country', null)
  .select('id')

Reading the column first and then writing would let two callbacks both see null and both claim it. The same shape as a payment webhook race I wrote about earlier — check-then-act is check-then-act wherever it appears.

The capture moved to the root layout, so it runs wherever the visitor lands, including on pages written next year.

And the migration moved. The five columns lived in docs/migrations/, which nothing replays — applied by hand to production and absent from a fresh database, where the signup insert would have failed outright. I have a test that reads every table referenced in the source and requires a migration to create it. It checks tables. It says nothing about columns.

What I take from this

Absence of data is not evidence of anything. It is exactly as consistent with "no traffic" as with "no code", and the two demand opposite responses. The only way to tell them apart is a query that would look different in each case — here, splitting by signup provider and by date, which took thirty seconds and was ten days late.

When a feature reports on itself, it needs a first read early. Not a test — a look at the real table, on the day it ships, with the question "does this contain anything at all". I would have found both bugs on 9 August.

A number that answers your question deserves more suspicion than one that does not. direct fitted a story I could believe. Null did too, in its way. The check I actually ran was prompted by a small inconsistency — some of these users are newer than the feature — and that is usually how it goes.

None of this is exotic. It is the ordinary way instrumentation fails: quietly, in the direction of looking fine.