What is embedding drift between annual imagery refreshes?

What drift actually means here

When you add an embedding as a feature to a scoring model, you're treating it like any other variable: distance to coast, year built, construction class. The difference is that nobody hand-defined what the embedding measures. It's a vector learned from imagery, and the only real guarantee is that two sites that look similar from above land close together in that space.

Embedding drift is what happens when the same site, same footprint, same physical building, produces a meaningfully different vector between one annual refresh and the next, for reasons that have nothing to do with the site itself changing. A new sensor, a different processing run, an imagery vintage with more cloud cover or a different sun angle, a model retrain somewhere upstream. The vector moves. Your feature moves. If you haven't separated that from real change on the ground, your model is being fed noise and calling it signal.

This matters more for embeddings than for a hand-picked attribute like roof age, because roof age is anchored to something you can audit independently against a permit record. A vector isn't anchored to anything you can read off a label. You can't eyeball one dimension of it and know whether it shifted because the parking lot got repaved or because the pipeline changed underneath you.

A year-over-year stability check worth running

Before this year's refresh goes into the scoring model, run the comparison against last year's file for every site on the book.

1. Cosine distance, not raw difference. Compute cosine similarity between each site's current and prior vector. Most of the book should cluster tightly. Plot the distribution instead of eyeballing individual sites. You're looking for the shape of it, not a single number.

2. Flag the tail, not the whole book. Pull the sites sitting in the lowest band of similarity, the ones whose vector moved the most. On a large book that's a shortlist of a few dozen or a few hundred sites, not a re-audit of everything.

3. Cross-reference against what you already know changed. New construction, demolition, an expansion, a peril event that altered the structure. If a flagged site matches something in your underwriting notes or claims file, the drift is real and the embedding is doing exactly what it's there to do. If a flagged site has no known change on record, that's where you look harder.

4. Check for a systematic shift, not just outliers. If the whole book moves in roughly the same direction between refreshes, the cause sits upstream: a sensor swap, a different imagery provider in that region, a model version bump somewhere in the processing chain. Fix the pipeline before spending time re-checking individual sites one by one.

5. Re-run your calibration on last year's book using this year's vectors before swapping the feature into production. If the coefficient on the embedding feature swings hard without a matching swing in loss experience, hold off and find out where the vector moved before you trust it.

What a clean refresh looks like

Sites change for real reasons, and the embedding is supposed to pick that up: a roof gets replaced, a lot gets paved over, an addition goes up. The check exists to let you say, for each flagged site, whether the movement tracks something real or something upstream. Doing that site by site turns the embedding into a feature you trust instead of one you're quietly hoping holds up through next year's loss review.

If you're building this check from scratch, the hard part usually isn't the math. It's getting two annual vector files that are actually comparable: same dimensionality, same site keys, same provider, so a cosine distance means something instead of comparing apples to a different sensor's oranges. Imagery Embeddings delivers a vector per site, keyed to your own site list, on the same annual cadence each year, so a year-over-year stability check is a join and a distance calculation rather than a reconciliation project.

Worth putting this check on the refresh calendar now, before next year's file lands and you're doing it under deadline instead of on your own schedule.

Get in touch if you want to see what a vector file keyed to your site list looks like before you build the comparison pipeline around it.