What is nearest-neighbor site search in portfolio risk scoring?
A nearest-neighbor site search starts from a simple question: given one site in your book, which other sites in your book look most like it? Not which ones sit in the same CRESTA zone, and not which ones share a construction code on the bordereau. Which ones actually resemble it, structurally and texturally, when you look straight down at them.
To do that search you need every site reduced to the same kind of object: a vector. An embedding model reads the imagery for a site and outputs a fixed-length list of numbers that encodes what's going on in that frame: roof geometry, surrounding density, yard layout, the things an underwriter would notice on a site visit but that never make it into a data field. Once every site in the portfolio has one of these vectors, "nearest neighbor" just means finding the vectors sitting closest to each other in that space.
Why cosine similarity, specifically
Distance between two embedding vectors can be measured a few ways, but cosine similarity between site vectors is the one that holds up across a large book. It measures the angle between two vectors rather than the raw distance between them, so it's not thrown off by scale differences the way a straight Euclidean measure can be. Two sites can produce vectors of different magnitude for reasons that have nothing to do with risk (image brightness, slight crop differences, seasonal vegetation) and cosine similarity mostly cancels that out. What's left is a score from roughly -1 to 1 that tells you how aligned two sites are in the features the model picked up on. In practice, a quant pulls the top-k nearest neighbors for a given site by cosine score and treats that as a look-alike shortlist.
Look-alike site detection as a QA step
The most direct use is a sanity check on your own book. Pull a site that had a large loss, run a nearest-neighbor search against the rest of the portfolio, and see what comes back. If the top matches are other sites your model scored as low risk, that's worth a second look, because the imagery is telling you those sites share something structural with a site that already paid out. It surfaces candidates for engineering review out of a book too large to walk site by site, running alongside whatever review process you already run, not instead of it.
Accumulation risk by imagery signature
The less obvious use is accumulation. Geographic accumulation modeling already tells you when too many sites sit inside one peril footprint. What it doesn't tell you is when a large cluster of sites across unrelated regions all share the same underlying construction or site pattern, the kind of thing that wouldn't show up as a geographic concentration at all but would still move together if the driver behind it turns out to be a loss driver. Grouping sites by imagery signature, by running nearest-neighbor search across the whole book rather than against one reference site, surfaces exactly that: clusters of visually similar sites that your current attribute set treats as unrelated.
Where this sits relative to attribute-based scoring
None of this requires replacing the attributes you already feed your model. It sits alongside them. The embedding vector is just one more feature per site, derived from imagery instead of from a form someone filled in, and nearest-neighbor search is one way of using it before you've even decided which dimensions of the vector matter for your specific scoring question. That's the appeal for a quant feeding a model that already has twenty years of attritional loss history behind it: you're not re-architecting the pipeline, you're adding a signal that didn't exist before and seeing if it earns its place.
Getting there without a custom computer-vision build is the part most teams underestimate. Imagery Embeddings delivers exactly that vector per site, keyed to your own site list and refreshed annually, so the nearest-neighbor work starts from a file you already have rather than a model you'd otherwise have to build.
If your book has a loss history worth testing a look-alike search against, that's a reasonable place to start.