Myles MellorCommercial, digital & marketing operator
← Notes

18 May 2026

When retrieval is wrong, change the structure, not the knobs

A run of failed experiments on a search system taught me to reach for the structural fix before the tuning dial. Weights lift noise as fast as signal.

I spent a few sessions trying to fix one stubborn query in a retrieval system. The right document existed; it just wasn't ranking. The obvious move was to tune the scoring — boost headings, weight certain sections, adjust the blend.

Every global weight change I tried lifted noise about as much as it lifted signal. A tweak that rescued the broken query quietly regressed three that already worked. Two separate scoring experiments came back negative and got reverted.

What actually worked was structural: typing each chunk of content by the kind of section it came from, so the system could tell a heading from a body paragraph from a list. No weights moved. The fix held first try, with no regressions.

The lesson I keep: a global knob applies your change everywhere, including the places that were already right. A structural change is targeted — it adds information rather than re-weighting what's there. When something retrieves badly, I now look for the missing structure before I touch a dial.