Use storedFieldsSpec to load stored fields for highlighting #91841
Conversation
|
Pinging @elastic/es-search (Team:Search) |
|
Hi @romseygeek, I've created a changelog YAML for you. |
…ds' into highlight/report-stored-fields
|
@romseygeek just found this sitting in my queue of PRs assigned for review and wanted to say sorry this fell through the cracks. Seems stale at the moment, do you think it should get updated or is it better to close at this point? |
|
Still worth doing, I think. I'll update. |
|
@cbuescher this is ready for review again when you get time. |
There was a problem hiding this comment.
I took a look and left two questions. Mostly I'm trying to understand through which part of this change unified and plain highlighters make use of the pre-loading. I think I found the spot but want to make sure I understand this right. Also not sure how difficult it would be to test that the optimization actually gets used.
|
|
||
| Map<String, HighlightField> highlights = highlight(mapperService, doc, search); | ||
| assertHighlights(highlights, "field", "this is <em>some</em> text"); | ||
| } |
There was a problem hiding this comment.
These tests already pass without the modifications in this PR. Do we have other ways to verify that the new optimization is actually used, e.g. check frequency of stored field readers access? Some asserting mocks etc?
There was a problem hiding this comment.
That's interesting, I think something has happened since I opened the PR that makes these pass. But we can definitely add checks to the highlight method that asserts we only load stored fields once per document. I will work on that.
| hitContext.reader().document(hitContext.docId(), fieldVisitor); | ||
| List<Object> textsToHighlight = fieldVisitor.fields().get(fieldType.name()); | ||
| return Objects.requireNonNullElse(textsToHighlight, Collections.emptyList()); | ||
| List<Object> values = hitContext.loadedFields().get(fieldType.name()); |
There was a problem hiding this comment.
To check my understanding: is this the part where we actually save having to open stored field readers more than once? Does the hitContext now always already contain the loaded fields?
Since #91269, fetch phase subprocessors can report any stored fields that they
need back to the top-level FetchPhase so that all stored fields can be loaded
up front. This commit switches the unified and plain highlighters to use this
functionality so that highlighting does not need to open stored field readers
twice.