GeezerVison is showing github.com — test_indexing_item_not_front test is failing · Issue #258 · openzim/python-scraperlib
test_indexing_item_not_front test is failing #258DescriptionI had to skip this test which is failing for now with most recent libzim:
|
def test_indexing_item_not_front(tmp_path: pathlib.Path, png_image: pathlib.Path): |
|
fpath = tmp_path / "test.zim" |
|
main_path = "welcome" |
|
with Creator(fpath, main_path).config_dev_metadata() as creator: |
|
creator.add_item( |
|
StaticItem( |
|
filepath=png_image, |
|
path="welcome", |
|
title="brain food", # title used for suggestions |
|
index_data=IndexData( |
|
title="screen", content="car" # title and content used for search |
|
), |
|
hints={libzim.writer.Hint.FRONT_ARTICLE: False}, # mark as not front |
|
) |
|
) |
|
assert fpath.exists() |
|
|
|
reader = Archive(fpath) |
|
# "brain" works as a suggestion but "food" doesn't work because since no front |
|
# article is present in the zim file, libzim doesn't create a title xapian index. |
|
# so, when searching suggestion, libzim is fallback to a binary search on the title |
|
# and return only article starting by the query. |
|
# see https://github.com/openzim/libzim/issues/902#issuecomment-2223050129 |
|
assert "welcome" in list(reader.get_suggestions("brain")) |
|
assert "welcome" not in list(reader.get_suggestions("food")) |
|
assert "welcome" not in list(reader.get_suggestions("screen")) |
|
assert "welcome" not in list(reader.get_suggestions("car")) |
|
assert reader.get_search_results_count("screen") >= 1 |
|
assert reader.get_search_results_count("car") >= 1 |
|
assert reader.get_search_results_count("brain") == 0 |
|
assert reader.get_search_results_count("food") == 0 |
This looks like an upstream issue, hopefully only at read time: openzim/libzim#981 Activitybenoit74 commented on May 13, 2025 CollaboratorAuthor More actions We've agreed upstream that this was more a kind of bug / unwanted behavior. I will hence fix the test on scraperlib side. benoit74 commented on May 13, 2025 CollaboratorAuthor More actions I will move this out of scraperlib 5.2.0:
- it is not requiring any code change on the lib itself, only on tests
- we need to test / document / align on more things around all these searches, and this is going to take time ; I would like to be sure about the full-text search behavior when non-front
text/html article is passed for instance
benoit74 commented on Oct 20, 2026 CollaboratorAuthor More actions If someone wants to clarify this issue around full-text search behavior by fixing the failing test and adding more e.g. when non-front text/html article is passed, you're welcomed. benoit74 commented on Oct 20, 2026 CollaboratorAuthor More actions nota: I feel like sufficient unit tests can cover current libzim behavior and we can consider again that this is the spec for the time being ; if libzim behavior changes, then we can report it upstream and decides where the bug is. rgaudin commented on Oct 21, 2026 Yes, I think the reasonning was that we are exposing our own API shadowing part of the libzim but I agree we should consider libzim as a dependency and only test our code. mcwanza commented on Nov 7, 2026 I'd like to take this one, following the last comments: treat libzim's current behavior as the spec. Plan: un-skip test_indexing_item_not_front and assert what libzim 3.13 does today (no suggestions for a non-front item, full-text search still matches), plus two tests for a non-front text/html item, one with custom IndexData and one with libzim's own indexing. Tests only, no library change. Could you assign it to me?
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|