Skip to content

Issue #768 extract URIs nested inside JSON arrays - #771

Merged
ato merged 1 commit into
internetarchive:masterfrom
frmoretto:fix-extractorjson-array-traversal
Sep 22, 2026
Merged

ato merged 1 commit into
internetarchive:masterfrom
frmoretto:fix-extractorjson-array-traversal

Conversation

@frmoretto

Copy link
Copy Markdown
Contributor

Fixes #768.

parse() called propertyStream() on array nodes, which Jackson returns empty
for anything that is not an object, so URIs inside an array held as an object
property value were never extracted. Those array elements now go through the same
value handling as object values, which is also what lets a bare URI string inside
an array be extracted and not only an object holding one.

A known limit this does not change: parse() still starts from
rootNode.properties(), so a document whose root is itself an array yields no
links.

valueStream() rather than elements(): both iterate the elements of an array,
and neither is deprecated in the 2.22.2 Jackson this builds against, so the
choice is one of style β€” valueStream() matches the 2.19-style node API this
file already uses, properties() and the propertyStream() being replaced here.

The extractor already recurses through nested objects without a depth guard.
This patch applies the same recursive approach to arrays; adding a depth limit
is outside this defect fix.

ExtractorJsonTest covers it: the object-property case passes before and after
as a control, and the two array cases go from failing to passing. Full modules
suite, JDK 17 on Ubuntu 22.04 with LANG=C.UTF-8: 254 run, 2 failures, 1 skipped
before the change; 254 run, 0 failures, 1 skipped after. The single skip
(FetchHTTPTest) is the same in both runs.

Built and measured against abd17fa.

@ato
ato merged commit ed2805f into internetarchive:master Sep 22, 2026
3 checks passed
@ato

ato commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ExtractorJson does not extract URIs nested inside JSON arrays

2 participants