LSM: bounded range scans over multi-series compacted indexes drop rows and mis-order descending results #5214

Summary

Bounded range scans over an LSM index that has compacted into 3+ series silently drop rows, and the descending bounded iterator returns results in the wrong order. Data on disk is intact (a full scan returns everything); only the bounded/range read paths over the multi-series compacted layer are affected.

Multiple compacted series in a single file are produced when a compaction is RAM-bounded (INDEX_COMPACTION_RAM_MB), i.e. on large indexes whose immutable pages exceed the compaction budget. Small indexes compact into a single series, where these paths happen to work, which is why this has stayed latent.

This is independent of #5211 (which fixed only the unbounded full descending scan).

Reproduction

Non-unique LSM index over a String property, 20,000 keys, INDEX_COMPACTION_RAM_MB=1 so one compaction emits several series:

-- correct
SELECT count(*) FROM Doc;
-- => 20000

-- BUG: range query drops ~2,500 rows
SELECT count(*) FROM Doc WHERE email >= 'K...00005000' AND email <= 'K...00015000';
-- => 7514   (expected 10001)

At the index API level:

Root causes

1. Interior series dropped from bounded scans - LSMTreeIndexCompacted.newIterators(ascending, fromKeys, toKeys)

For a bounded scan each series only yields a cursor when fromKeys or toKeys falls inside that series (searchInCurrentPage returns null when resultInRootPage.outside). A series lying entirely within [lo, hi] - containing neither endpoint - fails both lookups, contributes no cursor, and all of its keys are lost. A wide range spanning many series drops every interior series.

2. Descending bounded iterator returns ascending order - LSMTreeIndexMutable.iterator(false, fromKeys, inclusive)

It delegates to the auto-detecting range(null, true, fromKeys, inclusive). With the begin-key null, the direction auto-detect defaults to ascending = true, so a descending request iterates ascending. Masked today because existing tests assert only element counts, never order.

3. Latent NPE on a null bound in the descending path

The straightforward fix for (2) - range(false, fromKeys, inclusive, null, true) - surfaces a NullPointerException (Cannot read the array length because "convertedKeys" is null) when the lower bound is null on a descending scan. Needs handling as part of the fix.

Impact

• Range predicates (>=/<=/BETWEEN) and descending ordered index reads over large (multi-series-compacted) indexes can return incomplete or mis-ordered results.
• Correctness only; no data is lost on disk.

Proposed fix

1. In newIterators, emit a full-series cursor when a series lies wholly inside [fromKeys, toKeys] (both endpoints outside but the series is within range) instead of dropping it.
2. Route iterator(false, fromKeys, …) through an explicit-direction descending range, and fix the null-bound NPE it exposes.
3. Add regression tests asserting completeness and order (not just counts) for multi-series bounded and range scans, both directions, inclusive/exclusive bounds.

Found while adding follow-up coverage for #5211.