ArcadeStateMachine.takeSnapshot() reports a compaction index to Ratis but never writes a snapshot file — log purge can permanently orphan applied state #4829

Affected version: 26.6.1
Component: ArcadeStateMachine.takeSnapshot / reinitialize
Severity: HIGH

Summary

takeSnapshot() returns lastAppliedIndex — the Ratis contract for "state up to
this index is durable, you may purge the log" — but writes no snapshot file.
SimpleStateMachineStorage.getLatestSnapshot() discovers snapshots by scanning for
snapshot.<term>_<index> files, so it always returns null. With auto-snapshot +
purgeUptoSnapshotIndex enabled, Ratis purges log entries up to the returned
index even though no snapshot exists.

Code

ArcadeStateMachine.java:360-367 (return currentIndex; no file written),
:189-225 (reinitialize → getLatestSnapshot() null → lastAppliedIndex=-1);
RaftPropertiesBuilder.java:84,89-95 (auto-trigger + purge enabled).

Impact

After log purge + restart, the node restores lastAppliedIndex = -1 and Ratis
replays from the start of a log whose entries were purged → cannot self-recover,
forced into full snapshot download or divergence. The reinitialize gap-detection
branch is effectively dead code since getLatestSnapshot() is always null. The
hand-rolled .raft/applied-index is used only for comparison, never to seed
lastAppliedIndex.

Suggested fix

Persist a real snapshot marker file via storage at (term,index) in
takeSnapshot(); or seed lastAppliedIndex from readPersistedAppliedIndex()
when getLatestSnapshot() is null (and make that file durable — see HA-27);
otherwise do not enable purgeUptoSnapshotIndex.