Skip to content

huff0: Allow building tables from histogram. - #1155

Merged
klauspost merged 2 commits into
masterfrom
huff0-buildtables
May 21, 2026
Merged

klauspost merged 2 commits into
masterfrom
huff0-buildtables

Conversation

@klauspost

@klauspost klauspost commented May 21, 2026 •

Copy link
Copy Markdown
Owner

Add interface to build tables from the histograms themselves.

Adds direct method to export the current table and helpers for size reuse (also via histograms).

Summary by CodeRabbit

  • New Features

    • Build Huffman compression tables from symbol frequency data
    • Estimate compressed size and validate table compatibility with input histograms
    • Serialize and append compression tables for reuse and transfer between encoder/decoder
  • Tests

    • Comprehensive tests covering table building, error cases, estimation/eligibility, serialization roundtrips, and end-to-end compression/decompression workflows

Review Change Stack

Add interface to build tables from the histograms themselves.

Adds direct method to export the current table and helpers for size reuse (also via histograms).
@coderabbitai

coderabbitai Bot commented May 21, 2026 •

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 3853e93b-44a3-4c67-925f-9f3648608d70

📥 Commits

Reviewing files that changed from the base of the PR and between 7e29a67 and 6382a30.

📒 Files selected for processing (2)
  • huff0/build_table.go
  • huff0/build_table_test.go

📝 Walkthrough

Walkthrough

This PR adds table-building capabilities to the huff0 Huffman compression codec by introducing four methods to Scratch: BuildCTable constructs tables from histograms with validation, EstimateSize and CanUseTable probe table viability against histograms, and AppendTable serializes tables to self-delimiting headers. Comprehensive tests validate basic cases, error handling, table reuse in compression workflows, serialization roundtrips, and payload consistency.

Changes

Huffman Table Building and Serialization

Layer / File(s) Summary
Core table-building API and basic validation
huff0/build_table.go (lines 17–136), huff0/build_table_test.go (lines 1–78)
BuildCTable validates histogram input and constructs Huffman tables, rejecting single-symbol and uniform cases via ErrUseRLE; EstimateSize and CanUseTable provide table-based probing returning -1 and false when encoding is impossible; AppendTable serializes prevTable to a self-delimiting header with lazy FSE state initialization. Tests validate basic success, empty/nil histogram error handling, and single-symbol ErrUseRLE.
Table integration and roundtrip tests
huff0/build_table_test.go (lines 80–259)
EstimateSize and CanUseTable reject foreign histograms; end-to-end roundtrip builds table, transfers to encoder with ReusePolicyMust, compresses with verified table reuse (no emitted table bytes), then deserializes and decompresses using ReadTable and Decompress4X to recover original input; AppendTable header roundtrips preserving non-zero symbols; payload-consistency test ensures reused-compression payload matches reference non-reuse payload.

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: adding functionality to build Huffman tables from histograms, which is the core feature introduced in both the new build_table.go file and its comprehensive test suite.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch huff0-buildtables

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@huff0/build_table.go`:
- Around line 66-73: The EstimateSize(hist *[256]uint32) int method (and the
companion probe API that also accepts hist *[256]uint32 around lines 86-92)
currently dereference and iterate hist without a nil guard; add an early nil
check in both functions and return the appropriate sentinel (for EstimateSize
return -1; for the probe API return false) before any iteration or access to
hist to avoid panics when callers pass nil.
- Around line 17-27: BuildCTable currently dereferences the pointer parameter
count (s.count = *count) without checking for nil; add a nil guard at the start
of BuildCTable to return a clear error (e.g., errors.New("huff0: nil count
passed to BuildCTable")) when count == nil before any dereference or use. Locate
the function BuildCTable and insert the check before s.count = *count (and
before any other use of count) so callers passing nil receive a proper error
instead of a panic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: c8610f78-59b3-4b57-9f35-3011db506c49

📥 Commits

Reviewing files that changed from the base of the PR and between 38efa45 and 7e29a67.

📒 Files selected for processing (2)
  • huff0/build_table.go
  • huff0/build_table_test.go

Comment thread huff0/build_table.go Resolved
Comment thread huff0/build_table.go Resolved
@klauspost
klauspost merged commit ecdb779 into master May 21, 2026
22 checks passed
@klauspost
klauspost deleted the huff0-buildtables branch May 21, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant