LZ4 v1.11.0 is a major update that integrates 850+ commits since v1.10.0. Its headline feature is multithreaded compression, which is now directly integrated into the liblz4 library. This version is also substantially faster on small data, especially when using dictionaries, and includes a large number of fixes for corner cases scenarios. liblz4 remains binary compatible with previous versions, and upgrading is recommended.

Multithreaded compression in liblz4

v1.10.0 introduced multithreaded compression in the lz4 command line, but applications using liblz4 were not benefiting from it. Starting with v1.11.0, multithreading is part of the lz4frame API: setting a single new field, LZ4F_preferences_t.nbWorkers, is enough to compress with multiple threads, in one-shot or streaming mode.

Speed scales remarkably linearly with the number of workers. The following table shows LZ4F_compressFrame() compressing silesia.tar (202 MB) on an AMD EPYC Genoa server, compiled with gcc 11:

Given enough threads, one can now hope compressing enwik9 in less than a second at max level...

The compressed output only depends on the data and on compression parameters: it doesn't change with the number of workers, nor with the scheduling of threads, so results are reproducible from one system to another.

The lz4 command line now relies on this capability, and also gains a --memory option, to keep multithreaded compression within a memory budget.

The lz4 command line is also faster than v1.10.0. With enough threads, it reaches a ceiling, set by the work which remains sequential in the main thread: mostly the frame's content checksum (XXH32), computed over the whole input, but also reading and writing data. v1.11.0 pushes this ceiling further. Compressing silesia.tar at level 1, wall time, on a consumer desktop (Intel Core i7-9700K, 8 cores, gcc 13) and on a datacenter server (AMD EPYC Genoa, gcc 11):

This ceiling is reached with about 4 threads on the desktop, at ~2.6 GB/s (v1.10.0: ~2.2 GB/s), and with about 8 threads on the server, at ~3 GB/s (v1.10.0: ~2.1 GB/s). Without the content checksum (--no-frame-crc), v1.11.0 keeps scaling: ~4.2 GB/s with all 8 cores of the desktop (v1.10.0: ~3.3 GB/s), and ~6 GB/s with 24 threads on the server (v1.10.0: ~2.7 GB/s at best).
Requesting more threads than useful also wastes fewer resources: v1.11.0 only starts worker threads when jobs are waiting for them, and no more than the file has jobs. With 16 threads on the 8-core desktop, or all 166 threads of the server, its speed doesn't change, while v1.10.0 slows down. Getting the best performance requires less tuning.
These performance gains extend to higher levels, where, given enough threads, the same limits will be reached.

Multi-threading support is enabled by default in the dynamic library. The static library remains single-threaded by default, so that it doesn't require linking with -pthread: it still accepts nbWorkers, and produces the same output, with a single thread. nbWorkers is labelled "experimental" in this release, giving it time to collect feedback before it is promoted to stable status.

Faster on small data, and with dictionaries

Many applications compress data in small pieces, such as database pages, cache entries, cold pages, network messages or log records. For them, the cost of preparing each compression matters. This version makes it much cheaper, in particular when reusing a compression state, or when compressing with a dictionary.

silesia.tar cut into 4 KB blocks, 64 KB dictionary, level 1 unless noted, AMD EPYC Genoa. Speeds with gcc 11, improvements with gcc 11 and clang 22:

Gains are largest for blocks of 4 KB to 16 KB, fade with larger blocks, and depend on the compiler:

These gains reward the recommended practices for small data: reusing states and contexts, and employing dictionaries. For example, a reused state kept the entries of previous compressions in its tables, which the compressor then had to filter out at each step: v1.11.0 clears them instead, which is cheaper from 1 KB blocks.
Measured functions: LZ4_resetStream_fast() + LZ4_compress_fast_continue(), LZ4F_compressBegin() / LZ4F_compressUpdate() / LZ4F_compressEnd() with a reused context, LZ4_attach_dictionary(), LZ4F_compressBegin_usingCDict(), LZ4_attach_HC_dictionary(), LZ4F_decompress_usingDict().

Other notable performance improvements:

• Decompression is faster on aarch64, by up to +19% on Neoverse-V2, thanks to @Nicoshev.
• Level 2 compresses a bit better (~1%) thanks to a better search algorithm
• LZ4 is quite sensible to the "JCC erratum", a security fix which slows down loops depending on where the compiler places their branches, which is notably present on Intel Skylake family. On these cpus, LZ4 speed used to vary a lot, from one compiler version to another. v1.11.0 builds now avoid this penalty: on a Core i7-9700K, decompression now reaches ~5 GB/s with all tested compilers, instead of varying between 3.7 to 4.8 GB/s.
• Compression ratio is slightly better at level 1 (on average). This comes from a fix, where periodic data could trap fast levels into a chain of short matches. Reported by @martinus: : a 4 KB page compressed with an 8-byte dictionary could produce 1543 bytes instead of 49. now fixed.

Reliability

LZ4 runs at a scale where even rare corner cases eventually happen, so this release invested heavily in testing. A new unit test suite checks the documented behavior of the API, contract by contract. Mutation testing, which deliberately introduces bugs into the code to verify that some test catches them, made it possible to find additional coverage space for new tests. Fuzzing now also covers streaming frame compression, the lz4file API and multithreaded compression. And every change is validated by more than 100 CI jobs.

This effort found and fixed a number of bugs. Many of them concern invalid arguments, such as negative sizes, NULL pointers or extreme parameters, which no longer crash nor overflow, with contributions from @arthuz-th, @jaypatrickhoward, @louislafosse, @akashdeshmukh-git and @rinsuki. A few could produce incorrect output in rare
streaming scenarios, such as decoding into a ring buffer of minimal size, or resuming LZ4_compress_HC_continue_destSize() after a partial consumption, reported by @freedom-winds. For the ring buffer case, updating the library is enough: applications compiled with older headers don't need to be recompiled.

Errors are also easier to diagnose. When decompression fails, LZ4_decompress_safe() now tells why: malformed input (-1), destination buffer too small (-2), or missing dictionary (-3). And the lz4file API reports truncated files and write failures, by @axiqia.

The lz4 command line handles files more safely: it refuses to write over its own input, detects a source file modified during compression, and --rm cannot delete the source when an operation fails, thanks to @Furan666, @Nexory and @immanuwell.

Miscellaneous

• lz4 can now decompress Firefox's mozlz4 files, as proposed by @UnitedMarsupials
• Random access within frames: LZ4F_resumeFrame() resumes decoding from any block of a frame, using information saved during a previous pass, and the lz4file API gains LZ4F_seek() and LZ4F_tell(). These functions are available in the experimental section of the API.
• When LZ4 is included as a cmake subproject, the lz4 program is no longer built by default, by @crueter. cmake also exports a lz4::lz4 target, by @dg0yt, and supports CMake 4, by @GeorgH93.
• Functions deprecated since v1.7 and v1.8 are no longer declared in headers (the library still exports them), and LZ4F_compressOptions_t must now be zero-initialized.

Changes list

Here is a more detailed list of updates introduced in v1.11.0 :

• fix : data corruption in valid usage : ring buffer decoding (LZ4_decoderRingBufferSize(), since v1.9.0), rare cases of streaming or destSize compression, one reported by @freedom-winds
• fix : single-thread lz4 could truncate the decoding of concatenated frames with block checksums
• fix : multithreaded decompression could write out of bounds after 2^31 frames (since v1.10.0), by @arshsmith1
• fix : LZ4F_compressBegin_usingDict() ignored the dictionary with independent blocks
• fix : some periodic data compressed up to 30x worse at fast levels, and very slowly at level 12, reported by @martinus
• fix : invalid arguments (negative sizes, NULL, invalid blockSizeID, extreme levels) no longer crash nor overflow, by @arthuz-th, @jaypatrickhoward, @louislafosse, @akashdeshmukh-git, @rinsuki
• fix : lz4file reports truncated files and write failures, and accepts files < 19 bytes, by @axiqia
• perf : faster decompression on aarch64 and x86-64 (up to +19%, +4% on silesia.tar), LZ4_FAST_DEC_LOOP on aarch64 with clang and on riscv64, by @Nicoshev and @arpadpanyik-arm
• perf : faster compression of small blocks at fast levels, with reused states (up to +80%) or dictionaries (up to +35%), partly proposed by @BiplabRaut
• perf : faster levels 11 and 12 (+8% to +27%), level 2 compresses ~0.8% better
• perf : x86 builds avoid the JCC erratum : up to +12% compression and +31% decompression on Intel Skylake-family cpus
• api : multithreaded lz4frame compression (experimental LZ4F_preferences_t.nbWorkers) : output independent of the nb of workers and of the context's history
• api : random access within frames and lz4 files : LZ4F_resumeFrame(), LZ4F_seek(), LZ4F_tell() (static)
• api : LZ4_decompress_safe() returns distinct error codes : -1 malformed input, -2 dstCapacity too small, -3 missing dictionary
• api : lz4frame compression accepts any dstCapacity, and only fails if the output doesn't fit
• api : removed from headers : functions deprecated since v1.7.x / v1.8.0 (still exported by liblz4), unprefixed lz4frame enums ; LZ4_resetStream() and LZ4_resetStreamHC() are deprecated
• api : LZ4F_compressOptions_t must be zero-initialized ; reserved fields are scalars (flatten positional initializers) ; LZ4F_uncompressedUpdate() deprecated (use .uncompressed)
• api : custom allocators for lz4file (_advanced() functions), LZ4_USER_MEMORY_FUNCTIONS also for lz4frame and lz4file ; LZ4F_cctx_size() and LZ4F_dctx_size() by @felixhandte
• api : HC states include the buffer of levels 10+ (+64 KB per state) : no allocation during compression
• cli : multithreaded compression relies on liblz4 : same output with any nb of threads and without multithreading support ; -T# within a memory budget (--memory=#) ; new --single-thread
• cli : decodes Firefox's mozlz4 files, proposed by @UnitedMarsupials, and legacy frames with the Linux kernel end marker, requested by @benjarobin (frame format documentation by @Artoria2e5)
• cli : safer file handling : an output identical to the input is refused, a source modified during compression is detected, --rm keeps the source after failures, by @Furan666 and @Nexory
• cli : benchmark (-b) measures frames, compressed as lz4 does with the same options
• cli : various fixes (--list, legacy format, benchmark, -t -m, numeric arguments), by @tansy, @jmestwa-coder, @akashdeshmukh-git, @arshsmith1, @jlee303 and @Nexory
• build: dynamic library multithreaded by default, static library single-threaded ; cmake lz4 multithreaded
• build: cmake : exported lz4::lz4 target by @dg0yt, CMake 4 support by @GeorgH93, CMAKE_POSITION_INDEPENDENT_CODE by @ds-sloth
• build: cmake : as a subproject, the lz4 program is no longer built by default (-DLZ4_BUILD_CLI=ON restores it), by @crueter
• build: reproducible static library by @rpurdie, AIX support by @shubhamhii, meson fixes by @grobian, @bgilbert and @eli-schwartz, other fixes by @hotschmoe, @iwamatsu, @doraorak, @JCash and @brainy
• port : RISC-V with Zicclsm uses unaligned memory accesses, by @Polaris-911

Auto-generated liste of changes

• Add an lz4::lz4 target to the exported package by @dg0yt in #1470
• Fix stack alignment of _start() in freestanding test by @hhoffstaette in #1468
• remove "deploy" paragraph from Appveyor CI script by @Cyan4973 in #1471
• docs: update lz4conf.h by @eltociear in #1475
• Fix file size check in LZ4F_readOpen by @axiqia in #1472
• Fix building freestanding test with clang & enabled stack protector by @hhoffstaette in #1469
• Bump ossf/scorecard-action from 2.3.3 to 2.4.0 by @dependabot[bot] in #1482
• -h only documents -T# when multithreading is enabled by @Cyan4973 in #1481
• add back lz4c target but default to OFF by @heitbaum in #1479
• fix process time counter on Windows for very large files by @Cyan4973 in #1486
• slighly better cpu time measurement for Windows by @Cyan4973 in #1487
• meson: do not force c99 mode by @grobian in #1485
• Fix vs2022 by @Cyan4973 in #1521
• Update FreeBSD VM image to 14.1 by @dearblue in #1520
• minor: fix test name by @Cyan4973 in #1508
• update -H documentation regarding compression levels by @Cyan4973 in #1506
• lib/Makefile: Fix static library reproducibility by @rpurdie in #1497
• build: use uname -m to detect processor by @bgilbert in #1523
• build/meson: avoid passing long options to ln by @bgilbert in #1522
• build/meson: force LZ4_DEBUG=0 for tests/freestanding by @bgilbert in #1524
• CMakeLists.txt: respect global CMAKE_POSITION_INDEPENDENT_CODE flag by @ds-sloth in #1510
• disable x32 tests on Github Actions CI by @Cyan4973 in #1546
• Fix examples dictionary random access by @t-mat in #1544
• Bump github/codeql-action from 3.26.2 to 3.27.5 by @dependabot[bot] in #1547
• Specify parameters for repology status by @dearblue in #1545
• fix lz4_opt behavior when input is (Null,0) by @Cyan4973 in #1556
• fixed minor static analyzer warning in fileCompress example by @Cyan4973 in #1564
• meson: fix broken oss-fuzz support by @eli-schwartz in #1568
• fix FreeBSD test on Cirrus CI by @Cyan4973 in #1578
• Fix theoretical risk of double-free in UTIL_realloc() by @Cyan4973 in #1577
• fix(memory): add memory deallocation for srcBuff in roundTripCheck and loadFile functions by @louislafosse in #1587
• fix(segv): add null pointer assertions for file pointers in example files by @louislafosse in #1586
• build(deps): bump actions/setup-python from 5.4.0 to 5.5.0 by @dependabot[bot] in #1592
• AIX build failure fix by @shubhamhii in #1598
• updated list of compilers for Github Actions by @Cyan4973 in #1602
• Add a cmake 3.10 build and install test by @Cyan4973 in #1604
• Update cmake minimum required version by @GeorgH93 in #1601
• minor: removed year from header by @Cyan4973 in #1605
• Add License variable to pkg-config file by @iwamatsu in #1603
• build(deps): bump actions/setup-python from 5.5.0 to 5.6.0 by @dependabot[bot] in #1607
• [LZ4] Limit disabling LZ4_FAST_DEC_LOOP to aarch64+clang+android by @Nicoshev in #1610
• Refactor make build by @Cyan4973 in #1612
• Make build update by @Cyan4973 in #1614
• Refactor lz4file by @Cyan4973 in #1617
• cmake readability improvements by @Cyan4973 in #1618
• refactor appveyor test by @Cyan4973 in #1619
• Lz4 ci minor optimization by @Cyan4973 in #1620
• Github Actions script refactoring by @Cyan4973 in #1621
• Fixed small typo by @TomasLeTrain in #1623
• fix(null) : improve error handlings when passing a null pointer to some functions from lz4frame by @louislafosse in #1593
• Replaced uint8_t with unsigned char in BYTE typedef for post C99/C++ by @KSSAB in #1581
• Add Functions that Report LZ4F Contexts' Memory Footprints by @felixhandte in #1595
• remove outdated debian/ files that uses EOL CDBS by @a-detiste in #1649
• Configure LZ4_FORCE_MEMORY_ACCESS=2 for RISC-V with Zicclsm extension by @Polaris-911 in #1648
• lz4_Frame_format.md: document kernel legacy last block by @Artoria2e5 in #1634
• update LZ4F_decompress() code documentation by @Cyan4973 in #1654
• Touch up double buffer example by @Benjamin-Philip in #1643
• CircleCI: Test using Python 3.14 instead of 3.4 by @cclauss in #1665
• [ci] update FreeBSD image to 15 by @Cyan4973 in #1700
• Remove extra $ causing bug #1672 by @jlee303 in #1698
• reduced tests on circle ci by @Cyan4973 in #1703
• removed broken appveyor badge by @Cyan4973 in #1702
• bench: align display when speed > 10000 MB/s by @Cyan4973 in #1705
• Update datagen to return error if usage is not expected by @jlee303 in #1701
• Updated lz4hc documentation: LZ4_saveDictHC() expects valid arguments by @Cyan4973 in #1699
• make improvements by @Cyan4973 in #1706
• narrower concurrency groups by @Cyan4973 in #1707
• Add warning message when content size flag is used with stdin by @jlee303 in #1697
• fix typedef redefinition warning in strict C90 mode by @Cyan4973 in #1717
• AArch64: Enable LZ4_FAST_DEC_LOOP for Android by @arpadpanyik-arm in #1713
• Update appveyor with .dll files by @jlee303 in #1715
• Added -Wmissing-prototypes to the list of warning flags by @Cyan4973 in #1716
• Fix compress bound off by one by @jlee303 in #1714
• Introduce LZ4_wildCopy64 for aarch64 by @Nicoshev in #1723
• Quickly branch on stream's end by @Nicoshev in #1724
• Use 2-byte loads on LZ4_memcpy_using_offset_base by @Nicoshev in #1725
• Avoid resetting make jobserver in submakes by @Cyan4973 in #1754
• Remove Cirrus CI configuration by @Cyan4973 in #1755
• Fix read oob by @Cyan4973 in #1753
• examples: cast division denominators to double (fixes macOS -Werror build) by @Alb3e3 in #1795
• cli: keep source after decompression failure by @Furan666 in #1798
• Lz4file block size test by @Cyan4973 in #1819
• fix unvalidated blockSizeID in LZ4F_compressBegin_internal() by @jaypatrickhoward in #1788
• fix dst capacity accounting by @Cyan4973 in #1820
• Custom block sizes in MT mode by @Cyan4973 in #1821
• Allow LZ4F compression on smaller output buffers by @Cyan4973 in #1822
• 32-bit fuzz producer by @Cyan4973 in #1826
• fix HC --favor-dec-speed by @Cyan4973 in #1827
• Optimization LZ4F by @Cyan4973 in #1828
• Speed optimization for compression of small blocks by @Cyan4973 in #1829
• Fix favorDecSpeed being lost when a dictionary is loaded or attached by @Cyan4973 in #1831
• Hc opt reproducible by @Cyan4973 in #1835
• Attach dict cleared by @Cyan4973 in #1836
• cmake: enable multithreading in lz4 programs on pthread platforms by @Cyan4973 in #1837
• Ring buffer margin by @Cyan4973 in #1838
• Compress overlap by @Cyan4973 in #1839
• Ring buffer by @Cyan4973 in #1840
• ossfuzz: check cstreamHC after LZ4_createStreamHC() in the stream fuzzer by @Mohil-Ahuja in #1832
• ossfuzz: check dictStream allocations in the stream fuzzer by @Cyan4973 in #1842
• lz4hc: simplify the optimal parser by @Cyan4973 in #1843
• Fix freestanding test selection when host and target differ by @Cyan4973 in #1844
• lz4: fix LZ4_loadDict() with a dictionary too small by @Cyan4973 in #1846
• lz4hc: optimal parser: remove fullUpdate by @Cyan4973 in #1845
• lz4frame: fix decompression hints, test random access within a frame by @Cyan4973 in #1847
• lz4hc: fix LZ4_compressHC2_continue() by @Cyan4973 in #1849
• lz4hc: stop searching once a match reaches the maximum possible length by @Cyan4973 in #1848
• NEWS: changes since v1.10.0 by @Cyan4973 in #1851
• lz4frame: new experimental API LZ4F_resumeFrame() by @Cyan4973 in #1850
• lz4hc: simplify the hash chain parser by @Cyan4973 in #1852
• ci: test meson shared library build with MSVC by @Cyan4973 in #1853
• lz4file: new functions LZ4F_seek() and LZ4F_tell() by @Cyan4973 in #1854
• lz4hc: favorDecSpeed : optimal parser prefers fewer sequences by @Cyan4973 in #1855
• doc: state match copy semantics in block format by @Cyan4973 in #1856
• lz4hc: simplify LZ4HC_InsertAndGetWiderMatch() by @Cyan4973 in #1860
• doc: --favor-decSpeed size cost, NEWS entry for #1855 by @Cyan4973 in #1861
• lz4frame: skipChecksums also skips block checksums of compressed blocks by @Cyan4973 in #1862
• fix LZ4_compress_destSize_extState() with out-of-range acceleration by @Cyan4973 in #1864
• Fix test mode writing decoded content to stdout with multiple files by @Cyan4973 in #1865
• lz4: factor repeated conditions of compression entry points by @Cyan4973 in #1866
• lz4file: fix error paths of LZ4F_writeOpen() and LZ4F_writeClose() by @Cyan4973 in #1867
• lz4file: LZ4F_read() reports files ending within a frame by @Cyan4973 in #1868
• make: install depends on lib and lz4 by @hotschmoe in #1869
• lz4: simplify LZ4_compress_fast_extState() by @Cyan4973 in #1871
• fix short header read for dictID frames in lz4 --list by @arshsmith1 in #1769
• ci: pin cmake-test.yml actions to commit SHAs, NEWS updates by @Cyan4973 in #1872
• avoid hang by using predicate-based wait in TPool_jobsCompleted by @jmestwa-coder in #1740
• lz4: LZ4_putPosition() registers a position in any table type by @Cyan4973 in #1873
• lz4frame: don't redefine DEBUGLOG when already defined by @Cyan4973 in #1874
• lz4: reject negative originalSize in LZ4_decompress_fast*() by @Cyan4973 in #1875
• lz4: factor the decoder extDict match copy by @Cyan4973 in #1876
• lz4frame: FORWARD_IF_ERROR evaluates its argument only once by @Cyan4973 in #1877
• tests: fix memory leak in tests/decompress-partial-usingDict by @Cyan4973 in #1878
• lz4: remove two LZ4_decompress_generic() checks that can never fail by @Cyan4973 in #1880
• lz4: reject negative compressedSize in LZ4_decompress_safe_partial() by @Cyan4973 in #1879
• util: check UTIL_realloc result before computing *bufEnd in UTIL_prep… by @h-mole in #1775
• lz4io: size multi-file output names for the suffix by @Cyan4973 in #1881
• build(deps): bump actions/checkout from 6.0.2 to 7.0.1 by @dependabot[bot] in #1768
• --list on truncated frames by @Cyan4973 in #1883
• lz4: LZ4_decompress_generic() fast loop hand over by @Cyan4973 in #1882
• README: refresh text, links and build pointers by @Cyan4973 in #1884
• lz4: LZ4_decompress_generic() safe loop by @Cyan4973 in #1885
• lz4: LZ4_decompress_generic() safe loop: remove the _copy_match label by @Cyan4973 in #1887
• enable LZ4_FAST_DEC_LOOP on riscv64 when unaligned accesses are fast by @Cyan4973 in #1886
• lz4: LZ4_decompress_generic(): name literal and match lengths ll and ml by @Cyan4973 in #1888
• lz4: LZ4_decompress_generic(): simplify the last literals copy by @Cyan4973 in #1893
• Cli numeric overflow by @Cyan4973 in #1894
• lz4: reject negative dictSize in dictionary entry points by @arshsmith1 in #1892
• Save dict negative by @Cyan4973 in #1895
• ci: bump ossf/scorecard-action to v2.5.0 by @Cyan4973 in #1896
• Touch up line by line compression example by @Benjamin-Philip in #1642
• lz4: LZ4_decompress_generic(): fast loop refactor by @Cyan4973 in #1897
• LZ4_wildCopy64 on x86-64 by @Cyan4973 in #1898
• cli: decode Firefox's mozlz4 files by @Cyan4973 in #1900
• lz4: LZ4_decompress_generic(): safe loop refactor by @Cyan4973 in #1899
• lz4hc: LZ4HC_compress_hashChain(): simplify flow by @Cyan4973 in #1901
• lz4: decoder supports rings using former LZ4_DECODER_RING_BUFFER_SIZE() by @Cyan4973 in #1902
• bench: fix compressed buffer size with small blocks by @Cyan4973 in #1903
• lz4hc: LZ4MID_compress() optimization by @Cyan4973 in #1904
• cli: accept the Linux kernel's legacy end marker by @Cyan4973 in #1905
• [ci] restore FreeBSD testing on GitHub Actions by @neilpang in #1782
• bench: fix decode-only result buffer overflow by @Cyan4973 in #1906
• lz4hc: LZ4MID_compress(): factor match evaluation by @Cyan4973 in #1907
• headers: include dependencies outside of extern "C" by @Cyan4973 in #1908
• lz4frame: LZ4F_decompress(): factor block decoding and history move by @Cyan4973 in #1909
• lz4: set the compressor's end limits after the input size check by @Cyan4973 in #1910
• lz4frame: LZ4F_decompress() refactor by @Cyan4973 in #1911
• xxhash: no -Wbad-function-cast warning by @Cyan4973 in #1913
• lz4: LZ4_WILDCOPY64 can be set from the build command by @Cyan4973 in #1912
• lz4io: detect truncated skippable frames in regular files by @akashdeshmukh-git in #1914
• cli: reject malformed --fast= and --threads arguments by @akashdeshmukh-git in #1915
• cli: legacy format honors --rm and keeps file attributes by @Cyan4973 in #1918
• cli: share file name building of multiple-files modes by @Cyan4973 in #1920
• lz4frame: share state initialization, LZ4F_makeBlock() takes the context by @Cyan4973 in #1921
• cli: main() : clearer check of the .lz4 extension for automatic output names by @Cyan4973 in #1922
• Copy the tables of an attached dictionary from 2 KB inputs by @Cyan4973 in #1924
• fast levels : first test the position following the previous occurrence of ip-1 by @Cyan4973 in #1925
• liblz4 : multithreaded frame compression by @Cyan4973 in #1929
• level 2 : check ip+1 after any short match by @Cyan4973 in #1928
• tests: unittest, a unit test framework with autonomous tests by @Cyan4973 in #1926
• cli: --rm skips -t and stdin by @Cyan4973 in #1931
• lz4hc : copy the tables of an attached dictionary from 4 KB inputs by @Cyan4973 in #1930
• lz4io: use unsigned counters for multithreaded decode buffer pool by @arshsmith1 in #1917
• tests: move the simple block unit tests from fuzzer.c to unittest by @Cyan4973 in #1932
• lz4frame : multi-threaded jobs of 1 MB with linked blocks too by @Cyan4973 in #1933
• lz4hc.h : document the compression levels of LZ4_compress_HC() by @Cyan4973 in #1934
• tests: organize tests/ into subdirectories by @Cyan4973 in #1938
• AGENTS.md : notes for coding agents, notably where to add tests by @Cyan4973 in #1939
• lz4frame : the calling thread no longer compresses jobs when worker threads exist by @Cyan4973 in #1940
• lz4io: errors met by jobs are reported once all jobs are done by @Cyan4973 in #1941
• tests: move the remaining inline unit tests of frametest.c, and decompress-partial, to unittest by @Cyan4973 in #1942
• lz4hc: LZ4_compress_HC_continue_destSize() can resume after a partial consumption by @Cyan4973 in #1943
• lz4frame : error when no worker thread can be created by @Cyan4973 in #1944
• lz4frame : multi-threading resources allocated on demand by @Cyan4973 in #1946
• tests: move 7 testXxx() functions of frametest.c to unittest by @Cyan4973 in #1945
• lz4file: skip zero-length fwrite calls in LZ4F_write by @akashdeshmukh-git in #1927
• unittest: tests register themselves, no more table to maintain by @Cyan4973 in #1948
• lz4io: --list: require complete frame header reads by @akashdeshmukh-git in #1937
• lz4 CLI : benchmark measures frames with the CLI options and -T# ; new option --single-thread by @Cyan4973 in #1949
• tests: move the last testXxx() functions of frametest.c to unittest by @Cyan4973 in #1950
• cli: ignore LZ4_CLEVEL and LZ4_NBWORKERS values with trailing garbage by @akashdeshmukh-git in #1951
• tests: streaming_smallDst replays each pass on a new context by @Cyan4973 in #1952
• lz4hc: set the compressor's end limits after the input size check by @anishagg17 in #1919
• lz4frame : copied jobs are submitted before hashing their last piece by @Cyan4973 in #1954
• unittest: -r selects a random seed, used by make test-unittest by @Cyan4973 in #1953
• tests: move the last unit tests of fuzzer.c to unittest by @Cyan4973 in #1955
• lz4io: report decompression speed on success, not on failure by @akashdeshmukh-git in #1956
• ci: build the tests with MSVC, and run unittest by @Cyan4973 in #1957
• tests: remove roundTripTest by @Cyan4973 in #1958
• lz4frame: LZ4F_compressBegin_usingDict() uses the dictionary for each independent block by @Cyan4973 in #1959
• Fix portability warnings reported in #1465 by @Cyan4973 in #1960
• tests: unittest-mt, first unit tests of multi-threaded compression by @Cyan4973 in #1961
• doc: the matchlength bits of the last token are unused by @Cyan4973 in #1962
• build: x86 builds keep branches within 32-byte boundaries (JCC erratum) by @Cyan4973 in #1963
• lz4: fast levels without dictionary compare the candidate before its distance by @Cyan4973 in #1968
• cli: -r doesn't follow symbolic links to directories by @akashdeshmukh-git in #1965
• tests: move frameMT contract checks to unittest_lz4frameMT by @Cyan4973 in #1966
• move the Windows package files out of lib/ by @Cyan4973 in #1970
• build: Windows : VS2022 projects mitigate the JCC erratum, MinGW lz4 without -pthread by @Cyan4973 in #1971
• lz4.c: faster extDict compression (branchless match selection) by @Cyan4973 in #1967
• cli: legacy format uses HC from level 2 by @Nexory in #1969
• Windows package: lz4.exe, README, NEWS and licenses by @Cyan4973 in #1972
• lz4.c: extDict compression continues in prefix mode beyond reach of the dictionary by @Cyan4973 in #1973
• lz4frame: one place to start a stream, one place to invalidate a frame by @Cyan4973 in #1974
• ci: test with recent MinGW-w64 toolchains (MSYS2) by @Cyan4973 in #1979
• lz4frame: simpler job ring, LZ4F_st_update() by @Cyan4973 in #1977
• bench: -d decodes all frames of a file by @akashdeshmukh-git in #1975
• lz4frame: a frame cut into jobs always has a pool by @Cyan4973 in #1981
• lz4hc: attached dictionaries are copied from 2 KB at level 11, 1 KB at level 12 by @Cyan4973 in #1980
• cli: keep new output files private until permissions are copied by @arshsmith1 in #1976
• tests: frame format and header validation, found by mutation testing by @Cyan4973 in #1983
• appveyor: build x86 with the MinGW-w64 toolchain of MSYS2 by @Cyan4973 in #1984
• tests: compression choices and streaming state, found by mutation testing (+ LZ4F_cctx_size() fix) by @Cyan4973 in #1985
• tests: decoder history, hints and allocation failures, found by mutation testing by @Cyan4973 in #1987
• lz4frame, lz4file : LZ4_USER_MEMORY_FUNCTIONS applies to them too by @Cyan4973 in #1988
• lz4frame: autoFlush keeps 64 KB of history with small unstable updates by @Cyan4973 in #1989
• lz4frame: remove dead code from LZ4F_compressBound_internal() and LZ4F_decodeHeader() by @Cyan4973 in #1991
• unittest: compressBegin_usingDict no longer depends on the random seed by @Cyan4973 in #1993
• lz4hc : extended states include the temporary buffer of levels >= 10 by @Cyan4973 in #1990
• cli: refuse an output which is the input itself by @immanuwell in #1992
• tests: lz4 streaming and dictionaries, found by mutation testing by @Cyan4973 in #1994
• tests: content-dependent checks on fixed content by @Cyan4973 in #1997
• lz4file: custom allocators, with LZ4F_readOpen_advanced() and LZ4F_writeOpen_advanced() by @Cyan4973 in #1996
• tests: lz4 API contracts, found by mutation testing by @Cyan4973 in #1995
• lz4file: LZ4F_read() no longer reuses its decoder after an error by @Cyan4973 in #1998
• ossfuzz: add a fuzz target for the file API (lz4file.c) by @17krishna8 in #1982
• lz4frame: uncompressed input as an option of LZ4F_compressUpdate() by @Cyan4973 in #1999
• lz4hc: LZ4_compressHC2_continue() no longer crashes after LZ4_slideInputBufferHC() by @Cyan4973 in #2000
• tests: lz4hc API contracts, dictionaries, and long streams, found by mutation testing by @Cyan4973 in #2001
• tests: lz4file API contracts and resume points of LZ4F_seek(), found by mutation testing by @Cyan4973 in #2002
• lz4file: no cache of frame parameters, no handle reset after release by @Cyan4973 in #2003
• lz4frame: no history save after the last block of a frame by @Cyan4973 in #2004
• lz4hc: no full reset after a failed compression by @Cyan4973 in #2005
• lz4: LZ4_decompress_safe() returns distinct error codes by @Cyan4973 in #2006
• lz4frame: raw dictionary copied ahead of buffered input (linked blocks) by @Cyan4973 in #2007
• lz4frame: avoid a zero offset on a null pending pointer by @immanuwell in #2008
• tests: unit tests pass with any LZ4_MEMORY_USAGE, and LZ4_DISTANCE_MAX >= 2048 by @Cyan4973 in #2009
• lz4, lz4hc: LZ4_resetStream() and LZ4_resetStreamHC() are deprecated by @Cyan4973 in #2011
• tests: CI runs unit tests with LZ4_MEMORY_USAGE MIN and MAX, and LZ4_DISTANCE_MAX=8000 by @Cyan4973 in #2012
• version: v1.11.0 by @Cyan4973 in #2013
• lz4, lz4hc, lz4frame: remove deprecated functions and enums from the headers by @Cyan4973 in #2014
• lz4: small blocks compress the same from a used state by @Cyan4973 in #2022
• tests: ABI test checks that liblz4.so exports the symbols of the latest release by @Cyan4973 in #2021
• lib: liblz4.so exports only its API again by @Cyan4973 in #2023
• cli: long commands with an argument are no longer matched by their prefix by @Cyan4973 in #2024
• cmake: disable LZ4_BUILD_CLI by default if lz4 is a subproject by @crueter in #2018
• NEWS: v1.11.0 lists significant changes only by @Cyan4973 in #2025
• build: multiconf.make 2.0.0 by @Cyan4973 in #2027
• lz4frame: at flush points, the calling thread compresses the last job by @Cyan4973 in #2028
• build: flags the build needs stay with CFLAGS, CPPFLAGS or LDFLAGS on the command line by @Cyan4973 in #2029
• lz4frame: the calling thread compresses a flushed job only when no other job runs by @Cyan4973 in #2031
• doc: API manuals regenerated for v1.11.0 by @Cyan4973 in #2032
• build: -m32, -DNDEBUG and -std=c90 stay with CFLAGS or CPPFLAGS on the command line by @Cyan4973 in #2033
• v1.11.0 : man page, versionsTest includes v1.10.0 by @Cyan4973 in #2035
• fix: keep the source when the output cannot be written by @immanuwell in #2034
• cli: one function ends the processing of every file by @Cyan4973 in #2036
• ci: meson job installs meson from the distribution by @Cyan4973 in #2038
• update published benchmark figures by @Cyan4973 in #2040
• ci: CircleCI config version 2.1 by @Cyan4973 in #2041

New Contributors

• @dg0yt made their first contribution in #1470
• @hhoffstaette made their first contribution in #1468
• @eltociear made their first contribution in #1475
• @axiqia made their first contribution in #1472
• @heitbaum made their first contribution in #1479
• @grobian made their first contribution in #1485
• @dearblue made their first contribution in #1520
• @rpurdie made their first contribution in #1497
• @bgilbert made their first contribution in #1523
• @ds-sloth made their first contribution in #1510
• @louislafosse made their first contribution in #1587
• @shubhamhii made their first contribution in #1598
• @GeorgH93 made their first contribution in #1601
• @iwamatsu made their first contribution in #1603
• @TomasLeTrain made their first contribution in #1623
• @KSSAB made their first contribution in #1581
• @a-detiste made their first contribution in #1649
• @Polaris-911 made their first contribution in #1648
• @Artoria2e5 made their first contribution in #1634
• @Benjamin-Philip made their first contribution in #1643
• @cclauss made their first contribution in #1665
• @jlee303 made their first contribution in #1698
• @arpadpanyik-arm made their first contribution in #1713
• @Alb3e3 made their first contribution in #1795
• @Furan666 made their first contribution in #1798
• @jaypatrickhoward made their first contribution in #1788
• @Mohil-Ahuja made their first contribution in #1832
• @hotschmoe made their first contribution in #1869
• @arshsmith1 made their first contribution in #1769
• @jmestwa-coder made their first contribution in #1740
• @h-mole made their first contribution in #1775
• @neilpang made their first contribution in #1782
• @akashdeshmukh-git made their first contribution in #1914
• @anishagg17 made their first contribution in #1919
• @Nexory made their first contribution in #1969
• @immanuwell made their first contribution in #1992
• @17krishna8 made their first contribution in #1982
• @crueter made their first contribution in #2018

Full Changelog: v1.10.0...v1.11.0