Skip to content

ADR-077 - Multi-Format Region Support: Pluggable Region Readers (Linear via Addon)

Status: Accepted (revised 2026-09-23, revised again 2026-10-06 - see Revisions) Date: 2026-08-26 Extends: ADR-016 (Anvil Read-Only Subsystem) Related: ADR-028 (L3 Backlog Cache), ADR-067 (Adaptive Scan Rate and Header Generation Check)

Revision (2026-10-06) - Linear withdrawn from core; ships later as a guarded addon

The built-in LinearRegionReader did not implement the real Linear format. The reference implementation (xymb LinearRegionFileFormatTools, linear.py, used by LinearPaper and Leaves) uses signature 0xc3ff13183cca9d9a at both the start and the end of the file, a 32-byte header, and keeps the 1024-entry (size, timestamp) table inside the compressed blob. The reader expected a different signature, a 22-byte header and plain size/timestamp tables before the compressed payload, so every real .linear file failed its signature check and returned UNKNOWN after a whole-file read. Its tests built fixtures in the reader's own layout and so passed against it. The reader also summed untrusted chunk lengths into an int without bounds and let the decompressor skip up to ~2 GiB.

Decision: Linear is removed from core.

  • LinearRegionReader, its tests and its fuzz target are deleted.
  • RegionFormatRegistry registers only .mca by default.
  • RegionFileResolver keeps no Linear special case (no isLinear flag, no hard-coded .linear fallback).
  • AnvilRegionScanner and PregenBiomeExtractor read only .mca and registered extensions. A file in an unregistered format is never handed to AnvilReader.
  • io.airlift:aircompressor is dropped from anvil-api and from both shaded jars, and zstd-jni is dropped from the test classpath. The plugin jar shades no third-party library.
  • The RegionFileReader / RegionFileReaderProvider / RegionFormatRegistry SPI stays. A future addon shall provide a Linear reader that is guarded:
  • validate both signatures and the header;
  • decompress into a buffer of capped size;
  • add up chunk sizes as a long, with a per-chunk cap;
  • test against files produced by the reference converter.

On a Linear world, core resolves no region file, so the probe returns UNKNOWN and the chunk goes to the live async load path (S-004, S-005). That was already the effective behaviour, minus the wasted whole-file read. None of the Linear code reached a release (3.2.1 shipped Anvil only). This revision supersedes the 2026-09-23 revision below, Decision items 2, 3 and 5 for Linear, and Option F.

Revision (2026-09-23) - Linear folded into anvil-api via a pure-Java decoder (superseded 2026-10-06)

The original decision (Option E) packaged the Linear decoder as a standalone LeafRTPLinearAddon specifically to keep the native com.github.luben:zstd-jni library (~6.3 MiB of multi-platform native binaries) out of the core jar. That size concern no longer applies: RTP only ever decompresses Linear frames (read-only, off-tick), so the native library was replaced with the pure-Java io.airlift:aircompressor:2.0.3 ZStandard decoder (Java-8/21-compatible maintenance line, no native binaries). The shaded decoder footprint dropped from ~6.3 MiB to ~0.26 MiB.

Given that footprint, LinearRegionReader was moved into api/anvil-api (package io.github.dailystruggle.rtp.anvil) and .linear is now registered by default in RegionFormatRegistry alongside .mca. The standalone addons/LeafRTPLinearAddon module was retired. .linear off-tick pre-filtering is therefore built in for both the Lite and Pro editions with no operator action and no native-binary bloat. zstd-jni is retained only as a testImplementation in anvil-api to author synthetic .linear fixtures (cross-validated against the pure-Java decode path). The pluggable RegionFileReader / RegionFileReaderProvider SPI is unchanged and still available for third-party formats. This supersedes Option E below and the addon-specific wording in the Decision/Consequences.

Context

High-performance server forks (such as Leaves and Gale) and modded environments (Fabric/NeoForge running the Linear Region Format mod) frequently utilize the Linear region format (.linear). Linear replaces Mojang's 4 KiB sector-aligned Anvil (.mca) allocation scheme with continuous ZStandard (zstd) compressed streams to eliminate sector quantization padding, reduce world disk space by 30-60%, and speed up sequential I/O.

Under ADR-016, RTP's off-tick region pre-filter subsystem (api/anvil-api) hardcodes file resolution to r.X.Z.mca and expects an 8 KiB Anvil sector header table. On servers with .linear region storage: 1. AnvilPrefilter reports UNKNOWN:no-region-file(r.X.Z.mca) because the files on disk are named r.X.Z.linear. 2. Even if renamed or targeted, standard Anvil sector calculations fail, producing corrupted sector offsets. 3. As a result, all location probes return Verdict.UNKNOWN and fall through to live chunk loading. While S-004 safe, this disables off-tick pre-filtering and pre-scan optimizations on Linear-backed worlds.

Decision

  1. Pluggable Region Reader SPI: Generalize api/anvil-api to parse multiple on-disk region formats behind a unified RegionFileReader SPI: 
    public interface RegionFileReader {
    byte[] readChunkNbt(byte[] regionBytes, int rx, int rz) throws IOException;
    boolean isChunkGenerated(byte[] regionBytes, int rx, int rz);
}
  2. Format Resolution and Auto-Detection:
  3. RegionFileResolver shall probe for r.X.Z.linear first, falling back to r.X.Z.mca.
  4. File format verification shall validate magic header bytes (0xC370ACDE22013702 for Linear v1/v2 vs. Anvil sector structures).
  5. Linear Decoder Implementation (revised - see Revision 2026-09-23):
  6. Built into api/anvil-api (LinearRegionReader), using the pure-Java io.airlift:aircompressor ZStandard decoder to decompress Linear chunk frames with zero native binaries.
  7. Registered for .linear by default in RegionFormatRegistry (the RegionFileReaderProvider SPI remains available for third-party formats).
  8. Reads Linear v1/v2 headers, parses chunk entry index tables, and decompresses the requested chunk's NBT payload.
  9. Reuses existing zero-dependency Nbt.readRootCompound to construct standard AnvilChunkView objects.
  10. Safety and Fallback Guarantees (S-004 and S-005):
  11. All decompression and file reads shall remain asynchronous on ForkJoinPool.commonPool().
  12. If the ZStandard decoder classes are unavailable, or if a region file is malformed, the probe shall catch the error, emit diagnostic logging, and return Verdict.UNKNOWN to safely fall through to runtime chunk loading. (With the pure-Java decoder there is no native linkage to fail, but the guard is retained defensively.)
  13. Cache Compatibility:
  14. AnvilRegionByteCache shall be renamed/aliased to RegionByteCache to pool raw byte buffers for both .mca and .linear files.

Alternatives Considered

Alternative Why Rejected
Option A: Require operators to use .mca Breaks compatibility with Leaves/Gale servers and forced conversion negates disk-saving benefits for server operators.
Option B: Disable pre-filter on .linear worlds Causes 100% fallback to live chunk loads, increasing server tick pressure and losing pre-scan acceleration.
Option C: Inline Linear parsing into AnvilReader Violates single-responsibility principle; mixing Anvil 4 KiB sector arithmetic with ZSTD stream decoding creates tight coupling and testing complexity.
Option D: Bundle zstd-jni into anvil-api directly Bundles multi-platform native binaries into the core jar, increasing overall jar size by ~6.3 MiB for all users even though 98%+ use standard .mca Anvil.
Option E: Pluggable SPI with dedicated LeafRTPLinearAddon (originally selected; superseded 2026-09-23) Kept the native zstd-jni out of core, but required operators on .linear worlds to install a separate addon jar. Superseded once the native dependency was replaced by the tiny pure-Java aircompressor decoder (see Revision).
Option F: Build Linear into anvil-api using a pure-Java ZStandard decoder (selected 2026-09-23; superseded 2026-10-06) The shipped reader did not match the real Linear layout and lacked input bounds, and the decoder was the plugin's only shaded third-party library.
Option G: Anvil-only core; Linear as a guarded addon on the existing SPI (Selected, 2026-10-06) Keeps core free of shaded dependencies and of an unverified parser. Linear worlds fall back to live async loads until the addon ships, which is what already happened in practice.

Consequences

The 2026-10-06 revision supersedes the Linear-specific points below. Core pre-filters only .mca; .linear worlds fall back to live async chunk loads until a Linear addon registers a reader.

  • Positive:
  • Leaves, Gale, and modded servers using .linear retain full off-tick biome and safety pre-filtering out of the box, in both the Lite and Pro editions, with no operator action.
  • Core plugin JAR remains lightweight (the pure-Java decoder adds ~0.26 MiB) with zero native binary bloat for any server.
  • /rtp scan and the Backlog cache (ADR-028) can inspect .linear files without triggering server chunk loads.
  • Clean modular SPI architecture (RegionFormatRegistry / RegionFileReaderProvider) remains available for future region formats.
  • Negative / Trade-offs:
  • The pure-Java ZStandard decoder is somewhat slower than the native zstd-jni path, but decode runs off-tick on ForkJoinPool.commonPool() for a prefilter, so it is not tick-critical.
  • Linear frame decompression may allocate slightly larger temporary byte buffers during initial decode compared to individual 4 KiB sector slices (buffered and bounded by RegionByteCache).

References

  • ADR-016 - Anvil Read-Only Subsystem
  • ADR-067 - Adaptive Scan Rate and Generation Checks
  • REQ-RTP-S-004 (No silently discarded failures)
  • REQ-RTP-S-005 (No chunk loading on main thread)