Module object_manager 

Metadata Reservations in Fxfs

Fxfs maintains a metadata reservation (M_total) in the allocator so that space-freeing operations and journal compactions can proceed even when the filesystem has zero free space:

• Funded transactions: Normal transactions reserve metadata space from the allocator up front (4x their journal byte size) and transfer that reservation into M_total on commit.
• Borrowing transactions: Operations that free space (such as unlinking files, purging graveyard tombstones, or trimming extents) must succeed even when the disk is full, so they borrow from M_total (Borrowed) instead of reserving from the allocator.
• Minor vs. Major Compaction: Flushing an object store normally performs a minor compaction, writing recent journal mutations (including deletion tombstones) into a new small layer file stacked on top of existing immutable layers. Minor compactions consume blocks from M_total to write the new layer file and do not free older layers. Borrowed metadata space is only reclaimed when a store performs a major compaction, which merges all layers into a single base layer, drops all tombstones, and deletes the old layer files. To prevent repeated minor compactions from draining M_total, Fxfs forces a major compaction once borrowed space reaches 50% of the available borrowing headroom.

Why Per-Store Borrowing Tracking Is Required (and the Locked-Store Problem)

Encrypted volumes can be locked (ObjectStore::lock), dropping their decryption keys from memory. If a locked store has unflushed journal mutations when the journal compacts, ObjectStore::flush_locked appends them to a sidecar encrypted_mutations file in root_store; critically, a locked store cannot decrypt and merge its layers to perform a major compaction until it is unlocked again.

If borrowed space and major-compaction thresholds were tracked as a single global scalar across the entire filesystem (borrowed = R_total - M_total >= R_total / 2, as Fxfs did prior to V61), a locked store could tie up the filesystem’s metadata space in two ways:

1. Consuming other stores’ headroom before locking: Because a global R_total / 2 threshold includes journal::RESERVED_SPACE plus 1.5 * L for root_store, the allocator, and all child volumes, a single volume S_locked could perform many borrowing deletions and minor compactions while unlocked without ever crossing global R_total / 2, borrowing far more than its own 0.5 * L headroom and eating into the reservations of other volumes and root_store. If S_locked was then locked, all of that borrowed space remained frozen in its unmerged layers while locked.
2. Starving other stores and forcing endless major compactions: With S_locked tying up most of global R_total / 2, any small borrow by an active unlocked store S_active or root_store pushed global borrowed past R_total / 2. Every subsequent journal flush then forced a major compaction on every unlocked store (S_active and root_store) even when they had no tombstones to reclaim, while S_locked (the store actually holding the debt) was locked and skipped (or worse, exhausted M_total altogether).

Assigning each child store S its own borrowing budget req_S and borrowed counter B_S prevents both failure modes:

• While S_locked is unlocked, it is forced to major-compact as soon as its own borrowing reaches B_S >= req_S / 2 (resetting B_S = 0 and freeing its tombstone layers), regardless of how large other stores are. When S_locked is locked, the uncompacted debt it can hold is strictly bounded by its own req_S share (which S_locked itself funded when its layers were created) and can never dip into req_other or req_sys.
• While S_locked is locked, each unlocked store S_active is evaluated solely against its own B_{S_active} >= req_{S_active} / 2, root_store subtracts sum(B_S) when computing system borrowed space, and S_locked is ignored by compactions until unlocked.

Reservation Model

Each LSM tree (the allocator, the root store, and each child volume) with total layer size L has a target reservation budget of 1.5 * L, which decomposes into:

• Compaction scratch space (1.0 * L): Worst-case temporary space needed to write a new compacted layer file before tombstoning the old layer files. Because only one tree is compacted at a time, this scratch space is shared across all trees (max(L) across all trees).
• Borrowing and system headroom (0.5 * L): Space that can be borrowed across compactions (e.g. unmerged minor layers of deletion tombstones, or encrypted_mutations in a locked volume) before a major compaction is required to reclaim it.

To isolate borrowing between volumes while sharing compaction scratch space, ObjectManager computes required reservations as:

1. System Requirement (req_sys): Covers shared compaction scratch space across all trees (max(L)), the allocator and root store headroom (0.5 * (L_alloc + L_root)), a scaling share of child stores (1/8 * sum(L_child) for root_parent_store, root_store, and allocator records of the child stores’ layer files themselves; if extreme layer file fragmentation ever exceeded this share, R_total is unaffected and root_store would simply major-compact slightly earlier), unflushed system journal usage (4 * J_sys), and journal::RESERVED_SPACE.
2. Per-Child-Store Requirement (req_S): For each child volume S, covers that volume’s borrowing headroom (3/8 * L_S) and its unflushed journal mutations (4 * J_S).
3. Total Required Reservation (R_total): req_sys + sum(req_S).

Rather than partitioning the physical reservation M_total (metadata_reservation.amount()) into per-store balances (which would require scaling per-store balances down whenever M_total has a shortfall and could cause repeated major compactions on subsequent transactions), ObjectManager tracks:

• M_total: The total physical bytes currently held in metadata_reservation.
• store_borrowed[S]: The metadata space borrowed by each child store S’s flushed non-base layers (or encrypted_mutations), while journal_checkpoints[S] tracks unflushed journal borrowing before (Old) and after (Current) BeginFlush. Store S’s total borrowed space is B_S = store_borrowed[S] + 4 * journal_checkpoints[S].borrowed_usage().

Accounting Lifecycle

• Transaction Commit: Non-borrowing transactions (New / Hold) transfer 4 * txn_size into M_total and record usage in journal_checkpoints[S]. Borrowing transactions (Borrowed) transfer nothing into M_total and record both usage and borrowed_usage in journal_checkpoints[S] for each mutated child store S.
• Tree Compaction (BeginFlush & EndFlush): When BeginFlush is applied to child store S, journal_checkpoints[S] moves Current into Old so any concurrent transactions between BeginFlush and EndFlush accumulate into a fresh Current span. At EndFlush(borrowed), Old is removed (leaving only any post-BeginFlush borrowing in Current) and store_borrowed[S] is set to borrowed (0 for a full compaction, or the updated flushed layer borrowing computed by store_borrowed_for_flush for a minor or locked flush).
• Major Compaction: Each child store S independently forces a major compaction when its borrowed space exceeds half of its layer headroom (B_S > h_S / 2, where h_S = 3/8 * L_S) and the store can be compacted (needs_flush(S) or layers.len() > 1). root_store forces a major compaction when system borrowed space ((R_total - M_total).saturating_sub(sum(B_S))) exceeds half of the system’s non-scratch layer and journal headroom (0.5 * (L_alloc + L_root) + 1/8 * sum(L_child) + RESERVED_SPACE). Because a full compaction of S unconditionally resets store_borrowed[S] = 0 and drops Checkpoints::Old even when M_total < R_total, a single subsequent transaction on S will not trigger another major compaction.

Journal Replay and Software Upgrades

The target reservation formulas (such as journal::RESERVED_SPACE or the layer/journal multipliers) can change between software versions, and during journal replay the full set of child volumes and their layer sizes are not yet known. If the super-block only recorded global borrowed space B and computed the mount-time reservation as R_total - B, any software update that increased R_total would try to reserve more space from the allocator at mount time than was actually set aside before the unmount or crash, which could fail with NoSpace on a nearly full filesystem.

Instead, SuperBlockHeader::reserved_metadata_space records M_total at key 0 and each child store’s flushed layer borrowing store_borrowed[S] at key store_id as of super_block_journal_file_offset, and the journal distinguishes non-borrowing (Commit) from borrowing (CommitBorrowed(u64)) transactions:

1. Seeding from the Super-Block: init_replay_metadata_reservation seeds M_total from key 0 and store_borrowed[S] from each store_id != 0.
2. Replaying Journal Transactions (replay_mutations):
   • All mutations in transactions (which Journal::read_transactions already filtered to those unflushed as of super_block.journal_file_offsets) run through Inner::apply_mutation and record_journal_usage, taking journal_checkpoints and store_borrowed through the same BeginFlush -> Old -> EndFlush(borrowed) state transitions as live mode.
   • For transactions after super_block_journal_file_offset:
     • Commit (non-borrowing): Adds 4 * txn_size to M_total.
     • CommitBorrowed(reserved) (borrowing): Updates M_total to reserved (the post-apply_transaction metadata_reservation.amount() recorded in live mode).
3. Reconciling with Current Software Requirements (init_metadata_reservation): Once replay completes and all child volumes are opened, min(M_total, R_total) is reserved from the allocator (while total_store_borrowed clamps each store_borrowed[S] to h_S on read):
   • If required reservations increased (M_total < R_total): Only M_total is reserved from the allocator at mount time, never asking for extra space.
   • If required reservations decreased (M_total > R_total): The reservation is capped at R_total, leaving the excess unreserved in the allocator.
   • Legacy (< V61) migration: Pre-V61 super-blocks only recorded global borrowed space B computed against the legacy required reservation (legacy_required = max(1.5 * L_i) + 4 * J_total + reserved_space), so M_total = min(legacy_required - B, R_total) is reserved and any non-journal deficit is distributed proportionally across child stores’ layer headroom to seed store_borrowed[S].

Structs

ObjectManager

ObjectManager is a global loading cache for object stores and other special objects.

ReservationUpdate

ReservationUpdate is an associated object attached to Mutation::EndFlush that updates the tree layer size (layer_size) for an object.

Enums

ReplayMetadataReservation

Tracks the metadata reservation state during journal replay before init_metadata_reservation initializes the live Reservation.

Functions

reserved_space_from_journal_usage