Behavioral change/Performance issue in lx_nand_flash_sector_release.

[ChunBoo]

Hi Team,

I am writing to report a performance issue caused by a behavioral change in file deletion.

• Current Behavior (Latest): lx_nand_flash_sector_release invokes lx_nand_flash_driver_pages_copy (causing actual page writes) even when >90% of blocks are free.
• Previous Behavior (v6.1.12): The function simply marked corresponding pages as obsolete, completely avoiding R/W overhead.
• Performance Impact: Introduces unnecessary I/O during file deletion, causing a noticeable performance drop in mostly-empty flash scenarios.

Proposed Solution:
Could we revert to or re-implement the v6.1.12 logic to only trigger page copies when free blocks fall below a certain threshold?

Looking forward to hearing your thoughts on this.

Best regards,
Bob

[fdesbiens]

Hi @ChunBoo. Thank you for reporting this. I will look into it.

[fdesbiens]

Thank you for the detailed report! We've investigated the issue and have a proposed fix ready for review and testing.

---

What caused the regression

In LevelX v6.1.12, releasing a sector from a full block was an O(1) operation: it simply marked the page as obsolete in the spare bytes of theblock's existing pages.

Commit 27e3f5d ("Fix ECC issues by new NAND logic", January 2023) introduced a fundamentally different block model to properly support ECC. Under the new model, a block is written strictly append-only — pages are never modified after being written. As a result, releasing a sector from a full block now requires allocating a fresh block, copying all surviving pages into it, and erasing the old block. That is correct and intentional behaviour.

The unintended consequence: when you delete N sectors that all happen to live in the same full block, you trigger N copy+erase cycles on that block— one per sector released — yielding O(N²) flash I/O instead of O(N).

---

The proposed fix

A tentative fix is now available on the dev branch (see PR #71) . It will ship in the Q2 2026 release but the code needs proper field testing. It introduces an opt-in lazy sector release mechanism. To enable it, define LX_NAND_FLASH_ENABLE_LAZY_SECTOR_RELEASE in your lx_user.h (or via a compiler flag). When active:

• Releasing a sector from a full block defers the copy+erase. Instead, a small new block is allocated, and a single tombstone page is written to it. The original full block is flagged as a compaction source.
• The actual merge (copy surviving pages → fresh block, erase both sources) is performed lazily, just-in-time when the logical group is next written to, when wear-levelling selects it, or when the free-block list runs low.
• lx_nand_flash_defragment() — previously a stub returning LX_NOT_SUPPORTED — now iterates all pending compactions and completes them eagerly.

When the flag is not defined, all code paths are completely unchanged.

You can also tune the threshold below which the eager path is still used (i.e., when free blocks are scarce):

#define LX_NAND_FLASH_SECTOR_RELEASE_LAZY_THRESHOLD  10  /* default */

This is a percentage value, which enables the feature to scale according to storage capacity.

---

How to test — what we especially need

⚠️ This is brand-new code. It has only been validated against the existing LevelX regression suite. Real-world testing before any production use is essential.

We are particularly interested in feedback on the following scenarios, roughly in order of priority:

1. Bulk delete / O(N²) regression — delete a large number of sectors (hundreds to thousands) that reside in the same full block and measure totalflash write/erase operations. The count should now scale linearly, not quadratically.
2. Power-loss / crash recovery — cut power (or simulate it) at arbitrary points during a delete-heavy workload, then re-open the flash instance.All data that was committed before the interruption should be intact and readable; the instance should recover cleanly without corruption or loss of the free-block pool.
3. Sustained mixed workload — interleave writes, reads, and releases over many thousands of cycles. Verify that wear counts remain balanced andthat the free-block list does not shrink unexpectedly over time.
4. Defragmentation — call lx_nand_flash_defragment() explicitly after a batch of lazy releases and confirm it completes all pending compactions,freeing the deferred source blocks.
5. Near-full flash — exercise the feature when the device is close to capacity (free blocks ≤ threshold). The code falls back to the eager path inthis case; please confirm the transition is seamless.
6. Read-back correctness — after lazy releases, verify that released sectors return all-0xFF and that unreleased sectors in the same logical groupstill return their correct data.

Please report your findings directly in this issue, including your device geometry (block count, pages per block, page size), workload description, and any assertion failures, data corruption, or unexpected behaviour. Even a "tested successfully" reply is valuable.

Thank you!