Motivation
The Java implementation can reopen a normally closed (complete) TsFile for appending through:
RestorableTsFileIOWriter.getWriterForAppendingDataOnCompletedTsFile(File)
This method detects a complete file, locates the start of FileMetadata from the footer, truncates the separator marker and all tail metadata, recovers the existing schema/chunk metadata, and then allows new data to be appended. Closing the writer generates a new metadata section and tail magic.
The C++ implementation currently supports recovery and continued writing only when the file is incomplete or its tail is damaged. When the tail magic is valid, RestorableTsFileIOWriter::self_check() treats the file as complete, sets can_write_ = false, and closes the write handle.
As a result, users of the C++ API and the C wrapper cannot append to a TsFile that was closed normally, even though the equivalent workflow is available in Java.
Expected behavior
Provide a supported C++ API equivalent to the Java completed-file append workflow, and expose it through the C API as well.
The workflow should:
- Validate that the input is a compatible TsFile.
- Detect whether the file is complete by checking the tail magic.
- For a complete file, read the footer metadata size and locate the start of
FileMetadata.
- Safely truncate the separator marker, metadata/index section, footer size, and tail magic while retaining all completed chunk groups.
- Recover the existing tree/table schema, alignment information, chunk metadata, statistics, and per-device last timestamps.
- Allow users to append data using the recovered schema.
- Allow registration of supported new schemas after recovery, with clear behavior for:
- a new device or measurement in the tree model;
- a new table;
- attempts to replace or evolve an existing table/aligned schema.
- On close, write a new metadata section, footer, and tail magic so that the resulting file is complete and readable.
- Reject out-of-order timestamps relative to recovered data.
The operation should preserve the original file when validation or recovery fails before truncation. The API documentation should make the destructive/in-place nature of a successful append-open explicit.
Current C++ behavior
For an incomplete or damaged-tail file, the following path already works:
RestorableTsFileIOWriter::open(path, true)
→ recover schema and chunk metadata
→ TsFileWriter::init(&restorable_writer)
→ append data
→ close and rebuild metadata
For a normally closed file, open() returns successfully but can_write() is false, so it cannot be passed to TsFileWriter::init().
Suggested tests
- Append to a normally closed tree-model TsFile and verify old and new rows.
- Append to a normally closed table-model TsFile.
- Preserve recovered encoding/compression/data types and alignment.
- Register a new device and a new non-aligned measurement after recovery.
- Verify documented behavior for aligned-series and existing-table schema evolution.
- Reject duplicate or incompatible schemas.
- Reject timestamps not newer than the recovered last timestamp.
- Handle empty, incomplete, incompatible, encrypted, and large-metadata files.
- Verify that a no-op append followed by close produces a valid file without losing data.
- Add coverage for the C wrapper API.
Related implementation
- Java:
RestorableTsFileIOWriter.getWriterForAppendingDataOnCompletedTsFile(File)
- C++:
RestorableTsFileIOWriter::self_check()
- C++ recovery integration:
TsFileWriter::init(RestorableTsFileIOWriter*)
Motivation
The Java implementation can reopen a normally closed (complete) TsFile for appending through:
RestorableTsFileIOWriter.getWriterForAppendingDataOnCompletedTsFile(File)This method detects a complete file, locates the start of
FileMetadatafrom the footer, truncates the separator marker and all tail metadata, recovers the existing schema/chunk metadata, and then allows new data to be appended. Closing the writer generates a new metadata section and tail magic.The C++ implementation currently supports recovery and continued writing only when the file is incomplete or its tail is damaged. When the tail magic is valid,
RestorableTsFileIOWriter::self_check()treats the file as complete, setscan_write_ = false, and closes the write handle.As a result, users of the C++ API and the C wrapper cannot append to a TsFile that was closed normally, even though the equivalent workflow is available in Java.
Expected behavior
Provide a supported C++ API equivalent to the Java completed-file append workflow, and expose it through the C API as well.
The workflow should:
FileMetadata.The operation should preserve the original file when validation or recovery fails before truncation. The API documentation should make the destructive/in-place nature of a successful append-open explicit.
Current C++ behavior
For an incomplete or damaged-tail file, the following path already works:
RestorableTsFileIOWriter::open(path, true)→ recover schema and chunk metadata
→
TsFileWriter::init(&restorable_writer)→ append data
→ close and rebuild metadata
For a normally closed file,
open()returns successfully butcan_write()is false, so it cannot be passed toTsFileWriter::init().Suggested tests
Related implementation
RestorableTsFileIOWriter.getWriterForAppendingDataOnCompletedTsFile(File)RestorableTsFileIOWriter::self_check()TsFileWriter::init(RestorableTsFileIOWriter*)