![]() |
NDEVR
API Documentation
|
Logic for reading or writing to a binary file including logic for compressing or decompressing the file. More...
Public Member Functions | |
| BinaryFile () | |
| Constructs an empty BinaryFile backed by an in-memory cache. | |
| BinaryFile (BinaryFile &&bf) noexcept | |
| Move constructor. | |
| BinaryFile (const Buffer< uint01 > &data) | |
| Constructs a BinaryFile from a Buffer of bytes. | |
| BinaryFile (const File &file) | |
| Constructs a BinaryFile associated with the given file on disk. | |
| BinaryFile (const HighCapacityBuffer< uint01 > &data) | |
| Constructs a BinaryFile from an existing data buffer (copy). | |
| BinaryFile (const uint01 *data, uint08 size) | |
| Constructs a BinaryFile from a raw pointer and size. | |
| BinaryFile (HighCapacityBuffer< uint01 > &&data) | |
| Constructs a BinaryFile from an existing data buffer (move). | |
| ~BinaryFile () | |
| Destructor. | |
| HighCapacityBuffer< uint01 > & | cachedData () |
| Returns a mutable reference to the in-memory cached data. | |
| const HighCapacityBuffer< uint01 > & | cachedData () const |
| Returns a const reference to the in-memory cached data. | |
| void | cachedOpen (bool read) |
| Opens the file and reads its entire contents into an in-memory cache. | |
| template<class t_type> | |
| bool | canRead () const |
| Checks whether enough data remains to read a value of the given type. | |
| void | close () |
| Closes the file and flushes any pending data. | |
| BinaryCompressionObject | createCompressionObject () |
| Creates a BinaryCompressionObject initialized with this file's compression settings. | |
| File & | file () |
| Returns a mutable reference to the underlying File object. | |
| const File & | file () const |
| Returns a const reference to the underlying File object. | |
| uint08 | fileSize () const |
| Returns the size of the underlying file in bytes. | |
| bool | isFinished () |
| Checks whether the file has been fully read. | |
| void | open (bool read, bool safe) |
| Opens the file for reading or writing. | |
| BinaryFile & | operator= (BinaryFile &&bf) noexcept |
| Move assignment operator. | |
| fltp04 | percent () const |
| Returns the current read progress as a percentage of the total file size. | |
| uint08 | position () const |
| Returns the current read or write position in the file or cache. | |
| template<class t_type> | |
| t_type | read () |
| Reads a single value of the given type from the file or cache. | |
| template<class t_type, class t_memory_manager> | |
| void | read (Buffer< t_type, t_memory_manager > &data) |
| Reads a compressed buffer using the file's multithreading setting. | |
| template<class t_type, class t_memory_manager> | |
| std::enable_if<!ObjectInfo< t_type >::Buffer >::type | read (Buffer< t_type, t_memory_manager > &data, bool &allow_multithread) |
| Reads a buffer of non-buffer elements with decompression. | |
| template<class t_type, class t_memory_manager> | |
| std::enable_if< ObjectInfo< t_type >::Buffer >::type | read (Buffer< t_type, t_memory_manager > &data, bool &allow_multithread) |
| Reads a buffer of buffer elements (nested buffers) with decompression. | |
| uint04 | readData (char *string, uint04 max_size) |
| Reads raw data into a character buffer up to a maximum size. | |
| String | readData (uint04 max_size) |
| Reads raw data up to a maximum size and returns it as a String. | |
| uint08 | readDirect (uint01 *ptr, uint08 size) |
| Reads raw bytes directly from the file or cache. | |
| bool | readDirectlyTo (File file) |
| Reads the cached data directly to a file on disk. | |
| template<class t_type, class t_memory_manager> | |
| void | readNow (Buffer< t_type, t_memory_manager > &data) |
| Reads a compressed buffer synchronously, disabling multithreaded decompression. | |
| uint04 | readString (char *string, char terminator='\0') |
| Reads a string into a character buffer up to a terminating character. | |
| void | readString (String &string, char terminator='\0') |
| Reads a string up to a terminating character. | |
| template<class t_memory_manager> | |
| void | readStringBuffer (Buffer< String, t_memory_manager > &data, bool &allow_multithread) |
| Reads a buffer of strings using the string reference table for decompression. | |
| void | readStringData (bool has_reference_table) |
| Reads the string reference table from the file. | |
| void | seek (uint08 location) |
| Seeks to an absolute byte position in the file or cache. | |
| void | setMaxBufferSize (uint08 max_buffer_size) |
| Sets the maximum buffer size for cached operations. | |
| void | setUseLargeCompression (bool use_large_compression) |
| Enables or disables large (64-bit) compression size headers. | |
| template<class t_type> | |
| void | skip () |
| Advances the read position by the size of the given type without reading. | |
| void | switchToFile (File file, bool is_write, bool is_safe) |
| Switches the underlying file to a different file and reopens it. | |
| void | uncompressAll () |
| Decompresses all pending multithreaded compression objects. | |
| void | uncompressSections (uint04 start, uint04 count) |
| Decompresses a range of pending multithreaded compression objects. | |
| template<class t_type, class t_memory_manager> | |
| std::enable_if< ObjectInfo< t_type >::Buffer >::type | write (const Buffer< t_type, t_memory_manager > &data, CompressionMode compression_mode) |
| Writes a buffer of buffer elements (nested buffers) with optional compression. | |
| template<class t_type, class t_memory_manager> | |
| std::enable_if<!ObjectInfo< t_type >::Buffer >::type | write (const Buffer< t_type, t_memory_manager > &data, CompressionMode compression_mode=CompressionMode::e_no_compression) |
| Writes a buffer of non-buffer elements with optional compression. | |
| void | write (const StringView &data, CompressionMode compression_mode) |
| Writes a string view with optional compression. | |
| template<class t_type> | |
| std::enable_if<!ObjectInfo< t_type >::Buffer >::type | write (const t_type &data) |
| Writes a non-buffer type as raw bytes to the file. | |
| void | writeCompression (BinaryCompressionObject &compression_object) |
| Writes a compression object's data to the file. | |
| void | writeDirect (const uint01 *ptr, uint08 size) |
| Writes raw bytes directly to the file or cache. | |
| bool | writeDirectlyFrom (File file) |
| Writes data from a file on disk directly into the cache. | |
| void | writeRawData (const StringView &data, uint04 size, char fill_space='\0') |
| Writes a string as raw data padded or truncated to a fixed size. | |
| void | writeStringData (CompressionMode compression_mode) |
| Writes the accumulated string reference table to the file. | |
| void | writeTerminatingString (const StringView &string, char terminator='\0') |
| Writes a string followed by a terminating character. | |
Protected Attributes | |
| fltp08 | fltp_error = Constant<fltp08>::Invalid |
| Floating-point error tolerance used during compression. | |
| uint08 | m_cache_location = Constant<uint08>::Invalid |
| Current read/write position within the cached data. | |
| HighCapacityBuffer< uint01 > | m_cached_data |
| In-memory cache of the file's raw byte contents. | |
| HighCapacityBuffer< uint01 > | m_compressed_data |
| Temporary buffer for compressed data during read/write operations. | |
| Buffer< BinaryCompressionObject > | m_compressions |
| Pending compression objects for multithreaded decompression. | |
| File | m_file |
| The underlying file handle for disk-based I/O. | |
| uint08 | m_file_size = Constant<uint08>::Invalid |
| Cached file size to avoid repeated filesystem queries. | |
| bool | m_is_read = false |
| Whether the file was opened for reading. | |
| bool | m_large_compression = false |
| Whether to use 64-bit size fields for compression headers. | |
| uint08 | m_max_buffer_size = Constant<uint04>::Max |
| Maximum buffer size for cached read/write operations. | |
| bool | m_multithreaded_compression = true |
| Whether multithreaded decompression is enabled. | |
| Dictionary< String, uint04 > | m_string_reference |
| Maps strings to their index in the string reference table. | |
| Buffer< Bounds< 1, uint04 > > | m_string_size_info |
| Stores the start and end bounds of each string in m_strings. | |
| String | m_strings |
| Accumulated string data for the string reference table. | |
| bool | m_using_cache = false |
| Whether the file is operating in cached (in-memory) mode. | |
Logic for reading or writing to a binary file including logic for compressing or decompressing the file.
Definition at line 135 of file BinaryFile.h.
|
inlineexplicit |
Constructs a BinaryFile associated with the given file on disk.
| [in] | file | The file to read from or write to. |
Definition at line 143 of file BinaryFile.h.
References file(), and m_file.
Referenced by BinaryFile(), and operator=().
|
inlineexplicit |
Constructs a BinaryFile from an existing data buffer (copy).
| [in] | data | The buffer of raw bytes to use as cached data. |
Definition at line 159 of file BinaryFile.h.
References m_cache_location, m_cached_data, m_is_read, and m_using_cache.
|
inlineexplicit |
Constructs a BinaryFile from an existing data buffer (move).
| [in] | data | The buffer of raw bytes to move into the cached data. |
Definition at line 169 of file BinaryFile.h.
References m_cache_location, m_cached_data, m_is_read, and m_using_cache.
Constructs a BinaryFile from a raw pointer and size.
| [in] | data | Pointer to the raw byte data. |
| [in] | size | Number of bytes to copy from the pointer. |
Definition at line 181 of file BinaryFile.h.
References m_cache_location, m_cached_data, m_is_read, and m_using_cache.
Constructs a BinaryFile from a Buffer of bytes.
| [in] | data | The buffer of bytes to copy into the cached data. |
Definition at line 192 of file BinaryFile.h.
References m_cache_location, m_cached_data, m_is_read, and m_using_cache.
|
noexcept |
|
inline |
Destructor.
Closes the file if open.
Definition at line 213 of file BinaryFile.h.
References close().
|
inline |
Returns a mutable reference to the in-memory cached data.
Definition at line 244 of file BinaryFile.h.
References m_cached_data.
|
inline |
Returns a const reference to the in-memory cached data.
Definition at line 236 of file BinaryFile.h.
References m_cached_data.
| void BinaryFile::cachedOpen | ( | bool | read | ) |
|
inlinenodiscard |
Checks whether enough data remains to read a value of the given type.
Definition at line 427 of file BinaryFile.h.
References m_cache_location, m_cached_data, m_file, m_using_cache, and position().
| BinaryCompressionObject BinaryFile::createCompressionObject | ( | ) |
Creates a BinaryCompressionObject initialized with this file's compression settings.
|
inline |
Returns a mutable reference to the underlying File object.
Definition at line 719 of file BinaryFile.h.
References m_file.
|
inline |
Returns a const reference to the underlying File object.
Definition at line 714 of file BinaryFile.h.
References m_file.
Referenced by BinaryFile(), readDirectlyTo(), switchToFile(), and writeDirectlyFrom().
| uint08 BinaryFile::fileSize | ( | ) | const |
Returns the size of the underlying file in bytes.
| bool BinaryFile::isFinished | ( | ) |
Checks whether the file has been fully read.
| void BinaryFile::open | ( | bool | read, |
| bool | safe ) |
|
noexcept |
Move assignment operator.
| [in] | bf | The BinaryFile to move from. |
References BinaryFile().
| fltp04 BinaryFile::percent | ( | ) | const |
Returns the current read progress as a percentage of the total file size.
| uint08 BinaryFile::position | ( | ) | const |
Returns the current read or write position in the file or cache.
Referenced by canRead().
|
inlinenodiscard |
Reads a single value of the given type from the file or cache.
Definition at line 441 of file BinaryFile.h.
References cast(), m_cache_location, m_cached_data, m_file, and m_using_cache.
Referenced by cachedOpen(), open(), read(), read(), readNow(), and readStringBuffer().
|
inline |
Reads a compressed buffer using the file's multithreading setting.
| [in] | data | The buffer to populate with decompressed data. |
Definition at line 510 of file BinaryFile.h.
References m_multithreaded_compression, and read().
|
inline |
Reads a buffer of non-buffer elements with decompression.
| [in] | data | The buffer to populate with decompressed data. |
| [in] | allow_multithread | Whether multithreaded decompression is allowed. |
Definition at line 578 of file BinaryFile.h.
References cast(), BinaryCompressionObject::compressed_data, BinaryCompressionObject::compressed_size, BinaryCompressionObject::compression_mode, createCompressionObject(), Compressor::DecompressData(), e_no_compression, e_string_compression, e_string_reference, m_cache_location, m_cached_data, m_compressed_data, m_compressions, m_file, m_large_compression, m_multithreaded_compression, m_using_cache, read(), BinaryCompressionObject::uncompressed_data, and BinaryCompressionObject::uncompressed_size.
|
inline |
Reads a buffer of buffer elements (nested buffers) with decompression.
| [in] | data | The buffer of buffers to populate. |
| [in] | allow_multithread | Whether multithreaded decompression is allowed. |
Definition at line 531 of file BinaryFile.h.
References read().
Reads raw data into a character buffer up to a maximum size.
| [in] | string | The character buffer to populate. |
| [in] | max_size | The maximum number of bytes to read. |
Reads raw bytes directly from the file or cache.
| [in] | ptr | Pointer to the destination buffer. |
| [in] | size | Number of bytes to read. |
| bool BinaryFile::readDirectlyTo | ( | File | file | ) |
|
inline |
Reads a compressed buffer synchronously, disabling multithreaded decompression.
| [in] | data | The buffer to populate with decompressed data. |
Definition at line 520 of file BinaryFile.h.
References read().
| uint04 BinaryFile::readString | ( | char * | string, |
| char | terminator = '\0' ) |
Reads a string into a character buffer up to a terminating character.
| [in] | string | The character buffer to populate. |
| [in] | terminator | The character that marks the end of the string. |
| void BinaryFile::readString | ( | String & | string, |
| char | terminator = '\0' ) |
Reads a string up to a terminating character.
| [in] | string | The string to populate with the read data. |
| [in] | terminator | The character that marks the end of the string. |
|
inline |
Reads a buffer of strings using the string reference table for decompression.
| [in] | data | The buffer of strings to populate. |
| [in] | allow_multithread | Whether multithreaded decompression is allowed. |
Definition at line 556 of file BinaryFile.h.
References m_string_size_info, m_strings, read(), and Bounds< t_dims, t_type, t_vertex >::span().
| void BinaryFile::readStringData | ( | bool | has_reference_table | ) |
Reads the string reference table from the file.
| [in] | has_reference_table | Whether the file contains a string reference table. |
| void BinaryFile::seek | ( | uint08 | location | ) |
Seeks to an absolute byte position in the file or cache.
| [in] | location | The byte offset to seek to. |
|
inline |
Sets the maximum buffer size for cached operations.
| [in] | max_buffer_size | The maximum number of bytes to buffer. |
Definition at line 729 of file BinaryFile.h.
References m_max_buffer_size.
|
inline |
Enables or disables large (64-bit) compression size headers.
| [in] | use_large_compression | Whether to use 64-bit size fields for compression. |
Definition at line 667 of file BinaryFile.h.
References m_large_compression.
| void BinaryFile::switchToFile | ( | File | file, |
| bool | is_write, | ||
| bool | is_safe ) |
Decompresses a range of pending multithreaded compression objects.
| [in] | start | The starting index in the compression list. |
| [in] | count | The number of compression objects to decompress. |
|
inline |
Writes a buffer of buffer elements (nested buffers) with optional compression.
| [in] | data | The buffer of buffers to write. |
| [in] | compression_mode | The compression mode to use. |
Definition at line 309 of file BinaryFile.h.
References write().
|
inline |
Writes a buffer of non-buffer elements with optional compression.
| [in] | data | The buffer to write. |
| [in] | compression_mode | The compression mode to use. |
Definition at line 265 of file BinaryFile.h.
References BinaryCompressionObject::buffer_size, cast(), Compressor::CompressData(), BinaryCompressionObject::compressed_data, BinaryCompressionObject::compressed_size, BinaryCompressionObject::compression_mode, createCompressionObject(), e_no_compression, e_string_compression, e_string_reference, GetTypeInfo(), m_compressed_data, m_large_compression, BinaryCompressionObject::object_type, Compressor::PickCompressionMode(), BinaryCompressionObject::uncompressed_data, BinaryCompressionObject::uncompressed_size, write(), and writeCompression().
|
inline |
Writes a string view with optional compression.
| [in] | data | The string view to write. |
| [in] | compression_mode | The compression mode to use. |
Definition at line 333 of file BinaryFile.h.
References StringView::begin(), BinaryCompressionObject::buffer_size, cast(), Compressor::CompressData(), BinaryCompressionObject::compressed_data, BinaryCompressionObject::compressed_size, BinaryCompressionObject::compression_mode, createCompressionObject(), e_string_compression, e_string_reference, GetTypeInfo(), m_compressed_data, m_large_compression, BinaryCompressionObject::object_type, Compressor::PickCompressionMode(), StringView::size(), BinaryCompressionObject::uncompressed_data, BinaryCompressionObject::uncompressed_size, write(), and writeCompression().
|
inline |
Writes a non-buffer type as raw bytes to the file.
| [in] | data | The value to write. |
Definition at line 254 of file BinaryFile.h.
References writeDirect().
| void BinaryFile::writeCompression | ( | BinaryCompressionObject & | compression_object | ) |
| bool BinaryFile::writeDirectlyFrom | ( | File | file | ) |
|
inline |
Writes a string as raw data padded or truncated to a fixed size.
| [in] | data | The string data to write. |
| [in] | size | The fixed size in bytes to write. |
| [in] | fill_space | The character used to pad if data is shorter than size. |
Definition at line 390 of file BinaryFile.h.
References StringView::begin(), getMin(), m_cache_location, m_cached_data, m_file, m_using_cache, and StringView::size().
| void BinaryFile::writeStringData | ( | CompressionMode | compression_mode | ) |
Writes the accumulated string reference table to the file.
| [in] | compression_mode | The compression mode to use for the string data. |
| void BinaryFile::writeTerminatingString | ( | const StringView & | string, |
| char | terminator = '\0' ) |
Writes a string followed by a terminating character.
| [in] | string | The string to write. |
| [in] | terminator | The terminating character appended after the string. |