NDEVR
API Documentation
BinaryFile

Logic for reading or writing to a binary file including logic for compressing or decompressing the file. More...

Collaboration diagram for BinaryFile:
[legend]

Public Member Functions

 BinaryFile ()
 Constructs an empty BinaryFile backed by an in-memory cache.
 BinaryFile (BinaryFile &&bf) noexcept
 Move constructor.
 BinaryFile (const Buffer< uint01 > &data)
 Constructs a BinaryFile from a Buffer of bytes.
 BinaryFile (const File &file)
 Constructs a BinaryFile associated with the given file on disk.
 BinaryFile (const HighCapacityBuffer< uint01 > &data)
 Constructs a BinaryFile from an existing data buffer (copy).
 BinaryFile (const uint01 *data, uint08 size)
 Constructs a BinaryFile from a raw pointer and size.
 BinaryFile (HighCapacityBuffer< uint01 > &&data)
 Constructs a BinaryFile from an existing data buffer (move).
 ~BinaryFile ()
 Destructor.
HighCapacityBuffer< uint01 > & cachedData ()
 Returns a mutable reference to the in-memory cached data.
const HighCapacityBuffer< uint01 > & cachedData () const
 Returns a const reference to the in-memory cached data.
void cachedOpen (bool read)
 Opens the file and reads its entire contents into an in-memory cache.
template<class t_type>
bool canRead () const
 Checks whether enough data remains to read a value of the given type.
void close ()
 Closes the file and flushes any pending data.
BinaryCompressionObject createCompressionObject ()
 Creates a BinaryCompressionObject initialized with this file's compression settings.
Filefile ()
 Returns a mutable reference to the underlying File object.
const Filefile () const
 Returns a const reference to the underlying File object.
uint08 fileSize () const
 Returns the size of the underlying file in bytes.
bool isFinished ()
 Checks whether the file has been fully read.
void open (bool read, bool safe)
 Opens the file for reading or writing.
BinaryFileoperator= (BinaryFile &&bf) noexcept
 Move assignment operator.
fltp04 percent () const
 Returns the current read progress as a percentage of the total file size.
uint08 position () const
 Returns the current read or write position in the file or cache.
template<class t_type>
t_type read ()
 Reads a single value of the given type from the file or cache.
template<class t_type, class t_memory_manager>
void read (Buffer< t_type, t_memory_manager > &data)
 Reads a compressed buffer using the file's multithreading setting.
template<class t_type, class t_memory_manager>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type read (Buffer< t_type, t_memory_manager > &data, bool &allow_multithread)
 Reads a buffer of non-buffer elements with decompression.
template<class t_type, class t_memory_manager>
std::enable_if< ObjectInfo< t_type >::Buffer >::type read (Buffer< t_type, t_memory_manager > &data, bool &allow_multithread)
 Reads a buffer of buffer elements (nested buffers) with decompression.
uint04 readData (char *string, uint04 max_size)
 Reads raw data into a character buffer up to a maximum size.
String readData (uint04 max_size)
 Reads raw data up to a maximum size and returns it as a String.
uint08 readDirect (uint01 *ptr, uint08 size)
 Reads raw bytes directly from the file or cache.
bool readDirectlyTo (File file)
 Reads the cached data directly to a file on disk.
template<class t_type, class t_memory_manager>
void readNow (Buffer< t_type, t_memory_manager > &data)
 Reads a compressed buffer synchronously, disabling multithreaded decompression.
uint04 readString (char *string, char terminator='\0')
 Reads a string into a character buffer up to a terminating character.
void readString (String &string, char terminator='\0')
 Reads a string up to a terminating character.
template<class t_memory_manager>
void readStringBuffer (Buffer< String, t_memory_manager > &data, bool &allow_multithread)
 Reads a buffer of strings using the string reference table for decompression.
void readStringData (bool has_reference_table)
 Reads the string reference table from the file.
void seek (uint08 location)
 Seeks to an absolute byte position in the file or cache.
void setMaxBufferSize (uint08 max_buffer_size)
 Sets the maximum buffer size for cached operations.
void setUseLargeCompression (bool use_large_compression)
 Enables or disables large (64-bit) compression size headers.
template<class t_type>
void skip ()
 Advances the read position by the size of the given type without reading.
void switchToFile (File file, bool is_write, bool is_safe)
 Switches the underlying file to a different file and reopens it.
void uncompressAll ()
 Decompresses all pending multithreaded compression objects.
void uncompressSections (uint04 start, uint04 count)
 Decompresses a range of pending multithreaded compression objects.
template<class t_type, class t_memory_manager>
std::enable_if< ObjectInfo< t_type >::Buffer >::type write (const Buffer< t_type, t_memory_manager > &data, CompressionMode compression_mode)
 Writes a buffer of buffer elements (nested buffers) with optional compression.
template<class t_type, class t_memory_manager>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type write (const Buffer< t_type, t_memory_manager > &data, CompressionMode compression_mode=CompressionMode::e_no_compression)
 Writes a buffer of non-buffer elements with optional compression.
void write (const StringView &data, CompressionMode compression_mode)
 Writes a string view with optional compression.
template<class t_type>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type write (const t_type &data)
 Writes a non-buffer type as raw bytes to the file.
void writeCompression (BinaryCompressionObject &compression_object)
 Writes a compression object's data to the file.
void writeDirect (const uint01 *ptr, uint08 size)
 Writes raw bytes directly to the file or cache.
bool writeDirectlyFrom (File file)
 Writes data from a file on disk directly into the cache.
void writeRawData (const StringView &data, uint04 size, char fill_space='\0')
 Writes a string as raw data padded or truncated to a fixed size.
void writeStringData (CompressionMode compression_mode)
 Writes the accumulated string reference table to the file.
void writeTerminatingString (const StringView &string, char terminator='\0')
 Writes a string followed by a terminating character.

Protected Attributes

fltp08 fltp_error = Constant<fltp08>::Invalid
 Floating-point error tolerance used during compression.
uint08 m_cache_location = Constant<uint08>::Invalid
 Current read/write position within the cached data.
HighCapacityBuffer< uint01m_cached_data
 In-memory cache of the file's raw byte contents.
HighCapacityBuffer< uint01m_compressed_data
 Temporary buffer for compressed data during read/write operations.
Buffer< BinaryCompressionObjectm_compressions
 Pending compression objects for multithreaded decompression.
File m_file
 The underlying file handle for disk-based I/O.
uint08 m_file_size = Constant<uint08>::Invalid
 Cached file size to avoid repeated filesystem queries.
bool m_is_read = false
 Whether the file was opened for reading.
bool m_large_compression = false
 Whether to use 64-bit size fields for compression headers.
uint08 m_max_buffer_size = Constant<uint04>::Max
 Maximum buffer size for cached read/write operations.
bool m_multithreaded_compression = true
 Whether multithreaded decompression is enabled.
Dictionary< String, uint04m_string_reference
 Maps strings to their index in the string reference table.
Buffer< Bounds< 1, uint04 > > m_string_size_info
 Stores the start and end bounds of each string in m_strings.
String m_strings
 Accumulated string data for the string reference table.
bool m_using_cache = false
 Whether the file is operating in cached (in-memory) mode.

Detailed Description

Logic for reading or writing to a binary file including logic for compressing or decompressing the file.


See also
File, Compressor, BinaryFileTableInfo

Definition at line 135 of file BinaryFile.h.

Constructor & Destructor Documentation

◆ BinaryFile() [1/6]

BinaryFile::BinaryFile ( const File & file)
inlineexplicit

Constructs a BinaryFile associated with the given file on disk.

Parameters
[in]fileThe file to read from or write to.

Definition at line 143 of file BinaryFile.h.

References file(), and m_file.

Referenced by BinaryFile(), and operator=().

◆ BinaryFile() [2/6]

BinaryFile::BinaryFile ( const HighCapacityBuffer< uint01 > & data)
inlineexplicit

Constructs a BinaryFile from an existing data buffer (copy).

Parameters
[in]dataThe buffer of raw bytes to use as cached data.

Definition at line 159 of file BinaryFile.h.

References m_cache_location, m_cached_data, m_is_read, and m_using_cache.

◆ BinaryFile() [3/6]

BinaryFile::BinaryFile ( HighCapacityBuffer< uint01 > && data)
inlineexplicit

Constructs a BinaryFile from an existing data buffer (move).

Parameters
[in]dataThe buffer of raw bytes to move into the cached data.

Definition at line 169 of file BinaryFile.h.

References m_cache_location, m_cached_data, m_is_read, and m_using_cache.

◆ BinaryFile() [4/6]

BinaryFile::BinaryFile ( const uint01 * data,
uint08 size )
inlineexplicit

Constructs a BinaryFile from a raw pointer and size.

Parameters
[in]dataPointer to the raw byte data.
[in]sizeNumber of bytes to copy from the pointer.

Definition at line 181 of file BinaryFile.h.

References m_cache_location, m_cached_data, m_is_read, and m_using_cache.

◆ BinaryFile() [5/6]

BinaryFile::BinaryFile ( const Buffer< uint01 > & data)
inlineexplicit

Constructs a BinaryFile from a Buffer of bytes.

Parameters
[in]dataThe buffer of bytes to copy into the cached data.

Definition at line 192 of file BinaryFile.h.

References m_cache_location, m_cached_data, m_is_read, and m_using_cache.

◆ BinaryFile() [6/6]

BinaryFile::BinaryFile ( BinaryFile && bf)
noexcept

Move constructor.

Parameters
[in]bfThe BinaryFile to move from.

References BinaryFile().

◆ ~BinaryFile()

BinaryFile::~BinaryFile ( )
inline

Destructor.

Closes the file if open.

Definition at line 213 of file BinaryFile.h.

References close().

Member Function Documentation

◆ cachedData() [1/2]

HighCapacityBuffer< uint01 > & BinaryFile::cachedData ( )
inline

Returns a mutable reference to the in-memory cached data.

Returns
Mutable reference to the cached byte buffer.

Definition at line 244 of file BinaryFile.h.

References m_cached_data.

◆ cachedData() [2/2]

const HighCapacityBuffer< uint01 > & BinaryFile::cachedData ( ) const
inline

Returns a const reference to the in-memory cached data.

Returns
Const reference to the cached byte buffer.

Definition at line 236 of file BinaryFile.h.

References m_cached_data.

◆ cachedOpen()

void BinaryFile::cachedOpen ( bool read)

Opens the file and reads its entire contents into an in-memory cache.

Parameters
[in]readWhether to open for reading (true) or writing (false).

References read().

◆ canRead()

template<class t_type>
bool BinaryFile::canRead ( ) const
inlinenodiscard

Checks whether enough data remains to read a value of the given type.

Returns
True if there are sufficient bytes remaining to read a value of type t_type.

Definition at line 427 of file BinaryFile.h.

References m_cache_location, m_cached_data, m_file, m_using_cache, and position().

◆ createCompressionObject()

BinaryCompressionObject BinaryFile::createCompressionObject ( )

Creates a BinaryCompressionObject initialized with this file's compression settings.

Returns
A new BinaryCompressionObject configured for this file.

Referenced by read(), write(), and write().

◆ file() [1/2]

File & BinaryFile::file ( )
inline

Returns a mutable reference to the underlying File object.

Returns
Mutable reference to the File.

Definition at line 719 of file BinaryFile.h.

References m_file.

◆ file() [2/2]

const File & BinaryFile::file ( ) const
inline

Returns a const reference to the underlying File object.

Returns
Const reference to the File.

Definition at line 714 of file BinaryFile.h.

References m_file.

Referenced by BinaryFile(), readDirectlyTo(), switchToFile(), and writeDirectlyFrom().

◆ fileSize()

uint08 BinaryFile::fileSize ( ) const

Returns the size of the underlying file in bytes.

Returns
The file size in bytes.

◆ isFinished()

bool BinaryFile::isFinished ( )

Checks whether the file has been fully read.

Returns
True if the current position is at or past the end of the file data.

◆ open()

void BinaryFile::open ( bool read,
bool safe )

Opens the file for reading or writing.

Parameters
[in]readWhether to open for reading (true) or writing (false).
[in]safeWhether to use safe mode, which writes to a temporary file first.

References read().

◆ operator=()

BinaryFile & BinaryFile::operator= ( BinaryFile && bf)
noexcept

Move assignment operator.

Parameters
[in]bfThe BinaryFile to move from.
Returns
A reference to this BinaryFile.

References BinaryFile().

◆ percent()

fltp04 BinaryFile::percent ( ) const

Returns the current read progress as a percentage of the total file size.

Returns
A value between 0.0 and 1.0 representing the read progress.

◆ position()

uint08 BinaryFile::position ( ) const

Returns the current read or write position in the file or cache.

Returns
The current byte offset position.

Referenced by canRead().

◆ read() [1/4]

template<class t_type>
t_type BinaryFile::read ( )
inlinenodiscard

Reads a single value of the given type from the file or cache.

Returns
The value read from the current position.

Definition at line 441 of file BinaryFile.h.

References cast(), m_cache_location, m_cached_data, m_file, and m_using_cache.

Referenced by cachedOpen(), open(), read(), read(), readNow(), and readStringBuffer().

◆ read() [2/4]

template<class t_type, class t_memory_manager>
void BinaryFile::read ( Buffer< t_type, t_memory_manager > & data)
inline

Reads a compressed buffer using the file's multithreading setting.

Parameters
[in]dataThe buffer to populate with decompressed data.

Definition at line 510 of file BinaryFile.h.

References m_multithreaded_compression, and read().

◆ read() [3/4]

template<class t_type, class t_memory_manager>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type BinaryFile::read ( Buffer< t_type, t_memory_manager > & data,
bool & allow_multithread )
inline

◆ read() [4/4]

template<class t_type, class t_memory_manager>
std::enable_if< ObjectInfo< t_type >::Buffer >::type BinaryFile::read ( Buffer< t_type, t_memory_manager > & data,
bool & allow_multithread )
inline

Reads a buffer of buffer elements (nested buffers) with decompression.

Parameters
[in]dataThe buffer of buffers to populate.
[in]allow_multithreadWhether multithreaded decompression is allowed.

Definition at line 531 of file BinaryFile.h.

References read().

◆ readData() [1/2]

uint04 BinaryFile::readData ( char * string,
uint04 max_size )

Reads raw data into a character buffer up to a maximum size.

Parameters
[in]stringThe character buffer to populate.
[in]max_sizeThe maximum number of bytes to read.
Returns
The number of bytes actually read.

◆ readData() [2/2]

String BinaryFile::readData ( uint04 max_size)

Reads raw data up to a maximum size and returns it as a String.

Parameters
[in]max_sizeThe maximum number of bytes to read.
Returns
The data read as a String.

◆ readDirect()

uint08 BinaryFile::readDirect ( uint01 * ptr,
uint08 size )

Reads raw bytes directly from the file or cache.

Parameters
[in]ptrPointer to the destination buffer.
[in]sizeNumber of bytes to read.
Returns
The number of bytes actually read.

◆ readDirectlyTo()

bool BinaryFile::readDirectlyTo ( File file)

Reads the cached data directly to a file on disk.

Parameters
[in]fileThe destination file to write to.
Returns
True if the operation was successful.

References file().

◆ readNow()

template<class t_type, class t_memory_manager>
void BinaryFile::readNow ( Buffer< t_type, t_memory_manager > & data)
inline

Reads a compressed buffer synchronously, disabling multithreaded decompression.

Parameters
[in]dataThe buffer to populate with decompressed data.

Definition at line 520 of file BinaryFile.h.

References read().

◆ readString() [1/2]

uint04 BinaryFile::readString ( char * string,
char terminator = '\0' )

Reads a string into a character buffer up to a terminating character.

Parameters
[in]stringThe character buffer to populate.
[in]terminatorThe character that marks the end of the string.
Returns
The number of characters read.

◆ readString() [2/2]

void BinaryFile::readString ( String & string,
char terminator = '\0' )

Reads a string up to a terminating character.

Parameters
[in]stringThe string to populate with the read data.
[in]terminatorThe character that marks the end of the string.

◆ readStringBuffer()

template<class t_memory_manager>
void BinaryFile::readStringBuffer ( Buffer< String, t_memory_manager > & data,
bool & allow_multithread )
inline

Reads a buffer of strings using the string reference table for decompression.

Parameters
[in]dataThe buffer of strings to populate.
[in]allow_multithreadWhether multithreaded decompression is allowed.

Definition at line 556 of file BinaryFile.h.

References m_string_size_info, m_strings, read(), and Bounds< t_dims, t_type, t_vertex >::span().

◆ readStringData()

void BinaryFile::readStringData ( bool has_reference_table)

Reads the string reference table from the file.

Parameters
[in]has_reference_tableWhether the file contains a string reference table.

◆ seek()

void BinaryFile::seek ( uint08 location)

Seeks to an absolute byte position in the file or cache.

Parameters
[in]locationThe byte offset to seek to.

◆ setMaxBufferSize()

void BinaryFile::setMaxBufferSize ( uint08 max_buffer_size)
inline

Sets the maximum buffer size for cached operations.

Parameters
[in]max_buffer_sizeThe maximum number of bytes to buffer.

Definition at line 729 of file BinaryFile.h.

References m_max_buffer_size.

◆ setUseLargeCompression()

void BinaryFile::setUseLargeCompression ( bool use_large_compression)
inline

Enables or disables large (64-bit) compression size headers.

Parameters
[in]use_large_compressionWhether to use 64-bit size fields for compression.

Definition at line 667 of file BinaryFile.h.

References m_large_compression.

◆ switchToFile()

void BinaryFile::switchToFile ( File file,
bool is_write,
bool is_safe )

Switches the underlying file to a different file and reopens it.

Parameters
[in]fileThe new file to use.
[in]is_writeWhether to open for writing.
[in]is_safeWhether to use safe write mode.

References file().

◆ uncompressSections()

void BinaryFile::uncompressSections ( uint04 start,
uint04 count )

Decompresses a range of pending multithreaded compression objects.

Parameters
[in]startThe starting index in the compression list.
[in]countThe number of compression objects to decompress.

◆ write() [1/4]

template<class t_type, class t_memory_manager>
std::enable_if< ObjectInfo< t_type >::Buffer >::type BinaryFile::write ( const Buffer< t_type, t_memory_manager > & data,
CompressionMode compression_mode )
inline

Writes a buffer of buffer elements (nested buffers) with optional compression.

Parameters
[in]dataThe buffer of buffers to write.
[in]compression_modeThe compression mode to use.

Definition at line 309 of file BinaryFile.h.

References write().

◆ write() [2/4]

template<class t_type, class t_memory_manager>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type BinaryFile::write ( const Buffer< t_type, t_memory_manager > & data,
CompressionMode compression_mode = CompressionMode::e_no_compression )
inline

◆ write() [3/4]

◆ write() [4/4]

template<class t_type>
std::enable_if<!ObjectInfo< t_type >::Buffer >::type BinaryFile::write ( const t_type & data)
inline

Writes a non-buffer type as raw bytes to the file.

Parameters
[in]dataThe value to write.

Definition at line 254 of file BinaryFile.h.

References writeDirect().

Referenced by write(), write(), and write().

◆ writeCompression()

void BinaryFile::writeCompression ( BinaryCompressionObject & compression_object)

Writes a compression object's data to the file.

Parameters
[in]compression_objectThe compression object containing data to write.

Referenced by write(), and write().

◆ writeDirect()

void BinaryFile::writeDirect ( const uint01 * ptr,
uint08 size )

Writes raw bytes directly to the file or cache.

Parameters
[in]ptrPointer to the data to write.
[in]sizeNumber of bytes to write.

Referenced by write().

◆ writeDirectlyFrom()

bool BinaryFile::writeDirectlyFrom ( File file)

Writes data from a file on disk directly into the cache.

Parameters
[in]fileThe source file to read from.
Returns
True if the operation was successful.

References file().

◆ writeRawData()

void BinaryFile::writeRawData ( const StringView & data,
uint04 size,
char fill_space = '\0' )
inline

Writes a string as raw data padded or truncated to a fixed size.

Parameters
[in]dataThe string data to write.
[in]sizeThe fixed size in bytes to write.
[in]fill_spaceThe character used to pad if data is shorter than size.

Definition at line 390 of file BinaryFile.h.

References StringView::begin(), getMin(), m_cache_location, m_cached_data, m_file, m_using_cache, and StringView::size().

◆ writeStringData()

void BinaryFile::writeStringData ( CompressionMode compression_mode)

Writes the accumulated string reference table to the file.

Parameters
[in]compression_modeThe compression mode to use for the string data.

◆ writeTerminatingString()

void BinaryFile::writeTerminatingString ( const StringView & string,
char terminator = '\0' )

Writes a string followed by a terminating character.

Parameters
[in]stringThe string to write.
[in]terminatorThe terminating character appended after the string.

The documentation for this class was generated from the following file: