FileFormat

ParquetFileFormat

IpcFileFormat

A <code>FileFormat</code> holds information about how to read and parse the files
included in a <code>Dataset</code>. There are subclasses corresponding to the supported
file formats (<code>ParquetFileFormat</code> and <code>IpcFileFormat</code>).

'Apache' 'Arrow' <https://arrow.apache.org/> is a cross-language
development platform for in-memory data. It specifies a standardized
language-independent columnar memory format for flat and hierarchical data,
organized for efficient analytic operations on modern hardware. This
package provides an interface to the 'Arrow C++' library.

Neal Richardson

arrow

Integration to 'Apache' 'Arrow'

Romain François

Jeroen Ooms

Javier Luraschi

Jeffrey Wong

Apache Arrow 

FileFormat function

<code>FileFormat$create()</code> takes the following arguments:<ul>
<li><code>format</code>: A string identifier of the format of the files in <code>path</code>.
Currently "parquet" and "ipc"/"arrow"/"feather" (aliases for each other)
are supported. For Feather, only version 2 files are supported.</li>
<li><code>...</code>: Additional format-specific options
format="parquet":<ul>
<li><code>use_buffered_stream</code>: Read files through buffered input streams rather than
loading entire row groups at once. This may be enabled
to reduce memory overhead. Disabled by default.</li>
<li><code>buffer_size</code>: Size of buffered stream, if enabled. Default is 8KB.</li>
<li><code>dict_columns</code>: Names of columns which should be read as dictionaries.</li>
</ul></li>
</ul>It returns the appropriate subclass of <code>FileFormat</code> (e.g. <code>ParquetFileFormat</code>)

Factory

Dataset file formats — FileFormat

Dataset file formats

FileFormat: Dataset file formats

Description

Arguments

Factory