A wrapper around the core iotools functions to easily apply a function over chunks of a large file. Results can be either written to a file or returned as an internal list.
chunk.map(input, output = NULL, formatter = .default.formatter,
FUN, key.sep = NULL, max.line = 65536L,
max.size = 33554432L, output.sep = ",", output.nsep = "\t",
output.keys = FALSE, skip = 0L, ...)A list of results when output is NULL; otherwise no output is returned.
an input connection or character vector describing a local file.
an optional output connection or character vector describing a local file.
If NULL, the results are returned internally as a list.
a function that takes raw input and produces the input given to FUN
a user provided function to map over the chunks. The result of FUN is either
wrapper in a list item, when output is NULL, or written to the output file
using as.output
optional key separator given to chunk.reader
maximum number of lines given to chunk.reader
maximum size of a block as given to read.chunk
single character giving the field separator in the output.
single character giving the key separator in the output.
logical. Whether as.output should interpret row names as keys.
integer giving the number of lines to strip off the input before reading. Useful when the input contains a row a column headers
additional parameters to pass to FUN
Taylor Arnold