datadr (version 0.8.6.1)

drPersist: Persist a Transformed 'ddo' or 'ddf' Object

Description

Persist a transformed 'ddo' or 'ddf' object by making a deferred transformation permanent

Usage

drPersist(x, output = NULL, overwrite = FALSE, control = NULL)

Arguments

x

an object of class 'ddo' or 'ddf'

output

a "kvConnection" object indicating where the output data should reside (see localDiskConn, hdfsConn). If NULL (default), output will be an in-memory "ddo" object.

overwrite

logical; should existing output location be overwritten? (also can specify overwrite = "backup" to move the existing output to _bak)

control

parameters specifying how the backend should handle things (most-likely parameters to rhwatch in RHIPE) - see rhipeControl and localDiskControl

Value

a 'ddo' or 'ddf' object with the transformation evaluated on the data

Details

When a transformation is added to a ddf/ddo via addTransform, the transformation is deferred until the some action is taken with the data (e.g. a call to recombine). See the documentation of addTransform for more information about the nature of transformations.

Calling drPersist() on the ddo/ddf makes the transformation permanent (persisted). In the case of a local disk connection (via localDiskConn) or HDFS connection (via hdfsConn), the transformed data are written to disk.

See Also

addTransform

Examples

# NOT RUN {
bySpecies <- divide(iris, by = "Species")

# Create the transformation and add it to bySpecies
bySpeciesSepal <- addTransform(bySpecies, function(x) x[,c("Sepal.Length", "Sepal.Width")])

# Note the transformation is 'pending' a data action
bySpeciesSepal

# Make the tranformation permanent (persistent)
bySpeciesSepalPersisted <- drPersist(bySpeciesSepal)

# The transformation no longer pending--but a permanent part of the new ddo
bySpeciesSepalPersisted
bySpeciesSepalPersisted[[1]]
# }