collapse (version 1.5.0)

qF, qG, finteraction: Fast Factor Generation, Interactions and Vector Grouping

Description

qF, shorthand for 'quick-factor' implements very fast (ordered) factor generation from atomic vectors using either radix ordering or index hashing.

qG, shorthand for 'quick-group', generates a kind of factor-light without the levels attribute but instead an attribute providing the number of levels. Optionally the levels / groups can be attached, but without converting them to character. Objects have a class 'qG'.

finteraction generates a factor by interacting multiple vectors or factors. In that process missing values are always replaced with a level and unused levels are always dropped.

Usage

qF(x, ordered = FALSE, na.exclude = TRUE, sort = TRUE, drop = FALSE,
   keep.attr = TRUE, method = c("auto", "radix", "hash"))
qG(x, ordered = FALSE, na.exclude = TRUE, sort = TRUE,
   return.groups = FALSE, method = c("auto", "radix", "hash"))
is.qG(x)
as.factor_qG(x, ordered = FALSE, na.exclude = TRUE)
finteraction(…, ordered = FALSE, sort = TRUE)

Arguments

x

a atomic vector, factor or quick-group.

ordered

logical. Adds a class 'ordered'.

na.exclude

logical. TRUE preserves missing values (i.e. no level is generated for NA).

sort

logical. TRUE sorts the levels in ascending order (like factor); FALSE provides the levels in an arbitrary order (determined by the hash map if method = "hash"), which can be slightly faster.

drop

logical. If x is a factor, TRUE efficiently drops unused factor levels beforehand using fdroplevels.

keep.attr

logical. If TRUE and x has additional attributes apart from 'levels' and 'class', these are preserved in the conversion to factor.

method

an integer or character string specifying the method of computation:

Int.		String		Description
1		"auto"		automatic selection: hash for character, logical or if length(x) < 500, else radix.
2		"radix"		use radix ordering to generate factors. See Details.
3		"hash"		use index hashing to generate factors. See Details.

return.groups

logical. TRUE returns the unique elements / groups / levels of x in an attribute called 'groups'. Unlike qF, they are not converted to character.

…

multiple atomic vectors or factors, or a single list of equal-length vectors or factors. See Details.

Value

qF returns an (ordered) factor. qG returns an object of class 'qG': an integer grouping vector with an attribute 'N.groups' indicating the number of groups, and, if return.groups = TRUE, an attribute 'groups' containing the vector of unique groups / elements in x corresponding to the integer-id.

Details

These functions are quite important. Whenever a vector is passed to a collapse function such as fmean(mtcars, mtcars$cyl), is is grouped using qF or qG.

qF is a combination of as.factor and factor. Applying it to a vector i.e. qF(x) gives the same result as as.factor(x). qF(x, ordered = TRUE) generates and ordered factor (same as factor(x, ordered = TRUE)), and qF(x, na.exclude = FALSE) generates a level for missing values (same as factor(x, exclude = NULL)). An important addition is that qF(x, na.exclude = FALSE) also adds a class 'na.included'. This prevents collapse functions from checking missing values in the factor, and is thus computationally more efficient. Therefore factors used in grouped operations should preferably be generated using qF(x, na.exclude = FALSE). Setting sort = FALSE gathers the levels in a random order (unless method = "radix" and x is numeric, in which case the levels are always sorted). This can provide a speed improvement, particularly for character data.

There are two methods of computation: radix ordering and index hashing. Radix ordering is done through combining the functions radixorder and groupid. It is generally faster than index hashing for large numeric data (although there are exceptions). Index hashing is done using Rcpp::sugar::sort_unique and Rcpp::sugar::match. It is generally faster for character data. For logical data, a super fast one-pass method was written which is subsumed in the hash method. Regarding speed: In general qF is around 5x faster than as.factor on character data and about 30x faster on numeric data. Automatic method dispatch typically does a good job delivering optimal performance.

qG is in the first place a programmers function. It generates a factor-'light' consisting of only an integer grouping vector and an attribute providing the number of groups. It is faster and more memory efficient than GRP for grouping atomic vectors, which is the main reason it exists. The fact that it (optionally) returns the unique groups / levels without converting them to character is an added bonus (this also provides a small performance gain compared to qF).

finteraction is simply a wrapper around as.factor_GRP(GRP.default(X, sort = TRUE)), where X is replaced by the arguments in '…' combined in a list. See GRP for computational details. In general: All vectors, factors, or lists of vectors / factors passed can be interacted. Interactions always create a level for missing values and always drop any unused levels.

See Also

groupid, GRP, Fast Grouping and Ordering, Collapse Overview

Examples

# NOT RUN {
cylF <- qF(mtcars$cyl)     # Factor from atomic vector
cylG <- qG(mtcars$cyl)     # Quick-group from atomic vector
cylG                       # See the simple structure of this object

cf  <- qF(wlddev$country)  # Bigger data
cf2 <- qF(wlddev$country, na.exclude = FALSE)  # With na.included class
dat <- num_vars(wlddev)
# }
# NOT RUN {
 <!-- % No code relying on suggested package -->
# cf2 is faster in grouped operations because no missing value check is performed
library(microbenchmark)
microbenchmark(fmax(dat, cf), fmax(dat, cf2))
# }
# NOT RUN {
finteraction(mtcars$cyl, mtcars$vs)  # Interacting two variables (can be factors)
head(finteraction(mtcars))           # A more crude example..

# }