dataPreparation (version 0.4.3)

findAndTransformDates: Identify date columns

Description

Find and transform dates that are hidden in a character column. It use a bunch of default formats, and you can also add your own formats.

Usage

findAndTransformDates(
  dataSet,
  cols = "auto",
  formats = NULL,
  n_test = 30,
  ambiguities = "IGNORE",
  verbose = TRUE
)

Arguments

dataSet

Matrix, data.frame or data.table

cols

List of column(s) name(s) of dataSet to look into. To check all all columns, set it to "auto". (characters, default to "auto")

formats

List of additional Date formats to check (see strptime)

n_test

Number of non-null rows on which to test (numeric, default to 30)

ambiguities

How ambiguities should be treated (see details in ambiguities section) (character, default to IGNORE)

verbose

Should the algorithm talk? (Logical, default to TRUE)

Value

dataSet set (as a data.table) with identified dates transformed by reference.

Ambiguity

Ambiguities are often present in dates. For example, in date: 2017/01/01, there is no way to know if format is YYYY/MM/DD or YYYY/DD/MM. Some times ambiguity can be solved by a human. For example 17/12/31, a human might guess that it is YY/MM/DD, but there is no sure way to know. To be safe, findAndTransformDates doesn't try to guess ambiguities. To answer ambiguities problem, param ambiguities is now available. It can take one of the following values

• IGNORE function will then take the first format which match (fast, but can make some mistakes)

• WARN function will try all format and tell you - via prints - that there are multiple matches (and won't perform date transformation)

• SOLVE function will try to solve ambiguity by going through more lines, so will be slower. If it is able to solve it, it will transform the column, if not it will print the various acceptable formats.

If there are some columns that have no chance to be a match think of removing them from cols to save some computation time.

Details

This function is using identifyDates to find formats. Please see it's documentation. In case identifyDates doesn't find wanted formats you can either provide format in param formats or use setColAsDate to force transformation.

Examples

# NOT RUN {
# Load exemple set
data(messy_adult)
head(messy_adult)
# using the findAndTransformDates
findAndTransformDates(messy_adult, n_test = 5)
head(messy_adult)

# Example with ambiguities
# }
# NOT RUN {
require(data.table)
data(messy_adult) # reload data
# Add an ambiguity by sorting date1
messy_adult$date1 = sort(messy_adult$date1, na.last = TRUE)
# Try all three methods:
result_1 = findAndTransformDates(copy(messy_adult))
result_2 = findAndTransformDates(copy(messy_adult), ambiguities = "WARN")
result_3 = findAndTransformDates(copy(messy_adult), ambiguities = "SOLVE")
# }
# NOT RUN {
# "##NOT RUN:" mean that this example hasn't been run on CRAN since its long. But you can run it!
# }