unnest_character_shingles(
tbl,
output,
input,
n = 3L,
n_min = n,
strip_non_alphanum = TRUE,
format = c("text", "man", "latex", "html", "xml"),
to_lower = TRUE,
drop = TRUE,
collapse = NULL,
...
)
Arguments
tbl
A data frame
output
Output column to be created as string or symbol.
input
Input column that gets split as string or symbol.
The output/input arguments are passed by expression and support
quasiquotation; you can unquote strings and symbols.
strip_non_alphanum
Should punctuation and white space be stripped?
format
Either "text", "man", "latex", "html", or "xml". If not text,
this uses the hunspell tokenizer, and can tokenize only by "word"
to_lower
Whether to convert tokens to lowercase. If tokens include
URLS (such as with token = "tweets"), such converted URLs may no
longer be correct.
drop
Whether original input column should get dropped. Ignored
if the original input and new output column have the same name.
collapse
Whether to combine text with newlines first in case tokens
(such as sentences or paragraphs) span multiple lines. If NULL, collapses
when token method is "ngrams", "skip_ngrams", "sentences", "lines",
"paragraphs", or "regex".
# NOT RUN {library(dplyr)
library(janeaustenr)
d <- tibble(txt = prideprejudice)
d %>%
unnest_characters(word, txt)
d %>%
unnest_character_shingles(word, txt, n = 3)
# }