quanteda (version 0.99.22)

regex2fixed: convert regex and glob patterns to type IDs or fixed patterns

Description

regex2fixed converts regex and glob patterns to fixed patterns.

regex2id converts regex or glob to type IDs to allow C++ function to perform fast searches in tokens object. C++ functions use a list of type IDs to construct a hash table, against which sub-vectors of tokens object are matched. This function constructs index of glob patterns for faster matching.

This is an internal function for regex2id() that select types using an index of types by reular expressions.

An internal function for regex2id that constructs an index of regex patterns (e.g. ^xxxx, xxxx$ and ^xxxx$) to avoid expensive sequential search by stri_detect_regex.

internal function for select_types() to search the index using fastmatch.

simpler and faster version of expand.grid() in base package

internal function for select_types() to check if a glob pattern is indexed by index_types()

Usage

regex2fixed(pattern, types = NULL, valuetype = NULL,
  case_insensitive = NULL, index = NULL)
regex2id(pattern, types = NULL, valuetype = NULL, case_insensitive = NULL,
  index = NULL)
search_glob(patterns, types_search, index)
search_regex(patterns, types_search, case_insensitive)
search_fixed(patterns, types_search, index)
index_types(types, valuetype, case_insensitive, max_len = NULL)
search_index(pattern, index)
expand(elem)
is_indexed(pattern)

Arguments

pattern

a character vector, list of character vectors, dictionary, collocations, or dfm. See pattern for details.

types

unique types of tokens obtained by quanteda:::types()

valuetype

the type of pattern matching: "glob" for "glob"-style wildcard expressions; "regex" for regular expressions; or "fixed" for exact matching. See valuetype for details.

case_insensitive

ignore case when matching, if TRUE

index

If TRUE, index is constructed automatically. It also accept index constructed by index_types().

patterns

a list of regular expressions

types_search

lowercased types when case_insensitive=TRUE, but not used in glob and fixed matching as types are in the index.

max_len

maximum length of types to be indexed

elem

list of elements to be combined

case_insensitive

ignore case when matching, if TRUE, but not used in glob and fixed matching as types are lowercased in the index.

index

index object created by index_types()

types

types of tokens to index

case_insensitive

ignore case when matching, if TRUE

regex

a glob expression to search

index

an index object created by index_types()

x

a glob pattern to be tested

Value

regex2fixed returns a list of character vectors containing types

regex2id returns a list of integer vectors containing type IDs

a list of integer vectors containing type IDs with index keys as an attribute

See Also

index_types

Examples

# NOT RUN {
pattern <- list(c('^a$', '^b'), c('c'), c('d'))
types <- c('A', 'AA', 'B', 'BB', 'BBB', 'C', 'CC')
quanteda:::regex2fixed(pattern, types, 'regex', case_insensitive = TRUE)
index <- quanteda:::index_types(types, 'regex', case_insensitive = TRUE)
quanteda:::regex2fixed(pattern, index = index)
types <- c('A', 'AA', 'B', 'BB', 'BBB', 'C', 'CC')

pats_regex <- list(c('^a$', '^b'), c('c'), c('d'))
quanteda:::regex2id(pats_regex, types, 'regex', case_insensitive = TRUE)

pats_glob <- list(c('a*', 'b*'), c('c'), c('d'))
quanteda:::regex2id(pats_glob, types, 'glob', case_insensitive = TRUE)

index <- quanteda:::index_types(c('xxx', 'yyyy', 'ZZZ'), 'glob', FALSE, 3)
quanteda:::search_glob('yy*', attr(index, 'type_search'), index)
quanteda:::expand(list(c('a', 'b', 'c'), c('x', 'y')))
# }