FuzzExtract: Fuzzy extraction from a sequence

Description

Fuzzy extraction from a sequence

Usage

# init <- FuzzExtract$new(decoding = NULL)

Arguments

Methods

FuzzExtract$new(decoding = NULL)

--------------

Extract(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, limit = 5L)

--------------

ExtractBests(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L, limit = 5L)

--------------

ExtractWithoutOrder(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L)

--------------

ExtractOne(string = NULL, sequence_strings = NULL, processor = NULL, scorer = NULL, score_cutoff = 0L)

--------------

Dedupe(contains_dupes = NULL, threshold = 70L, scorer = NULL)

Methods

Public methods

Method `new()`

Usage

FuzzExtract$new(decoding = NULL)

Arguments

decoding: either NULL or a character string. If not NULL then the decoding parameter takes one of the standard python encodings (such as 'utf-8'). See the details and references link for more information.

Method `Extract()`

Usage

FuzzExtract$Extract(
  string = NULL,
  sequence_strings = NULL,
  processor = NULL,
  scorer = NULL,
  limit = 5L
)

Arguments

string: a character string.

sequence_strings

a character string vector

processor

either NULL or a function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. See the examples for more details.

scorer

a function for scoring matches between the query and an individual processed choice. This should be a function of the form f(query, choice) -> int. By default, FuzzMatcher.WRATIO() is used and expects both query and choice to be strings. See the examples for more details.

limit

An integer value for the maximum number of elements to be returned. Defaults to 5L

Method `ExtractBests()`

Usage

FuzzExtract$ExtractBests(
  string = NULL,
  sequence_strings = NULL,
  processor = NULL,
  scorer = NULL,
  score_cutoff = 0L,
  limit = 5L
)

Arguments

string: a character string.

sequence_strings

a character string vector

processor

either NULL or a function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. See the examples for more details.

scorer

score_cutoff

an integer value for the score threshold. No matches with a score less than this number will be returned. Defaults to 0

limit

An integer value for the maximum number of elements to be returned. Defaults to 5L

Method `ExtractWithoutOrder()`

Usage

FuzzExtract$ExtractWithoutOrder(
  string = NULL,
  sequence_strings = NULL,
  processor = NULL,
  scorer = NULL,
  score_cutoff = 0L
)

Arguments

string: a character string.

sequence_strings

a character string vector

processor

either NULL or a function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. See the examples for more details.

scorer

score_cutoff

an integer value for the score threshold. No matches with a score less than this number will be returned. Defaults to 0

Method `ExtractOne()`

Usage

FuzzExtract$ExtractOne(
  string = NULL,
  sequence_strings = NULL,
  processor = NULL,
  scorer = NULL,
  score_cutoff = 0L
)

Arguments

string: a character string.

sequence_strings

a character string vector

processor

either NULL or a function of the form f(a) -> b, where a is the query or individual choice and b is the choice to be used in matching. See the examples for more details.

scorer

score_cutoff

an integer value for the score threshold. No matches with a score less than this number will be returned. Defaults to 0

Method `Dedupe()`

Usage

FuzzExtract$Dedupe(contains_dupes = NULL, threshold = 70L, scorer = NULL)

Arguments

contains_dupes: a vector of strings that we would like to dedupe

threshold

the numerical value (0, 100) point at which we expect to find duplicates. Defaults to 70 out of 100

scorer

Method `clone()`

The objects of this class are cloneable with this method.

Usage

FuzzExtract$clone(deep = FALSE)

Arguments

deep: Whether to make a deep clone.

Details

the decoding parameter is useful in case of non-ascii character strings. If this parameter is not NULL then the force_ascii parameter (if applicable) is internally set to FALSE. Decoding applies only to python 2 configurations, as in python 3 character strings are decoded to unicode by default.

the Extract method selects the best match of a character string vector. It returns a list with the match and it's score.

the ExtractBests method returns a list of the best matches for a sequence of character strings.

the ExtractWithoutOrder method returns the best match of a character string vector (in python it returns a generator of tuples containing the match and it's score).

the ExtractOne method finds the single best match above a score for a character string vector. This is a convenience method which returns the single best choice.

the Dedupe is a convenience method which takes a character string vector containing duplicates and uses fuzzy matching to identify and remove duplicates. Specifically, it uses the Extract method to identify duplicates that score greater than a user defined threshold. Then, it looks for the longest item in the duplicate vector since we assume this item contains the most entity information and returns that. It breaks string length ties on an alphabetical sort. Note: as the threshold DECREASES the number of duplicates that are found INCREASES. This means that the returned deduplicated list will likely be shorter. Raise the threshold for fuzzy_dedupe to be less sensitive.

References

https://github.com/seatgeek/fuzzywuzzy/blob/master/fuzzywuzzy/process.py, https://docs.python.org/3/library/codecs.html#standard-encodings

Examples

Run this code

# NOT RUN {
try({
  if (reticulate::py_available(initialize = FALSE)) {

    if (check_availability()) {

      library(fuzzywuzzyR)

      word = "new york jets"

      choices = c("Atlanta Falcons", "New York Jets", "New York Giants", "Dallas Cowboys")

      duplicat = c('Frodo Baggins', 'Tom Sawyer', 'Bilbo Baggin', 'Samuel L. Jackson',

                   'F. Baggins', 'Frody Baggins', 'Bilbo Baggins')

      #------------
      # processor :
      #------------

      init_proc = FuzzUtils$new()

      PROC = init_proc$Full_process    # class process-method

      PROC1 = tolower                  # base R function

      #---------
      # scorer :
      #---------

      init_scor = FuzzMatcher$new()

      SCOR = init_scor$WRATIO


      init <- FuzzExtract$new()

      init$Extract(string = word, sequence_strings = choices, processor = PROC, scorer = SCOR)

      init$ExtractBests(string = word, sequence_strings = choices, processor = PROC1,

                        scorer = SCOR, score_cutoff = 0L, limit = 2L)

      init$ExtractWithoutOrder(string = word, sequence_strings = choices, processor = PROC,

                               scorer = SCOR, score_cutoff = 0L)

      init$ExtractOne(string = word, sequence_strings = choices, processor = PROC,

                      scorer = SCOR, score_cutoff = 0L)

      init$Dedupe(contains_dupes = duplicat, threshold = 70L, scorer = SCOR)

    }
  }
}, silent=TRUE)
# }

Run the code above in your browser using DataLab

Description

Usage

Arguments

Methods

Methods

Public methods

Method new()

Usage

Arguments

Method Extract()

Usage

Arguments

Method ExtractBests()

Usage

Arguments

Method ExtractWithoutOrder()

Usage

Arguments

Method ExtractOne()

Usage

Arguments

Method Dedupe()

Usage

Arguments

Method clone()

Usage

Arguments

Details

References

Examples

Method `new()`

Method `Extract()`

Method `ExtractBests()`

Method `ExtractWithoutOrder()`

Method `ExtractOne()`

Method `Dedupe()`

Method `clone()`