preprocText

Preprocess text data such as names and addresses.

Implements a Fellegi-Sunter probabilistic record linkage model that allows for missing data
and the inclusion of auxiliary information. This includes functionalities to conduct a merge of two
datasets under the Fellegi-Sunter model using the Expectation-Maximization algorithm. In addition,
tools for preparing, adjusting, and summarizing data merges are included. The package implements methods
described in Enamorado, Fifield, and Imai (2019) ''Using a Probabilistic Model to Assist Merging of
Large-scale Administrative Records'' <doi:10.1017/S0003055418000783> and is available
at <https://imai.fas.harvard.edu/research/linkage.html>.

Ted Enamorado

fastLink

Fast Probabilistic Record Linkage with Missing Data

Ben Fifield

Kosuke Imai

preprocText function

<dl><dt>text</dt>
<dd>A vector of text data to convert.</dd>
<dt>convert_text</dt>
<dd>Whether to convert text to the desired encoding, where
the encoding is specified in the 'convert_text_to' argument. Default is
TRUE</dd>
<dt>tolower</dt>
<dd>Whether to normalize the text to be all lowercase. Default is
TRUE.</dd>
<dt>soundex</dt>
<dd>Whether to convert the field to the Census's soundex encoding.
Default is FALSE.</dd>
<dt>usps_address</dt>
<dd>Whether to use USPS address standardization rules to clean address fields.
Default is FALSE.</dd>
<dt>remove_whitespace</dt>
<dd>Whether to remove leading and trailing whitespace, and to convert multiple
spaces to a single space. Default is TRUE.</dd>
<dt>remove_punctuation</dt>
<dd>Whether to remove punctuation from a string. Default is TRUE.</dd>
<dt>convert_text_to</dt>
<dd>Which encoding to use when converting text. Default is 'Latin-ASCII'.
Full list of encodings in the <code>stri_trans_list()</code> function in the <code>stringi</code> package.</dd></dl>

Arguments

Ben Fifield &lt;benfifield@gmail.com&gt;

Author

preprocText — preprocText

<dl>

<dt>text</dt>
<dd>A vector of text data to convert.</dd>


<dt>convert_text</dt>
<dd>Whether to convert text to the desired encoding, where
the encoding is specified in the 'convert_text_to' argument. Default is
TRUE</dd>


<dt>tolower</dt>
<dd>Whether to normalize the text to be all lowercase. Default is
TRUE.</dd>


<dt>soundex</dt>
<dd>Whether to convert the field to the Census's soundex encoding.
Default is FALSE.</dd>


<dt>usps_address</dt>
<dd>Whether to use USPS address standardization rules to clean address fields.
Default is FALSE.</dd>


<dt>remove_whitespace</dt>
<dd>Whether to remove leading and trailing whitespace, and to convert multiple
spaces to a single space. Default is TRUE.</dd>


<dt>remove_punctuation</dt>
<dd>Whether to remove punctuation from a string. Default is TRUE.</dd>


<dt>convert_text_to</dt>
<dd>Which encoding to use when converting text. Default is 'Latin-ASCII'.
Full list of encodings in the <code>stri_trans_list()</code> function in the <code>stringi</code> package.</dd>

</dl>

preprocText: preprocText

Description

Usage

Value

Arguments

Author