kgrams (version 0.1.0)

EOS: Special Tokens

Description

Return Begin-Of-Sentence, End-Of-Sentence and Unknown-Word special tokens.

Usage

EOS()
BOS()
UNK()

Value

a string representing the appropriate special token.

Details

These functions return the internal representation of BOS, EOS and UNK tokens respectively. Their actual returned values are irrelevant and their only purpose is to simplify queries of k-gram counts and probabilities involving the special tokens, as shown in the examples.

Examples

# NOT RUN {
f <- kgram_freqs("a b b a b", 2)
query(f, c(BOS(), EOS(), UNK()))
probability(c("a", "b") %|% BOS(), f)
probability("a b b a" %+% EOS(), f)

# The actual values of BOS(), EOS() and UNK() are irrelevant
c(BOS(), EOS(), UNK())


# }