Object of class R6 which stores the text embeddings generated by an object of class
TextEmbeddingModel. The text embeddings are stored within memory/RAM. In the case of a high number of documents
the data may not fit into memory/RAM. Thus, please use this object only for a small sample of texts. In general, it
is recommended to use an object of class LargeDataSetForTextEmbeddings which can deal with any number of texts.
Returns an object of class EmbeddedText. These objects are used for storing and managing the text embeddings created with objects of class TextEmbeddingModel. Objects of class EmbeddedText serve as input for objects of class TEClassifierRegular, TEClassifierProtoNet, and TEFeatureExtractor. The main aim of this class is to provide a structured link between embedding models and classifiers. Since objects of this class save information on the text embedding model that created the text embedding it ensures that only embedding generated with same embedding model are combined. Furthermore, the stored information allows objects to check if embeddings of the correct text embedding model are used for training and predicting.
embeddings('data.frame()')
data.frame containing the text embeddings for all chunks. Documents are in the rows. Embedding dimensions are
in the columns.
configure()Creates a new object representing text embeddings.
EmbeddedText$configure(
model_name = NA,
model_label = NA,
model_date = NA,
model_method = NA,
model_version = NA,
model_language = NA,
param_seq_length = NA,
param_chunks = NULL,
param_features = NULL,
param_overlap = NULL,
param_emb_layer_min = NULL,
param_emb_layer_max = NULL,
param_emb_pool_type = NULL,
param_aggregation = NULL,
embeddings
)model_namestring Name of the model that generates this embedding.
model_labelstring Label of the model that generates this embedding.
model_datestring Date when the embedding generating model was created.
model_methodstring Method of the underlying embedding model.
model_versionstring Version of the model that generated this embedding.
model_languagestring Language of the model that generated this embedding.
param_seq_lengthint Maximum number of tokens that processes the generating model for a chunk.
param_chunksint Maximum number of chunks which are supported by the generating model.
param_featuresint Number of dimensions of the text embeddings.
param_overlapint Number of tokens that were added at the beginning of the sequence for the next chunk
by this model. #'
param_emb_layer_minint or string determining the first layer to be included in the creation of
embeddings.
param_emb_layer_maxint or string determining the last layer to be included in the creation of
embeddings.
param_emb_pool_typestring determining the method for pooling the token embeddings within each layer.
param_aggregationstring Aggregation method of the hidden states. Deprecated. Only included for backward
compatibility.
embeddingsdata.frame containing the text embeddings.
Returns an object of class EmbeddedText which stores the text embeddings produced by an objects of class TextEmbeddingModel.
save()Saves a data set to disk.
EmbeddedText$save(dir_path, folder_name, create_dir = TRUE)dir_pathPath where to store the data set.
folder_namestring Name of the folder for storing the data set.
create_dirbool If True the directory will be created if it does not exist.
Method does not return anything. It write the data set to disk.
is_configured()Method for checking if the model was successfully configured. An object can only be used if this
value is TRUE.
EmbeddedText$is_configured()bool TRUE if the model is fully configured. FALSE if not.
load_from_disk()loads an object of class EmbeddedText from disk and updates the object to the current version of the package.
EmbeddedText$load_from_disk(dir_path)dir_pathPath where the data set set is stored.
Method does not return anything. It loads an object from disk.
get_model_info()Method for retrieving information about the model that generated this embedding.
EmbeddedText$get_model_info()list contains all saved information about the underlying text embedding model.
get_model_label()Method for retrieving the label of the model that generated this embedding.
EmbeddedText$get_model_label()string Label of the corresponding text embedding model
get_times()Number of chunks/times of the text embeddings.
EmbeddedText$get_times()Returns an int describing the number of chunks/times of the text embeddings.
get_features()Number of actual features/dimensions of the text embeddings.In the case a
feature extractor was used the number of features is smaller as the original number of
features. To receive the original number of features (the number of features before applying a
feature extractor) you can use the method get_original_features of this class.
EmbeddedText$get_features()Returns an int describing the number of features/dimensions of the text embeddings.
get_original_features()Number of original features/dimensions of the text embeddings.
EmbeddedText$get_original_features()Returns an int describing the number of features/dimensions if no
feature extractor) is used or before a feature extractor) is
applied.
is_compressed()Checks if the text embedding were reduced by a feature extractor.
EmbeddedText$is_compressed()Returns TRUE if the number of dimensions was reduced by a feature extractor. If
not return FALSE.
add_feature_extractor_info()Method setting information on the feature extractor that was used to reduce the number of dimensions of the text embeddings. This information should only be used if a feature extractor was applied.
EmbeddedText$add_feature_extractor_info(
model_name,
model_label = NA,
features = NA,
method = NA,
noise_factor = NA,
optimizer = NA
)model_namestring Name of the underlying TextEmbeddingModel.
model_labelstring Label of the underlying TextEmbeddingModel.
featuresint Number of dimension (features) for the compressed text embeddings.
methodstring Method that the TEFeatureExtractor applies for genereating the compressed text
embeddings.
noise_factordouble Noise factor of the TEFeatureExtractor.
optimizerstring Optimizer used during training the TEFeatureExtractor.
Method does nothing return. It sets information on a feature extractor.
get_feature_extractor_info()Method for receiving information on the feature extractor that was used to reduce the number of dimensions of the text embeddings.
EmbeddedText$get_feature_extractor_info()Returns a list with information on the feature extractor. If no
feature extractor was used it returns NULL.
convert_to_LargeDataSetForTextEmbeddings()Method for converting this object to an object of class LargeDataSetForTextEmbeddings.
EmbeddedText$convert_to_LargeDataSetForTextEmbeddings()Returns an object of class LargeDataSetForTextEmbeddings which uses memory mapping allowing to work with large data sets.
n_rows()Number of rows.
EmbeddedText$n_rows()Returns the number of rows of the text embeddings which represent the number of cases.
get_all_fields()Return all fields.
EmbeddedText$get_all_fields()Method returns a list containing all public and private fields
of the object.
clone()The objects of this class are cloneable with this method.
EmbeddedText$clone(deep = FALSE)deepWhether to make a deep clone.
Other Data Management:
DataManagerClassifier,
LargeDataSetForText,
LargeDataSetForTextEmbeddings