epitweetr (version 2.2.16)

json2lucene: Function used for migrating tweets from to old to the new file system

Description

migrates geolocated tweets from the old to the new file system allowing full text search using Apache Lucene Indexes

Usage

json2lucene(tasks = get_tasks(), chunk_size = 400)

Value

the updated tasks.

Arguments

tasks

named list, current tasks for logging and updating progress default: get_tasks()

chunk_size,

integer, the chunk size for indexing tweets, default: 400

Details

This function can be called manually to perform the migration of tweets between v0.0.x to v2+ It iterates over existing tweets collected with epitweetr v0.0.x series joins base tweets and geolocated tweets and then sends themes to the Lucene index via the dedicated REST API. Migrated files will be moved to search_archive and geo_archive folders. Users can backup and remove these folders when migration ends to gain disk space. Series folders are maintained for migrated tweets

Examples

if(FALSE){
   library(epitweetr)
   # setting up the data folder
   message('Please choose the epitweetr data directory')
   setup_config(file.choose()) 
   # runnint the migration
   json2lucene()
}