read_civis: Read tables and files from Civis Platform

Description

read_civis loads a table from Redshift as a data frame if given a "schema.table" or sql("query") as the first argument, or loads a file from Amazon S3 (the files endpoint) if a file id is given. Run outputs from any Civis platform script are returned if a civis_script is given.

A default database can be set using options(civis.default_db = "my_database"). If there is only one database available, this database will automatically be used as the default.

Usage

read_civis(x, ...)
# S3 method for numeric
read_civis(x, using = read.csv, verbose = FALSE, ...)
# S3 method for character
read_civis(x, database = NULL, ...)
# S3 method for sql
read_civis(
  x,
  database = NULL,
  using = utils::read.csv,
  job_name = NULL,
  hidden = TRUE,
  verbose = FALSE,
  ...
)
# S3 method for civis_script
read_civis(x, using, regex = NULL, ...)

Arguments

x: "schema.table", sql("query"), or a file id.
...: arguments passed to using.
using: function, Function to convert the file to a data frame or to unserialize. the file (e.g. read.csv or readRDS).
verbose: bool, Set to TRUE to print intermediate progress indicators.
database: string, Name of database where data frame is to be uploaded. If no database is specified, uses options(civis.default_db).
job_name: string, Name of the job (default: "Civis Export Via R Client").
hidden: bool, Whether the job is hidden.
regex: Regex of matching run output names.

Methods (by class)

numeric: Return a file as a data frame
character: Return all columns from a table as a data frame.
sql: Return a SQL query as a data frame.
civis_script: Return run outputs of a civis_script as a named list.

Details

By default, read_civis.numeric assumes the file is a CSV. For reading a serialized R object, set using = readRDS for example.

If using = NULL, read_civis.civis_script will return all JSONValues with name matching regex. Otherwise all File run outputs matching regex will be read into memory with using. Results are always a named list. If the script has no outputs, an empty list will be returned.

Examples

Run this code

if (FALSE) {
# Read all columns in a single table
df <- read_civis("schema.my_table", database = "my_database")

# Read data from a SQL select statement
query <- sql("SELECT * FROM table JOIN other_table USING id WHERE var1 < 23")
df <- read_civis(query, database = "my_database")

# Read an R object from the files endpoint.
id <- write_civis_file(df)
df <- read_civis(id)

# Read a text file or csv from the files endpoint.
id <- write_civis_file("my_csv.csv")
df <- read_civis(id)

# Read JSONValues from a civis script
vals <- read_civis(civis_script(1234))

# Read File run outputs from a civis script
df <- read_civis(civis_script(1234), regex = '.csv', using = read.csv)
obj <- read_civis(civis_script(1234), regex = '.rds', using = readRDS)

# Gracefully handle when read_civis.sql returns no rows
query <- sql("SELECT * FROM table WHERE 1 = 2")
mean_x <- tryCatch({
  df <- read_civis(query, database = "my_database")
  mean(df$x)
}, empty_result_error = function(e) {
   NA
})
}