gar_batch_walk: Walk data through batches

Description

Convenience function for walking through data in batches

Usage

gar_batch_walk(f, walk_vector, gar_pars = NULL, gar_paths = NULL,
  the_body = NULL, pars_walk = NULL, path_walk = NULL, body_walk = NULL,
  batch_size = 10, batch_function = NULL, data_frame_output = TRUE, ...,
  batch_endpoint = getOption("googleAuthR.batch_endpoint", default =
  "https://www.googleapis.com/batch"))

Arguments

a function from gar_api_generator

walk_vector

a vector of the parameter or path to change

gar_pars

a list of parameter arguments for f

gar_paths

a list of path arguments for f

the_body

a list of body arguments for f

pars_walk

a character vector of the parameter(s) to modify for each walk of f

path_walk

a character vector of the path(s) to modify for each walk of f

body_walk

a character vector of the body(s) to modify for each walk of f

batch_size

size of each request to Google /batch API

batch_function

a function that will act on the result list of each batch API call

data_frame_output

if the list of lists are dataframes, you can bind them all by setting to TRUE

...

further arguments passed to the data parse function of f

batch_endpoint

the batch API endpoint to send

Value

if data_frame_output is FALSE: A list of lists. Outer list the length of number of batches required, inner lists the results from the calls

if data_frame_output is TRUE: The list of lists will attempt to rbind all the results

Details

You can modify more than one parameter or path arg, but it must be the same walked vector e.g. start = end = x

Many Google APIs have batch_size limits greater than 10, 1000 is common.

The `f` function needs to be a `gar_api_generator()` function that uses one of `path_args`, `pars_args` or `body_args` to construct the URL (rather than say using `sprintf()` to create the API URL).

You don't need to set the headers in the Google docs for batching API functions - those are done for you.

The argument `walk_vector` needs to be a vector of the values of the arguments to walk over, which you indicate will walk over the pars/path or body arguments on the function via on of the `*_walk` arguments e.g. if walking over id=1, id=2, for a path argument then it would be `path_walk="id"` and `walk_vector=c(1,2,3,4)`

The `gar_*` parameter is required to pass intended for other arguments to the function `f` you may need to pass through.

`gar_batch_walk()` only supports changing one value at a time, for one or multiple arguments (I think only changing the `start-date`, `end-date` example would be the case when you walk through more than one per call)

`batch_size` should be over 1 for batching to be of any benefit at all

The `batch_function` argument gives you a way to operate on the parsed output of each call

Examples

Run this code

# NOT RUN {
# }
# NOT RUN {
                               

# get a webproperty per account 
getAccountInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/accounts",
  "GET", data_parse_function = function(x) unique(x$items$id))

getWebpropertyInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/", # don't use sprintf to construct this
  "GET",
  path_args = list(accounts = "default", webproperties = ""),
  data_parse_function = function(x) x$items)

walkData <- function(){

  # here due to R lazy evaluation  
  accs <- getAccountInfo()
  gar_batch_walk(getWebpropertyInfo, 
                 walk_vector = accs,
                 gar_paths = list("webproperties" = ""),
                 path_walk = "accounts",
                 batch_size = 100, data_frame_output = FALSE)
                 }
                 
# do the walk
walkData()

# }
# NOT RUN {


# }