crul (version 0.9.0)

Paginator: Paginator client

Description

A client to help you paginate

Arguments

client

an object of class HttpClient, from a call to HttpClient

by

(character) how to paginate. Only 'query_params' supported for now. In the future will support 'link_headers' and 'cursor'. See Details.

limit_param

(character) the name of the limit parameter. Default: limit

offset_param

(character) the name of the offset parameter. Default: offset

limit

(numeric/integer) the maximum records wanted

limit_chunk

(numeric/integer) the number by which to chunk requests, e.g., 10 would be be each request gets 10 records

progress

(logical) print a progress bar, using utils::txtProgressBar. Default: FALSE.

Value

a list, with objects of class HttpResponse(). Responses are returned in the order they are passed in.

Methods to paginate

Supported now:

• query_params: the most common way, so is the default. This method involves setting how many records and what record to start at for each request. We send these query parameters for you.

Supported later:

• link_headers: link headers are URLS for the next/previous/last request given in the response header from the server. This is relatively uncommon, though is recommended by JSONAPI and is implemented by a well known API (GitHub).

• cursor: this works by a single string given back in each response, to be passed in the subsequent response, and so on until no more records remain. This is common in Solr

Details

Methods

get(path, query, ...)

make a paginated GET request

post(path, query, body, encode, ...)

make a paginated POST request

put(path, query, body, encode, ...)

make a paginated PUT request

patch(path, query, body, encode, ...)

make a paginated PATCH request

delete(path, query, body, encode, ...)

make a paginated DELETE request

head(path, ...)

make a paginated HEAD request - not sure if this makes any sense or not yet

responses()

list responses - returns: a list of HttpResponse objects, empty list before requests made

parse(encoding = "UTF-8")

parse content - returns: character vector, empty character vector before requests made

status_code()

(integer) HTTP status codes - returns: numeric vector, empty numeric vector before requests made

status()

(list) HTTP status objects - returns: a list of http_code objects, empty list before requests made

content()

raw content - returns: raw list, empty list before requests made

times()

curl request times - returns: list of named numeric vectors, empty list before requests made

url_fetch(path, query)

get URLs that would be sent (i.e., before executing the request). the only things that change the URL are path and query parameters; body and any curl options don't change the URL - returns: character vector of URLs

See HttpClient() for information on parameters.

Examples

# NOT RUN {
(cli <- HttpClient$new(url = "https://api.crossref.org"))
cc <- Paginator$new(client = cli, limit_param = "rows",
   offset_param = "offset", limit = 50, limit_chunk = 10)
cc
cc$get('works')
cc
cc$responses()
cc$status()
cc$status_code()
cc$times()
cc$content()
cc$parse()
lapply(cc$parse(), jsonlite::fromJSON)

# get full URLs for each request to be made
cc$url_fetch('works')
cc$url_fetch('works', query = list(query = "NSF"))

# progress bar
(cli <- HttpClient$new(url = "https://api.crossref.org"))
cc <- Paginator$new(client = cli, limit_param = "rows",
   offset_param = "offset", limit = 50, limit_chunk = 10, 
   progress = TRUE)
cc
cc$get('works')
# }