A client to help you paginate
a list, with objects of class HttpResponse().
Responses are returned in the order they are passed in.
Supported now:
query_params: the most common way, so is the default. This method
involves setting how many records and what record to start at for each
request. We send these query parameters for you.
Supported later:
link_headers: link headers are URLS for the next/previous/last
request given in the response header from the server. This is relatively
uncommon, though is recommended by JSONAPI and is implemented by a
well known API (GitHub).
cursor: this works by a single string given back in each response, to
be passed in the subsequent response, and so on until no more records
remain. This is common in Solr
http_reqan object of class HttpClient
by(character) how to paginate. Only 'query_params' supported for now. In the future will support 'link_headers' and 'cursor'. See Details.
limit_chunk(numeric/integer) the number by which to chunk requests, e.g., 10 would be be each request gets 10 records
limit_param(character) the name of the limit parameter. Default: limit
offset_param(character) the name of the offset parameter. Default: offset
limit(numeric/integer) the maximum records wanted
progress(logical) print a progress bar, using utils::txtProgressBar.
Default: FALSE.
print()print method for Paginator objects
Paginator$print(x, ...)
xself
...ignored
new()Create a new Paginator object
Paginator$new( client, by = "query_params", limit_param, offset_param, limit, limit_chunk, progress = FALSE )
clientan object of class HttpClient, from a call to HttpClient
by(character) how to paginate. Only 'query_params' supported for now. In the future will support 'link_headers' and 'cursor'. See Details.
limit_param(character) the name of the limit parameter. Default: limit
offset_param(character) the name of the offset parameter. Default: offset
limit(numeric/integer) the maximum records wanted
limit_chunk(numeric/integer) the number by which to chunk requests, e.g., 10 would be be each request gets 10 records
progress(logical) print a progress bar, using utils::txtProgressBar.
Default: FALSE.
A new Paginator object
get()make a paginated GET request
Paginator$get(path = NULL, query = list(), ...)
pathURL path, appended to the base URL
queryquery terms, as a named list
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
post()make a paginated POST request
Paginator$post( path = NULL, query = list(), body = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list
bodybody as an R list
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
put()make a paginated PUT request
Paginator$put( path = NULL, query = list(), body = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list
bodybody as an R list
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
patch()make a paginated PATCH request
Paginator$patch( path = NULL, query = list(), body = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list
bodybody as an R list
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
delete()make a paginated DELETE request
Paginator$delete( path = NULL, query = list(), body = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list
bodybody as an R list
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
head()make a paginated HEAD request
Paginator$head(path = NULL, ...)
pathURL path, appended to the base URL
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
not sure if this makes any sense or not yet
responses()list responses
Paginator$responses()
a list of HttpResponse objects, empty list before requests made
status_code()Get HTTP status codes for each response
Paginator$status_code()
numeric vector, empty numeric vector before requests made
status()List HTTP status objects
Paginator$status()
a list of http_code objects, empty list before requests made
parse()parse content
Paginator$parse(encoding = "UTF-8")
encoding(character) the encoding to use in parsing. default:"UTF-8"
character vector, empty character vector before requests made
content()Get raw content for each response
Paginator$content()
raw list, empty list before requests made
times()curl request times
Paginator$times()
list of named numeric vectors, empty list before requests made
url_fetch()get the URL that would be sent (i.e., before executing the request) the only things that change the URL are path and query parameters; body and any curl options don't change the URL
Paginator$url_fetch(path = NULL, query = list())
pathURL path, appended to the base URL
queryquery terms, as a named list
URLs (character)
\dontrun{
cli <- HttpClient$new(url = "https://api.crossref.org")
cc <- Paginator$new(client = cli, limit_param = "rows",
offset_param = "offset", limit = 50, limit_chunk = 10)
cc$url_fetch('works')
cc$url_fetch('works', query = list(query = "NSF"))
}
clone()The objects of this class are cloneable with this method.
Paginator$clone(deep = FALSE)
deepWhether to make a deep clone.
See HttpClient() for information on parameters
# NOT RUN {
(cli <- HttpClient$new(url = "https://api.crossref.org"))
cc <- Paginator$new(client = cli, limit_param = "rows",
offset_param = "offset", limit = 50, limit_chunk = 10)
cc
cc$get('works')
cc
cc$responses()
cc$status()
cc$status_code()
cc$times()
cc$content()
cc$parse()
lapply(cc$parse(), jsonlite::fromJSON)
# progress bar
(cli <- HttpClient$new(url = "https://api.crossref.org"))
cc <- Paginator$new(client = cli, limit_param = "rows",
offset_param = "offset", limit = 50, limit_chunk = 10,
progress = TRUE)
cc
cc$get('works')
# }
# NOT RUN {
## ------------------------------------------------
## Method `Paginator$url_fetch`
## ------------------------------------------------
# }
# NOT RUN {
cli <- HttpClient$new(url = "https://api.crossref.org")
cc <- Paginator$new(client = cli, limit_param = "rows",
offset_param = "offset", limit = 50, limit_chunk = 10)
cc$url_fetch('works')
cc$url_fetch('works', query = list(query = "NSF"))
# }
Run the code above in your browser using DataLab