HttpClient: HTTP client

Description

HTTP client

Arguments

url

(character) A url. One of url or handle required.

opts

(list) curl options, a named list. See curl_options for available curl options

proxies

an object of class proxy, as returned from the proxy function. Supports one proxy for now

auth

result of a call to the auth function, e.g. auth(user = "foo", pwd = "bar")

headers

(list) a named list of headers

handle

A handle, see handle

progress

a function with logic for printing a progress bar for an HTTP request, ultimately passed down to curl. only supports httr::progress() for now

hooks

(list) a named list (accepts: request, response) of functions (callbacks) to run on request and response objects. See hooks for more details.

Value

an HttpResponse object

handles

curl handles are re-used on the level of the connection object, that is, each HttpClient object is separate from one another so as to better separate connections.

If you don't pass in a curl handle to the handle parameter, it gets created when a HTTP verb is called. Thus, if you try to get handle after creating a HttpClient object only passing url parameter, handle will be NULL. If you pass a curl handle to the handle parameter, then you can get the handle from the HttpClientobject. The response from a http verb request does have the handle in thehandle` slot.

Details

Methods

get(path, query, disk, stream, ...): Make a GET request
post(path, query, body, disk, stream, ...): Make a POST request
put(path, query, body, disk, stream, ...): Make a PUT request
patch(path, query, body, disk, stream, ...): Make a PATCH request
delete(path, query, body, disk, stream, ...): Make a DELETE request
head(path, query, ...): Make a HEAD request
verb(verb, ...): Use an arbitrary HTTP verb supported on this class Supported verbs: get, post, put, patch, delete, head. Also supports retry
retry(verb, ..., pause_base = 1, pause_cap = 60, pause_min = 1, times = 3, terminate_on, retry_only_on, onwait): Retries the request given by verb until successful (HTTP response status < 400), or a condition for giving up is met. Automatically recognizes Retry-After and X-RateLimit-Reset headers in the response for rate-limited remote APIs.
handle_pop(): reset your curl handle
url_fetch(path, query): get the URL that would be sent (i.e., before executing the request). the only things that change the URL are path and query parameters; body and any curl options don't change the URL - returns: URL as a character vector

Possible parameters (not all are allowed in each HTTP verb):

path - URL path, appended to the base URL
query - query terms, as a named list
body - body as an R list
encode - one of form, multipart, json, or raw
disk - a path to write to. if NULL (default), memory used. See curl::curl_fetch_disk() for help.
stream - an R function to determine how to stream data. if NULL (default), memory used. See curl::curl_fetch_stream() for help
verb - an HTTP verb supported on this class: get, post, put, patch, delete, head. Also supports retry.
... - For retry, the options to be passed on to the method implementing the requested verb, including curl options. Otherwise, curl options, only those in the acceptable set from curl::curl_options() except the following: httpget, httppost, post, postfields, postfieldsize, and customrequest
pause_base,pause_cap,pause_min - basis, maximum, and minimum for calculating wait time for retry. Wait time is calculated according to the exponential backoff with full jitter algorithm. Specifically, wait time is chosen randomly between pause_min and the lesser of pause_base * 2 and pause_cap, with pause_base doubling on each subsequent retry attempt. Use pause_cap = Inf to not terminate retrying due to cap of wait time reached.
times - the maximum number of times to retry. Set to Inf to not stop retrying due to exhausting the number of attempts.
terminate_on,retry_only_on - a vector of HTTP status codes. For terminate_on, the status codes for which to terminate retrying, and for retry_only_on, the status codes for which to retry the request.
onwait - a callback function if the request will be retried and a wait time is being applied. The function will be passed two parameters, the response object from the failed request, and the wait time in seconds. Note that the time spent in the function effectively adds to the wait time, so it should be kept simple.

Examples

Run this code

# NOT RUN {
# set your own handle
(h <- handle("https://httpbin.org"))
(x <- HttpClient$new(handle = h))
x$handle
x$url
(out <- x$get("get"))
x$handle
x$url
class(out)
out$handle
out$request_headers
out$response_headers
out$response_headers_all

# if you just pass a url, we create a handle for you
#  this is how most people will use HttpClient
(x <- HttpClient$new(url = "https://httpbin.org"))
x$url
x$handle # is empty, it gets created when a HTTP verb is called
(r1 <- x$get('get'))
x$url
x$handle
r1$url
r1$handle
r1$content
r1$response_headers
r1$parse()

(res_get2 <- x$get('get', query = list(hello = "world")))
res_get2$parse()
library("jsonlite")
jsonlite::fromJSON(res_get2$parse())

# post request
(res_post <- x$post('post', body = list(hello = "world")))

## empty body request
x$post('post')

# put request
(res_put <- x$put('put'))

# delete request
(res_delete <- x$delete('delete'))

# patch request
(res_patch <- x$patch('patch'))

# head request
(res_head <- x$head())

# arbitrary verb
(x <- HttpClient$new(url = "https://httpbin.org"))
x$verb('get')
x$verb('GET')
x$verb('GET', query = list(foo = "bar"))
x$verb('retry', 'GET', path = "status/400")

# retry, by default at most 3 times
(res_get <- x$retry("GET", path = "status/400"))

# retry, but not for 404 NOT FOUND
(res_get <- x$retry("GET", path = "status/404", terminate_on = c(404)))

# retry, but only for exceeding rate limit (note that e.g. Github uses 403)
(res_get <- x$retry("GET", path = "status/429", retry_only_on = c(403, 429)))

# query params are URL encoded for you, so DO NOT do it yourself
## if you url encode yourself, it gets double encoded, and that's bad
(x <- HttpClient$new(url = "https://httpbin.org"))
res <- x$get("get", query = list(a = 'hello world'))

# get full url before the request is made
(x <- HttpClient$new(url = "https://httpbin.org"))
x$url_fetch()
x$url_fetch('get')
x$url_fetch('post')
x$url_fetch('get', query = list(foo = "bar"))

# access intermediate headers in response_headers_all
x <- HttpClient$new("https://doi.org/10.1007/978-3-642-40455-9_52-1")
bb <- x$get()
bb$response_headers_all
# }

Run the code above in your browser using DataLab