Async: Simple async client

Description

A client to work with many URLs, but all with the same HTTP method

Arguments

urls

(character) one or more URLs (required)

Value

a list, with objects of class HttpResponse(). Responses are returned in the order they are passed in. We print the first 10.

Failure behavior

HTTP requests mostly fail in ways that you are probably familiar with, including when there's a 400 response (the URL not found), and when the server made a mistake (a 500 series HTTP status code).

But requests can fail sometimes where there is no HTTP status code, and no agreed upon way to handle it other than to just fail immediately.

When a request fails when using synchronous requests (see HttpClient) you get an error message that stops your code progression immediately saying for example:

"Could not resolve host: https://foo.com"
"Failed to connect to foo.com"
"Resolving timed out after 10 milliseconds"

However, for async requests we don't want to fail immediately because that would stop the subsequent requests from occurring. Thus, when we find that a request fails for one of the reasons above we give back a HttpResponse object just like any other response, and:

capture the error message and put it in the content slot of the response object (thus calls to content and parse() work correctly)
give back a 0 HTTP status code. we handle this specially when testing whether the request was successful or not with e.g., the success() method

Details

Methods

get(path, query, disk, stream, ...): make async GET requests for all URLs
post(path, query, body, encode, disk, stream, ...): make async POST requests for all URLs
put(path, query, body, encode, disk, stream, ...): make async PUT requests for all URLs
patch(path, query, body, encode, disk, stream, ...): make async PATCH requests for all URLs
delete(path, query, body, encode, disk, stream, ...): make async DELETE requests for all URLs
head(path, ...): make async HEAD requests for all URLs
verb(verb, ...): make async requests with an arbitrary HTTP verb

See HttpClient() for information on parameters.

Examples

Run this code

# NOT RUN {
cc <- Async$new(
  urls = c(
    'https://httpbin.org/',
    'https://httpbin.org/get?a=5',
    'https://httpbin.org/get?foo=bar'
  )
)
cc
(res <- cc$get())
res[[1]]
res[[1]]$url
res[[1]]$success()
res[[1]]$status_http()
res[[1]]$response_headers
res[[1]]$method
res[[1]]$content
res[[1]]$parse("UTF-8")
lapply(res, function(z) z$parse("UTF-8"))

# curl options/headers with async
urls = c(
 'https://httpbin.org/',
 'https://httpbin.org/get?a=5',
 'https://httpbin.org/get?foo=bar'
)
cc <- Async$new(urls = urls, 
  opts = list(verbose = TRUE),
  headers = list(foo = "bar")
)
cc
(res <- cc$get())

# using auth with async
dd <- Async$new(
  urls = rep('https://httpbin.org/basic-auth/user/passwd', 3),
  auth = auth(user = "foo", pwd = "passwd"),
  opts = list(verbose = TRUE)
)
dd
res <- dd$get()
res
vapply(res, function(z) z$status_code, double(1))
vapply(res, function(z) z$success(), logical(1))
lapply(res, function(z) z$parse("UTF-8"))

# failure behavior
## e.g. when a URL doesn't exist, a timeout, etc.
urls <- c("http://stuffthings.gvb", "https://foo.com", 
  "https://httpbin.org/get")
conn <- Async$new(urls = urls)
res <- conn$get()
res[[1]]$parse("UTF-8") # a failure
res[[2]]$parse("UTF-8") # a failure
res[[3]]$parse("UTF-8") # a success

# use arbitrary http verb
cc <- Async$new(
  urls = c(
    'https://httpbin.org/',
    'https://httpbin.org/get?a=5',
    'https://httpbin.org/get?foo=bar'
  )
)
method <- 'get'
(res <- cc$verb(method))
lapply(res, function(z) z$parse("UTF-8"))
# }