curl (version 4.3)

curl: Curl connection interface

Description

Drop-in replacement for base url that supports https, ftps, gzip, deflate, etc. Default behavior is identical to url, but request can be fully configured by passing a custom handle.

Usage

curl(url = "http://httpbin.org/get", open = "", handle = new_handle())

Arguments

url

character string. See examples.

open

character string. How to open the connection if it should be opened initially. Currently only "r" and "rb" are supported.

handle

a curl handle object

Details

As of version 2.3 curl connections support open(con, blocking = FALSE). In this case readBin and readLines will return immediately with data that is available without waiting. For such non-blocking connections the caller needs to call isIncomplete to check if the download has completed yet.

Examples

# NOT RUN {
con <- curl("https://httpbin.org/get")
readLines(con)

# Auto-opened connections can be recycled
open(con, "rb")
bin <- readBin(con, raw(), 999)
close(con)
rawToChar(bin)

# HTTP error
curl("https://httpbin.org/status/418", "r")

# Follow redirects
readLines(curl("https://httpbin.org/redirect/3"))

# Error after redirect
curl("https://httpbin.org/redirect-to?url=http://httpbin.org/status/418", "r")

# Auto decompress Accept-Encoding: gzip / deflate (rfc2616 #14.3)
readLines(curl("http://httpbin.org/gzip"))
readLines(curl("http://httpbin.org/deflate"))

# Binary support
buf <- readBin(curl("http://httpbin.org/bytes/98765", "rb"), raw(), 1e5)
length(buf)

# Read file from disk
test <- paste0("file://", system.file("DESCRIPTION"))
readLines(curl(test))

# Other protocols
read.csv(curl("ftp://cran.r-project.org/pub/R/CRAN_mirrors.csv"))
readLines(curl("ftps://test.rebex.net:990/readme.txt"))
readLines(curl("gopher://quux.org/1"))

# Streaming data
con <- curl("http://jeroen.github.io/data/diamonds.json", "r")
while(length(x <- readLines(con, n = 5))){
  print(x)
}

# Stream large dataset over https with gzip
library(jsonlite)
con <- gzcon(curl("https://jeroen.github.io/data/nycflights13.json.gz"))
nycflights <- stream_in(con)
# }
# NOT RUN {
# }