curl (1) - Linux Manuals

curl: transfer a URL

Command to display curl manual in Linux: $ man 1 curl

NAME

curl - transfer a URL

SYNOPSIS

curl [options / URLs]

DESCRIPTION

curl is a tool for transferring data from or to a server. It supports these protocols: DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET or TFTP. The command is designed to work without user interaction.

curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume and more. As you will see below, the number of features will make your head spin.

curl is powered by libcurl for all transfer-related features. See libcurl(3) for details.

URL

The URL syntax is protocol-dependent. You find a detailed description in RFC 3986.

You can specify multiple URLs or parts of URLs by writing part sets within braces and quoting the URL as in:

  "http://site.{one,two,three}.com"

or you can get sequences of alphanumeric series by using [] as in:

  "ftp://ftp.example.com/file[1-100].txt"

  "ftp://ftp.example.com/file[001-100].txt"    (with leading zeros)

  "ftp://ftp.example.com/file[a-z].txt"

Nested sequences are not supported, but you can use several ones next to each other:

  "http://example.com/archive[1996-1999]/vol[1-4]/part{a,b,c}.html"

You can specify any amount of URLs on the command line. They will be fetched in a sequential manner in the specified order. You can specify command line options and URLs mixed and in any order on the command line.

You can specify a step counter for the ranges to get every Nth number or letter:

  "http://example.com/file[1-100:10].txt"

  "http://example.com/file[a-z:2].txt"

When using [] or {} sequences when invoked from a command line prompt, you probably have to put the full URL within double quotes to avoid the shell from interfering with it. This also goes for other characters treated special, like for example '&', '?' and '*'.

Provide the IPv6 zone index in the URL with an escaped percentage sign and the interface name. Like in

  "http://[fe80::3%25eth0]/"

If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with "ftp." curl will assume you want to speak FTP.

curl will do its best to use what you pass to it as a URL. It is not trying to validate it as a syntactically correct URL by any means but is fairly liberal with what it accepts.

curl will attempt to re-use connections for multiple file transfers, so that getting many files from the same server will not do multiple connects / handshakes. This improves speed. Of course this is only done on files specified on a single command line and cannot be used between separate curl invocations.

OUTPUT

If not told otherwise, curl writes the received data to stdout. It can be instructed to instead save that data into a local file, using the -o, --output or -O, --remote-name options. If curl is given multiple URLs to transfer on the command line, it similarly needs multiple options for where to save them.

curl does not parse or otherwise "understand" the content it gets or writes as output. It does no encoding or decoding, unless explicitly asked to with dedicated command line options.

PROTOCOLS

curl supports numerous protocols, or put in URL terms: schemes. Your particular build may not support them all.

DICT: Lets you lookup words using online dictionaries.
FILE: Read or write local files. curl does not support accessing file:// URL remotely, but when running on Microsoft Windows using the native UNC approach will work.
FTP(S): curl supports the File Transfer Protocol with a lot of tweaks and levers. With or without using TLS.
GOPHER(S): Retrieve files.
HTTP(S): curl supports HTTP with numerous options and variations. It can speak HTTP version 0.9, 1.0, 1.1, 2 and 3 depending on build options and the correct command line options.
IMAP(S): Using the mail reading protocol, curl can "download" emails for you. With or without using TLS.
LDAP(S): curl can do directory lookups for you, with or without TLS.
MQTT: curl supports MQTT version 3. Downloading over MQTT equals "subscribe" to a topic while uploading/posting equals "publish" on a topic. MQTT over TLS is not supported (yet).
POP3(S): Downloading from a pop3 server means getting a mail. With or without using TLS.
RTMP(S): The Realtime Messaging Protocol is primarily used to server streaming media and curl can download it.
RTSP: curl supports RTSP 1.0 downloads.
SCP: curl supports SSH version 2 scp transfers.
SFTP: curl supports SFTP (draft 5) done over SSH version 2.
SMB(S): curl supports SMB version 1 for upload and download.
SMTP(S): Uploading contents to an SMTP server means sending an email. With or without TLS.
TELNET: Telling curl to fetch a telnet URL starts an interactive session where it sends what it reads on stdin and outputs what the server sends it.
TFTP: curl can do TFTP downloads and uploads.

PROGRESS METER

curl normally displays a progress meter during operations, indicating the amount of transferred data, transfer speeds and estimated time left, etc. The progress meter displays number of bytes and the speeds are in bytes per second. The suffixes (k, M, G, T, P) are 1024 based. For example 1k is 1024 bytes. 1M is 1048576 bytes.

curl displays this data to the terminal by default, so if you invoke curl to do an operation and it is about to write data to the terminal, it disables the progress meter as otherwise it would mess up the output mixing progress meter and response data.

If you want a progress meter for HTTP POST or PUT requests, you need to redirect the response output to a file, using shell redirect (>), -o, --output or similar.

This does not apply to FTP upload as that operation does not spit out any response data to the terminal.

If you prefer a progress "bar" instead of the regular meter, -#, --progress-bar is your friend. You can also disable the progress meter completely with the -s, --silent option.

OPTIONS

Options start with one or two dashes. Many of the options require an additional value next to them.

The short "single-dash" form of the options, -d for example, may be used with or without a space between it and its value, although a space is a recommended separator. The long "double-dash" form, -d, --data for example, requires a space between it and its value.

Short version options that do not need any additional values can be used immediately next to each other, like for example you can specify all the options -O, -L and -v at once as -OLv.

In general, all boolean options are enabled with --option and yet again disabled with --no-option. That is, you use the same option name but prefix it with "no-". However, in this list we mostly only list and show the --option version of them.

--abstract-unix-socket <path>

(HTTP) Connect through an abstract Unix domain socket, instead of using the network. Note: netstat shows the path of an abstract socket prefixed with '@', however the <path> argument should not have this leading character.

Example:

 curl --abstract-unix-socket socketpath https://example.com

See also --unix-socket. Added in 7.53.0.

--alt-svc <file name>

(HTTPS) This option enables the alt-svc parser in curl. If the file name points to an existing alt-svc cache file, that will be used. After a completed transfer, the cache will be saved to the file name again if it has been modified.

Specify a "" file name (zero length) to avoid loading/saving and make curl just handle the cache in memory.

If this option is used several times, curl will load contents from all the files but the last one will be used for saving.

Example:

 curl --alt-svc svc.txt https://example.com

See also --resolve and --connect-to. Added in 7.64.1.

--anyauth

(HTTP) Tells curl to figure out authentication method by itself, and use the most secure one the remote site claims to support. This is done by first doing a request and checking the response-headers, thus possibly inducing an extra network round-trip. This is used instead of setting a specific authentication method, which you can do with --basic, --digest, --ntlm, and --negotiate.

Using --anyauth is not recommended if you do uploads from stdin, since it may require data to be sent twice and then the client must be able to rewind. If the need should arise when uploading from stdin, the upload operation will fail.

Used together with -u, --user.

Example:

 curl --anyauth --user me:pwd https://example.com

See also --proxy-anyauth, --basic and --digest.

-a, --append

(FTP SFTP) When used in an upload, this makes curl append to the target file instead of overwriting it. If the remote file does not exist, it will be created. Note that this flag is ignored by some SFTP servers (including OpenSSH).

Example:

 curl --upload-file local --append ftp://example.com/

See also -r, --range and -C, --continue-at.

--aws-sigv4 <provider1[:provider2[:region[:service]]]>

Use AWS V4 signature authentication in the transfer.

The provider argument is a string that is used by the algorithm when creating outgoing authentication headers.

The region argument is a string that points to a geographic area of a resources collection (region-code) when the region name is omitted from the endpoint.

The service argument is a string that points to a function provided by a cloud (service-code) when the service name is omitted from the endpoint.

Example:

 curl --aws-sigv4 "aws:amz:east-2:es" --user "key:secret" https://example.com

See also --basic and -u, --user. Added in 7.75.0.

--basic

(HTTP) Tells curl to use HTTP Basic authentication with the remote host. This is the default and this option is usually pointless, unless you use it to override a previously set option that sets a different authentication method (such as --ntlm, --digest, or --negotiate).

Used together with -u, --user.

Example:

 curl -u name:password --basic https://example.com

FILES

~/.curlrc

: Default config file, see -K, --config for details.

ENVIRONMENT

The environment variables can be specified in lower case or upper case. The lower case version has precedence. http_proxy is an exception as it is only available in lower case.

Using an environment variable to set the proxy has the same effect as using the -x, --proxy option.

http_proxy [protocol://]<host>[:port]

Sets the proxy server to use for HTTP.

HTTPS_PROXY [protocol://]<host>[:port]

Sets the proxy server to use for HTTPS.

[url-protocol]_PROXY [protocol://]<host>[:port]

Sets the proxy server to use for [url-protocol], where the protocol is a protocol that curl supports and as specified in a URL. FTP, FTPS, POP3, IMAP, SMTP, LDAP, etc.

ALL_PROXY [protocol://]<host>[:port]

Sets the proxy server to use if no protocol-specific proxy is set.

NO_PROXY <comma-separated list of hosts/domains>

list of host names that should not go through any proxy. If set to an asterisk '*' only, it matches all hosts. Each name in this list is matched as either a domain name which contains the hostname, or the hostname itself.

This environment variable disables use of the proxy even when specified with the -x, --proxy option. That is NO_PROXY=direct.example.com curl -x http://proxy.example.com http://direct.example.com accesses the target URL directly, and NO_PROXY=direct.example.com curl -x http://proxy.example.com http://somewhere.example.com accesses the target URL through the proxy.

The list of host names can also be include numerical IP addresses, and IPv6 versions should then be given without enclosing brackets.

IPv6 numerical addresses are compared as strings, so they will only match if the representations are the same: "::1" is the same as "::0:1" but they do not match.

APPDATA <dir>

On Windows, this variable is used when trying to find the home directory. If the primary home variable are all unset.

COLUMNS <terminal width>

If set, the specified number of characters will be used as the terminal width when the alternative progress-bar is shown. If not set, curl will try to figure it out using other ways.

CURL_CA_BUNDLE <file>

If set, will be used as the --cacert value.

CURL_HOME <dir>

If set, is the first variable curl checks when trying to find its home directory. If not set, it continues to check XDG_CONFIG_HOME.

CURL_SSL_BACKEND <TLS backend>

If curl was built with support for "MultiSSL", meaning that it has built-in support for more than one TLS backend, this environment variable can be set to the case insensitive name of the particular backend to use when curl is invoked. Setting a name that is not a built-in alternative will make curl stay with the default.

SSL backend names (case-insensitive): bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, schannel, secure-transport, wolfssl

HOME <dir>

If set, this is used to find the home directory when that is needed. Like when looking for the default .curlrc. CURL_HOME and XDG_CONFIG_HOME have preference.

QLOGDIR <directory name>

If curl was built with HTTP/3 support, setting this environment variable to a local directory will make curl produce qlogs in that directory, using file names named after the destination connection id (in hex). Do note that these files can become rather large. Works with both QUIC backends.

SHELL

Used on VMS when trying to detect if using a DCL or a "unix" shell.

SSL_CERT_DIR <dir>

If set, will be used as the --capath value.

SSL_CERT_FILE <path>

If set, will be used as the --cacert value.

SSLKEYLOGFILE <file name>

If you set this environment variable to a file name, curl will store TLS secrets from its connections in that file when invoked to enable you to analyze the TLS traffic in real time using network analyzing tools such as Wireshark. This works with the following TLS backends: OpenSSL, libressl, BoringSSL, GnuTLS, NSS and wolfSSL.

USERPROFILE <dir>

On Windows, this variable is used when trying to find the home directory. If the other, primary, variable are all unset. If set, curl will use the path "$USERPROFILE\Application Data".

XDG_CONFIG_HOME <dir>

If CURL_HOME is not set, this variable is checked when looking for a default .curlrc file.

PROXY PROTOCOL PREFIXES

The proxy string may be specified with a protocol:// prefix to specify alternative proxy protocols.

If no protocol is specified in the proxy string or if the string does not match a supported one, the proxy will be treated as an HTTP proxy.

The supported proxy protocol prefixes are as follows:

http://: Makes it use it as an HTTP proxy. The default if no scheme prefix is used.
https://: Makes it treated as an HTTPS proxy.
socks4://: Makes it the equivalent of --socks4
socks4a://: Makes it the equivalent of --socks4a
socks5://: Makes it the equivalent of --socks5
socks5h://: Makes it the equivalent of --socks5-hostname

EXIT CODES

There are a bunch of different error codes and their corresponding error messages that may appear under error conditions. At the time of this writing, the exit codes are:

1: Unsupported protocol. This build of curl has no support for this protocol.
2: Failed to initialize.
3: URL malformed. The syntax was not correct.
4: A feature or option that was needed to perform the desired request was not enabled or was explicitly disabled at build-time. To make curl able to do this, you probably need another build of libcurl.
5: Could not resolve proxy. The given proxy host could not be resolved.
6: Could not resolve host. The given remote host could not be resolved.
7: Failed to connect to host.
8: Weird server reply. The server sent data curl could not parse.
9: FTP access denied. The server denied login or denied access to the particular resource or directory you wanted to reach. Most often you tried to change to a directory that does not exist on the server.
10: FTP accept failed. While waiting for the server to connect back when an active FTP session is used, an error code was sent over the control connection or similar.
11: FTP weird PASS reply. Curl could not parse the reply sent to the PASS request.
12: During an active FTP session while waiting for the server to connect back to curl, the timeout expired.
13: FTP weird PASV reply, Curl could not parse the reply sent to the PASV request.
14: FTP weird 227 format. Curl could not parse the 227-line the server sent.
15: FTP cannot use host. Could not resolve the host IP we got in the 227-line.
16: HTTP/2 error. A problem was detected in the HTTP2 framing layer. This is somewhat generic and can be one out of several problems, see the error message for details.
17: FTP could not set binary. Could not change transfer method to binary.
18: Partial file. Only a part of the file was transferred.
19: FTP could not download/access the given file, the RETR (or similar) command failed.
21: FTP quote error. A quote command returned error from the server.
22: HTTP page not retrieved. The requested url was not found or returned another error with the HTTP error code being 400 or above. This return code only appears if -f, --fail is used.
23: Write error. Curl could not write data to a local filesystem or similar.
25: FTP could not STOR file. The server denied the STOR operation, used for FTP uploading.
26: Read error. Various reading problems.
27: Out of memory. A memory allocation request failed.
28: Operation timeout. The specified time-out period was reached according to the conditions.
30: FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT command, try doing a transfer using PASV instead!
31: FTP could not use REST. The REST command failed. This command is used for resumed FTP transfers.
33: HTTP range error. The range "command" did not work.
34: HTTP post error. Internal post-request generation error.
35: SSL connect error. The SSL handshaking failed.
36: Bad download resume. Could not continue an earlier aborted download.
37: FILE could not read file. Failed to open the file. Permissions?
38: LDAP cannot bind. LDAP bind operation failed.
39: LDAP search failed.
41: Function not found. A required LDAP function was not found.
42: Aborted by callback. An application told curl to abort the operation.
43: Internal error. A function was called with a bad parameter.
45: Interface error. A specified outgoing interface could not be used.
47: Too many redirects. When following redirects, curl hit the maximum amount.
48: Unknown option specified to libcurl. This indicates that you passed a weird option to curl that was passed on to libcurl and rejected. Read up in the manual!
49: Malformed telnet option.
51: The peer's SSL certificate or SSH MD5 fingerprint was not OK.
52: The server did not reply anything, which here is considered an error.
53: SSL crypto engine not found.
54: Cannot set SSL crypto engine as default.
55: Failed sending network data.
56: Failure in receiving network data.
58: Problem with the local certificate.
59: Could not use specified SSL cipher.
60: Peer certificate cannot be authenticated with known CA certificates.
61: Unrecognized transfer encoding.
62: Invalid LDAP URL.
63: Maximum file size exceeded.
64: Requested FTP SSL level failed.
65: Sending the data requires a rewind that failed.
66: Failed to initialise SSL Engine.
67: The user name, password, or similar was not accepted and curl failed to log in.
68: File not found on TFTP server.
69: Permission problem on TFTP server.
70: Out of disk space on TFTP server.
71: Illegal TFTP operation.
72: Unknown TFTP transfer ID.
73: File already exists (TFTP).
74: No such user (TFTP).
75: Character conversion failed.
76: Character conversion functions required.
77: Problem reading the SSL CA cert (path? access rights?).
78: The resource referenced in the URL does not exist.
79: An unspecified error occurred during the SSH session.
80: Failed to shut down the SSL connection.
82: Could not load CRL file, missing or wrong format.
83: Issuer check failed.
84: The FTP PRET command failed.
85: Mismatch of RTSP CSeq numbers.
86: Mismatch of RTSP Session Identifiers.
87: Unable to parse FTP file list.
88: FTP chunk callback reported error.
89: No connection available, the session will be queued.
90: SSL public key does not matched pinned public key.
91: Invalid SSL certificate status.
92: Stream error in HTTP/2 framing layer.
93: An API function was called from inside a callback.
94: An authentication function returned an error.
95: A problem was detected in the HTTP/3 layer. This is somewhat generic and can be one out of several problems, see the error message for details.
96: QUIC connection error. This error may be caused by an SSL library error. QUIC is the protocol used for HTTP/3 transfers.
XX: More error codes will appear here in future releases. The existing ones are meant to never change.

BUGS

If you experience any problems with curl, submit an issue in the project's bug tracker on GitHub: https://github.com/curl/curl/issues

AUTHORS / CONTRIBUTORS

Daniel Stenberg is the main author, but the whole list of contributors is found in the separate THANKS file.

WWW

https://curl.se

curl (1) - Linux Manuals

curl: transfer a URL

NAME

SYNOPSIS

DESCRIPTION

URL

OUTPUT

PROTOCOLS

PROGRESS METER

OPTIONS

FILES

ENVIRONMENT

PROXY PROTOCOL PREFIXES

EXIT CODES

BUGS

AUTHORS / CONTRIBUTORS

WWW

SEE ALSO

Pages related to curl