cclive (1) - Linux Manuals

cclive: command line video download tool

NAME

cclive - command line video download tool

SYNOPSIS

cclive [options] [url...]

DESCRIPTION

cclive is a command line video download tool for Youtube and similar websites. It is a rewrite of the clive software in C++.

OPTIONS

-h, --help

Print help and exit.

-v, --version

Print version and exit.

--hosts

Print supported hosts and exit.

--background

Go to background immediately after startup. Output will be written to cclive.log unless --logfile is used.

Output

-q, --quiet

Turn off all output.

--debug

Turn on libcurl verbose mode.

--emit-csv

Print video details in CSV format and exit. Prepends ``csv:'' for each video. Ignores "--quiet" for CSV.

--print-fname

Print filename on a separate line before each download starts. Prepends ``file:'' for each video.

-o, --logfile=file

Write output to file while in the background. Defaults to cclive.log.

-i, --logfile-interval=seconds

Update logfile every seconds while in the background. Defines how frequently the download progress information is written to the log. Defaults to 10.

HTTP

--agent=agentstring

Identify cclive as agentstring to the HTTP servers.

--proxy=proxyhost[:port]

Use the specified HTTP proxy. Overrides http_proxy definition.

--no-proxy

Do not use HTTP proxy, even if http_proxy environment variable is defined.

--connect-timeout=seconds

Maximum time in seconds allowed for connection to take. Defaults to 30.

--connect-timeout-socks=seconds

Otherwise identical to the above option but works around the known ``SOCKS proxy connect timeout'' bug in libcurl. Defaults to 30.

-t, --retry=number

number of retries. Specify 0 for infinite. Default is 5 times, with the exception of errors such as ``forbidden'' (403) or ``not found'' (404).

--retry-wait=seconds

Wait 1..seconds between the retries. Default is 1.

Download

-O, --output-video=file

Write the video to file. Overrides "--filename-format".

-c, --continue

Resume a partially downloaded video file. Affects the file transfers prior to this invocation of cclive. You do not need to specify this option if you want the current invocation of cclive to retry downloading should the connection be lost midway through. This is the default behaviour.

-W, --overwrite

Overwrite the existing file. Negates "--continue".

-n, --no-extract

Do not actually extract any videos, simulate only. cclive parses, verifies the video link and exits without downloading it.

l, --limit-rate=amount

Limit download speed to amount (KB/s). Ignored for video page fetches.

-f, --format=formatid

Download formatid of the video. If set to "best", cclive attempts to download the best quality of the video. See also ``FORMATS''.

-M, --format-map=hostid:formatid|hostid:formatid...

Like "--format" but allows specifying the format for multiple hosts. Note that "--format" setting overrides this setting. See also ``EXAMPLES''.

Filename formatting

-N, --number-videos

Prepend a numeric prefix (e.g. ``001_'') to the output filenames.

-r, --regexp=regexp

Regular expression used to clean up video titles before they are used for output filenames. Supports (and mimics) Perl's /g (global, find all) and /i (case-insensitive). See also ``EXAMPLES''.

-S, --substitute=regexps

Mimics Perl's s/old/new/(gi) substitution. This option can be used to replace occurences in output filenames. Supports /g (global, find all occurences) and /i (case-insensitive).

Use of multiple regular expressions is also supported. You can separate each with with a whitespace. The number of regular expressions is currently unlimited.

See also ``EXAMPLES''.

-F, --filename-format=formatstring

Use the specified formatstring to format the output filenames. cclive defaults to ``%h_%i.%s''. Any of the following specifiers appearing in the format string will be replaced accordingly:

  %t = video title
  %i = video id
  %h = website id
  %s = file suffix

See also ``EXAMPLES''.

Subsequent

--exec=expression;

Execute expression after each file transfer. Optional arguments may be passed to the command. The expression must be terminated by a semicolon (``;'').

If the specifier ``%i'' appears anywhere in expression it will be replaced with the pathname of the extracted video file.

--exec=expression+

Same as "--exec", except that ``%i'' will be replaced with as many pathnames as possible for the invocation of expression.

-e, --exec-run

Invokes the expression specified with "--exec" when download finishes.

Streaming

--stream-exec=expression

Expression to be invoked with "--stream" and "--stream-pass".

If a ``%i'' specifier is used in the expression, it will be replaced with either the video pathname ("--stream") or the parsed video link ("--stream-pass").

-s, --stream-pass

In addition to the ``%i'' specifier, if the specifier ``%f'' appears anywhere in "--stream-exec" expression it will be replaced with the pathname of the extracted video file.

See also ``EXAMPLES''. This feature is based on clive wrapper script contributed by Bill Squire.

EXAMPLES

cclive URL

Download video from URL.

cclive -f best Youtube_URL

Download best available format from Youtube_URL.

cclive -F %t.%s URL

Use video titles in filenames. cclive uses ``%i_%h.%s'' by default. Note that cclive will apply --regexp pattern, if any, to the title before using it. For the supported specifiers, see the "--filename-format" description.

cclive -F %t.%s -r /(\w+)/ URL

Match a string of ``word'' character from the video title and use it in the filename replacing the ``%t'' specifier. cclive replaces the ``%s'' specifier with appropriate file suffix string (e.g. ``flv'').

cclive -F %t.%s -r /(\w|\s)/g URL

Match all ``word'' and ``whitespace'' characters, and use them in the filename replacing the ``%t''. Note the use of ``/g'' (global, find all).

cclive -S s/old/new/i URL

Replace all occurences of ``old'' with ``new'' in the output filename. Note the use of ``i'' (case-insensitive).

cclive -S s/old/new/i s/:/_/g URL

Same but replaces also ':' with '_'. Note the use of ``g'' (global, find all) and the use of a whitespace to separate the used regular expressions.

cclive --exec=mplayer -really-quiet %i; -e URL

Play the downloaded video with mplayer(1) when download finishes.

cclive --exec=ffmpeg -i %i -acodec libvorbis %i.ogg; -e URL

Similar but re-encode audio from the downloaded video to a vorbis audio file, using ffmpeg(1).

echo 'stream-exec = mplayer -really-quiet %i' >> ~/.ccliverc

Adds "--stream-exec" to the config file for permanent use. Saves typing as shown below:

cclive -s URL

Stream and play the video from the URL using mplayer(1), as defined with "--stream-exec" above.

Note that cclive itself does not stream or play the media, it only parses and passes the video link to mplayer(1).

You can use the above as an alternative to the Adobe flash player if you cannot view the streamed videos otherwise, or if you want to play them in an external player while streaming.

The above should work with other player software, like vlc(1) and totem(1).

cclive -s URL --stream-exec=wget %i -O %f

Use wget(1) to download the video. Note the use of the ``%f'' specifier which is unique to "--pass-stream, -s".

cat > url.lst

  http://en.sevenload.com/videos/IUL3gda-Funny-Football-Clips
  http://youtube.com/watch?v=3HD220e0bx4
  http://break.com/index/beach-tackle-whip-lash.html
  http://www.liveleak.com/view?i=704_1228511265

Create a file that will contain multiple URLs, each separated with a newline. We can use the created file with cclive as shown below:

cclive < url.lst

Or:

cat url.lst | cclive

This may save you some typing, as you would normally have to type each URL to the command line as an argument.

echo 'format-map = youtube:best|dailymotion:hq' >> ~/.ccliverc

Save "--format-map" to config file for permanent use.

cclive Youtube_URL Dailymotion_URL

Would set --format=best for Youtube_URL and --format=hq for Dailymotion_URL. Note that the use of "--format" overrides --format-map setting.

FORMATS

cclive uses libquvi <http://quvi.googlecode.com> to parse the video download links. You can get a complete list of the supported formats with the "--hosts" option.

If you have any additional info regarding the formats, report them to the quvi project (see above link) as this is strictly quvi territory.

FILES

$HOME/.ccliverc

Most of the program options can be specified in the $HOME/.ccliverc config file. For example:

 agent      = JBond/1.0       # --agent
 proxy      = http://foo:1234 # --proxy
 limit-rate = 50              # --limit-rate
 no-extract                   # --no-extract

You can also use $CCLIVE_HOME instead of $HOME.

UNICODE

If you are seeing mangled characters in output filenames (titles), this may be because of an invalid locale setting or a sign of terminal incapable of displaying unicode characters. On a typical Unix-like system, try running ``locale -a'' to get a list of the available locale names.

For example, in bash and urxvt terms:

  % LANG=en_US.UTF8 urxvt&
  % cclive ... # in new terminal

cclive (libquvi) converts the characters to unicode if the video HTML specifies the charset meta tag. Otherwise the characters are copied as they are.

If you are missing the unicode characters when using "--regexp" and "--filename-format", make sure the regular expression includes ``\pL''. For example:

  % cclive -F "%t.%s" -r "/(\w|\s|\pL)/g" URL

``In UTF-8 mode, characters with values greater than 128 never match \d, \s, or \w, and always match \D, \S, and \W. This is true even when Uni- code character property support is available. These sequences retain their original meanings from before UTF-8 support was available, mainly for efficiency reasons. Note that this also affects \b, because it is defined in terms of \w and \W.'' --- man pcrepattern

BUGS

Report them at <http://cclive.googlecode.com/>. Please see if the issue has already been reported (or closed) before you submit another. You can use the issue tracker's search feature for this, just be sure to search ``All issues''.

If your bug report contains an error message starting with ``error: libquvi:'', report the bug to the quvi tracker instead (<http://quvi.googlecode.com/>).

You can use the issue trackers for submitting your patches.

DEBUGGING

The following lists some of the cclive options that may be useful while debugging. Other tools, like strace(1), gdb(1) and valgrind(1) may also prove helpful.

cclive --debug URL

Enable libcurl verbose mode.

cclive -n URL

Simulate only. Fetch, parse but skip get.

$prefix/share/quvi/lua/README

$prefix/share/quvi/lua/website/README

cclive relies on libquvi for parsing the video download links. If the parsing ever breaks, the above two files cover some of the essential details.

The latter directory also contains the website specific Lua scripts that libquvi calls to parse the video links.

$prefix/share/doc/quvi

The above directory contains quvi related HOWTOs which may also prove helpful reading for those interested in cclive. The guidelines that the HOWTOs list are also followed in this project.

EXIT STATUS

cclive exits 0 on success, and >0 if an error occurs.

  CCLIVE_OK           = 0
  CCLIVE_OPT          = 1  // cmdline option parsing error
  CCLIVE_OPTARG       = 2  // cmdline option arg error
  CCLIVE_CURLINIT     = 3  // curl init error (unused since 0.6.0+)
  CCLIVE_NOTHINGTODO  = 4  // file already retrieved
  CCLIVE_SYSTEM       = 5  // system call failed
  CCLIVE_NOSUPPORT    = 6  // host not supported
  CCLIVE_NET          = 7  // network error
  CCLIVE_FETCH        = 8  // fetch error
  CCLIVE_PARSE        = 9  // parse error
  CCLIVE_INTERNAL     = 10 // internal error (see return code)

OTHER

Project page:

<http://cclive.googlecode.com/>

AUTHOR

Toni Gundogdu <legatvs [at] gmail.com>.