pq2-ana-dist (1) - Linux Manuals

pq2-ana-dist: Analyse the file distribution of a dataset (or a set of datasets) from a dataset meta-repository based on ROOT files

NAME

pq2-ana-dist - Analyse the file distribution of a dataset (or a set of datasets) from a dataset meta-repository based on ROOT files

SYNOPSIS

pq2-ana-dist [options] datasets

DESCRIPTION

This manual page documents briefly the pq2-ana-dist program.

pq2-ana-dist is a script invoking the pq2 ROOT application to analyse the file distribution of a dataset (or a set of datasets) over the file servers, eiher in terms of files or of file sizes. The output is a text file with the the file movements needed to make the file distribution even in the chosen metrics to be used, for example, as in input to pq2-redistribute(1). Optionally the internal objects can be saved so that they can be used as starting point for a subsequent run. An histogram and a plot can also be saved to visualize the file distribution. The repository with the dataset information can be accessed via the local file system or a remote file server daemon or a PROOF facility.

More details about the underlying 'pq2' application can be found in the man page pq2(1).

ARGUMENTS

datasets

Comma-separated list of datasets to be analysed; the '*' wild card in the items (in such a case the full string - as shown by pq2-ls - should be given in quotes, e.g. '/default/ganis/h1-set5*'.

OPTIONS

-h, --help

Display help information.

-k, --keep

Keep the temporary files created during the analysis under $TMPDIR

-v

Verbose mode

--dataset=<datasets>

Alternative way to define the datasets to be analysed.

-s <servers>, --servers=<servers>

Comma-separated list of servers to be used (-s) in the analysis; a '+' in front of the list adds the specified servers to the existing ones: this can be useful when determining file movements to empty or new servers

-e <excsrvs>, --exclude=<excsrvs>

Comma-separated list of servers to be excluded from the target servers; this can be used, for example, to determine the files movements to drain a server.

-i <ignsrvs>, --ignore=<ignsrvs>

Comma-separated list of servers to be ignored in the analysis; this can be used, for example, to skip the redirector.

-m <metrics>, --metrics=<metrics>

Metrics to be used to calculate the degree of evenness:

    F          use the number of files (default)
    S          use the file size

-f <filemv>, --filemv=<filemv>

Defines the file where to save the result of the analysis; by the default the result is send to the screen. The output contains one line per each file that needs to be moved with the format 'file source destination' where 'file' is the file name, 'source' is the source server URL and 'destination' is the destination server URL. The file can be used as input, for example, to pq2-redistribute(1).

--plot[=<fileplot.fmt>]

Defines the file with the output plot with the original distribution with the server names and the +-10% limits; the extension (if known) defines the format; the default format is 'png' and the default name 'plot.png'. The plot can also be obtained directly from a binary output file (saved with '--fout=<outfile>.root') but just specifying '--fin=<outfile>.root --plot'. The available formats are those known by ROOT: png (default), eps, ps, pdf, svg, gif, xpm, jpg, tiff.

--fout[=<outfile>]

Defines the file where to save the output of the analysis in binary form (ROOT file); this output can be used as starting point for a next run, allowing to run over many datasets in separate steps.

--fin[=<infile>]

Defines the ROOT file from where to fetch the output of a previous run (saved with --fout=infile).

-u <serverurl>, --url=<serverurl>

URL of the PROOF master or data server providing the information; for data servers, it must include the directory. Can also be specified via the environment variables PQ2PROOFURL or PQ2DSSRVURL (see ENVIRONMENT VARIABLES)."

-t <dir>, --tmpdir=<dir>

Directory for temporary files; defualt is /tmp/<username>.

FORMAT OF THE FILE WITH MOVE DIRECTIVES

The out file constist of one line per file to be moved with the following format:

file-name       source-server-URL       destination-server-URL

ENVIRONMENT VARIABLES

See setup-pq2(1).

ORIGINAL AUTHORS

Gerardo Ganis for the ROOT team.

COPYRIGHT

This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version.

This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA

AUTHOR

This manual page was originally written by Gerardo Ganis <gerardo.ganis [at] cern.ch>, for ROOT version 5.