File
Residency Manager Reference
4-December-2013
Andrew Hanushevsky
© 2009-2013 by the Board of Trustees of the Leland
Stanford, Jr., University; all rights reserved.
Produced by Andrew Hanushevsky for Stanford University under contract
DE-AC02-76-SFO0515 with the Department of Energy.
1.1.1 File
Administration Using frm_admin
1.1.2 File Purging
Using frm_purged
1.1.3 Transfering
Files Using frm_xfrd
1.1.3.3 Staging
Files Across A Firewall
1.1.3.4 Staging
via a Global Redirector
1.1.3.4.1 Using The oss.remoteroot
Directive
2.1.1 Backward
Compatability Notes
4 The frm_purged Command for
Purging
4.1 Exported Environment Variables
4.2 The
frm_purged Configuration directives
4.2.1 Directives
used but documented in the ofs/oss Reference
4.2.2 Directives
used but documented in the xrd/xrootd Reference
4.4 Temporarily
Stopping frm_purged
5 frm_xfrd Command for the Transfer
Daemon
5.1 Exported Environment Variables
5.2 The
frm_xfrd Configuration directives
5.2.1 Directives
used but documented in the ofs/oss Reference
5.2.2 Directives
used but documented in the xrd/xrootd Reference
5.2.3 Sample
Configuration Files
5.4 Logged
Transfer Statistics
5.5 Temporarily
Stopping frm_xfrd
5.6 The
frm_xfrd Notification Messages
6 The frm_xfragent Command
Interface To frm_xfrd
7.1 The
hpsscp pftp Command Streams
This document describes File Residency Manager (FRM) commands and configuration directives. FRM was designed to
enhance the Open Storage System (oss) scalable file system, an xrootd feature. FRM can be used to
administer the local file system (i.e., placing files and maintaining symbolic
links to these files), and for copying files to and from other servers,
including a Mass Storage System. In effect, FRM
provides the infrastructure to fully implement Multi-Tiered Storage.
The FRM components are
fully configurable so that virtually any file transfer mechanisms can be used.
The FRM
configuration is described by a configuration file. This configuration file
must hold the configuration directives describing the xrootd configuration as well as a few FRM specific directives.
Using the oss component does not
require that you use FRM. In fact, there is no reason to use FRM if you do not need
Multi-Tiered Storage, do not need server-to-server copy functionality, nor have
enabled scalable file system support using the oss.space
configuration directive.
The FRM consists of three
configurable components:
File administration via the frm_admin command,
·
Inactive file
purging via the frm_purged daemon.
and
·
File Transfer
via the frm_xfragent command and the
frm_xfrd daemon.
The FRM deals with two
major mechanisms:
·
local disk, and
·
remote servers
(e.g., Mass Storage System) that implement a Unix-like name space, a file transfer mechanism, and interface to supply
metadata about stored files.
The frm_admin command allows you to
manipulate the file space and file attributes for file accessed via xrootd on a particular server. Hence,
it can only be run on an actual data server and requires that you use the
standard underlying xrootd-provided mechanisms to access the underlying local
file system. If this is not the case,
the some frm_admin functions may not
be available.
To
properly work, the command scans your xrootd
configuration file to determine how the data server is configured. Once this
occurs you can
·
Verify
that the name space matches the data space, that usage statistics are correct,
and make any needed repairs if they are inconsistent (see audit)
·
Calculate,
reset, and vie file checksums (see chksum),
·
Find
files that have (e.g. a memory mapped or pinned) or lack (e.g. are not check-summed
or migrated) certain attributes (see find),
·
Designate
one or more files as migratable or purgeable (see mark),
·
Designate
one or more files are file as capable of being memory mapped or remove such
designations (see mmap),
·
Designate
one or more files are non-purgeable for a variable amount of time or remove
such designation (see pin),
·
Display
various information about the name and data spaces, transfer queues, and
perform name conversions (see query),
·
Relocate
files from one partition to another (see reloc), and
·
Remove
files from the data server and automatically remove any references to the files
from redirector caches (see rm).
All
of these functions can perform properly only when the total configuration
exists in a single file. This is because frm_admin
is a cross-component command and needs to determine how the components interact
to provide a seamless view.
The
frm_admin command can run as a
command line tool (i.e. a single command specified on the command line), as an
interactive command (i.e., prompting for sub-commands), and in batch mode (i.e.
piping multiple commands to its standard input while in interactive mode). The
command always exits with a zero status code if no errors occurred.
To
make administration easier, the frm_admin
command can run on a live system. There is no need to stop the data server
during its execution.
The
xrootd system allows you to setup
data servers in file caching mode, also known as staging. In this setup, you bring in files as a clients
request them, either in real-time or pre-run mode. Because the data server is
constantly ingesting files the File Residency Manager provides a tool that
allows you to remove files that are no longer popular (i.e., not referenced for
some time). This is accomplished by running the frm_purged daemon.
This
daemon perform the following steps:
·
Reads
the xrootd configuration file
o
looking
for exported paths that have been designated as purgeable (e.g. all.export path purge),
o
determine
the purge policy (i.e., when a file should be purge) for each configured space
(i.e. dirhold, policy, polprog directives), and
o
how
often to attempt purging (i.e. waittime
directive).
·
It
scans all purgeable paths and constructs a list of purgeable candidate files,
ordering them in least used order.
·
At
each purge interval, if the amount of free space is below the desired level,
candidates files still eligible for purging are removed (subject to any
real-time policy as implemented by the polprog
directive) until there
is sufficient free space available.
·
The
above step is repeated until no candidate files remain. In this case, a new
candidate list is constructed (i.e., step 2 above).
While
it is normal to run frm_purged as a
daemon, you can also run it as a one-time command (see the O command line option). This allows you to make additional space
available on an ad hoc basis. When you to run frm_purged as a one-time command and you are also running it as a
daemon, you should pause the daemon during the one-time run. You can do this in
a number of ways (e.g., sending it a SIGSTOP kill signal or temporarily
creating a stop file).
While not absolutely necessary, it does keep the log file clean of false error
messages as the two processes compete for purgeable files.
With
the T command line option you can
see which files would be removed without actually removing the files. This is
useful if you wish to verify that you have correctly specified various purge options.
A
file is deemed purgeable when all of the following conditions are true:
·
resides
in a purgeable path,
·
has
not been accessed for a specified amount of time,
·
is
not pinned (see the frm_admin pin),
·
has
been migrated if it exists in migratable space and has been modified (see Migrating Files), and
·
the
specified real-time policy, if any, allows the file to be purged.
As
files are removed, it is likely that empty directories are created. Normally, frm_purged keeps empty directories for
40 hours and then removes them. You can specify a longer or shorter time using
the dirhold directive.
If
you are also running a simple server inventory daemon (i.e. XrdCnsd) you should use the cnsd directive to tell frm_purged to notify the daemon to
delete removed files from its inventory.
The
frm_xfrd command runs as a daemon
and co-ordinates copying files out of and into a data server. The daemon, itself,
does not actually copes files but instead relies on site-supplied scripts to
perform the actual transfer operation. This allows you to use any copy command
and any source or target location.
The
notion of copying files out of a data server is known as migration. It allows
you to make sure that modified files are backed up on more permanent media.
Migration is completely optional.
The
notion of copying files into a data server is known as staging. It allows you
to automatically fetch missing files into a data server from another local
(e.g. MSS) or global (e.g. remote server) location.
When
frm_xfrd is started, it reads the
xrootd configuration file looking for paths that have been designated as
migratable (i.e. all.export path migrate). If it find no such paths, the migration component of frm_xfrd is disabled and migration
activity occurs. Migration activity can also be disabled, even though
migratable paths exists, if you have not specified how files are to be copied
out of the server using the copycmd directive with the out attribute.
Otherwise,
the daemon sets up an internal process to look for files that should be
migrated. In some sense, the use of the word migrate is historic and rather
inaccurate. Historically, migration meant that the file is merely copied to
another location but not necessarily removed (i.e. purged).
For
each migratable path, the daemon constructs a candidate list of files to be
copied out of the server. A file is deemed migratable if all of the following
are true.
·
It
has not be accessed for the time specified by the idlehold directive
(default is 10 minutes) and
·
it
has been modified but not yet been successfully copied out of the server.
For
each file in the candidate list, the transfer command specified by the copycmd out directive
is invoked to copy the file. If the copy is successful, the file is marked as
migrated and will not be migrated again unless it is modified. If the copy
fails, the copy will be retried on the next construction of a candidate list.
Since
transfers can take a long time, frm_xfrd
rechecks if a file is still eligible for migration prior to invoking the copy
command.
Candidate
lists are periodically constructed. The time interval is specified by the waittime directive which, by
default, is 60 minutes. To avoid over-loading the server, a maximum number of
transfer are allowed to occur at the same time. This is specified by the copymax directive (default is
2). Since migration competes with staging, frm_xfrd
employs a fair share algorithm to prevent starvation of either activity.
You
can manage migration using the frm_admin command with
two subcommands:
·
find to display eligible files that have not yet been
migrated, and
·
mark to force a file to be migrated or to cancel a
pending migration.
You
can always pause frm_xfrds
migration component by creating a special stop file named STOPMIGR.
The
frm_xfrd may also co-ordinate
requests to copy files into the data server. This mechanism is known as
staging. It occurs automatically for stageable paths (i.e., all.export path stage) and it is
possible to manually request that a staging operation occur using the frm_xfragent
command.
Staging
is driven by a disk-based queue. Requests may be added to the queue by the oss component of xrootd or by the frm_xfragent
command. The queue can be listed using frm_admin
query xfrq command. The
control flow is shown in the following diagram.
The
relevant steps are
1. The xrootd daemon and frm_xfragent
command adds stage-in requests to the transfer queue.
2. The frm_xfrd daemon reads these requests
from the queue.
3. Then it
normally pre-allocates a null file to mark where the file is to be placed on
local disk. The pre-allocation occurs just as if the file were created via xrootd less any security checks since
these would have been performed prior to placing the request in the transfer
queue.
4. After possible
file pre-allocation frm_xfrd launches
(i.e. fork/exec) the program identified in the copycmd directive
with the in attribute. The program
is passed the data source and destination, plus additional information, as
configured.
5. The copy
program is responsible for copying the data from the remote source to the local
destination, overlaying the pre-allocated file (i.e. truncate/copy).
6. For xrootd placed entries, frm_xfrd notifies xrootd whether the copy succeeded or failed.
7. If server-side
inventory (i.e. XrdCnsd) is
configured via the cnsd directive and the copy succeeded,
the inventory daemon is notified to add the file to its inventory.
The
transfer queue is a set of files located in the directory identified by the adminpath directive (default is
/tmp); possible modified by the instance name (-n command line option). Normally, there is a 1- to
1-correspondence between an xrootd
instance and a transfer queue. However, you can use the qcheck directive
to establish a single transfer queue for any number of xrootd instances.
The
transfer queue is a set of files because the queue keeps track of requests to
stage files, frm_xfragent initiated
requests to migrate files, and xrootd
3rd copy file requests. Each of these requests can be individually
paused using special stop files. Also,
each request can be given a low, medium, or high priority. High priority
requests execute three times as often as low priority requests; medium priority
requests twice as often. By default, all requests are placed in the low
priority queue.
The
copy command can be any command you desire, including a script that can make
arbitrary decisions on how to accomplish the copy. In all cases, the copy
command must end with a 0 status code is the copy was successful. A status code of 2 if the source file was not
found, and any non-zero status code for any other type of error.
The
frm_xrfd daemon maintains numerous
variables holding information to pass to the command. Perhaps the two most
important variable are $SRC which
contains the source path and $DST
which contains the destination path. For instance, the directive
frm.xfr.copycmd in xrdcp
root:/redirhost/$SRC $DST
Would
copy a file whose name is contained in $SRC
from an xrootd cluster control by
the redirector redirhost to the local
disk at $DST. View some of the sample configuration files to get
additional ideas on how to specify the handling of stage-in copies.
For
many sites, staging causes data to be copied from some internal source (e.g.,
slow file server, Mass Storage System, etc). When you wish to stage from an
external source (e.g. a globally federated cluster) then the server needs to be
able to establish an outgoing connection on the public internet. If there is a
firewall that disallows out-going connections, then additional you need to
bridge the firewall. The usual accepted solution to such a problem is to setup
a border machine that has connectivity to the external internet as well as the
local network. This becomes the proxy host and the actual copy must occur on
the border machine since it is the only machine that has access to the inside
and outside worlds.
This
situation is illustrated in the following diagram.
The
trick is to execute the copy command on the border machine. This can be easily
accomplished by simply using ssh. For instance,
frm.xfr.copycmd in ssh border copycmd
Of
course, you would need to setup ssh host keys to prevent ssh from prompting for a password (see ssh-keygen).
The
only problem here is that the copy command needs to be able to access the local
disk that resides at the launching host. Some copy programs can do this but not
all. Notably, xrdcp cannot be used
to directly access the local disk from a border machine. There are two solutions
to this problem.
This
solution simply sends the data from fetched by xrdcp back to the launching host via the established ssh tunnel and directs it to the
destination file. The basic concept is shown below along with the copycmd directive that would be
specified.
The
wallhop script, running on the
border machine, simply creates a local pipe to capture data from xrdcp and sends it back across the
private network to the originating host where it gets piped into the
destination file. The wallhop script
is so simple that it is shown below.
#!/bin/perl
$infn = $ARGV[0]; # Input source
$fifo = "/tmp/.wallhop.$$"; # The fifo we will be using
print STDERR `mkfifo $fifo`; # Create the
temporary fifo
# Copy fifo to standard out (i.e. source file
-> stdout)
#
exec("/bin/cat $fifo") if !($pid =
fork());
# Copy source file to the fifo, delete the
fifo, and exit
#
$resp = `xrdcp -f -np $infn $fifo`;
kill(KILL,$pid) if $rc = $?;
unlink($fifo);
exit $rc;
The wallhop Script
The
second solution avoids the performance loss due to ssh tunneling by sending the data directly to the xrootd server running on the launching
host, as shown below.
There
are several things to note. First, we must disable frm_xfrd from pre-creating the destination file using the noalloc option. This is because the xrootd will create the file and the
copy operation will fail if the file already exists.
Secondly,
we need to authorize the border to
write into directories that are typically exported as read-only. This is a bit
of a complication because the only way to do this is to export stageable paths
as read-write and prohibit everyone but the border
machine from writing into those paths. If you are already implementing a
complex security model, you may wish to run another instance of an unclustered xrootd server that can be only be used
by the border machine and allows
read-write access to stageable paths. Otherwise, the following diagram shows
how to transform a simple configuration file to one that allows the border machine access.
In
the previous configuration file snippet,
·
the
r/o attribute in the all.export directive was changed to
indicate that the path /data was now
r/w (i.e. read-write) but it should
be exported as r/o to the redirector (i.e. globalro).
This allows data to be locally written to the path but keeps a consistent
read-only view of the path in the cluster.
·
The
ofs.authorize directive was added to
indicate that access privileges are to be enforced.
·
The
acc.authdb directive indicates where
the authorization information is located (i.e. /authfile).
·
In
the authfile the u record indicate that, by default,
everyone has read-only access to data on this server. The h record establishes an read-write exception for any client coming
from the border machine.
Because
anyone on the border machine is given
read-write access, the machine is not shareable with other xrootd clients. If this is too restrictive then you need to run
another instance of an unclustered xrootd
server that can be only be used by the border
machine and allows read-write access to stageable paths.
Staging
in data from a globally federated xrootd
cluster is not much different from staging data an arbitrary remote source.
Here, the data source for the xrdcp
command points to the global redirector as the source of data and care has to be
taken to prevent redirection loops. A redirection loop occurs when your cluster
is federated with a global redirector and uses the global redirector to fetch a
missing file using xrd_frmd. If the
global redirector thinks that the requesting cluster may have the file, it may redirect
xrdcp back to its cluster. However,
since the file is actually missing the request proceeds to the xrd_frmd which simply asks the global
redirector again. Hence, a redirection loop that slowly but eventually ends.
Each
xrootd cluster is automatically
assigned a globally unique cluster identifier. The frm_xfrd sets the variable $CID
to contain this identifier. To avoid redirection loops via the global
redirector you must tell it to ignore your cluster when looking for a file.
This is done by adding additional cgi data (i.e., tried=+$CGI) to the source
path, as shown below.
frm.xfr.copycmd
in noalloc ssh border xrdcp np \
xroot://globalrdr/$SRC?tried=+$CGI
root://$HOST/$DST
The oss.remoteroot directive allows you to automatically direct data source at a remote location and segregate them from the normal copy stream. This works only if all of the following are true
· the default oss plug-in to access the underlying storage is being used, and
· the oss.namelib directive is not specified.
If you are using a name2name plug-in, you can still supply the correct remote location but you will have to programmatically do so.
For instance, assume the remote location is host globalrdr. Then including the directive
oss.remoteroot xrootd://globalrdr/
The oss plug-in would automatically prefix the logical file name with xrootd://globalrdr/ when handing the file to be staged in to frm_xfrd. Consequently, the frm.xfr.copycmd directives can directly refer to the source of the data. For instance,
frm.xfr.copycmd in noalloc ssh border
xrdcp np \
xroot://globalrdr/$SRC?tried=+$CGI
root://$HOST/$DST
becomes
oss.remoteroot xroot://globalrdr/
frm.xfr.copycmd in noalloc url ssh border xrdcp np \
$SRC?tried=+$CGI
root://$HOST/$DST
Notice that we added the url
option to the copycmd directive.
This is because the frm_xfrd will
receive an actual url as the copy
source, not a plain logical file name. A url
formatted name needs no additional qualification as it contains all the
required information.
To help you manage url-based copies, the frm_xfrd places them is a special input
url queue referenced by the copycmd directive. If such a directive
does not exist, no url copies are
performed. This allows you to segregate url
vs lfn based copy requests.
frm_admin [options] [cmd [cmdopts] cmdparms]
options: [-c cfn
] [-d] [-h] [-n name]
[-v]
command: help | audit | chksum | exit | f[ind] | makelf |
mark | mmap | pin | q[uery] | quit | reloc | rm
Function
Execute local file residency maintenance commands.
Options
-c cfn The
configuration file to be used. The default configuration file, /opt/xrootd/etc/xrootd.cf,
is read if it exists.
-d Turn
on debugging.
-h Print
usage information.
-n name
Is the name of the server
instance for which the command is being performed. See the notes for more
information.
-v Prints
additional messages where relevant.
Commands
cmd Is one of the following:
help - provide additional usage information.
audit - verify file system consistency.
chksum - calculate, set, display, or verify a files
checksum.
exit - terminate the program (used in
interactive mode).
find - find existing or missing residency
management file attributes.
makelf - create missing lock meta-files or modifies
existing ones.
mmap - set file memory mapping attributes.
mark - set file migration or purging attributes
pin - set pin attributes.
query - display various information.
reloc - relocate files from one type of space to
another.
rm - remove one or more files or
directories.
cmdopts
Are command specific options. Refer to each command
description for more information.
cmdparms
Are command specific parameters. Refer to each
command description for more information.
Notes
1)
When
frm_admin is invoked with no
parameters, it enters interactive prompt mode. In this mode you can execute a
series of commands by entering new line separated functions and parameters in
response to the prompt. Command-line editing tools are enabled on platforms
that support them.
2)
To
leave interactive mode, use the exit
or quit commands. Alternatively, a
null line in response to the prompt causes the program to exit.
audit [options] what
what: names ldir | space sname[:sdir] | usage [sname]
options: [-fix] [-force] [-migratable] [-purgeable]
[-recursive]
Function
Inventory the logical name
space, physical data space, or usage information for validity.
Parameters
names
Audits the logical name space, ldir.
ldir Is the logical name of the
directory whose contents are to be audited.
space Audits
the physical data space named sname.
usage Audits
the space usage associated with physical data space named sname.
sname Is the name of the space to be
audited for contents or usage. This is the name that is specified using the oss.cache or oss.space directive. By default, all directories associated with
the named space are processed. For usage,
if no sname is specified, all spaces
are audited for usage.
sdir Is a specific directory
associated with sname, as specified
using the oss.cache or oss.space directive. Only that
directory is processed.
Options
-fix Attempts
to fix problems encountered during the audit.
-force When
used in conjunction with fix,
automatically answers yes to every
question posed. This, in effect, corrects all mistakes without intervention.
-migratable
With fix, sets missing
migration-purge file attributes
in ldir to indicate that the file
with the missing attributes must be migrated prior to being purged. This is the
default when ldir is exported with
the mig option.
-purgeable
With fix, sets missing migration-purge
file attributes
in ldir to indicate that the file
with the missing attributes may be purged.
-recursive
Recursively audits the names in
all directories starting with ldir.
Notes on audit names
1)
A
copy-time attribute is associated with each data file to track file
modifications. This attribute is automatically set when the space is exported
with the mig or stage options.
2)
When
a names audit is requested the following occurs
a.
The
directory is scanned for any files that lack a required copy-time attribute. Files
in paths exported with the mig or purge and r/w options must have a copy-time attribute to make them eligible
for migration and purging.
b.
The
default fix action is to set the copy-time attribute that requires the base
file to be migrated or allows it to be purged; depending on the subcommand
options or defaults.
c.
Should
a base file be a symbolic link to a named space, the links consistency is
verified. If the link point to a non-existent data file; the default fix action
is to remove the dangling link.
d.
Additionally
for XA spaces, the data file must be associated with pfn metadata. If the pfn
metadata is missing or incorrect, the default fix action is to recreate the
metadata.
Notes on audit space
1)
Generally,
space auditing is a deterministic process. Occasionally, irresolvable conflicts
can arise. You should not use the fix
option to repair spaces without first running the audit names subcommand.
2)
When
a space is audited the following occurs
a.
The
directory containing the space is scanned to verify that each data file in the
space is pointed to by a symbolic link in the associated name space. When a
missing link is found and the physical name of the file can be determined, the
default fix action is to create the symbolic link. If the physical name cannot
be determined, the default fix action is to delete the data file and its
associated metadata.
b.
Additionally
for XA spaces, the data file must be associated with pfn metadata. If the pfn
metadata is missing; the default fix action is to delete the data file. If the
pfn metadata is incorrect, the default fix action is to create a new symbolic
link in the logical name space pointing to the data file that is consistent
with the pfn metadata.
Notes on audit usage
1)
Usage
auditing calculates the amount of space used by files associated with a
particular space. It is only meaningful for XA spaces.
2)
The
fix option is only meaningful when
usage logging is enabled (see the oss.usage
directive).
3)
Space
usage can be fixed while the xrootd server is running. Applying multiple fixes
is allowed.
When
the oss.runmodeold directive is in
effect, the following notes apply.
Notes on audit names
1)
Lock
meta-files are special files placed in the logical name space that are used to
track file modifications. These files are automatically created when the space
is exported with the mig or stage options.
2)
When
a names audit is requested the following occurs
a.
The
directory is scanned for any orphaned meta-files. These files have a suffix of
.lock and .pin. An orphaned meta-file is a file that exist without the
presence of its associated base (i.e., data) file. The default fix action is to
remove orphaned files.
b.
If
the name space has been exported with the mig
or stage options, the audit verifies
that each base file has an associated lock meta-file. The default fix action is
to create missing lock files that either requires the base file to be migrated
or allows it to be purged; depending on the subcommand options or defaults.
c.
Should
a base file be a symbolic link to a named space, the links consistency is
verified. If the link point to a non-existent data file); the default fix
action is to remove the dangling link.
d.
Additionally
for XA spaces, the data file must be
associated with a pfn meta-file. If the pfn meta-file is missing or incorrect,
the default fix action is to create a new one.
chksum [options] func path
options: [-force]
[-pfn]
[-type digest] [-verbose]
func: calc | ls | set csval | unset | verify csval
Function
Get, set, and verify a files
checksum.
Parameters
calc Calculates
the checksum unless the file already has a valid checksum. Specifying force will force the calculation of
the checksum. When the checksum is calculated, the files checksum is also
updated with the calculated value. The files checksum is also printed.
ls Lists
all the checksums associated with the file unless type specifies a particular checksum.
set Sets
the files checksum to the specified value, csval.
The csval must be specified as an
ASCII sequence of hexadecimal digits consistent with the associated checksum.
unset Deletes
the files default checksum information or the checksum specified with the type option.
verify Effectively
issues calc on the file and then compares the resulting checksum with csval; indicating whether the values are
the same or not. The csval must be
specified as an ASCII sequence of hexadecimal digits consistent with the
associated checksum.
path Is the path of the file to be
acted upon. The path represents a
logical name unless pfn is
specified, in which case path is
taken as a physical filename.
Options
-force forces the calculation of
a checksum even when the file has a valid checksum associated with it.
-pfn treats path as a physical file name and does not apply logical file name
transformations.
-type specifies the checksum
digest to be used. The adler32, crc32, and md5 checksums are natively supported. An additional checksum may be
defined with the ofs.ckslib
directive. The default checksum corresponds to the digest specified on the xrootd.chksum directive. In absence of
that directive, the default becomes adler32.
-verbose
Displays additional information
when the checksum is printed. The default format is:
csval digest
Where csval is
the checksum value as an ASCII sequence of hexadecimal digits and digest is the
algorithmic name for csval. When verbose is specified the format is:
csval digest mm/dd/yy
hh:mm:ss path
Where mm/dd/yy
hh:mm:ss is the date and
time the checksum was calculated and path
is the filename.
Notes
1)
Checksum
values are recorded in the files extended attributes.
find [options] what ldir [ldir [. . .]]
what: failfiles | mmapped | nochksum
digest | pinned |
unmigrated | old
options: [-recursive]
old: nolkfiles
Function
Search the logical name space
for certain kinds of anomalies.
Parameters
failfiles
Finds all fail meta-files (i.e., files suffixed with
.fail).
mmapped
Finds all data files that have individual memory-map
attributes and displays their memory-map properties. See the mmap subcommand for details.
nochksum digest
Finds all data files that do not have a valid checksum
for digest (e.g. md5); nocs is a synonym for nochksum.
nolkfiles
Finds all base files that have missing lock meta-files
(i.e., files suffixed with .lock). This option is provided for installation
using oss.runmode old for backward
compatibility reasons.
pinned
Finds all data files that are potentially pinned and
displays their pinned properties. See the pin
subcommand for details.
unmigrated
Finds all data files whose attributes indicate an
un-migrated status.
ldir Is the logical name of the
directory whose contents are to be searched. More than one logical directory
name may be specified.
Options
-recursive
Recursively processes all
subdirectories of ldir.
Notes
1)
Except
for the mmapped option, the find subcommand is useful only for name
spaces that have been exported with the mig
or stage options.
mark [options] lspec [lspec [. . .]]
options: [-force]
[-migratable]
[-purgeable] [-recursive]
lspec: lfn | ldir[/*]
Function
Set migration-purge attributes
for one or more files.
Parameters
lfn Is the logical name of a file whose
migration-purge attributes are to be set. Multiple lfns may be specified.
ldir Is the logical name of a
directory. When ldir is suffixed by /* then migration and purge metadata is
created or altered for all data files in ldir.
Specifying recursive applies mark to all files in ldir and all of its descendants.
Multiple ldirs may be specified.
Options
-force Does
not ask questions regarding possible conflicts when setting or over-riding
migration-purge attributes.
-migratable
Sets the migration-purge
attributes to indicate that lfn must
be migrated prior to being purged. This is the default.
-purgeable
Sets the migration-purge
attributes to indicate that lfn may
be purged.
-recursive
Recursively processes all
subdirectories of ldir.
Notes
1)
The
mark command is useful only for name
spaces that have been exported with the mig
or stage options.
2)
You
must quote or escape ldir when
specifying ldir with an asterisk on
the frm_admin command line.
mmap [options] lspec [lspec [. . .]]
options: [-keep]
[-lock]
[-off] [-recursive]
lspec: lfn | ldir[/*]
Function
Set memory mapping attributes
for one or more files.
Parameters
lfn Is the logical name of a file whose
memory map attributes are to be set. Multiple lfns may be specified.
ldir Is the logical name of a
directory. When ldir is suffixed by /* then memory map attributes are set
for all data files in ldir.
Specifying recursive applies mmap to all files in ldir and all of its descendants.
Multiple ldirs may be specified.
Options
-keep requests that once the
memory mapping is established, it should be kept even when the file is not
open.
-lock requests that memory pages
be locked in memory, if possible.
-off removes all memory map
attributes. This option takes precedence over any other attribute option.
-recursive
Recursively processes all
subdirectories of ldir.
Notes
1)
The
mmap command without merely allows
the file to be mapped in memory.
2)
The
mmap command is useful only for name
spaces that have been exported with the mcheck
option that allows individual file mmap
attributes to be honored.
3)
You
must quote or escape ldir when specifying
ldir with an asterisk on the frm_admin command line.
4)
The
mmap subcommand is supported only
for file systems that support extended attributes.
mv oldlfn
newlfn
Function
Rename an existing file.
Parameters
oldlfn Is the logical name of the file
to be renamed.
newlfn Is the new logical name. The
new name may not exist.
pin [options] lspec [lspec [. . .]]
options: [-keep tspec]
[-recursive]
tspec: [+]num[d|h|m|s]
| mm/dd/yy | forever
lspec: lfn | ldir[/*]
Function
Set pin attributes for one or
more files to control file purging.
Parameters
lfn Is the logical name of a file
for which pin attributes are to be set. Multiple lfns may be specified.
ldir Is the logical name of a
directory. When ldir is suffixed by /* then pin attributes are set for all
data files in ldir. Specifying recursive applies pin to all files in ldir
and all of its descendants. Multiple ldirs
may be specified.
Options
-keep Sets
amount of time a file must be kept in local disk. To keep a file on disk
unless it has not been accessed for a period of time, specify the period as +num[d | h | m | s] where num is the
quantity and d
is for days, h
for hours, m
for minutes, and s
for seconds (the default). To keep a file on disk for a certain amount of time
past midnight, specify the time as num[d | h | m | s] without the leading plus
sign. To keep a file on disk until a particular date, specify the time in date
format (i.e., keep until this date) as mm/dd/yy
where mm
is the month, dd
is the day, and yy
is the year. To keep the file on disk indefinitely, specify the time as forever. To unpin a file specify a
period of zero (i.e. 0).
-recursive
Recursively
processes all subdirectories of lspec
should it be a directory.
Notes
1)
The
pin command is useful only for name
spaces that have been exported with the purge
option.
2)
Default
pinning attributes are based on the configuration supplied to the purge daemon.
The file pin attributes over-ride the defaults.
3)
When
keep time is zero, the file pin
attributes are removed.
4)
You
must quote or escape ldir when
specifying ldir with an asterisk on
the frm_admin command line.
5)
The
pin subcommand is supported only for
file systems that support extended attributes.
query what
what: pfn lspec
[lspec [. . .]]
| rfn lspec [lspec [. . .]]
| space [[-recursive] lspec [lspec [...]]
| usage [sname]
| xfrq [qtypes] [prty] [vars]
lspec: lfn | ldir[/*]
qtypes: {all | get | migr | put | stage}
[qtypes]
vars: {lfn | lfncgi | mode | obj | objcgi
| op | prty |
qwt | rid | tod | note
| tid} [vars]
Function
Display various information.
Parameters
pfn Displays
the local disk physical file name corresponding to the specified logical name. The
name need not exist. More than one lspec
may be specified.
rfn Displays
the remote physical file name corresponding to the specified logical name. The
name need not exist. More than one lspec
may be specified.
space Without
arguments, displays the configured spaces as defined by the oss.space directive. When lspec is specified, displays the space
in which one or more files reside. The lspec may be preceded by recursive or r to recursively processes all subdirectories of lspec should it be a directory.
lfn Is the logical name to be
queried. Multiple lfns may be
specified.
ldir Is the logical name of a
directory to be queried. Multiple ldirs may be specified.
ldir/* Is the logical name of a
directory. For query space, the
query is applied to all data files in the directory. This is the default for query space when lspec is a directory name. Multiple ldir/*s may be
specified.
usage Displays
usage information based on the usage log file for the space named sname. If sname is not specified, usage for all named spaces is displayed.
xfrq Displays
frm_xfrd queue information. These
are files that need to be copied into or out of the server.
qtypes by default, all queues are listed. Otherwise, qtypes lists particular queues, as follows:
all - all queues, the default.
get - the copy-in queue for specific client-issued request.
migr - the migrate queue for the FRM migration service.
put - the copy-out queue for specific client-issued requests.
stage - the stage-in queue for the FRM pre-stage service.
prty by default, all priorities are listed. Otherwise, prty specified the particular priority
to be listed (i.e. 0, 1, or 2).
vars is an optional list of variable
names, as described below. Information is listed in the order in which the
variables are listed; each separated by a space. The default variable is lfn. Valid variables are:
Var |
Information |
Var |
Information |
Var |
Information |
lfn |
logical
filename |
obj |
lfn or url |
qwt |
seconds
in queue |
lfncgi |
lfn?[cgi string] |
objcgi |
lfncgi or url?[cgi] |
rid |
requestid |
mode |
processing
opts |
op |
Operation
(e.g., <) |
tid |
traceid |
note |
notification
string |
prty |
priority |
tod |
time
of day |
Notes
1)
The
query space subcommand is only
useful when a cached file system has been defined with the oss.cache or oss.space directive.
2)
The
query usage subcommand is only
useful with usage logging has been enabled with the oss.usage directive.
3)
Usage
for non-XA spaces contains limited information and is not necessarily accurate
relative to other non-XA spaces.
4)
Usage
information is displayed in bytes with the following tags:
a. Space the name of the space.
b. Used bytes currently recognized by xrootd as being used.
c. Staged bytes staged by the pre-stage daemon.
d. Purged bytes purged by the purge daemon.
e. Adjust the correction factor determined by audit usage fix.
f. Effective (Used + Staged Purged + Adjust)
5)
The
staged, purged, and adjust
values are periodically reset to zero when the xrootd daemon re-computes the quantity it considers in-use based on
the previous values.
6)
You
must quote or escape ldir when
specifying ldir with an asterisk on
the frm_admin command line.
7)
The
output of query xfrq is largely
self-explanatory. Additional details can be found in the section describing frm_xfragent. For the variable op, the following designations are
used:
< - copy file lfn from a remote location to local
disk.
= - copy file lfn to a remote location and then remove
it from local disk.
> - copy file lfn to a remote location.
+ - stage file lfn from a remote location to local
disk.
^ - migrate file lfn to a remote location and then remove it from local disk.
& - migrate file lfn to a remote location.
reloc lfn sname[:sdir]
Function
Relocate a data file from one
space to another.
Parameters
lfn Is the logical name of the file
to be relocated.
sname Is the name of the space in
which the file is to reside.
sdir Is a specific directory
associated with sname, in which the
file is to reside.
Notes
1)
The
reloc subcommand is only supported
for XA space targets. That is, the file may reside in any space but may only be
relocated to an XA space.
2)
If
lfn already exists in sname, then you must specify sdir and it must differ from the
directory in which the data file resides.
rm [options] lspec [lspec [. . .]]
options: [-echo] [-force] [-recursive]
lspec: lfn | ldir[/*]
Function
Remove one or more files from
local disk.
Parameters
lfn Is the logical name of a file
to be removed. Multiple lfns may be
specified.
ldir Is the logical name of a
directory to be removed. The directory must be effectively empty. Multiple ldirs may be specified.
ldir/* Is the logical name of a
directory. All entries in the directory, but not the directory itself are
removed. Removal fails if the directory contains another directory. In this
case, you must use the recursive option.
Multiple ldir/*s may be specified.
Options
echo Displays
the physical name of every removed file.
force Does
not ask for permission prior to removing a file.
recursive
Recursively removes all files and directories starting
with ldir before removing ldir itself.
Notes
1)
The
recursive option only applies to
directories and is ignored if lspec
is not a directory.
2)
You
must quote or escape ldir when
specifying ldir with an asterisk on
the frm_admin command line.
3)
The
frm.all.cnsd configuration file
directive determines whether or not and how file removals are communicated to
the XrdCnsd. Refer to the Interface to XrdCnsd section for more information.
makelf [options] lspec [lspec [. . .]]
options: [-migratable] [-owner ospec] [-purgeable]
[-recursive]
ospec: usr[:grp] | [usr]:grp
lspec: lfn | ldir[/*]
Function
Create or alter a lock
meta-file.
Parameters
lfn Is the logical name of a file
for which a lock meta-file is to be created or altered. Multiple lfns may be specified.
ldir Is the logical name of a
directory. When ldir is suffixed by /* then lock meta-file are created or
altered for all data files in ldir.
Specifying recursive applies makelf to all files in ldir and all of its descendants.
Multiple ldirs may be specified.
Options
-migratable
Creates
or alters the lock meta-file to indicate that lfn must be migrated prior to being purged. This is the default
when lfn is in a directory exported
with the mig option and the lock
file does not exist.
-owner
Sets the lock meta-file ownership. The default ownership corresponds to the current
effective user and group id. To change the user id, specify usr as a login username or uid. To
change the group id, specify grp as a
group name or gid. Setting ownership may require root privileges.
-purgeable
Creates
or alters the lock meta-file to indicate that lfn may be purged.
-recursive
Recursively processes all
subdirectories of ldir.
Notes
1)
The
makelf command is useful only for
name spaces that have been exported with the mig or stage options.
2)
You
must quote or escape ldir when
specifying ldir with an asterisk on
the frm_admin command line.
3)
The
makelf is deprecated and offered
only for backward compatibility (i.e., oss.rumode
old directive has been specified). It will be removed in a future release.
frm_purged [ options ] [ args ]
args: [sname]
[path] [args]
options: [-b] [-c cfn] [-d] [-f]
[-k {num | sz{k|m|g}
| sig}] [-l [=]logfn]
[-n name] [-O free[,hold]] [-s pfn]
[-S site]
[-T] [-v] [-z]
sig: fifo|hup|rtmin|rtmin+1|rtmin+2|ttou|winch|xfsz
Function
Manage
the purging of files inactive files from local disk.
Arguments
sname Restricts
purging to the indicated space. Only files residing in sname are potentially purged. Any number of space names may be
specified but each specified name must have been defined via the oss.cache or oss.space directive.
path Restricts
purging to the indicated path and all of its descendants. Only files residing
on path and defined as purgeable via
the all.export directive are
potentially purged. Any number of path names may be specified but each
specified name must have been exported via the all.export directive.
Options
-b Runs in true daemon mode as a
background process.
-c configfn
The
configuration file name. The default name for the configuration file is
/opt/xrootd/etc/xrootd.cf.
-d Turns on debugging mode.
Additional information is printed to describe various actions.
-f Automatically removes (i.e., fixes)
orphaned fail, lock, and pin files.
-k
num | sz{k|m|g} | sig
Keep
no more than num old log files. If sz is specified, the number of log files
kept (excluding the current log file) is trimmed to not exceed sz bytes. The sz must be suffixed by k,
m, or g to indicate kilobytes,
megabyte, or gigbytes, respectively. If a sig
value is specified (i.e. hup etc),
then an external program is expected to handle log file rotation (e.g.
logrotate). Except for fifo, the
argument specifies signal that causes the daemon to close and re-open the log
file to allow rotation to occur. When fifo is specified, the daemon waits for
data to appear on a fifo whose path is identical to the log file path but whose
name is prefixed by a dot. Refer to the notes for manual rotation caveats.
-l logfn
Routes
error messages and any trace output to logfn. By default, messages are
directed to standard error. When fn
is prefixed by an equals sign, the fn
is not qualified by the instance name, if any. This allows log files to be
handled in an arbitrary manual way. For more information see the section on
fencing daemon instances.
-n
name
Assigns name to the frm_purged
instance. By default, the frm_purged
instance is unnamed. See the notes on how to use this option.
-O
free[,hold]
Runs frm_purged as a command and performs a one-time purge. Specify for free either a percentage of free space
(e.g., 10%) that is required or an actual amount optionally suffixed by k, m,
or g to indicate kilobytes, megabytes, or gigbytes,
respectively. Optionally specify for hold
the minimum amount of time must have not been accessed in order to be purged.
The time may be suffixed by h, m,
or s to indicate hours, minutes, or seconds (the
default), respectively. The one-time policy applies to all spaces selected for
purging. Unspecified values come from each configured policy.
-s pfn Specifies
the name of the file that is to hold the process id upon start-up.
-S site Specifies
a 1- to 15-character site name that is to be included in monitoring records.
The name may only contain letters, digits and the symbols _-:.; any other
characters are converted to a period.
-T Runs in test mode. No files are
actually deleted.
-v Increases verbosity by
displaying additional information about which files were purged.
-z provides microsecond resolution
for log file message timestamps.
Notes
1)
The
frm_purged also accepts additional
directives from the configuration file, configfn.
These directives are described in the following section.
2)
The
b and s options are meant to be used by start-up scripts (e.g. init.d). When s is specified, an additional pid-file is created.
3)
If
a log file is specified without a signal -k option, the file is closed at
midnight, renamed to have a date suffix (i.e., fn.yyyymmdd) and possible sequence number
(i.e. fn.yyyymmdd.n), and a new log file is
opened.
4)
When
a signal value is specified, log files are not automatically renamed at
midnight. Instead an external program must be used to properly rotate log
files. Make sure to choose a signal that is not
in use by any plug-in. If unsure,
choose one of the obscure signal names and monitor for any odd behavior.
Otherwise, use the fifo option. Be
aware that on some non-Linux platforms the fifo file descriptor may leak.
5)
When
fifo is specified the fifo file name
must not exists or exist as a fifo file. A simple echo x >> /path/.lfn
causes the log file to close and reopen.
6)
The
sig names, except for fifo, be fully capitalized as well
prefixed by sig or SIG when capitalized.
The following table shows the environment variable exported by frm_purged. These may be used by external programs and
plug-ins, as needed. They should never be modified.
Variable |
Contents |
XRDCONFIGFN |
The effective administrative path used for server management files. |
XRDDEBUG |
Set to one when the d command line option is specified. |
XRDHOST |
The current hosts DNS name. |
XRDLOGDIR |
Is the directory where log files are written. |
XRDNAME |
The name specified via n or anon if no instance name was specified. |
XRDPROG |
The executables name. |
XRDSITE |
The site name specified either via the s command line option or the
all.sitename directive. |
In
addition to the directives prefixed by frm.purge,
frm_purged also uses certain ofs, oss, and xrood-specific
directives, if already specified, for configuration. They are shown grayed-out
below.
all.adminpath apath
all.pidpath ppath
frm.all.cnsd {auto | ignore |
require} [cnsopts]
frm.all.monitor [ident sec] dest [ dest
]
frm.purge.dirhold hold
frm.purge.policy {* | sname}
[nopurge | polargs]
frm.purge.polprog [vars] |path [pargs]
frm.purge.waittime wsec
ofs.osslib libpath
[ parms ]
oss.localroot lpath
oss.namelib npath
[ parms ]
oss.remoteroot rpath
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
cnsopts: [apath apath]
dest: dest events host:port
events: [migr] [purge] [stage]
vars: atime | ctime | fname | fsize | fspace
mtime | pfn | sname | tspace | usage
polargs: [minfree [maxfree]] [hold hold] [polprog]
hold: forever | htime
Directives
frm.all.cnsd {auto|ignore|require} [apath apath]
Specifies
whether or not and how file removals are communicated to the XrdCnsd. Refer to the Interface to XrdCnsd section for more information.
frm.all.monitor [ident
sec] dest events dest [dest events
dest ]
Enables
monitoring of events (see the Scalla Monitoring reference for more
information). Event records can be sent via UDP to one or two destinations, dest (i.e. host:port) You select which records are sent by
specifying one more event types. Valid events
are:
migr files copied to remote storage by frm_xfrd.
purge files removed from local storage by frm_purged.
stage files copied from remote storage by frm_xfrd.
The ident option is used to specify the
number of seconds between each server identity transmission (i.e., the = map record). Specify a number optionally suffixed
by h
for hours, m for minutes, or s for seconds, the
default. A value of zero transmits the identity only once at start-up time. The
default is 1 hour (i.e. 3600 seconds).
frm.purge.dirhold htime
Specifies
how long empty directories are to be kept after being created. If htime is the word forever, empty directories are not removed. Otherwise, empty
directories are removed after htime
past the last modification. The htime
may be suffixed by h, m, or s to indicate hours, minutes, or seconds (the default), respectively. The default is 40h.
frm.purge.policy {* | sname} nopurge | [minfree [maxfree]] [hold hold]
[polprog]
Defines
a purging policy. An asterisk (*)
defines the default policy and is used to complete all subsequent policy
directives; while an sname defines a
policy for the named space. Specifying nopurge prevents file purging.
Otherwise, specify one or more of the following:
minfree the minimum amount of free space. Purging begins when the amount
of free space falls below this value. Specify a percentage of the total space
(e.g., 10%) or an absolute amount space, optionally suffixed by k, m,
or g to indicate kilobytes, megabytes, or gigbytes,
respectively. The default is 2%.
maxfree the maximum amount of free space. Purging stops when the amount
of free space goes above this value. Specify a percentage of the total space or
an absolute amount space which may be optionally suffixed by k, m,
or g to indicate kilobytes, megabytes, or gigbytes,
respectively. The maxfree value must
be greater than or equal to the minfree
value. The default is 3% if minfree
is not specified. Otherwise, if minfree
is a percentage the default is minfree+1
and if minfree is an absolute value
the default is minfree*120%.
htime the minimum amount of time the file must have not been
accessed in order to be purged. The htime
may be suffixed by h, m, or s to indicate hours, minutes, or seconds (the default), respectively. The default is 20h (hours).
polprog invokes the program defined by a previous polprog directive in order to determine if the file can be purged
once all other criteria have been met.
frm.purge.polprog [vars]
|path [pargs]
Defines
an external policy. The external policy is implemented by the program named by
path. The program is started with optional arguments pargs. Whenever a purge decision is needed the indicated variable
values, terminated by a newline character (\n), are sent to the programs
standard-in. The program must write a single character response, terminated by
a newline, to standard-out (see the usage notes). The following variables may
be specified:
atime files last access time in Unix seconds.
ctime files creation time in Unix seconds.
fname file name, excluding the directory path.
fsize size of the file in bytes.
fspace bytes of free space available in the files designated space.
mtime files last modification time in Unix seconds.
pfn physical file name, including its full path.
sname name of the space in which the file resides.
tspace total bytes allocated to the files designated space.
usage bytes used in the files designated space. This may be -1 if usage
information is not maintained by space name.
frm.purge.waittime wsec
wsec is the number of seconds to
wait before checking whether purging is necessary. The wsec may be suffixed by h,
m, or s to indicate hours, minutes, or seconds (the default), respectively. The default is 5 minutes.
ofs.osslib libpath
[ parms ]
libpath is the absolute path to the
shared library that contains the implementation of the storage system interface
that frm_xfrd is to use for file
system specific storage operations (e.g., create, rename, etc). The default is
to use a built-in mechanism that is identical to what xrootd uses by default.
oss.localroot lpath
lpath is the path that must prefix
any logical file name before using it as a local physical file name (i.e. to
map lfn to local pfn). By default, no prefix is used. Also see the oss.namelib directive.
oss.namelib npath [ parms ]
npath is the absolute path to the
shared library that contains the implementation of the name-to-name translation
interface that frm_xfrd is to use to
convert logical file names to local and remote physical file names. The default
is to use a built-in mechanism that relies on the oss.localroot and oss.remoteroot
directives.
oss.rmoteroot rpath
rpath is the path that must prefix
any logical file name before using it as a remote physical file name (i.e. to
map lfn to Mass Storage System pfn). By default, no prefix is used.
Also see the oss.namelib directive.
all.adminpath apath
apath is the fully qualified
administrative base path where various directories and special files may be
created to control execution. The default is /tmp. The n option augments the base path.
all.pidpath apath
ppath is the fully qualified base path
where the frm_purged.pid file is to
be written. The default is /tmp. The n
option augments the base path.
Notes
1)
The
frm_purged program periodically
performs a full file system namespace scan. The frequency of the scan is
determined by the smallest policy hold time. Full namespace scans are skipped as
long as sufficient files remain that meet the policy from the previous scan.
2)
The
polprog directive identifies an
external policy program. The program is started at initialization time and is
expected to remain active to accept input from standard-in and provide
responses to standard-out. It is automatically restarted should it fail. The
default set of tokens sent to the program are sname pfn fsize atime mtime\n (i.e., space name, physical file
name, file size, last access time, last modification time). The program must
respond with a single character followed by a newline character (i.e., x\n).
There are three possible response sequences:
a) n\n which prevents the associated file from being purged,
b) y\n which purges the associated file, and
c) Any other sequence which not
only prevents the associated file from being purged but also stops the purging
process associated with the space until the next purge cycle starts.
The following files are created by
the frm_xfrd:
Path |
Type |
Modified
by |
Purpose |
/tmp/[name/]frm_purged.pid |
File |
pidpath
and n
option |
Holds the process id of the associated
frm_purged server |
/var/adm/frm/core/[name/]core |
File |
n
option |
Core file via default in StartXRD.cf |
/var/adm/frm/logs/[name/]xfrlog |
File |
-l
option and n
option |
Log file via default in StartXRD.cf |
The adminpath directive
specifies the directory where the remaining files are written. The -n option specifies the
frm_purged instance name. If specified, the instance name is
automatically suffixed to the adminpath or /tmp, as shown by [name/]. A directory is also
create in the current working directory for core files and the log file
destination is modified by inserting [name/]. in the destination
specified by the l option. If necessary, the directory is created.
Each
time frm_purged is about to schedule
a purge scan, it check whether a stopfile
exists indicating that purging is not allowed.
The stopfiles is:
/apath/frm[/name]/STOPPURGE
Where
apath comes from the all.adminpath
directive or its default.
name comes from the n command line option and is empty if not
specified.
The
presence of the file stops purging.
frm_xfrd [ options ]
options: [-b] [-c cfn] [-d] [-k {num | sz{k|m|g}
| sig}]
[-l [=]logfn]
[-n name] [-s pfn] [-S site] [-T]
[-v] [-z]
sig: fifo|hup|rtmin|rtmin+1|rtmin+2|ttou|winch|xfsz
Function
Manage
the copying of files from a Mass Storage System to local disk.
Options
-b Runs in true daemon mode as a
background process.
-c configfn
The configuration file name. The default name for the
configuration file is /opt/xrootd/etc/xrootd.cf.
-d Turns on debugging mode. Additional
information is printed to describe various actions.
-k
num | sz{k|m|g} | sig
Keep
no more than num old log files. If sz is specified, the number of log files
kept (excluding the current log file) is trimmed to not exceed sz bytes. The sz must be suffixed by k,
m, or g to indicate kilobytes,
megabyte, or gigbytes, respectively. If a sig
value is specified (i.e. hup etc),
then an external program is expected to handle log file rotation (e.g.
logrotate). Except for fifo, the
argument specifies signal that causes the daemon to close and re-open the log
file to allow rotation to occur. When fifo is specified, the daemon waits for
data to appear on a fifo whose path is identical to the log file path but whose
name is prefixed by a dot. Refer to the notes for manual rotation caveats.
-l logfn
Routes
error messages and any trace output to logfn. By default, messages are
directed to standard error. When fn
is prefixed by an equals sign, the fn
is not qualified by the instance name, if any. This allows log files to be
handled in an arbitrary manual way. For more information see the section on
fencing daemon instances.
-n
name
Assigns name to the frm_xfrd
instance. By default, the frm_xfrd
instance is unnamed. See the notes on how to use this option.
-s pfn Specifies
the name of the file that is to hold the process id upon start-up.
-S site Specifies
a 1- to 15-character site name that is to be included in monitoring records.
The name may only contain letters, digits and the symbols _-:.; any other
characters are converted to a period.
-T Runs in test mode. In test
mode, no destructive actions (e.g., file removal) are taken.
-v Produces additional details
during executions (i.e., verbose mode).
Notes
1)
The
frm_xfrd also accepts additional
directives from the configuration file, configfn.
These directives are described in the following section.
2)
The n option allows you to run multiple
instances of frm_xfrd with a common
configuration file. This is possible because the specified name is used to
modify various file system paths frm_xfrd
uses for output files. By automatically differentiating such paths by instance
name prevents two frm_xfrd processes
from interfering with each other.
3)
The
frm_xfrd verifies that it is the
only one running with a given instance name. If another frm_xfrd is discovered, the program exits.
4)
The
chosen instance name must be the same as assigned to the frm_xfragent that supplies frm_xfrd
work.
5)
The
cmsd and xrootd have built-in mechanisms to communicate with frm_xfrd.
6)
The
b and s options are meant to be used by start-up scripts (e.g. init.d). When s is specified, an additional pid-file is created.
7)
If a log file
is specified without a signal -k
option, the file is closed at midnight, renamed to have a date suffix (i.e., fn.yyyymmdd)
and possible sequence number (i.e. fn.yyyymmdd.n),
and a new log file is opened.
8)
When a signal
value is specified, log files are not automatically renamed at midnight.
Instead an external program must be used to properly rotate log files. Make
sure to choose a signal that is not
in use by any plug-in. If unsure,
choose one of the obscure signal names and monitor for any odd behavior.
Otherwise, use the fifo option. Be
aware that on some non-Linux platforms the fifo file descriptor may leak.
9)
When fifo is specified the fifo file name must not exists or
exist as a fifo file. A simple echo x >> /path/.lfn causes the log file to close and reopen.
10) The sig
names, except for fifo, be fully
capitalized as well prefixed by sig
or SIG when capitalized.
The following table shows the environment variable exported by frm_xfrd. These may be used by external programs and
plug-ins, as needed. They should never be modified.
Variable |
Contents |
XRDCONFIGFN |
The effective administrative path used for server management files. |
XRDDEBUG |
Set to one when the d command line option is specified. |
XRDHOST |
The current hosts DNS name. |
XRDLOGDIR |
Is the directory where log files are written. |
XRDNAME |
The name specified via n or anon if no instance name was specified. |
XRDPROG |
The executables name. |
XRDSITE |
The site name specified either via the s command line option or the
all.sitename directive. |
In
addition to the directives prefixed by frm.xfr
and frm.xfr.migr (for the
automatic migration component), frm_xfrd
also uses certain ofs, oss, and xrootd-specific directives, if present, for configuration. They are
shown grayed-out below. Refer to the corresponding manuals for their full descriptions.
all.adminpath apath
all.pidpath ppath
frm.all.cnsd {auto | ignore |
require} [cnsopts]
frm.all.monitor [ident sec] dest [ dest
]
frm.xfr.copycmd [opts] cmd [ args ]
frm.xfr.copymax cmax
frm.xfr.qcheck [qsec] [qpath]
frm.xfr.migr.idlehold isec
frm.xfr.migr.waittime wsec
ofs.osslib libpath
[ parms ]
oss.localroot lpath
oss.namelib npath
[ parms ]
oss.remoteroot rpath
oss.xfr deny dt
xrootd.monitor dest stage host:port
cnsopts: [apath apath]
dest: dest events host:port
events: [migr] [purge] [stage]
opts: in | noalloc | out | stats | timeout tsec | url |
xpd [opts]
args: [text] [var] [args]
var: $CID | $CGI | $DST | $INS | $LFN | $PFN | $RFN |
$NOTIFY | $MDP | $OFLAG | $PRTY | $RID | $SRC |
$eVar
Directives
frm.all.cnsd {auto|ignore|require} [apath apath]
Specifies
whether or not and how file additions are communicated to the XrdCnsd. Refer to the Interface to XrdCnsd section for more information.
frm.all.monitor [ident
sec] dest events dest [dest events
dest ]
Enables
monitoring of events (see the Scalla Monitoring reference for more
information). Event records can be sent via UDP to one or two destinations, dest (i.e. host:port) You select which records are sent by
specifying one more event types. Valid events
are:
migr files copied to remote storage by frm_xfrd.
purge files removed from local storage by frm_purged.
stage files copied from remote storage by frm_xfrd.
The ident option is used to specify the
number of seconds between each server identity transmission (i.e., the = map record). Specify a number optionally suffixed
by h
for hours, m for minutes, or s for seconds, the
default. A value of zero transmits the identity only once at start-up time. The
default is 1 hour (i.e. 3600 seconds).
frm.xfr.copycmd [opts] cmd [ args
]
cmd is the absolute path of the
program that frm_xfrd is to use to
transfer a file. One or more commands may be specified, depending on opts:
in cmd is only used to
transfer files into the server. If copycmd
in is defined but a copycmd out is not defined; outgoing transfers will fail. If cmd is not qualified with in or out, cmd is used to
transfer a file regardless of direction.
noalloc
does not create a placeholder in the filesystem for
incoming files.
out cmd is only used to
transfer files out of the server. If copycmd
out is defined but an copycmd in is not defined; incoming transfers will fail. If cmd is not qualified with in or out, cmd is used to
transfer a file regardless of direction.
stats transfer statistics are written to the log file. See the next
section for the format and meaning of these statistics.
timeout
tsec
cmd is allowed
up to tsec seconds to complete before
it is killed. The tsec value can
suffixed by h, m, or s to indicate hours, minutes, or seconds (the
default), respectively. The default is zero, preventing any timeout from
occurring.
url cmd is only used if
the source or destination filename is in url-format. If copycmd url is not
defined, any url-based transfers will fail.
xpd includes the first line of output from the copy program into the
monitoring record tagged as pd=.
See the notes for more information.
args are arbitrary command-specific
tokens and may include special variables (see the notes for substitution rules)
that are substituted each time cmd is
invoked. The following variables may be specified:
$CID the global cluster identifier issuing this request.
$CGI all of the opaque information specified after the question
mark in the file path.
$DST the target file name. This may be $PFN or $RFN, depending
on the transfer direction. It may also be a url for url-based copy commands.
$INS instance name (n
option) issuing this request.
$HOST the hostname of where the daemon is running.
$LFN logical file name.
$PFN the local physical file name as modified by localroot or the namelib.
$RFN remote file name as modified by remoteroot or the namelib.
$NOTIFY notification string. See the cms.prepmsg or the oss.stagemsg directive for details.
$MDP the offset into the destination file path where directories
likely need to be created for the transfer to succeed. See the notes for more information
about this variable.
$OFLAG a character letter describing the file open processing flags:
w
O_WRONLY | O_RDWR r O_RDONLY
$PRTY request priority.
$RID request identifier.
$SRC the source file name. This may be $PFN or $RFN, depending
on the transfer direction. It may also be a url for url-based copy commands.
$TID original requesting clients trace identification.
$eVar any variable that has been passed along with the file name
as opaque information (i.e.,
contained in $CGI).
frm.xfr.copymax cmax
cmax specifies the maximum number
of files that may be transferred at the same time. The default is 2.
frm.xfr.qcheck [qsec] [qpath]
specifies
the queue check interval and/or the queue path. Specify qsec or qpath, or both,
as follows:
qsec specifies how often the transfer queues are to be checked for new
additions. Normally, xfr_xfrd is
notified when an addition is made and immediately checks the appropriate queue.
A manual check is periodically made in case a notification is lost. The qsec may be suffixed by h, m,
or s to indicate hours, minutes, or seconds (the
default), respectively. The default is 5 minutes.
qpath specifies the absolute location where the queue files are to be
written. The specified path is not augmented by the instance name. This allows
you to share a single instance of frm_xfrd
with multiple clusters running on the same host. By default, the queue files
are written using the adminpath
location.
frm.xfr.migr.idlehold isec
isec specifies the number of
seconds a file must be idle before it is eligible for automatic migration to a
Mass Storage System. This is part of automatic migration which is enabled when
migratable paths have been defined via the all.export
directive. The isec may be suffixed
by h, m, or s to indicate
hours, minutes, or seconds (the default), respectively.
The default is 10 minutes. See the usage notes before changing the default.
frm.xfr.migr.waittime wsec
wsec specifies the number of
seconds to wait between scans of the name space to find files eligible for
migration to a Mass Storage System. This is part of automatic migration which
is enabled when migratable paths have been defined via the all.export directive. The wsec
may be suffixed by h, m, or s to indicate hours, minutes, or seconds (the default), respectively. The default is 60 minutes. See
the usage notes before changing the default.
Notes
1)
You
must use the frm_xfragent command to
communicate with frm_xfrd. The xfr_agent command is responsible for
adding, removing, and listing transfer requests queued to frm_xfrd (see the next section).
2)
You
can have up to four different copy commands:
|
non-url
copy command |
url
based copy command |
In |
1 |
2 |
Out |
3 |
4 |
3)
The copy command must indicate success or
failure by its return code:
RC |
Meaning |
RC |
Meaning |
RC |
Meaning |
0 |
Copy succeeded |
2 |
file not found |
5 |
I/O error |
>0 |
fatal error |
|
|
|
|
4)
r.migr.idlehold
value sets the minimum amount of time that a file
must remain unmodified before it can be migrated. This minimizes migrations
that need to be restarted should the file change. Smaller values increase the
chance a file migration will have to be redone.
5)
The
frm.xfr.migr.waittime value sets the maximum
amount of time between full name space scans. Full name space scans are disk
I/O intensive and should be minimized. Since scans more often than the frm.xfr.migr.idlehold
value cant produce new files to migrate, the
minimum is automatically capped at the idlehold value.
6)
Using the $MDP variable causes frm_xfrd to keep track of
successful outgoing transfers by destination. When a transfer succeeds, the
destination path is recorded under the presumption that the transfer command
created the path at the destination. Subsequent transfers using the path (or
some part of it) are told which portion of the path should already exist at the
destination by having $MDP provide the offset into
the destination path of where the first unseen directory occurs. If all
directories should have been created the $MDP offset refers to the slash just before the file name.
7)
The oss component in xrootd provides an automatic
staging facility where missing files on local disk can be automatically fetched
from a remote location. This is done using the oss.remoteroot and oss.stagecmd directives along with the
stage option on the all.export directive. Refer to the ofs/oss reference for more information.
8)
The frm_xfrd creates queue files that describe pending requests in the same partition
as the adminpath. If you need to place
these files in a larger partition or, perhaps, in a more reliable partition,
use a symbolic link at the Queues directory (see the next section) to point
to where the queue files are to be placed. Alternatively, specify the path
using the qcheck directive. Be aware that
this directive allows all clusters running on the same machine to use the same
queue file and consequently the same frm_xfrd.
9)
If the copycmd causes data to flow back through the initiating xrootd, you should specify the noalloc option to prevent frm_xfrd from interfering with the
xrootd in terms of file creation.
10) Monitoring records are created when migr (outgoing data) or stage (incoming data) is
specified on the xrootd.monitor directive. The xpd option of the copycmd allows you to include
additional information from the copy command, typically a script wrapper. When xpd is specified, the first line of output (i.e. the first line ending with
a new line character, \n, or physically the only line) is added after the pd=
tag in the record. The data in the line should adhere to standard CGI syntax to
allow for uniform processing.
ofs.osslib libpath
[ parms ]
libpath is the absolute path to the
shared library that contains the implementation of the storage system interface
that frm_xfrd is to use for file
system specific storage operations (e.g., create, rename, etc). The default is
to use a built-in mechanism that is identical to what xrootd uses by default.
oss.localroot lpath
lpath is the path that must prefix
any logical file name before using it as a local physical file name (i.e. to
map lfn to local pfn). By default, no prefix is used. Also see the oss.namelib directive.
oss.namelib npath [ parms ]
npath is the absolute path to the
shared library that contains the implementation of the name-to-name translation
interface that frm_xfrd is to use to
convert logical file names to local and remote physical file names. The default
is to use a built-in mechanism that relies on the oss.localroot and oss.remoteroot
directives.
oss.rmoteroot rpath
rpath is the path that must prefix
any logical file name before using it as a remote physical file name (i.e. to
map lfn to Mass Storage System pfn). By default, no prefix is used.
Also see the oss.namelib directive.
oss.xfr deny dt
dt specifies how long a .fail
file may block a transfer. When a transfer fails file fn, a fn.fail file is
created. The presence of this file prevents additional transfer to be attempted
for the file. After dt seconds have
elapsed, the fail no longer prevents a transfer attempt. The dt may be suffixed by h, m,
or s to indicate hours, minutes, or seconds (the
default), respectively. The default is 3 hours.
all.adminpath apath
apath is the fully qualified
administrative base path where various directories and special files may be
created to control execution. The default is /tmp. The n option augments the base path.
all.pidpath apath
ppath is the fully qualified base
path where the frm_xfrd.pid file is
to be written. The default is /tmp. The n
option augments the base path.
xrootd.monitor dest
stage host:port
host:port is the logging destination for statistics about incoming
transfers. By default, incoming transfers are not externally logged.
To be added.
The following files are created by
the frm_xfrd:
Path |
Type |
Modified
by |
Purpose |
/tmp/[name/]frm/CIDS |
File |
adminpath
or qcheck and n option |
Cluster ID checkpoint file. |
/tmp/[name/]frm/frm_xfrd.lock |
File |
adminpath
or qcheck and n option |
Local frm_xfrd
execution lock. |
/tmp/[name/]frm/xfrd,udp |
UDP Socket |
adminpath
or qcheck and n option |
Local frm_xfrd
event notifications |
/tmp/[name/]frm_xfrd.pid |
File |
pidpath
and n
option |
Holds the process id of the associated
frm_xfrd server |
/tmp/[name/]frm/Queues/ |
Dir |
adminpath
or qcheck and n option |
Holds queue files and lock files for maintain
queued requests. |
/tmp/[name/]cmsd.superpid |
File |
pidpath
and n
option |
Holds the process id and the local path
prefix (i.e., localroot) for a
supervisor cmsd. |
/var/adm/frm/core/[name/]core |
File |
n
option |
Core file via default in StartXRD.cf |
/var/adm/frm/logs/[name/]xfrlog |
File |
-l
option and n
option |
Log file via default in StartXRD.cf |
The adminpath directive
specifies the directory where the remaining files are written. For certain
files, the frm.xfr.qcheck directive
overrides the adminpath directive. The
-n option
specifies the frm_xfrd instance name. If specified, the instance name is
automatically suffixed to the adminpath or /tmp, as shown by [name/]. The path is not augmented
if it was specified via the frm.xfr.qcheck
directive. A directory is also create in the current working directory for core
files and the log file destination is modified by inserting [name/].
in the destination specified by the l option. If necessary, the directory is created.
The Queues directory contains
numerous files that track pending requests. The files in this directory are
also modified by frm_xfragent.
When
the copycmd directive specifies stats, transfer statistics are written
to the log whenever the command successfully transfers a file. The preferred
alternative to getting this information is to use the frm.all.monitor directive. This mechanism provides a comprehensive
way to collect transfer statistics. The format is:
yymmdd
hh:mm:ss
tnum cpy:
fsz qt: qsec xt: xsec
up: usr lfn
Where
yymmdd
hh:mm:ss
year, month, day, hour (24-hour time),
minute, second the transfer completed.
tnum thread number used in the transfer.
cpy the token Got: for
and incoming and Put: for an
outgoing transfer.
fsz the size of the file in bytes.
qsec the number of seconds the transfer waited to start (i.e., queue
time).
xsec the number of seconds the transfer command actually ran (i.e.,
transfer time).
usr the identity of the client that requested the transfer.
lfn the transfer source or target logical file name.
Each
time frm_xfrd is about to schedule a
transfer operation, it check whether a stopfile
exists indicating that either an incoming or outgoing transfers are not
allowed. These stopfiles are:
/apath/frm[/name]/STOPCOPYIN
/apath/frm[/name]/STOPCOPYOUT
/apath/frm[/name]/STOPMIGR
/apath/frm[/name]/STOPSTAGE
Where
apath comes from the all.adminpath
directive or its default.
name comes from the n command line option and is empty if not specified.
The
presence of each file stops a particular type of transfer stream, as follows:
·
STOPCOPYIN file suspends new incoming url-based
transfers,
·
STOPCOPYOUT suspends new outgoing url-based
transfers,
·
STOPMIGR file suspends new outgoing
non-url transfers, and
·
STOPSTAGE suspends new incoming non-url
transfers.
In-progress
transfers are allowed to complete.
When frm_xfrd is asked to send a notification
it uses one or two message formats as shown in each diagram below.
Successful: optype OK lfn
Unsuccessful optype {ENOENT | BAD} lfn [ emsg ]
optype: get | migr | put | stage
Notification: file:////path
Where
optype is the name of the completed operation.
lfn logical filename of the file that was the source or target of
the operation.
ENOENT
staging
failed because the file could not be found.
BAD staging failed for other reasons (see emsg).
emsg an optional error message describing the nature of the failure.
Successful: ready requestid msg lfn
Unsuccessful: unprep requestid
msg lfn
Notification: udp://rhost:port/msg
Where
requestid
the
requestid associate with the staging request.
msg the message, if any, present in the notification string.
lfn the logical filename of the file that successfully staged in
or whose staging failed.
frm_xfragent [ options ]
options: [-c
cfn] [-d] [-k {num
| sz{k|m|g} | sig}]
[-l [=]logfn]
[-n name] [-z]
sig: fifo|hup|rtmin|rtmin+1|rtmin+2|ttou|winch|xfsz
Function
Add,
delete, and list entries in the frm_xfrd
request queue.
Options
-c configfn
The
configuration file name. The default name for the configuration file comes from
the environmental variable XRDCONFIGFN,
if set[1],
otherwise it defaults to /opt/xrootd/etc/xrootd.cf.
-d Turns on debugging mode.
Additional information is printed to describe various actions.
-k
num | sz{k|m|g} | sig
Keep
no more than num old log files. If sz is specified, the number of log files
kept (excluding the current log file) is trimmed to not exceed sz bytes. The sz must be suffixed by k,
m, or g to indicate kilobytes,
megabyte, or gigbytes, respectively. If a sig
value is specified (i.e. hup etc),
then an external program is expected to handle log file rotation (e.g.
logrotate). Except for fifo, the
argument specifies signal that causes the daemon to close and re-open the log
file to allow rotation to occur. When fifo is specified, the daemon waits for
data to appear on a fifo whose path is identical to the log file path but whose
name is prefixed by a dot. Refer to the notes for manual rotation caveats.
-l [=]logfn
Routes
error messages and any trace output to logfn. By default, messages are
directed to standard error unless the environmental viable XRDLOGDIR is set[2],
in which case the logfn defaults to $XRDLOGDIR/frm/xfralog. Refer to the
notes on how logfn is modified by the
n option. When logfn is prefixed by an equals sign, the logfn is not qualified by the instance name, if any. This allows
log files to be handled in an arbitrary manual way.
-n
name
Assigns name to the frm_xfragent
instance. The default name comes from
the environmental variable XRDNAME,
if set[3],
otherwise frm_xfragent is unnamed.
When frm_xfragent is used as a cmsd or xrootd service, you need not specify n as it defaults to the underlying systems instance name. See the
notes for additional information.
-z provides microsecond resolution
for log file message timestamps.
Notes
1)
The
frm_xfragent also accepts additional
directives from the configuration file, configfn.
These directives are described in the following section.
2)
The n option allows you to run multiple
instances of frm_xfragent with a
common configuration file. This is possible because the specified name is used
to modify various file system paths frm_xfragent
uses for output files. By automatically differentiating such paths by instance
name prevents two frm_xfragent
processes from interfering with each other.
3)
Generally,
you will only need to specify n when you invoke frm_xfragent outside the context of a cmsd or xrootd server.
For instance, you may wish to write scripts the use frm_xfragent to extend services outside the cmsd or xrootd context.
4)
The
n option modifies the logfn specified on the command line.
Assuming logfn is composed of /path/fn
and n name is specified, the log file name becomes /path/name/fn.
5)
The frm_xfragent command reads requests
from standard input and provides responses via standard out. This is compatible
with the requirements of the xrootds
oss.stagecmd and oss.stagemsg directives and cmsds cms.prep and cms.prepmsg
directives. These directives allow you to easily use the built-in version of frm_xfragent. Refer to each respective
manual for more information.
6)
The frm_xfragent command can also be used
with any other program to manage the transfer queue. Refer to the description of frm_xfrd command for more
information.
7)
The
frm_xfragent command has no specific
directive. All the information that it needs is to execute properly is tied to frm_xfrd directives.
8)
The
cmsd and xrootd have a built-in frm_xfragent.
These daemons do not require that you configure frm_xfragent to use frm_xfrd.
9)
If a log file
is specified without a signal -k
option, the file is closed at midnight, renamed to have a date suffix (i.e., fn.yyyymmdd)
and possible sequence number (i.e. fn.yyyymmdd.n),
and a new log file is opened.
10) When a signal value is specified, log files are not
automatically renamed at midnight. Instead an external program must be used to
properly rotate log files. Make sure to choose a signal that is not in use by any plug-in. If unsure, choose one of the obscure signal names and
monitor for any odd behavior. Otherwise, use the fifo option. Be aware that on some non-Linux platforms
the fifo file descriptor may leak.
11) When fifo is
specified the fifo file name must not exists or exist as a fifo file. A simple
echo x >> /path/.lfn
causes the log file to close and reopen.
12) The sig
names, except for fifo, be fully
capitalized as well prefixed by sig
or SIG when capitalized.
The following requests can be directed to frm_xfragent via standard-in:
Add to queue: aop[traceid] reqid notify prty
mode fn
Remove from queue: dop requestid
List the queue: ?[qtype] [varnames]
aop: < | = | > | + | ^ | &
dop: -
| ~
fn: [prot://dest/]lfn[?cgi]
qtype: op[qtype]
varnames: lfn
| lfncgi | mode | note | prty | qwt |
rid | tid | tod | [varnames]
traceid
a
1- to- 256 character identifier describing the entity that caused the entry to
be added. See the notes for an explanation of the standard traceid format. A generic default is used if traceid is not specified.
op requests an operation to be performed, as follows
:
< - copy file lfn from a remote location to local
disk.
= - copy file lfn to a remote location and then remove
it from local disk.
> - copy file lfn to a remote location.
+ - stage file lfn from a remote location to local
disk.
^ - migrate file lfn to a remote location and then
remove it from local disk.
& - migrate file lfn to a remote location.
requestid
a
1- to- 64 character identifier to be used to group this request into a unique
set of requests. This means that requestid
should be globally unique.
notify a
1- to- 512 character string describing who should get notified upon completion
of the request. Multiple strings, totaling 512 characters, can be specified by
separating each but the last one with a carriage return (\r) character. See
the section on frm_xfrd messages for
the actual message that is sent. The notify
format[4]
is:
- no notification is to be sent.
file:////path send a special message via FIFO named path
udp://rhost:port/msg send
msg via udp to rhost:port
udp://path/msg send msg via udp via local Unix path
path
prty the
character 0, 1, or 2, designating the requests priority (0 being the
lowest). If an invalid priority is specified, 0 is used. See frm_xfrd on how request priorities are
used.
mode The
processing mode as a sequence of characters:
r[x] - copy-in
a file in read-only mode. See below for the x values.
w[x] - copy-in
a file in read-write mode. See below for the x values.
The r and w functions can be
suffixed with additional letters to indicate how errors and successes are to be
handled. As shown above, x can be a combination of the
following:
f send notification if the file cannot be
copied. This option is
not affected by the q letter.
s send notification upon successfully
copying the file[5].
q do not send any notifications associated
with this entry. By default,
only failure notices are sent.
fn is either a simple logical file name of the file to be
copied or a URL describing the copy protocol to be used, the source or
destination location, as well as the simple logical file name.
cgi opaque information to be forwarded to the staging transfer
agent, if configured to accept this information (see frm_xfrds frm.xfr.copycmd
configuration directives). The total amount of information, including the lfn and ? separator, cannot exceed 2,175 characters.
Notes
1)
Currently,
there is no function difference between a copy-in and a stage operation.
However, future differences may occur so the correct operation code should be
used when adding a file to the queue.
Copy-out
operations are done without additional checks other than whether the files
changed during the copy operation. Migration operations
checks require the file not be modified for a certain period of time and not
have
dop applies the removal either to the a particular queue, as
follows:
- - removes requestid from the
stage-in and copy-in queues.
~ - removes requestid from the
migrate and copy-out queues.
requestid
a 1- to- 64 character identifier to be used to identify
the requests that are to be removed. All pending requests associated with this
identifier are removed. Requests already in progress remain are allowed to
complete even though they are no longer in the queue.
qtype by default, the stage-in and copy-in queues are listed. Otherwise,
qtype lists the particular queues
corresponding to the description associated with op.
varnames
is
an optional list of variable names, as described below. Information is listed
in the order in which the variables are listed; each separated by a space. The
default variable is lfn. Invalid varnames are ignored. Valid variables
are:
Var |
Information |
Var |
Information |
Var |
Information |
lfn |
logical
filename |
obj |
lfn or url |
qwt |
seconds
in queue |
lfncgi |
lfn?[cgi string] |
objcgi |
lfncgi or url?[cgi] |
rid |
requestid |
mode |
processing
opts |
op |
Operation
(e.g., <) |
tid |
traceid |
note |
notification
string |
prty |
priority |
tod |
time
of day |
hpsscp [ options ] [user@host:]sfn
[user@host:]tfn
options: [-C] [-d] [-f]
[-k keytab] [-m] [-N
npath] [-n]
[-o owner] [-p {r|w|mode}] [-P port] [-Q qpath]
[-t tries] [-W wcmd] [-x pftpcmd] [-z]
owner: user[:group]
| [user]:group
user: username |
uid
group: groupname | gid
Function
Copy
a file to local disk; typically, from a Mass Storage System.
Parameters
[user@host:]sfn
name
of the source file to read. If it is prefixed by user@host:,
then the file must exist in a pftp
accessible Mass Storage system located at host and reachable via login as user.
Otherwise, the file whose name is sfn
must be present on local disk.
[user@host:]tfn
name
of the source file to write. If it is prefixed by user@host:,
then the file is to be written into a pftp
accessible remote Mass Storage system located at host and reachable via login
as user. Otherwise, the file will be written to local disk using the name tfn.
targetfn
The
name of the file as it should exist on local disk.
Options
-C continues execution even when a
transfer slot (see Q) cannot be obtained.
-d Turns on debugging.
-f over-writes any local file,
even if it has zero length. The f option is the default for zero-length local
files. It has no effect on remote files.
-k keytab
specifies
the pftp passwords location. The keytab must contain a single password
that will be used when logging in as user@host.
The default location is /var/adm/xrootd/pftp/keyfile.
-m when tfn is local, verifies
that tfn does not exist. Should tfn exist, the command fails with an error
message.
-N npath
specifies
the location of the local name space directory when hpsscp is to record the directory path used to create a file in hpss. This allows hpsscp to avoid issuing duplicate mkdir requests for any paths that
exist in npath. If N is not specified, this optimization is suppressed.
-n does not redirect standard error to standard out when invoking the xfrcmd. This is meant purely for
debugging purposes.
-o
owner
sets
the ownership of the file. For non-root processes, the default ownership is
uid/gid of the process. For root process, the default ownership is the uid/gid assigned
to the file.
-p
{r | w | mode}
sets
the permission bits for the target file. r
implies a mode of 0440, w a mode of 0640, and mode is 1- to 4-character
octal mode. The default mode is determined by the xfrcmd that is used to transfer the file.
-P
port
is
the port number to contact when logging in using user@host. The default port number is 2021.
-Q qpath
specifies
the location of the execution pacing queue directory. The qpath directory must contain one or more writable zero-length
files. Each file represents a transfer queue slots implying that the maximum
number of transfers is equal to the number of files in qpath. Simultaneous requests over this limit wait until one of the queue
slots frees up (also see C). The
default directory is /var/hpss/xfrq
and is used if it exists. Transfer pacing does not apply if the default
directory does not exist.
-t tries
The
number of times to try a failing copy when the error indicates a transient
condition. The default is 2.
-W wcmd
is
the program that implements the transfer queue pacing mechanism. The default is
/opt/xrootd/utils/wait41.
-x xfrcmd
is
the pftp command to be used to copy sfn to tfn. The default is to use /opt/xrootd/utils/pftp_client.
-z does not establish a new process group prior to executing the xfrcmd.
user user pswd
delete mssfn
mkdir dirpath
●
●
●
binary
setpblocksize 2097152
site
setcos cos
pput localfn mssfn
site chuid uid mssfn
site chgid gid mssfn
quit
Transfer Command Stream When Sending a File to HPSS
Where:
cos The value specified with the s command line option.
dirpath The path segment that needs to exist in order to store the file. There are as many mkdir commands as there are path segments to ensure that the complete target path exists in HPSS.
gid The gid identified by o command line option. The site chgid command is only present when user if root.
localfn The name of the file to be placed in HPSS as it is known on local disk.
mssfn The name of the file as it must be stored in HPSS.
pswd The password in the file identified by k command line option.
uid The uid identified by o command line option. The site chuid command is only present when user if root.
user The user contained in user@host:tfn for the target file.
user xfruser pswd
binary
setpblocksize 2097152
pget mssfn localfn
quit
Transfer Command Stream When Retrieving a File from HPSS
Where:
user The user contained in user@host:tfn for the target file.
keyfn The password in the file identified by k command line option.
mssfn The name of the file as it exists in HPSS.
localfn The name of the file as it must be created on local disk.
The Cluster Name Space Daemon (XrdCnsd) is a daemon that may be started using the ofs.notify directive to capture file
creation and deletion requests and maintain a file inventory. Since frm_xfrd creates files and frm_purged removes files, these daemons
can also notify the XrdCnsd when
they add or remove files so that the inventory remains correct. Additionally,
the frm_admin rm subcommand (remove
file) can also communicate the removal to the XrdCnsd.
The frm.all.cnsd directive is used to specify how notifications are to
be processed.
frm.all.cnsd {auto
| ignore | require} [cnsopts]
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
cnsopts: [apath
apath]
Where
frm.all.cnsd {auto|ignore|require} [apath apath]
Specifies how create and
deletion requests are to be communicated to the Cluster Name Space Daemon (XrdCnsd). The apath option is used to specify the XrdCnsd administrative path should it
have been specified via the a
command line option. If the a
option was not specified, then the apath
option should be omitted. The processing modes are:
auto notify XrdCnsd only if it is running.
ignore never notify the XrdCnsd. This is the
default.
require always notify the XrdCnsd. If it is not running,
wait for it to start.
14
Jan 2009
·
New
Document.
23
Apr 2009
·
Document
frm_admin.
19
Nov 2009
·
Document
the o option of frm_xfr.hpss.
15
Dec 2009
·
Document
frm_purged.
5
Apr 2010
·
Document
frm_xfrd and frm_xfragent.
·
This
replaces documentation for frm_pstga
and frm_pstgd
12
Apr 2010
·
Document
the correct stopfile names.
·
Correct
documentation of actual messages sent by frm_xfrd.
14
Apr 2010
·
Better
document actual frm_xfragent queue
list options.
·
Indicate
that frm_xfragent is built into cmsd and xrootd.
21
Apr 2010
·
Remove
the frm.xfr.qpath directive and
document how to accomplish queue relocation via a symbolic link.
·
Document
created files.
·
Note
that cmsd and xrootd have a built-in frm_xfragent.
·
Clean-up
and reorganization.
·
Replace
the failhold directive with the oss.xfr directive as they provide the
same information.
6
Jul 2010
·
Add
$CID (cluster ID) and $INS (instance name) substitution
variables to frm_xfrd.
·
Document
that the frm_xfrd transfer queue can
be shared by multiple clusters on the same host.
·
Extend
the frm.xfr.qcheck directive to
allow the specification of an absolute queue path to be able to implement queue
sharing.
·
Rename
frm_xfr.hpss to be hpsscp to ease deployment in different
contexts.
·
Document
the C, N, Q, and W options enabled for hpsscp.
1
Sep 2010
·
Document
the new noalloc option for the frm.xfr.copycmd directive.
·
Document
the new query xfrq arguments for frm_admin.
7
Oct 2010
·
Document
how the STOPURGE file pauses frm_purged.
10
Nov 2010
·
Change
documentation to reflect the use of extended attributes instead of met-files to
control file residency. Most of these changes apply to the frm_admin command.
1
Feb 2011
·
Minor
editorial changes.
8
Mar 2011
·
Document
the b, p, and s command line
options.
30
May 2011
·
Document
the chksum command.
23
Jun 2011
·
Document
the frm.all.cnsd directive.
11
Jul 2011
·
Add
more descriptive information about purging and staging, especially with respect
to federated clusters.
-------------- Release 3.1.0
12
Oct 2011
·
Document
the new xpd option on the copycmd directive.
3
Dec 2011
·
Document
the new frm.all.monitor directive
for frm_purged and frm_xfrd.
-------------- Release 3.1.1
23
Apr 2012
·
Correct
description of the force option on
the mark subcommand.
-------------- Release 3.2.0
-------------- Release 3.2.1
-------------- Release 3.2.2
-------------- Release 3.2.3
-------------- Release 3.2.4
-------------- Release 3.2.5
-------------- Release 3.2.5
6
Feb 2013
· Remove wrong reference to $USER variable.
-------------- Release 3.3.0
-------------- Release 3.3.1
-------------- Release 3.3.2
-------------- Release 3.3.3
-------------- Release 3.3.4
-------------- Release 3.3.5
13 August 2013
· Document the extended k, l
and z command line options.
· Document exported environment variables.
· Document the environment information file contents.
· General clean-up and better explanations.
4 December 2013
· Document the frm_admin mv command.
[1] The cmsd and xrootd set this
environmental variable if passed the c
command line option.
[2] The cmsd and xrootd set this environmental variable if passed the l command line option.
[3] The cmsd and xrootd set this environmental
variable if passed the n command
line option.
[4] The old mps_prep and mps_PreStage mailto:// and tcp:// notify options are not supported.
[5] The letter n is a synonym
for s.