This part explains the syntax of calling the executable that
supports STORE FILE, FETCH
FILE and REMOVE FILE
operations.
put <pnfsID> <filename> -si=<storage-information> [<other-options>...]
get <pnfsID> <filename> -si=<storage-information> -uri=<storage-uri> [<other-options>...]
remove -uri=<storage-uri> [<other-options>...]
- put / get / remove:
these keywords indicate the operation to be performed.
- put: copy file from disk to TSS.
- get: copy file back from TSS to disk.
- remove: remove the file from TSS.
- <pnfsID>: The internal identifier (i-node) of the file within dCache. The <pnfsID> is unique within a single dCache instance and globally unique with a very high probability.
- <filename>: is the full path of the local file to be copied to the TSS (for put) and respectively into which the file from the TSS should be copied (for get).
- <storage-information>: the storage information of the file, as explained below.
- <storage-uri>:
the URI, which was returned by the
executable, after the file was written to tertiary storage. In order to get the file back from the TSS the information of the URI is preferred over the information in the <storage-information>. - <other-options>:
-<key> =
<value> pairs taken from the TSS
configuration commands of the pool 'setup' file. One of the
options, always provided is the option -command=<full
path of this
executable>.
The <storage-information> is a string in the format
-si=size=<bytes>;new=<true/false>;stored=<true/false>;sClass=<StorageClass>;\ cClass0<CacheClass>;hsm=<StorageType>;<key>=<value>;[<key>=<value>;[...]]
Example:
-si=size=1048576000;new=true;stored=false;sClass=desy:cms-sc3;cClass=-;hsm=osm;Host=desy;
Mandatory storage information’s keys
- <size>: Size of the file in bytes
- <new>: False if file already in the dCache; True otherwise
- <stored>: True if file already stored in the TSS; False otherwise
- <sClass>: HSM depended, is used by the
poolmanagerfor pool attraction. - <cClass>: Parent directory tag
(cacheClass). Used by the
poolmanagerfor pool attraction. May be '-'. - <hsm>: Storage manager name (enstore/osm). Can be overwritten by the parent directory tag (hsmType).
OSM specific storage information’s keys
- <group>: The storage group of the file to be stored as specified in the ".(tag)(sGroup)" tag of the parent directory of the file to be stored.
- <store>: The store name of the file to be stored as specified in the ".(tag)(OSMTemplate)" tag of the parent directory of the file to be stored.
- <bfid>: Bitfile ID (get and remove only) (e.g. 000451243.2542452542.25424524)
Enstore specific storage information’s keys
- <group>: The storage group (e.g. cdf, cms ...)
- <family>: The file family (e.g. sgi2test, h6nxl8, ...)
- <bfid>: Bitfile ID (get only) (e.g. B0MS105746894100000)
- <volume>: Tape Volume (get only) (e.g. IA6912)
- <location>: Location on tape (get only) (e.g. : 0000_000000000_0000117)
There might be more key values pairs which are used by the
dCache internally and which should not affect the
behaviour of the executable.
The <storage-uri> is formatted as follows:
hsmType://hsmInstance/?store=<storename>&group=<groupname>&bfid=<bfid>
- <hsmType>: The type of the Tertiary Storage System
- <hsmInstance>: The name of the instance
- <storename> and
<groupname> : The store and group
name of the file as provided by the arguments to this
executable. - <bfid>: The unique identifier needed to restore or remove the file if necessary.
Example:
A storage-uri:
osm://osm/?store=sql&group=chimera&bfid=3434.0.994.1188400818542| Return Code | Meaning | Behaviour for PUT FILE | Behaviour for GET FILE |
|---|---|---|---|
| 30 <= rc < 40 | User defined | Deactivates request |
Reports problem to poolmanager
|
| 41 | No space left on device | Pool retries |
Disables pool and reports problem to poolmanager
|
| 42 | Disk read I/O error | Pool retries |
Disables pool and reports problem to poolmanager
|
| 43 | Disk write I/O error | Pool retries |
Disables pool and reports problem to poolmanager
|
| other | - | Pool retries |
Reports problem to poolmanager
|
Whenever a disk file needs to be copied to a Tertiary Storage
System dCache automatically launches an executable on the
pool containing the file to be copied. Exactly one instance
of the executable is started for each file. Multiple
instances of the executable may run concurrently for
different files. The maximum number of concurrent instances of
the executablesexecutable can be configured in the ’setup’
file of the pool as described in the section called “The pool ’setup’ file”.
The following arguments are given to the executable of a
STORE FILE operation on startup.
put <pnfsID> <filename> -si=<storage-information> <more options>
Details on the meaning of certain arguments are described in the section called “Summary of command line options”.
With the arguments provided the executable is supposed to
copy the file into the Tertiary Storage System. The
executable must not terminate before the transfer of the
file was either successful or failed.
Success must be indicated by a 0 return of
the executable. All non-zero values are interpreted as a
failure which means, dCache assumes that the file has not
been copied to tape.
In case of a 0 return code the executable
has to return a valid storage URI to dCache in formate:
hsmType://hsmInstance/?store=<storename>&group=<groupname>&bfid=<bfid>Details on the meaning of certain parameters are described above.
The <bfid> can either be provided by
the TSS as result of the STORE FILE
operation or the pnfsID may be used. The
latter assumes that the file has to be stored with exactly
that pnfsID within the TSS. Whatever URI
is chosen, it must allow to uniquely identify the file within
the Tertiary Storage System.
Note
Only the URI must be printed to stdout by the
executable. Additional information printed either before
or after the URI will result in an error. stderr can be used
for additional debug information or error messages.
Whenever a disk file needs to be restored from a Tertiary
Storage System dCache automatically launches an
executable on the pool containing the file to be
copied. Exactly one instance of the executable is started
for each file. Multiple instances of the executable may run
concurrently for different files. The maximum number of
concurrent instances of the executable per pool as well as
the full path of the executable can be configured in the
’setup’ file of the pool as described in the section called “The pool ’setup’ file”.
The following arguments are given to the executable of a
FETCH FILE operation on startup:
get <pnfsID> <filename> -si=<storage-information> -uri=<storage-uri> <more options>
Details on the meaning of certain arguments are described in the section called “Summary of command line options”. For return codes see the section called “Summary of return codes”.
Whenever a file is removed from the dCache namespace (file
system) a process inside dCache makes sure that all copies
of the file are removed from all internal and external
media. The pool which is connected to the TSS which stores
the file is activating the executable with
the following command line options:
remove -uri=<storage-uri> <more options>
Details on the meaning of certain arguments are described in the section called “Summary of command line options”. For return codes see the section called “Summary of return codes”.
The executable is supposed to remove the
file from the TSS and report a zero return code. If a
non-zero error code is returned, the dCache will call the
script again at a later point in time.