dCache SRM is implemented as a Web Service running
in a Jetty servlet container and an
Axis Web Services
engine. The Jetty server is executed as a cell,
embedded in dCache and started automatically by the SRM service.
Other cells started automatically by SRM are
SpaceManager, PinManager and RemoteGSIFTPTransferManager.
Of these services only SRM and SpaceManager require special
configuration.
The SRM consists of the five categories of functions:
SRM version 2.2 introduces a concept of space
reservation. Space reservation guarantees that the requested
amount of storage space of a specified type is made
available by the storage system for a specified amount of
time.
We use three functions for space management:
srmReserveSpaceSrmGetSpaceMetadatasrmReleaseSpace
Space reservation is made using the
srmReserveSpace function. In case of
successful reservation, a unique name, called
space token is assigned to the
reservation. A space token can be used during the transfer
operations to tell the system to put the files being
manipulated or transferred into an associated space
reservation. A storage system ensures that the reserved
amount of the disk space is indeed available, thus providing
a guarantee that a client does not run out of space until
all space promised by the reservation has been used. When
files are deleted, the space is returned to the space
reservation.
dCache only manages write space, i.e. space on disk can be
reserved only for write operations. Once files are migrated
to tape, and if no copy is required on disk, space used by
these files is returned back into space reservation. When
files are read back from tape and cached on disk, they are
not counted as part of any space. SRM space reservation
can be assigned a non-unique description that can be used to
query the system for space reservations with a given
description.
Properties of
the SRM space reservations can be discovered using
the SrmGetSpaceMetadata function.
Space Reservations might be released with the function
srmReleaseSpace.
For a complete description of the available space management
functions please see the SRM
Version 2.2 Specification.
SRM defines a protocol named SRM, and introduces a way
to address the files stored in the SRM managed storage by
site URL (SURL of the format
srm://<host>:<port>/[<web service
path>?SFN=]<path>.
Example:
Examples of the SURLs a.k.a. SRM URLs are:
srm://fapl110.fnal.gov:8443/srm/managerv2?SFN=//pnfs/fnal.gov/data/test/file1 srm://fapl110.fnal.gov:8443/srm/managerv1?SFN=/pnfs/fnal.gov/data/test/file2 srm://srm.cern.ch:8443/castor/cern.ch/cms/store/cmsfile23
A transfer URL (TURL) encodes the file transport protocol in the URL.
Example:
gsiftp://gridftpdoor.fnal.gov:2811/data/test/file1
SRM version 2.2 provides three functions for performing
data transfers:
srmPrepareToGetsrmPrepareToPutsrmCopy
(in SRM version 1.1 these
functions were called get,
put and copy).
All three functions accept lists of SURLs as
parameters. All data transfer functions perform
file/directory access verification and
srmPrepareToPut and
srmCopy check if the receiving storage
element has sufficient space to store the files.
srmPrepareToGet prepares files for
read. These files are specified as a list of source SURLs,
which are stored in an SRM managed storage
element. srmPrepareToGet is used to
bring source files online and assigns transfer URLs
(TURLs) that are used for actual data transfer.
srmPrepareToPut prepares an SRM
managed storage element to receive data into the list of
destination SURLs. It prepares a list of TURLs where the
client can write data into.
Both functions support transfer protocol negotiation. A
client supplies a list of transfer protocols and the SRM
server computes the TURL using the first protocol from the list
that it supports. Function invocation on the Storage Element
depends on implementation and may range from simple SURL to
TURL translation to stage from tape to disk cache and
dynamic selection of transfer host and transfer protocol
depending on the protocol availability and current load on
each of the transfer server load.
The function srmCopy is used to copy
files between SRM managed storage elements. If both source
and target are local to the SRM, it performes a local
copy. There are two modes of remote copies:
PULL mode : The target SRM initiates an
srmCopy request. Upon the
client\u0411\u2500\u2265s
srmCopy request, the target
SRM makes a space at the target storage, executes
srmPrepareToGet on the source
SRM. When the TURL is ready at the source SRM,
the target SRM transfers the file from the source
TURL into the prepared target storage. After the
file transfer completes,
srmReleaseFiles is issued to the
source SRM.
PUSH mode : The source SRM initiates an
srmCopy request. Upon the
client\u0411\u2500\u2265s
srmCopy request, the source
SRM prepares a file to be transferred out to the
target SRM, executes
srmPrepareToPut on the target
SRM. When the TURL is ready at the target SRM,
the source SRM transfers the file from the prepared
source into the prepared target TURL. After the
file transfer completes,
srmPutDone is issued to the
target SRM.
When a specified target space token is provided, the files will be located in the space associated with the space token.
SRM Version 2.2 srmPrepareToPut and
srmCopy PULL mode transfers allow the
user to specify a space reservation token or a
retention policy and access latency. Any of these parameters
are optional, and it is up to the implementation to decide
what to do, if these properties are not specified. The
specification requires that if a space reservation is given,
then the specified access latency or
retention policy must match those of the space reservation.
The Data Transfer Functions are asynchronous, an initial
SRM call starts a request execution on the server side
and returns a request status that contains a unique
request token. The status of request is polled
periodically by SRM get request status functions. Once a
request is completed and the client receives the TURLs
the data transfers are initiated. When the transfers are
completed the client notifies the SRM server by
executing srmReleaseFiles in case of
srmPrepareToGet or
srmPutDone in case of
srmPrepareToPut. In case of
srmCopy, the system knows when the
transfers are completed and resources can be released, so
it requires no special function at the end.
Clients are free to cancel the requests at any time by
execution of srmAbortFiles or
srmAbortRequest.
The functions for checking the request status are:
srmStatusOfReserveSpaceRequestsrmStatusOfUpdateSpaceRequestsrmStatusOfChangeSpaceForFilesRequestsrmStatusOfChangeSpaceForFilesRequestsrmStatusOfBringOnlineRequestsrmStatusOfPutRequestsrmStatusOfCopyRequest
SRM Version 2.2, interface provides a complete set of
directory management functions. These are
srmLs, srmRmsrmMkDir, srmRmDirsrmMv
SRM Version 2.2 supports the following three file
permission functions:
srmGetPermissionsrmCheckPermission and
srmSetPermissiondCache contains an implementation of these functions that allows setting and checking of Unix file permissions.