dCache SRM
is implemented as a Web Service running
in a Jetty servlet container and an
Axis Web Services
engine. The Jetty server is executed as a cell,
embedded in dCache and started automatically by the SRM
service.
Other cells started automatically by SRM
are
SpaceManager
, PinManager
and RemoteGSIFTPTransferManager
.
Of these services only SRM
and SpaceManager
require special
configuration.
The SRM
consists of the five categories of functions:
SRM
version 2.2 introduces a concept of space
reservation. Space reservation guarantees that the requested
amount of storage space of a specified type is made
available by the storage system for a specified amount of
time.
We use three functions for space management:
srmReserveSpace
SrmGetSpaceMetadata
srmReleaseSpace
Space reservation is made using the
srmReserveSpace
function. In case of
successful reservation, a unique name, called
space token is assigned to the
reservation. A space token can be used during the transfer
operations to tell the system to put the files being
manipulated or transferred into an associated space
reservation. A storage system ensures that the reserved
amount of the disk space is indeed available, thus providing
a guarantee that a client does not run out of space until
all space promised by the reservation has been used. When
files are deleted, the space is returned to the space
reservation.
dCache only manages write space, i.e. space on disk can be
reserved only for write operations. Once files are migrated
to tape, and if no copy is required on disk, space used by
these files is returned back into space reservation. When
files are read back from tape and cached on disk, they are
not counted as part of any space. SRM
space reservation
can be assigned a non-unique description that can be used to
query the system for space reservations with a given
description.
Properties of
the SRM
space reservations can be discovered using
the SrmGetSpaceMetadata
function.
Space Reservations might be released with the function
srmReleaseSpace
.
For a complete description of the available space management
functions please see the SRM
Version 2.2 Specification.
SRM
defines a protocol named SRM
, and introduces a way
to address the files stored in the SRM
managed storage by
site URL (SURL of the format
srm://<host>:<port>/[<web service
path>?SFN=]<path>
.
Example:
Examples of the SURLs a.k.a. SRM
URLs are:
srm://fapl110.fnal.gov:8443/srm/managerv2?SFN=//pnfs/fnal.gov/data/test/file1 srm://fapl110.fnal.gov:8443/srm/managerv1?SFN=/pnfs/fnal.gov/data/test/file2 srm://srm.cern.ch:8443/castor/cern.ch/cms/store/cmsfile23
A transfer URL (TURL) encodes the file transport protocol in the URL.
Example:
gsiftp://gridftpdoor.fnal.gov:2811/data/test/file1
SRM
version 2.2 provides three functions for performing
data transfers:
srmPrepareToGet
srmPrepareToPut
srmCopy
(in SRM
version 1.1 these
functions were called get
,
put
and copy
).
All three functions accept lists of SURLs as
parameters. All data transfer functions perform
file/directory access verification and
srmPrepareToPut
and
srmCopy
check if the receiving storage
element has sufficient space to store the files.
srmPrepareToGet
prepares files for
read. These files are specified as a list of source SURLs,
which are stored in an SRM
managed storage
element. srmPrepareToGet
is used to
bring source files online and assigns transfer URLs
(TURLs) that are used for actual data transfer.
srmPrepareToPut
prepares an SRM
managed storage element to receive data into the list of
destination SURLs. It prepares a list of TURLs where the
client can write data into.
Both functions support transfer protocol negotiation. A
client supplies a list of transfer protocols and the SRM
server computes the TURL using the first protocol from the list
that it supports. Function invocation on the Storage Element
depends on implementation and may range from simple SURL to
TURL translation to stage from tape to disk cache and
dynamic selection of transfer host and transfer protocol
depending on the protocol availability and current load on
each of the transfer server load.
The function srmCopy
is used to copy
files between SRM
managed storage elements. If both source
and target are local to the SRM
, it performes a local
copy. There are two modes of remote copies:
PULL mode : The target
SRM
initiates ansrmCopy
request. Upon the client\u0411\u2500\u2265ssrmCopy
request, the targetSRM
makes a space at the target storage, executessrmPrepareToGet
on the sourceSRM
. When the TURL is ready at the sourceSRM
, the targetSRM
transfers the file from the source TURL into the prepared target storage. After the file transfer completes,srmReleaseFiles
is issued to the sourceSRM
.PUSH mode : The source
SRM
initiates ansrmCopy
request. Upon the client\u0411\u2500\u2265ssrmCopy
request, the sourceSRM
prepares a file to be transferred out to the targetSRM
, executessrmPrepareToPut
on the targetSRM
. When the TURL is ready at the targetSRM
, the sourceSRM
transfers the file from the prepared source into the prepared target TURL. After the file transfer completes,srmPutDone
is issued to the targetSRM
.
When a specified target space token is provided, the files will be located in the space associated with the space token.
SRM
Version 2.2 srmPrepareToPut
and
srmCopy
PULL mode transfers allow the
user to specify a space reservation token or a
retention policy and access latency. Any of these parameters
are optional, and it is up to the implementation to decide
what to do, if these properties are not specified. The
specification requires that if a space reservation is given,
then the specified access latency or
retention policy must match those of the space reservation.
The Data Transfer Functions are asynchronous, an initial
SRM
call starts a request execution on the server side
and returns a request status that contains a unique
request token. The status of request is polled
periodically by SRM
get request status functions. Once a
request is completed and the client receives the TURLs
the data transfers are initiated. When the transfers are
completed the client notifies the SRM
server by
executing srmReleaseFiles
in case of
srmPrepareToGet
or
srmPutDone
in case of
srmPrepareToPut
. In case of
srmCopy
, the system knows when the
transfers are completed and resources can be released, so
it requires no special function at the end.
Clients are free to cancel the requests at any time by
execution of srmAbortFiles
or
srmAbortRequest
.
The functions for checking the request status are:
srmStatusOfReserveSpaceRequest
srmStatusOfUpdateSpaceRequest
srmStatusOfChangeSpaceForFilesRequest
srmStatusOfChangeSpaceForFilesRequest
srmStatusOfBringOnlineRequest
srmStatusOfPutRequest
srmStatusOfCopyRequest
SRM
Version 2.2, interface provides a complete set of
directory management functions. These are
srmLs
,srmRm
srmMkDir
,srmRmDir
srmMv
SRM
Version 2.2 supports the following three file
permission functions:
srmGetPermission
srmCheckPermission
andsrmSetPermission
dCache contains an implementation of these functions that allows setting and checking of Unix file permissions.