SpaceManager is making reservations against free space
available in link
groups. The total free space in the given link
group is the sum of available spaces in all links. The
available space in each link is the sum of all sizes of
available space in all pools assinged to a given
link. Therefore for the space reservation to work correctly it
is essential that each pool belongs to one and only one link,
and each link belongs to only one link group. Link groups are
assigned several parameters that determine what kind of space
the link group corresponds to and who can make reservations
against this space.
Now that the SRM SpaceManager is activated you can make
a space reservation. As mentioned above you need link groups to
make a space reservation.
Login to the admin
interface and cd to the cell
SrmSpaceManager.
[user] $ssh -c blowfish -p 22223 -l admin headnode.example.org(local) admin >cd SrmSpaceManager
Type ls to get information about reservations and link groups.
(SrmSpaceManager) admin >lsReservations: total number of reservations: 0 total number of bytes reserved: 0 LinkGroups: total number of linkGroups: 0 total number of bytes reservable: 0 total number of bytes reserved : 0 last time all link groups were updated: Tue Sep 20 11:15:19 CEST 2011(1316510119634)
This output tells you that there are no reservations yet and no link groups. As there are no link groups no space can be reserved.
For a general introduction about link groups see the section called “Link Groups”.
Example:
In this example we will create a link group for the VO
desy. In order to do so we need to
have a pool, a pool group and a link. Moreover, we define
unit groups named any-store,
world-net and
any-protocol. (See the section called “Types of Units”.)
Define a pool in your layout file, add it to your pool
directory and restart the poolDomain.
[poolDomain] [poolDomain/pool] path=/srv/dcache/spacemanager-pool name=spacemanager-pool
[root] #mkdir -p /srv/dcache/spacemanager-pool[root] #/opt/d-cache/bin/dcache restart
In the Admin Interface cd to the
PoolManager and create a pool group, a link and a link
group.
(SrmSpaceManager) admin >..(local) admin >cd PoolManager(PoolManager) admin >psu create pgroup spacemanager_poolGroup(PoolManager) admin >psu addto pgroup spacemanager_poolGroup spacemanager-pool(PoolManager) admin >psu removefrom pgroup default spacemanager-pool(PoolManager) admin >psu create link spacemanager_WriteLink any-store world-net any-protocol(PoolManager) admin >psu set link spacemanager_WriteLink -readpref=10 -writepref=10 -cachepref=0 -p2ppref=-1(PoolManager) admin >psu add link spacemanager_WriteLink spacemanager_poolGroup(PoolManager) admin >psu create linkGroup spacemanager_WriteLinkGroup(PoolManager) admin >psu set linkGroup custodialAllowed spacemanager_WriteLinkGroup true(PoolManager) admin >psu set linkGroup replicaAllowed spacemanager_WriteLinkGroup true(PoolManager) admin >psu set linkGroup nearlineAllowed spacemanager_WriteLinkGroup true(PoolManager) admin >psu set linkGroup onlineAllowed spacemanager_WriteLinkGroup true(PoolManager) admin >psu addto linkGroup spacemanager_WriteLinkGroup spacemanager_WriteLink(PoolManager) admin >save(PoolManager) admin >..
Check whether the link group is available.
(local) admin >cd SrmSpaceManager(SrmSpaceManager) admin >lsReservations: total number of reservations: 0 total number of bytes reserved: 0 LinkGroups: 0 Name:spacemanager_WriteLinkGroup FreeSpace:7278624768 ReservedSpace:0 AvailableSpace:7278624768 VOs: onlineAllowed:true nearlineAllowed:false replicaAllowed:true custodialAllowed:true UpdateTime:Mon Nov 28 12:12:51 CET 2011(1322478771030) total number of linkGroups: 1 total number of bytes reservable: 7278624768 total number of bytes reserved : 0 last time all link groups were updated: Mon Nov 28 12:12:51 CET 2011(1322478771030)
The link group
spacemanager_WriteLinkGroup was created
and has the id 0.
Now you need to edit the
LinkGroupAuthorization.conf file. This
file contains a list of the link groups and all the VOs and
the VO Roles that are permitted to make reservations in a
given link group.
Specify the location of the
LinkGroupAuthorization.conf file in the
/opt/d-cache/etc/dcache.conf file.
SpaceManagerLinkGroupAuthorizationFileName=/path/to/LinkGroupAuthorization.conf
The file LinkGroupAuthorization.conf
has following syntax:
LinkGroup <NameOfLinkGroup> followed by the list of the Fully Qualified Attribute Names (FQANs). Each FQAN on a separate line, followed by an empty line, which is used as a record separator, or by the end of the file.
FQAN is usually a string of the form
<VO>/Role=<VORole>. Both
<VO> and
<VORole> could be set to
*, in this case all VOs or VO Roles will
be allowed to make reservations in this link group. Any line
that starts with # is a comment and may appear anywhere.
#SpaceManagerLinkGroupAuthorizationFile LinkGroup <NameOfLinkGroup> /<VO>/Role=<VORole>
Note
You do not need to restart the srm or dCache
after changing the
LinkGroupAuthorization.conf file. The
changes will be applied automatically after a few minutes.
Use update link groups to be sure that
the LinkGroupAuthorization.conf file
and the link groups have been updated.
(SrmSpaceManager) admin >update link groupsupdate started
Example:
In the example above you created the link group
spacemanager_WriteLinkGroup. Now you
want to allow members of the VO desy
with the role production to make a
space reservation in this link group.
#SpaceManagerLinkGroupAuthorizationFile # this is comment and is ignored LinkGroup spacemanager_WriteLinkGroup # /desy/Role=production
Example:
In this more general example for a
SpaceManagerLinkGroupAuthorizationFile
members of the VO desy with role
test get the right to make a space
reservation in a link group called
desy-test-LinkGroup. Moreover, all
members of the VO desy get the right to
make a reservation in the link group called
desy-anyone-LinkGroup and anyone will
get the right to make a space reservation in the link
group called default-LinkGroup.
#SpaceManagerLinkGroupAuthorizationFile # this is comment and is ignored LinkGroup desy-test-LinkGroup /desy/Role=/test LinkGroup desy-anyone-LinkGroup /desy/Role=* LinkGroup default-LinkGroup # allow anyone :-) */Role=*
Example:
Now you can make a space reservation for the VO desy.
(SrmSpaceManager) admin >reserve -vog=/desy -vor=production -desc=DESY_TEST 5000000 10000110000 voGroup:/desy voRole:production retentionPolicy:CUSTODIAL accessLatency:NEARLINE linkGroupId:0 size:5000000 created:Fri Dec 09 12:43:48 CET 2011 lifetime:10000000ms expiration:Fri Dec 09 15:30:28 CET 2011 description:DESY_TEST state:RESERVED used:0 allocated:0
The id of the space token is 110000.
Check the status of the reservation by
(SrmSpaceManager) admin >lsReservations: 110000 voGroup:/desy voRole:production retentionPolicy:CUSTODIAL accessLatency:NEARLINE linkGroupId:0 size:5000000 created:Fri Dec 09 12:43:48 CET 2011 lifetime:10000000ms expiration:Fri Dec 09 15:30:28 CET 2011 description:DESY_TEST state:RESERVED used:0 allocated:0 total number of reservations: 1 total number of bytes reserved: 5000000 LinkGroups: 0 Name:spacemanager_WriteLinkGroup FreeSpace:23747563520 ReservedSpace:5000000 AvailableSpace:23742563520 VOs:{/desy:*} onlineAllowed:true nearlineAllowed:true replicaAllowed:true custodialAllowed:true UpdateTime:Fri Dec 09 12:49:29 CET 2011(1323431369046) total number of linkGroups: 1 total number of bytes reservable: 23742563520 total number of bytes reserved : 5000000 last time all link groups were updated: Fri Dec 09 12:49:29 CET 2011(1323431369046)
You can now copy a file into that space token.
[user] $srmcp file:////bin/sh srm://<dcache.example.org>:8443/data/world-writable/space-token-test-file -space_token=110000
Now you can check via the Webadmin Interface or the Web Interface that the
file has been copied to the pool
spacemanager-pool.
There are several parameters to be specified for a space reservation.
(SrmSpaceManager) admin >reserve [-vog=voGroup] [-vor=voRole] [-acclat=AccessLatency] \ [-retpol=RetentionPolicy] [-desc=Description] [-lgid=LinkGroupId] [-lg=LinkGroupName] \ <sizeInBytes> <lifetimeInSecs>
- [-vog=voGroup]
voGroupshould match the VO you specified in theLinkGroupAuthorization.conffile. If you do not want to make a space reservation for a certain VO then the entry in theLinkGroupAuthorization.confshould readLinkGroup NameOfLinkGroup */Role=*
- [-vor=voRole]
voRolecan be specified if it is used in theLinkGroupAuthorization.conffile.- [-acclat=AccessLatency]
AccessLatencyneeds to match one of the access latencies allowed for the link group.- [-retpol=RetentionPolicy]
RetentionPolicyneeds to match one of the retention policies allowed for the link group.- [-desc=Description]
You can chose a value to describe your space reservation.
- [-lgid=LinkGroupId]
You can either use the
LinkGroupIdto make a space reservation or- [-lg=LinkGroupName]
you use the
LinkGroupNameto make a space reservation.- <sizeInBytes>
The size of the space reservation should be specified in bytes.
- <lifetimeInSecs>
The life time of the space reservation should be specified in seconds. Choose
"-1"for a space reservation that will never expire (use quotes around the negativeone).
If a space reservation is not needet anymore it can be released with
(SrmSpaceManager) admin >release <spaceTokenId>
Example:
(SrmSpaceManager) admin >reserve -vog=/desy -vor=production -desc=DESY_TEST 5000000 600110042 voGroup:/desy voRole:production retentionPolicy:CUSTODIAL accessLatency:NEARLINE linkGroupId:0 size:5000000 created:Thu Dec 15 12:00:35 CET 2011 lifetime:600000ms expiration:Thu Dec 15 12:10:35 CET 2011 description:DESY_TEST state:RESERVED used:0 allocated:0(SrmSpaceManager) admin >release 110042110042 voGroup:/desy voRole:production retentionPolicy:CUSTODIAL accessLatency:NEARLINE linkGroupId:0 size:5000000 created:Thu Dec 15 12:00:35 CET 2011 lifetime:600000ms expiration:Thu Dec 15 12:10:35 CET 2011 description:DESY_TEST state:RELEASED used:0 allocated:0
You can see that the value for state has
changed from RESERVED to
RELEASED
A user who has been given the right to make a space reservation
can make a space reservation. To achieve this the right entry in
the LinkGroupAuthorization.conf file is
required.
In order to be able to take advantage of the virtual organization (VO) infrastructure and VO based authorization and VO based access control to the space in dCache, certain things need to be in place:
User needs to be registered with the VO.
User needs to use voms-proxy-init to create a VO proxy.
dCache needs to use
gPlazmawith modules that extract VO attributes from the user’s proxy. (See Chapter 12, Authorization in dCache, have a look atgplazmalite-vorole-mappingplugin and see the section called “Authentication and Authorization in dCache” for an example withgplazmalite-vorole-mapping.
Only if these 3 conditions are satisfied the VO based
authorization of the SpaceManager will work.
As mentioned above dCache space
reservation functionality access control is currently
performed at the level of the link groups. Access to making
reservations in each link group is controlled by the
SpaceManagerLinkGroupAuthorizationFile.
This file contains a list of the link groups and all the VOs and the VO Roles that are permitted to make reservations in a given link group.
When a SRM Space Reservation request is executed, its
parameters, such as reservation size, lifetime,
AccessLatencyand
RetentionPolicy as well as user’s VO
membership information is forwarded to the SRM
SpaceManager.
Once a space reservation is created, no access control is performed, any user can store the files in this space reservation, provided he or she knows the exact space token.
A user who is given the rights in the
SpaceManagerLinkGroupAuthorizationFile can
make a space reservation by
[user] $srm-reserve-space -retention_policy=<RetentionPolicy> -lifetime=<lifetimeInSecs> -desired_size=<sizeInBytes> -guaranteed_size=<sizeInBytes> srm://<example.org>:8443Space token =SpaceTokenId
and release it by
[user] $srm-release-space srm://<example.org>:8443 -space_token=SpaceTokenId
Note
Please note that it is obligatory to specify the retention policy while it is optional to specify the access latency.
Example:
[user] $srm-reserve-space -retention_policy=REPLICA -lifetime=300 -desired_size=5500000 -guaranteed_size=5500000 srm://srm.example.org:8443Space token =110044
The space reservation can be released by:
[user] $srm-release-space srm://srm.example.org:8443 -space_token=110044
If a client uses a regular grid proxy, created with
grid-proxy-init, and not a VO proxy, which
is created with the voms-proxy-init, when
it is communicating with SRM server in dCache, then the VO
attributes can not be extracted from its credential. In this
case the name of the user is extracted from the Distinguished
Name (DN) to use name mapping. For the purposes of the space
reservation the name of the user as mapped by gplazma
is used as its VO Group name, and the VO Role is left
empty. The entry in the
SpaceManagerLinkGroupAuthorizationFile
should be:
#LinkGroupAuthorizationFile # <userName>
Edit the file /opt/d-cache/etc/dcache.conf to enable space
reservation for non SRM transfers.
SpaceManagerReserveSpaceForNonSRMTransfers=true
If the spacemanager is enabled,
SpaceManagerReserveSpaceForNonSRMTransfers
is set to true, and if the transfer
request comes from a door, and there was no prior space
reservation made for this file, the SpaceManager will
try to reserve space before satisfying the request.
Possible values are true or
false and the default value is
false.
There are a few parameters in
/opt/d-cache/share/defaults/*.properties
that you might find useful for nontrivial SRM deployment.
srmSpaceManagerEnabled tells if the space
management is activated in SRM.
Possible values are yes and
no. Default is yes.
Usage example:
srmSpaceManagerEnabled=yes
srmImplicitSpaceManagerEnabled tells if
the space should be reserved for SRM Version 1 transfers and
for SRM Version 2 transfers that have no space token
specified. Will have effect only if srmSpaceManagerEnabled.
Possible values are yes and
no. This is enabled by default. It has no
effect if srmSpaceManagerEnabled is set to
no.
Usage example:
srmImplicitSpaceManagerEnabled=yes
overwriteEnabled tells SRM and
GridFTP servers if the overwrite is allowed. If enabled on
the SRM node, should be enabled on all GridFTP nodes.
Possible values are yes and
no. Default is no.
Usage example:
overwriteEnabled=yes
srmOverwriteByDefault Set this to
true if you want overwrite to be enabled
for SRM v1.1 interface as well as for SRM v2.2 interface
when client does not specify desired overwrite mode. This
option will be considered only if
overwriteEnabled is set to
yes.
Possible values are true and
false. Default is false.
Usage example:
srmOverwriteByDefault=false
srmDatabaseHost tells SRM which
database host to connect to.
Default value is localhost.
Usage example:
srmDatabaseHost=database-host.example.org
spaceManagerDatabaseHost tells SpaceManager
which database host to connect to.
Default value is localhost.
Usage example:
spaceManagerDatabaseHost=database-host.example.org
pinManagerDbHost tells PinManager
which database host to connect to.
Default value is localhost.
Usage example:
pinManagerDbHost=database-host.example.org
srmDbName tells SRM which database
to connect to.
Default value is dcache.
Usage example:
srmDbName=dcache
srmDbUser tells SRM which database
user name to use when connecting to database. Do not change
unless you know what you are doing.
Default value is srmdcache.
Usage example:
srmDbUser=srmdcache
srmDbPassword tells SRM which
database password to use when connecting to database. The
default value is srmdcache.
Usage example:
srmDbPassword=NotVerySecret
srmPasswordFile tells SRM which
database password file to use when connecting to database. Do
not change unless you know what you are doing. It is
recommended that MD5 authentication method is used. To learn
about file format please see http://www.postgresql.org/docs/8.1/static/libpq-pgpass.html. To
learn more about authentication methods please visit http://www.postgresql.org/docs/8.1/static/encryption-options.html,
Please read "Encrypting Passwords Across A Network" section.
This option is not set by default.
Usage example:
srmPasswordFile=/root/.pgpass
srmRequestHistoryDatabaseEnabled enables
logging of the transition history of the SRM request in the
database. The request transitions can be examined through the
command line interface. Activation of this option might lead
to the increase of the database activity, so if the PostgreSQL
load generated by SRM is excessive, disable it.
Possible values are true and
false. Default is false.
Usage example:
srmRequestHistoryDatabaseEnabled=true
srmDbLogEnabled tells SRM to store
the information about the remote (copy, srmCopy) transfer
details in the database. Activation of this option might lead to
the increase of the database activity, so if the PostgreSQL load
generated by SRM is excessive, disable it.
Possible values are true and
false. Default is false.
Usage example:
srmDbLogEnabled=false
srmVersion is not used by SRM; it was
mentioned that this value is used by some publishing scripts.
Default is version1.
pnfsSrmPath tells SRM what the root
of all SRM paths is in pnfs. SRM will prepend path to all
the local SURL paths passed to it by SRM client. So if the
pnfsSrmPath is set to
/pnfs/fnal.gov/THISISTHEPNFSSRMPATH and
someone requests the read of
srm://srm.example.org:8443/file1, SRM will
translate the SURL path /file1 into
/pnfs/fnal.gov/THISISTHEPNFSSRMPATH/file1. Setting
this variable to something different from /
is equivalent of performing Unix chroot for
all SRM operations.
Default value is /.
Usage example:
pnfsSrmPath="/pnfs/fnal.gov/data/experiment"
parallelStreams specifies the number of the
parallel streams that SRM will use when performing third
party transfers between this system and remote GSI-FTP
servers, in response to SRM v1.1 copy or SRM V2.2 srmCopy
function. This will have no effect on srmPrepareToPut and
srmPrepareToGet command results and parameters of GridFTP
transfers driven by the SRM clients.
Default value is 10.
Usage example:
parallelStreams=20
srmBufferSize specifies the number of bytes
to use for the in memory buffers for performing third party
transfers between this system and remote GSI-FTP servers, in
response to SRM v1.1 copy or SRM V2.2 srmCopy
function. This will have no effect on srmPrepareToPut and
srmPrepareToGet command results and parameters of GridFTP
transfers driven by the SRM clients.
Default value is 1048576.
Usage example:
srmBufferSize=1048576
srmTcpBufferSize specifies the number of
bytes to use for the tcp buffers for performing third party
transfers between this system and remote GSI-FTP servers, in
response to SRM v1.1 copy or SRM V2.2 srmCopy
function. This will have no effect on srmPrepareToPut and
srmPrepareToGet command results and parameters of GridFTP
transfers driven by the SRM clients.
Default value is 1048576.
Usage example:
srmTcpBufferSize=1048576
srmAuthzCacheLifetime specifies the
duration that authorizations will be cached. Caching decreases
the volume of messages to the gPlazma cell or other
authorization mechanism. To turn off caching, set the value
to 0.
Default value is 120.
Usage example:
srmAuthzCacheLifetime=60
srmGetLifeTime,
srmPutLifeTime and
srmCopyLifeTime specify the lifetimes of
the srmPrepareToGet (srmBringOnline) srmPrepareToPut and
srmCopy requests lifetimes in millisecond. If the system is
unable to fulfill the requests before the request lifetimes
expire, the requests are automatically garbage collected.
Default value is 14400000 (4 hours)
Usage example:
srmGetLifeTime=14400000 srmPutLifeTime=14400000 srmCopyLifeTime=14400000
srmGetReqMaxReadyRequests and
srmPutReqMaxReadyRequests specify the
maximum number of the files for which the transfer URLs will
be computed and given to the users in response to SRM get
(srmPrepareToGet) and put (srmPrepareToPut) requests. The rest
of the files that are ready to be transfered are put on the
Ready queues, the maximum length of these
queues are controlled by
srmGetReqReadyQueueSize and
srmPutReqReadyQueueSize parameters. These
parameters should be set according to the capacity of the
system, and are usually greater than the maximum number of the
GridFTP transfers that this dCache instance GridFTP
doors can sustain.
Usage example:
srmGetReqReadyQueueSize=10000 srmGetReqMaxReadyRequests=2000 srmPutReqReadyQueueSize=10000 srmPutReqMaxReadyRequests=1000
srmCopyReqThreadPoolSize and
remoteGsiftpMaxTransfers.
srmCopyReqThreadPoolSize is used to specify how many parallel
srmCopy file copies to execute simultaneously. Once the
SRM contacted the remote SRM system, and obtained a Transfer
URL (usually GSI-FTP URL), it contacts a Copy Manager
module (usually RemoteGSIFTPTransferManager), and asks it to
perform a GridFTP transfer between the remote GridFTP server
and a dCache pool. The maximum number of simultaneous
transfers that RemoteGSIFTPTransferManager will support is
remoteGsiftpMaxTransfers, therefore it is important that
remoteGsiftpMaxTransfers is greater than or equal to
srmCopyReqThreadPoolSize.
Usage example:
srmCopyReqThreadPoolSize=250 remoteGsiftpMaxTransfers=260
srmCustomGetHostByAddr
srmCustomGetHostByAddr enables using the BNL developed
procedure for host by IP resolution if standard InetAddress
method failed.
Usage example:
srmCustomGetHostByAddr=true
RecursiveDirectoryCreation allows or
disallows automatic creation of directories via SRM,
allow=true, disallow=false.
Automatic directory creation is allowed by default.
Usage example:
RecursiveDirectoryCreation=true
This option allows you to control how often the SRM door will reload the
server’s host certificate from the filesystem. For the specified period,
the host certificate will be kept in memory. This speeds up the rate at
which the door can handle requests, but also causes it to be unaware of
changes to the host certificate (for instance in the case of renewal).
By changing this parameter you can control how long the host certificate is cached by the door and consequently how fast the door will be able to detect and reload a renewed host certificate.
Please note that the value of this parameter has to be specified in seconds.
Usage example:
hostCertificateRefreshPeriod=86400
The trustAnchorRefreshPeriod option is similar to
hostCertificateRefreshPeriod. It applies to the set
of CA certificates trusted by the SRM door for signing end-entity
certificates (along with some metadata, these form so called
trust anchors).
The trust anchors are needed to make a decision about
the trustworthiness of a certificate in X.509 client
authentication. The GSI security protocol used by SRM builds upon
X.509 client authentication.
By changing this parameter you can control how long the set of trust anchors remains cached by the door. Conversely, it also influences how often the door reloads the set of trusted certificates.
Please note that the value of this parameter has to be specified in seconds.
Tip
Trust-anchors usually change more often than the host certificate. Thus, it might be sensible to set the refresh period of the trust anchors lower than the refresh period of the host certificate.
Usage example:
trustAnchorRefreshPeriod=3600