“File Hopping on arrival” is a term, denoting the
possibility of initiating a pool to pool transfer as the result
of a file successfully arriving on a pool. The file must have
been written by an external client using any supported protocol
(dCap, FTP, xrootd). Files restored from HSM or arriving
on a pool as the result of a pool to pool transfer will not yet
be forwarded.
Forwarding of incoming files is enabled per pool in the
<hostname>.poollist file. The pool is
requested to send a “replicateFile” message to
either the PoolManager or to the HoppingManager, if
available. The different approaches are briefly described below
and in more detail in the subsequent sections.
The “replicateFile” message is sent to the
PoolManager. This happens for all files arriving at that pool from outside (no restore or p2p). No intermediateHoppingManageris needed. The restrictions areAll files are replicated. No pre-selection, e.g. on the storage class can be done.
The mode of the replicated file is determined by the destination pool and can’t be overwritten. See ’File mode of replicated files’.
The “replicateFile” message is sent to the
HoppingManager. TheHoppingManagercan be configured to replicate certain storage classes only and to set the mode of the replicated file according to rules. The file mode of the source file can’t be modified.
The mode of a replicated file can either be determined by
settings in the destination pool or by the HoppingManager.
If no
HoppingManageris used for replication, the mode of the replicated file is determined by the p2p=<cached|precious> setting in the <hostname>.poollist file of the destination pool. The default setting is cached.If a
HoppingManageris used for file replication, the mode of the replicated file is determined by theHoppingManagerrule responsible for this particular replication. If the destination mode is set to “keep” in the rule, the mode of the destination pool determines the final mode of the replicated file.
File hopping configuration instructs a pool to send a
’replicateFile’ request to the PoolManager as the result
of a file arriving on that pool from some external client. All
arriving files will be treated the same. The PoolManager
will process this request by trying to find a ’link’ with the
following attributes :
Table 9.1. PoolManager Hopping Request Attributes
| Data Flow Direction | Protocol | Storage Class | Client IP Number |
|---|---|---|---|
| Pool 2 Pool | dCap/3 | Class of file | Configurable |
In order to get pool 2 pool enabled for a particular pool, the
corresponding entry of that pool in the
XXX.poollist file has to be extended by
the replicateOnArrival key-value pair.
<PoolName> <PoolPath> replicateOnArrival=PoolManager,<ip-number> ...
where <ip-number> may be a real IP
number of a farm node which may be taken as example node for
others intending to read the file, or the IP number may be
taken from a non existing IP number range. This range can be
used to instruct the PoolManager to replicate files from
this pool to a special set of destination pools.
Please see the section "File mode of replicated files" for the mode of the file on the destination pool.
We assume that we want of have all files, arriving on pool
ocean of host earth to be immediately
replicated to a subset of read pools. This subset of pools is
described by pool group ocean-copies. No
other pool is member of pool group
ocean-copies. Other than that, files
arriving at pool mountain should be
replicated to all read pools from which farm nodes on the
131.169.10.0/24
subnet are allowed to read.
The earth.poollist file must be modified as followes.
ocean /bigdisk/pools/ocean replicateOnArrival=PoolManager,192.1.1.1 <more options> mountain /bigdisk/pools/mountain replicateOnArrival=PoolManager,131.169.10.1 <more options>
While 131.169.10.1
is a legal IP address e.g. of one of you farm nodes, the
192.1.1.1 IP
address must not exist anywhere at your site.
Add the following lines to the
PoolManager.conf in order to instruct
the PoolManager to replicate files, arriving at the
ocean pool to be replicated to the
ocean-copies subset of your read pools.
# # define the read-pools pool group and add pool members # psu create pgroup farm-read-pools # psu addto pgroup farm-read-pools read-pool-1 psu addto pgroup farm-read-pools read-pool-2 psu addto pgroup farm-read-pools read-pool-3 psu addto pgroup farm-read-pools read-pool-4 # psu create unit -net 131.169.10.0/255.255.255.0 # psu create ugroup farm-network # psu addto ugroup farm-network 131.169.10.0/255.255.255.0 # psu create link farm-read-link any-store any-protocol farm-network # psu addto link farm-read-link farm-read-pools # psu set link farm-read-link -p2ppref=100 -readpref=100 -writepref=0 -cachepref=XXX... # # #-------------------------------------------------------------- # # create the faked net unit # psu create unit -net 192.1.1.1/255.255.255.255 # psu create ugroup ocean-copy-network # psu addto ugroup ocean-copy-network 192.1.1.1/255.255.255.255 # # we assume that 'any-protocol' and 'any-store' is already defined. # psu create link ocean-copy-link any-store any-protocol ocean-copy-network # psu addto link ocean-copy-link ocean-copy-pools # # define the ocean-copy pool group and add pool members # psu create pgroup ocean-copy-pools # psu addto pgroup ocean-copy-pools read-pool-1 # psu set link ocean-copy-link -p2ppref=100 -readpref=100 -writepref=0 -cachepref=XXX... # # # #
The HoppingManager is not automatically started in
1.7.0. Please perform the following steps to get it started
:
Create a file
hopping.batchin the/opt/d-cache/configdirectory with the following content :set printout default 3 set printout CellGlue none onerror shutdown # check -strong setupFile # copy file:${setupFile} context:setupContext # # import the variables into our $context. # don't overwrite already existing variables. # import context -c setupContext # # Make sure we got what we need. # check -strong serviceLocatorPort serviceLocatorHost # create dmg.cells.services.RoutingManager RoutingMgr # # The LocationManager Part # create dmg.cells.services.LocationManager lm \ "${serviceLocatorHost} ${serviceLocatorPort}" # # create diskCacheV111.services.FileHoppingManager HoppingManager \ "${config}/HoppingManager.conf -export" #change to
/opt/d-cache/jobsRun ./initPackage.sh
Start the service ./hopping start
Initially no rules are configured for the hopping
manager. You may add rules by either edit the
/opt/d-cache/config/HoppingManager.conf
and restart the hopping service, or use the admin interface
and save the modifications by ’save’ into
HoppingManager.conf
In order to instruct the pool to send a ’replicateFile’ message to the HoppingManager service, modify the <hostname>.poolllist file as follows :
...
ocean /bigdisk/pools/ocean replicateOnArrival=HoppingManager <more options>
...The
HoppingManageressentially receives ’replicateFile’ messages from pools, configured to support file hopping, and either discards or modifies and forwards them to thePoolManager, depending on rules described below.The
HoppingManagerdecides on the action to perform, based on a set of configurable rules. Each rule has a name. Rules are checked in alphabetic order concerning their names.A rule it triggered if the storage class matches the storage class pattern assigned to that rule. If a rule is triggered, it is processed and no further rule checking is performed. If no rule is found for this request the file is not replicated.
If for whatever reason, a file couldn’t be replicated, NO RETRY is being performed.
Processing a triggered rule can be :
The message is discarded. No replication is done for this particular storage class.
The rule modifies the ’replicate message’, before it is forwarded to the PoolManager.
The ’destination ip number can be added to the ’replicate file’ message. This has the same effect as the ip-number following the "PoolManager" keyword in the <hostname>.poollist file in the ’unconditional replication section above. The rule assignes a ’destination’ ip number to the ’replicate message’, before it is forwarded to the PoolManager. This has the same effect as the ip-number following the "PoolManager" keyword in the <hostname>.poollist file in the ’unconditional replication section above.
The mode of the replicated file can be specified. This can either be ’precious’, ’cached’ or ’keep’. ’keep’ means that the pool mode determines the replicated file mode.
The requested protocol can be specified.
define hop OPTIONS <name> <pattern> precious|cached|keep
OPTIONS
-destination=<cellDestination> # default : PoolManager
-overwrite
-continue
-source=write|restore|* # !!!! for experts only StorageInfoOptions
-host=<destinationHostIp>
-protType=dCap|ftp...
-protMinor=<minorProtocolVersion>
-protMajor=<majorProtocolVersion> pattern is a storage class pattern which. If the incoming storage class matches this pattern, this rule is processed.
precious|cached|keep determines the mode of the replicated file. ’keep’ leaves the destination mode to the pool setting.
destination shouldn’t be used. For experts only.
overwrite In case, a rule with the same name already exists, it is overwritten. If this overwrite option is specified, the error will occure.
continue If a rule has triggered and the corresponding action has been performed, no other rules are checked. If the ’continue’ option is specified, rule checking continues. This is for debugging purposes only.
source Don’t use.
host, protType This is the ’host ip number’ and ’protocol type’used by the PoolManager in order find an appropriate pool for the replication request. Please note that ’host’ is not the host of the destination pool.
Define the HoppingManager as destination for the ’replicate
file’ requests on the pool(s).
...
ocean /bigdisk/pools/ocean replicateOnArrival=HoppingManager <more options>
...Replicate ’raw’ data files by all experiments.
#
define hop replicate-raw .*:raw@osm -host=<Farm Node Ip Number>
#Replicate all CMS files to pools assigned to CMS farm nodes and all ATLAS files to pools assigned to ATLAS farm nodes. Don’t replicate any other files.
#
define hop replicate-cms cms:.*@osm -host=<Farm Node Ip Number of cms farm>
define hop replicate-atlas atlase:.*@osm -host=<Farm Node Ip Number of atlas farm>
#