Client requests to a dCache system may have rather diverse bahaviour. Sometimes it is possible to classify them into several typical usage patterns. An example are the following two usage patterns:
Example 26.1. To Concurrent Usage Patterns
Data is copied with a high transfer rate to the dCache
system from an external source. This is done via the
GridFTP protocol. At the same time batch jobs on a local
farm process data. Since they only need a small part of each
file, they use the dCap protocol via the dCap library
and seek to the position in the file they are interested in,
read a few bytes, do a few hours of calculations, and
finally read some more data.
As long as the number of active requests do not exceed the
maximum number of allowed active requests, the two types of
requests are processed concurrently. The GridFTP transfers
complete at a high rate while the processing jobs take hours
to finish. This maximum number of allowed requests is set with
mover set max active and
should be tuned according to capabilities of the pool host.
However, if requests are queued, the slow processing jobs
might clog up the queue and not let the fast GridFTP request
through, even though the pool just sits there waiting for the
processing jobs to request more data. While this could be
temporarily remedied by setting the maximum active requests to
a higher value, then in turn GridFTP request would put a
very high load on the pool host.
The above example is pretty realistic: As a rule of thumb,
GridFTP requests are fastest, dCap requests with the
dccp program are a little slower and dCap requests
with the dCap library are very slow. However, the usage
patterns might be different at other sites and also might
change over time.
Use separate queues for the movers, depending on the door initiating them. This easily allows for a separation of requests of separate protocols. Up to 10 mover queues for client transfers are available since dCache version 1.6.6. Earlier versions support only one queue. (Transfers from an to a tape backend and pool-to-pool transfers are handled by separate queues, one for each of these transfers.)
A finer grained queue selection mechanism based on, e.g. the
IP address of the client or the file which has been
requested, is not possible with this mechanism. However, the
pool selection unit
(PSU) may provide a separation onto separate pools
using those criteria.
In the above example, two separate queues for fast GridFTP
transfers and slow dCap library access would solve the
problem. The maximum number of active movers for the GridFTP
queue should be set to a lower value compared to the dCap
queue since the fast GridFTP transfers will put a high load
on the system while the dCap requests will be mostly idle.
For a multi mover queue setup, the pools have to be told to
start several queues and the doors have to be configured to
use one of these. It makes sense to create the same queues on
all pools. This is done by the following change to the
config/pool.batch file:
Example 26.2. Modified config/pool.batch file
for multiple mover queues
...
define context startPools endDefine
create diskCacheV111.pools.MultiProtocolPool2 ${0} \
"!MoverMap \
${1} \
-io-queues=<queueName-1>[,<queueName-2>[,...,<queueName-10>]] \
-recover-control=yes \
-version=4 \
-sticky=allowed \
-sendHitInfoMessages=yes \
-${2} -${3} -${4} -${5} -${6} -${7} -${8} \
"
endDefine
...
The same can be achived by appending
-io-queues=<queueName-1>,...,<queueName-n>
to each line in the poollist
file. However, this only makes sense if the pools
should not all have the same queues.
The first in this list of queues
(<queueName-1>) is
the default mover
queue. Transfers not requesting a particular mover
queue or requesting a mover queue not existing on the selected
pool, are handled by this default queue.
The pool cell commands mover ls and mover set max active have an
-queue option to select the mover queue to
operate on. Without this option, mover set max active will act on the default
queue while mover ls will list
the requests of all pools for backward compatibility.
Each door may be configured to use a particular mover queue. The pool, selected for this request, doesn’t depend on the selected mover queue. So a request may go to a pool which doesn’t have the particular mover queue configured and will consequently end up in the default mover queue of that pool.
The doors are configured to use a particular mover queue as in the following example:
Example 26.3. Batch file for a GridFTP door using a mover queue
...
create dmg.cells.services.login.LoginManager GFTP \
"<portName> \
diskCacheV111.doors.GsiFtpDoorV1 \
-io-queue=<queueName> \
... \
"All requests send from this door will ask to be scheduled to the given mover queue. The selection of the pool is not affected.
For the dCap protocol, the corresponding door may be
configured to allow the client to determine the mover queue
name. In that case the client may use the extra option
facility to specify a mover queue. Whether the the dCap door
allows the client to request a particular mover queue or not
is configured with the
-io-queue={allowed|denied} option as in the
following example:
Example 26.4. Batch file for a dCap door for allowing the client to select the mover queue
...
create dmg.cells.services.login.LoginManager DCap \
"${dCapPort} \
diskCacheV111.doors.DCapDoor \
-io-queue=<queueName> \
-io-queue-overwrite=allowed \
... \
"With the dccp command the queue can now be specified as follows:
[user] $dccp -X-io-queue=queueName <source> <destination>
Since dccp requests may be quite different from other
requests with the dCap protocol, this feature may be used to
use separate queues for dccp requests and other dCap
library requests. Therefore, the dccp command may be
changed in future releases to request a special
dccp-queue by default.