In the Chimera namespace, each directory can have a number of tags. These directory tags may be used within dCache to control the file placement policy in the pools (see the section called “The Pool Selection Mechanism”). They might also be used by a tertiary storage system for similar purposes (e.g. controlling the set of tapes used for the files in the directory).
Note
Directory tags are not needed to control the behaviour of dCache. dCache works well without directory tags.
You can create tags with
[user] $echo "<content>" | /opt/d-cache/bin/chimera-cli.sh writetag <directory> <tagName>
list tags with
[user] $/opt/d-cache/bin/chimera-cli.sh lstag <directory>
and read tags with
[user] $/opt/d-cache/bin/chimera-cli.sh readtag <directory> <tagName>
Example:
Create tags for the directory data with
[user] $echo "myGroup" | /opt/d-cache/bin/chimera-cli.sh writetag /data sGroup[user] $echo "StoreName myStore" | /opt/d-cache/bin/chimera-cli.sh writetag /data OSMTemplate
list the existing tags with
[user] $/opt/d-cache/bin/chimera-cli.sh lstag /dataTotal: 2 OSMTemplate sGroup
and their content with
[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data OSMTemplateStoreName myStore[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data sGroupmyGroup
If the namespace is mounted, change to the directory for which the tag should be set and create a tag with
[user] $cd <directory>[user] $echo '<content1>' > '.(tag)(<tagName1>)'[user] $echo '<content2>' > '.(tag)(<tagName2>)'
Then the existing tags may be listed with
[user] $cat '.(tag)(<tagname1>) .(tag)(<tagname2>).(tags)()'
and the content of a tag can be read with
[user] $cat '<content1>.(tag)(<tagname1>)'[user] $cat '<content2>.(tag)(<tagName2>)'
Example:
Create tags for the directory data with
[user] $cd data[user] $echo 'StoreName myStore' > '.(tag)(OSMTemplate)'[user] $echo 'myGroup' > '.(tag)(sGroup)'
list the existing tags with
[user] $cat '.(tag)(OSMTemplate) .(tag)(sGroup).(tags)()'
and their content with
[user] $cat 'StoreName myStore.(tag)(OSMTemplate)'[user] $cat 'myGroup.(tag)(sGroup)'
A nice trick to list all tags with their contents is
[user] $grep "" $(cat ".(tags)()").(tag)(OSMTemplate):StoreName myStore .(tag)(sGroup):myGroup
When creating or changing directory tags by writing to the command file as in
[user] $echo '<content>' > '.(tag)(<tagName>)'
one has to take care not to treat the command files in the same way as regular files, because tags are different from files in the following aspects:
The <tagName> is limited to 62 characters and the <content> to 512 bytes. Writing more to the command file, will be silently ignored.
If a tag which does not exist in a directory is created by writing to it, it is called a primary tag.
Tags are inherited from the parent directory by a newly created directory. Changing a primary tag in one directory will change the tags inherited from it in the same way. Creating a new primary tag in a directory will not create an inherited tag in its subdirectories.
Moving a directory within the Chimera namespace will not change the inheritance. Therefore, a directory does not necessarily inherit tags from its parent directory. Removing an inherited tag does not have any effect.
Empty tags are ignored.
The following directory tags appear in the dCache context:
- OSMTemplate
Must contain a line of the form “
StoreName<storeName>” and specifies the name of the store that is used by dCache to construct the storage class if the HSM Type isosm.- HSMType
The
HSMTypetag is normally determined from the other existing tags. E.g., if the tagOSMTemplateexists,HSMType=osmis assumed. With this tag it can be set explicitly. A class implementing that HSM type has to exist. Currently the only implementations areosmandenstore.- sGroup
The storage group is also used to construct the storage class if the
HSMTypeisosm.- cacheClass
The cache class is only used to control on which pools the files in a directory may be stored, while the storage class (constructed from the two above tags) might also be used by the HSM. The cache class is only needed if the above two tags are already fixed by HSM usage and more flexibility is needed.
- hsmInstance
If not set, the
hsmInstancetag will be the same as theHSMTypetag. Setting this tag will only change the name as used in the storage class and in the pool commands.
The storage class is a
string of the form
<StoreName>:<StorageGroup>@<hsm-type>,
where <StoreName> is given by the
OSMTemplate tag,
<StorageGroup> by the
sGroup tag and
<hsm-type> by the
HSMType tag. As mentioned above the
HSMType tag is assumed to be
osm if the tag OSMTemplate
exists.
In the examples above two tags have been created.
Example:
[user] $/opt/d-cache/bin/chimera-cli.sh lstag /dataTotal: 2 OSMTemplate sGroup
As the tag OSMTemplate was created the tag
HSMType is assumed to be osm.
The storage class of the files which are copied into the
directory /data after
the tags have been set will be
myStore:myGroup@osm.
If directory tags are used to control the behaviour of dCache and/or a tertiary storage system, it is a good idea to plan the directory structure in advance, thereby considering the necessary tags and how they should be set up. Moving directories should be done with great care or even not at all. Inherited tags can only be created by creating a new directory.
Example:
Assume that data of two experiments,
experiment-a and
experiment-b is written into a namespace
tree with subdirectories /data/experiment-a and /data/experiment-b. As some pools
of the dCache are financed by
experiment-a and others by
experiment-b they probably do not like it
if they are also used by the other group. To avoid this the
directories of experiment-a and
experiment-b can be tagged.
[user] $echo "StoreName exp-a" | /opt/d-cache/bin/chimera-cli.sh writetag /data/experiment-a OSMTemplate[user] $echo "StoreName exp-b" | /opt/d-cache/bin/chimera-cli.sh writetag /data/experiment-b OSMTemplate
Data from experiment-a taken in 2010 shall
be written into the directory /data/experiment-a/2010 and data
from experiment-a taken in 2011 shall be
written into /data/experiment-a/2011. Data
from experiment-b shall be written into
/data/experiment-b. Tag
the directories correspondingly.
[user] $echo "run2010" | /opt/d-cache/bin/chimera-cli.sh writetag /data/experiment-a/2010 sGroup[user] $echo "run2011" | /opt/d-cache/bin/chimera-cli.sh writetag /data/experiment-a/2011 sGroup[user] $echo "alldata" | /opt/d-cache/bin/chimera-cli.sh writetag /data/experiment-b sGroup
List the content of the tags by
[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-a/2010 OSMTemplateStoreName exp-a[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-a/2010 sGrouprun2010[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-a/2011 OSMTemplateStoreName exp-a[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-a/2011 sGrouprun2011[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-b/2011 OSMTemplateStoreName exp-b[user] $/opt/d-cache/bin/chimera-cli.sh readtag /data/experiment-b/2011 sGroupalldata
As the tag OSMTemplate was created the
HSMType is assumed to be
osm.
The storage classes of the files which are copied into these directories after the tags have been set will be
exp-a:run2010@osmfor the files in/data/experiment-a/2010exp-a:run2011@osmfor the files in/data/experiment-a/2011exp-b:alldata@osmfor the files in/data/experiment-b
To see how storage classes are used for pool selection have a look at the example ’Reserving Pools for Storage and Cache Classes’ in the PoolManager chapter.
There are more tags used by dCache if the
HSMType is enstore.