In the following the installation of a single node dCache
instance will be described. The Chimera name space provider,
some management components, and the SRM
need a PostgreSQL server
installed. We recommend running this PostgreSQL on the local
node. The first section describes the configuration of a PostgreSQL
server. After that the installation of Chimera and of the
dCache components will follow. During the whole installation
process root access is required.
In order to install dCache the following requirements must be met:
An RPM-based Linux distribution is required for the following procedure. For Debian derived systems the RPM may be converted to a DEB using alien. Solaris is supported using either the Solaris package or the tarball.
dCache 1.9 requires Java 1.5 or 1.6 SDK. We recommend Java 1.6. It is recommended to use the newest Java release available within the release series used.
PostgreSQL must be installed and running. See the section called “Installing a PostgreSQL Server” for more details. It is strongly recommended to use version 8 or higher.
The RPM packages may be installed right away, for example using the command:
[root] #
rpm -ivh dcache-server-<version>-<release>.i386.rpm
[root] #
rpm -ivh dcache-client-<version>-<release>.i386.rpm
The actual sources lie at http://www.dcache.org/downloads.shtml. To install for example Version 1.9.4-2 of the server you would use this:
[root] #
rpm -ivh http://www.dcache.org/downloads/1.9/dcache-server-1.9.4-2.noarch.rpm
The Client can be found in the download-section of the above url, too.
You must configure PostgreSQL for use by dCache and create the necessary PostgreSQL user accounts and database structure. This section describes how to do this.
Using a PostgreSQL server with dCache places a number of requirements on the database. This section describes what configuration is necessary to ensure PostgreSQL operates so dCache can use it.
Restarting PostgreSQL
If you have edited PostgreSQL configuration files, you must restart PostgreSQL for those changes to take effect. On many systems, this can be done with the following command:
[root] #
/etc/init.d/postgresql restart
Important
For Versions of PostgreSQL newer than 8.0 theTCP
connections are already enabled and this section has to be
ignored.
When connecting to PostgreSQL, dCache will always use TCP
connections. So, for dCache to use PostgreSQL, support for
TCP
sockets must be enabled. We realize UNIX domain
sockets are easier to work with from a security point of
view, however there is no way to use UNIX domain sockets
from a Java application.
In contrast to dCache, the PostgreSQL stand-alone client
application psql can connect using
either a TCP
socket or via a UNIX domain socket.
Because of this, it is common for PostgreSQL to disable TCP
sockets by default, requiring the admin to explicitly
configure PostgreSQL so connecting via a TCP
socket is
supported.
To enable TCP
sockets, edit the PostgreSQL configuration
file postgresql.conf
. This is often
found in the /var/lib/pgsql/data
, but may
be located elsewhere. You should ensure that the line
tcpip_socket
is set to
true
; for example:
tcpip_socket = true
Perhaps the simplest configuration is to allow password-less access to the database and the following documentation assumes this is so.
To allow local users to access PostgreSQL without requiring a
password, ensure the file
pg_hba.conf
, usually located in
/var/lib/pgsql/data
,
contains the following lines.
local all all trust host all all 127.0.0.1/32 trust host all all ::1/128 trust
Note
Please note it is also possible to run dCache with all PostgreSQL accounts requiring passwords.
Chimera is a library providing a hierarhical name space with associated meta data. Where pools in dCache store the content of files, Chimera stores the names and meta data of those files. Chimera itself stores the data in a relational database. We will use PostgreSQL in this tutorial.
Note
dCache used to use another name space
implementation called pnfs
. pnfs
is still available, we
do however recommend that new installations use Chimera.
Create the Chimera user and database and add the Chimera-specific tables and stored procedures:
[root] #
createdb -U postgres chimera
CREATE DATABASE[root] #
createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt chimera
Enter password for new role: Enter it again: CREATE ROLE[root] #
psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/create.sql
psql:/opt/d-cache/libexec/chimera/sql/create.sql:23: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_inodes_pkey" for table "t_inodes" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:35: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_dirs_pkey" for table "t_dirs" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:45: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_inodes_data_pkey" for table "t_inodes_data" many more like this... INSERT 0 1 many more like this... INSERT 0 1 CREATE INDEX CREATE INDEX psql:/opt/d-cache/libexec/chimera/sql/create.sql:256: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_storageinfo_pkey" for table "t_storageinfo" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:263: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_access_latency_pkey" for table "t_access_latency" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:270: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_retention_policy_pkey" for table "t_retention_policy" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:295: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_locationinfo_pkey" for table "t_locationinfo" CREATE TABLE psql:/opt/d-cache/libexec/chimera/sql/create.sql:311: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_locationinfo_trash_pkey" for table "t_locationinfo_trash" CREATE TABLE CREATE INDEX psql:/opt/d-cache/libexec/chimera/sql/create.sql:332: NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "t_acl_pkey" for table "t_acl" CREATE TABLE CREATE INDEX[root] #
createlang -U postgres plpgsql chimera
[root] #
psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql
CREATE FUNCTION CREATE FUNCTION CREATE FUNCTION CREATE TRIGGER CREATE FUNCTION CREATE TRIGGER CREATE SEQUENCE CREATE FUNCTION CREATE TRIGGER
Database connection settings can be customized in
/opt/d-cache/config/chimera-config.xml
Specifically you should change the user to
“chimera”.
<?xml version="1.0"?> <config> <db fsid="0" url="jdbc:postgresql://localhost/chimera?prepareThreshold=3" drv="org.postgresql .Driver" user="chimera" pass="" dialect="PgSQL" /> <nfs> <port>2049</port> <logLevel>0</logLevel> <logFile>/tmp/himera.log</logFile> </nfs> </config>
Although most components in dCache access the Chimera database directly, some rely on a mounted file system for access. The mounted file system is also nice for administrative access. This offers the opportunity to use OS-level tools like ls and mkdir for Chimera. However, direct I/O-operations like cp are not possible, since the NFSV3 interface provides the namespace part only. This section describes how to start the Chimera NFS3 server and mount the name space.
Chimera NFS server uses
/etc/exports
file to manage
exports. So it has to exist or be created.
The typical exports
file looks like this:
/ localhost(rw) /pnfs # or # /pnfs *.my.domain(rw)
Since Chimera is coupled with dCache it uses the same configuration file
and won’t start without it. So copy the /opt/d-cache/etc/dCacheSetup.template
to
/opt/d-cache/config/dCacheSetup
.
Note
On some linux distributions you might have to switch the portmap daemon off before starting chimera:
[root] #
/etc/init.d/portmap stop
Stopping portmap: portmap
Start it via script:
[root] #
/opt/d-cache/libexec/chimera/chimera-nfs-run.sh start
To automate the launching of that script at startup time,
link to it from /etc/init.d/
. Then
announce it to chkconfig:
[root] #
chkconfig --add chimera-nfs-run.sh
[root] #
chkconfig chimera-nfs-run.sh on
First we create the root of the Chimera namespace, called ’pnfs’ for legacy reasons.
[root] #
/opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs
Now we need to add directory tags. For more information on tags see the section called “Directory Tags”.:
[root] #
/opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/<your domain>
[root] #
/opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/<your domain>/data
[root] #
echo "chimera" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/<your domain> /data sGroup
[root] #
echo "StoreName sql" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/<your do main>/data OSMTemplate
If you plan to use dCap
with mounted file system
instead of the URL-syntax (e.g. dccp
/pnfs/desy.de/data/file1
/tmp/file1
), we need to mount the
root of Chimera locally
(remote mounts are not allowed yet). This will allow us to
establish wormhole files so dCap
clients can discover the dCap
doors.
[root] #
mount localhost:/ /mnt
[root] #
mkdir /mnt/admin/etc/config/dCache
[root] #
touch /mnt/admin/etc/config/dCache/dcache.conf
[root] #
touch /mnt/admin/etc/config/dCache/'.(fset)(dcache.conf)(io)(on)'
[root] #
echo "<door host>:<port>" > /mnt/admin/etc/config/dCache/dcache.conf
The default values for ports can be found in Chapter 31, dCache default port values. They can be altered in
/opt/d-cache/config/dCacheSetup
The configuration is done now, so unmount Chimera:
[root] #
umount /mnt
Please note that whenever you need to change the
configuration, you have to remount the root
localhost:/
to a temporary location like /mnt
.
The “user's view” of Chimera is automatically mounted
by the dCache init script. You have to make sure that the mountpoint is
created on the machine (/pnfs
).
Chimera can be mounted manually with:
[root] #
mkdir /pnfs
[root] #
mount localhost:/pnfs /pnfs
The dCache components will access the database server with the user srmdcache which can be created with the createuser; for example:
[root] #
createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt srmdcache
Several management components running on the head node as well
as the SRM
will use the database
dcache for state information:
[root] #
createdb -U srmdcache dcache
There might be several of these on several hosts. Each is used by the dCache components running on the respective host.
[root] #
createdb -U srmdcache companion
[root] #
psql -U srmdcache companion -f /opt/d-cache/etc/psql_install_companion.sql
If the resilience feature provided by the replica manager is used, the database “replicas” has to be prepared on the head node with the command:
[root] #
createdb -U srmdcache replicas
[root] #
psql -U srmdcache replicas -f /opt/d-cache/etc/psql_install_replicas.sql
Note
Note that the disk space will at least be cut in half if the replica manager is used.
If the billing information should also be stored in a database (in addition to files) the database billing has to be created:
[root] #
createdb -U srmdcache billing
However, we strongly advise against using the same database
server for Chimera and the billing information. For how
to configure the billing
to write into this database,
see below.
The main configuration file of a dCache instance is
/opt/d-cache/config/dCacheSetup
. Set the
variable java
to the binary of the Java VM
and the variable serviceLocatorHost
to the
hostname of the single node running dCache.
Use the templates of the configuration files found in
/opt/d-cache/etc/
to
create the following files.
The installation and start-up scripts use the information in
/opt/d-cache/etc/node_config
. First copy it from the template.
For a setup with a single node, set NODE_TYPE
to
“admin”. To enable doors on this node, add the
respective doors to SERVICES
, for instance
“dcap” or “gridftp”. Set
NAMESPACE
to “chimera”.
For authorization of grid users the file
/opt/d-cache/etc/dcache.kpwd
is
needed. You can simply copy the template that is in the same directory. Note that it may be generated from the standard
/etc/grid-security/grid-mapfile
with the
tool grid-mapfile2dcache-kpwd
which is
distributed with the WLCG software.
We proceed by finalising the initial configuration by executing /opt/d-cache/install/install.sh, for example:
[root] #
/opt/d-cache/install/install.sh
INFO:Skipping ssh key generation Checking MasterSetup ./config/dCacheSetup O.k. Sanning dCache batch files Processing adminDoor Processing chimera Processing dCache Processing dir Processing door Processing gPlazma Processing gridftpdoor Processing gsidcapdoor Processing httpd Processing info Processing infoProvider Processing lm Processing maintenance Processing chimera Processing pool Processing replica Processing srm Processing statistics Processing utility Processing xrootdDoor Checking Users database .... Ok Checking Security .... Ok Checking JVM ........ Ok Checking Cells ...... Ok dCacheVersion ....... Version production-1-9-3-1
No pools have been created on the node yet. Adding pools to a node is a two step process:
The directory layout of the pool is created and filled with a skeleton configuration using dcache pool create <poolSize> <poolDirectory>, where <poolDirectory> is the full path to the directory which will contain the data files as well as some of the configuration of the pool, and <poolSize> is the size of the pool, specified in bytes or with a M, G, or T suffix (for mibibytes, gibibytes and tibibytes, respectively).
Make sure that there is always enough space under <poolDirectory>. Be aware that only pure data content is counted by dCache. Leave enough room for configuration files and filesystem overhead.
Creating a pool does not modify the dCache configuration.
The pool is given a unique name and added to the dCache configuration using dcache pool add <poolName> <poolDirectory>, where <poolDirectory> is the directory in which the pool was created and <poolName> is a name for the pool. The name must be unique throughout the whole dCache installation, not just on the node.
Adding a pool to a configuration does not modify the pool or the data in it and can thus safely be undone or repeated.
Note
The default gap for poolsizes is 4GiB. This means you should make a bigger pool than 4GiB otherwise you would have to change this gap in the dCache admin tool. See the example below. See also the section called “The Admin Interface”.
(local) admin >
cd <poolName>
(<poolname>) admin >
set gap 2G
(<poolname>) admin >
save
Example:
An example may help to clarify the use of these commands:
[root] #
/opt/d-cache/bin/dcache pool create 500G /q/pool1
Created a 500 GiB pool in /q/pool1. The pool cannot be used until it has been added to a domain. Use 'pool add' to do so. Please note that this script does not set the owner of the pool directory. You may need to adjust it.[root] #
/opt/d-cache/bin/dcache pool add myFirstPool
Added pool myFirstPool in /q/pool1 to dcache-vmDomain. The pool will not be operational until the domain has been started. Use 'start dcache-vmDomain' to start the pool domain./q/pool1/
[user] $
/opt/d-cache/bin/dcache pool ls
Pool Domain Size Free Path myFirstPool dcache-vmDomain 500 550 /q/pool1 Disk space is measured in GiB.
All configured components can now be starting with
dcache start
, for example:
[root] #
/opt/d-cache/bin/dcache start
Starting lmDomain Done (pid=7514) Starting dCacheDomain Done (pid=7574) Starting pnfsDomain Done (pid=7647) Starting dirDomain Done (pid=7709) Starting adminDomain Done (pid=7791) Starting httpdDomain Done (pid=7849) Starting utilityDomain Done (pid=7925) Starting gPlazma-dcache-vmDomain Done (pid=8002) Starting infoProviderDomain Done (pid=8081) Starting dcap-dcache-vmDomain Done (pid=8154) Starting gridftp-dcache-vmDomain Done (pid=8221) Starting gsidcap-dcache-vmDomain Done (pid=8296) Starting dcache-vmDomain Done (pid=8369)