release notes | Book: 1.9.5, 1.9.12 (opt, FHS), 2.11 (FHS), 2.12 (FHS), 2.13 (FHS), 2.14 (FHS), | Wiki | Q&A black_bg
Web: Multi-page, Single page | PDF: A4-size, Letter-size | eBook: epub black_bg

Installing a Single Node dCache Instance

In the following the installation of a single node dCache instance will be described. The Chimera name space provider, some management components, and the SRM need a PostgreSQL server installed. We recommend running this PostgreSQL on the local node. The first section describes the configuration of a PostgreSQL server. After that the installation of Chimera and of the dCache components will follow. During the whole installation process root access is required.

[return to top]

Prerequisites

In order to install dCache the following requirements must be met:

  • An RPM-based Linux distribution is required for the following procedure. For Debian derived systems the RPM may be converted to a DEB using alien. Solaris is supported using either the Solaris package or the tarball.

  • dCache 1.9 requires Java 1.5 or 1.6 SDK. We recommend Java 1.6. It is recommended to use the newest Java release available within the release series used.

  • PostgreSQL must be installed and running. See the section called “Installing a PostgreSQL Server” for more details. It is strongly recommended to use version 8 or higher.

[return to top]

Installation of the dCache Software

The RPM packages may be installed right away, for example using the command:

[root] # rpm -ivh dcache-server-<version>-<release>.i386.rpm
[root] # rpm -ivh dcache-client-<version>-<release>.i386.rpm

The actual sources lie at http://www.dcache.org/downloads.shtml. To install for example Version 1.9.4-2 of the server you would use this:

[root] # rpm -ivh http://www.dcache.org/downloads/1.9/dcache-server-1.9.4-2.noarch.rpm

The Client can be found in the download-section of the above url, too.

[return to top]

Readying the PostgreSQL server

You must configure PostgreSQL for use by dCache and create the necessary PostgreSQL user accounts and database structure. This section describes how to do this.

[return to top]

Configuring the PostgreSQL server

Using a PostgreSQL server with dCache places a number of requirements on the database. This section describes what configuration is necessary to ensure PostgreSQL operates so dCache can use it.

Restarting PostgreSQL

If you have edited PostgreSQL configuration files, you must restart PostgreSQL for those changes to take effect. On many systems, this can be done with the following command:

[root] # /etc/init.d/postgresql restart

[return to top]

Enabling TCP connections

Important

For Versions of PostgreSQL newer than 8.0 the TCP connections are already enabled and this section has to be ignored.

When connecting to PostgreSQL, dCache will always use TCP connections. So, for dCache to use PostgreSQL, support for TCP sockets must be enabled. We realize UNIX domain sockets are easier to work with from a security point of view, however there is no way to use UNIX domain sockets from a Java application.

In contrast to dCache, the PostgreSQL stand-alone client application psql can connect using either a TCP socket or via a UNIX domain socket. Because of this, it is common for PostgreSQL to disable TCP sockets by default, requiring the admin to explicitly configure PostgreSQL so connecting via a TCP socket is supported.

To enable TCP sockets, edit the PostgreSQL configuration file postgresql.conf. This is often found in the /var/lib/pgsql/data, but may be located elsewhere. You should ensure that the line tcpip_socket is set to true; for example:

tcpip_socket = true

[return to top]

Enabling local trust

Perhaps the simplest configuration is to allow password-less access to the database and the following documentation assumes this is so.

To allow local users to access PostgreSQL without requiring a password, ensure the file pg_hba.conf, usually located in /var/lib/pgsql/data, contains the following lines.

local   all         all                        trust
host    all         all         127.0.0.1/32   trust
host    all         all         ::1/128        trust

Note

Please note it is also possible to run dCache with all PostgreSQL accounts requiring passwords.

[return to top]

Configuring Chimera

Chimera is a library providing a hierarhical name space with associated meta data. Where pools in dCache store the content of files, Chimera stores the names and meta data of those files. Chimera itself stores the data in a relational database. We will use PostgreSQL in this tutorial.

Note

dCache used to use another name space implementation called pnfs. pnfs is still available, we do however recommend that new installations use Chimera.

[return to top]

Initialize the database

Create the Chimera user and database and add the Chimera-specific tables and stored procedures:

[root] # createdb -U postgres chimera
CREATE DATABASE
                
[root] # createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt chimera
Enter password for new role:
Enter it again:
CREATE ROLE

[root] # psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/create.sql
psql:/opt/d-cache/libexec/chimera/sql/create.sql:23: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_inodes_pkey" for table "t_inodes"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:35: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_dirs_pkey" for table "t_dirs"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:45: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_inodes_data_pkey" for table "t_inodes_data"
many more like this...
INSERT 0 1
many more like this...
INSERT 0 1
CREATE INDEX
CREATE INDEX
psql:/opt/d-cache/libexec/chimera/sql/create.sql:256: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_storageinfo_pkey" for table "t_storageinfo"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:263: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_access_latency_pkey" for table "t_access_latency"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:270: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_retention_policy_pkey" for table "t_retention_policy"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:295: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_locationinfo_pkey" for table "t_locationinfo"
CREATE TABLE
psql:/opt/d-cache/libexec/chimera/sql/create.sql:311: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_locationinfo_trash_pkey" for table "t_locationinfo_trash"
CREATE TABLE
CREATE INDEX
psql:/opt/d-cache/libexec/chimera/sql/create.sql:332: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "t_acl_pkey" for table "t_acl"
CREATE TABLE
CREATE INDEX

[root] # createlang -U postgres plpgsql chimera
[root] # psql -U chimera chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE TRIGGER
CREATE FUNCTION
CREATE TRIGGER
CREATE SEQUENCE
CREATE FUNCTION
CREATE TRIGGER

Database connection settings can be customized in /opt/d-cache/config/chimera-config.xml Specifically you should change the user to chimera.

<?xml version="1.0"?>
<config>
        <db fsid="0" url="jdbc:postgresql://localhost/chimera?prepareThreshold=3" drv="org.postgresql
.Driver" user="chimera" pass="" dialect="PgSQL" />
        <nfs>
                <port>2049</port>
                <logLevel>0</logLevel>
                <logFile>/tmp/himera.log</logFile>
        </nfs>
</config>

[return to top]

Mounting Chimera through NFS

Although most components in dCache access the Chimera database directly, some rely on a mounted file system for access. The mounted file system is also nice for administrative access. This offers the opportunity to use OS-level tools like ls and mkdir for Chimera. However, direct I/O-operations like cp are not possible, since the NFSV3 interface provides the namespace part only. This section describes how to start the Chimera NFS3 server and mount the name space.

Chimera NFS server uses /etc/exports file to manage exports. So it has to exist or be created. The typical exports file looks like this:

/ localhost(rw)
/pnfs
# or
# /pnfs *.my.domain(rw)

Since Chimera is coupled with dCache it uses the same configuration file and won’t start without it. So copy the /opt/d-cache/etc/dCacheSetup.template to /opt/d-cache/config/dCacheSetup.

Note

On some linux distributions you might have to switch the portmap daemon off before starting chimera:

[root] # /etc/init.d/portmap stop
Stopping portmap: portmap

Start it via script:

[root] # /opt/d-cache/libexec/chimera/chimera-nfs-run.sh start

To automate the launching of that script at startup time, link to it from /etc/init.d/. Then announce it to chkconfig:

[root] # chkconfig --add chimera-nfs-run.sh
[root] # chkconfig chimera-nfs-run.sh on

First we create the root of the Chimera namespace, called ’pnfs’ for legacy reasons.

[root] # /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs

Now we need to add directory tags. For more information on tags see the section called “Directory Tags”.:

[root] # /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/<your domain>
[root] # /opt/d-cache/libexec/chimera/chimera-cli.sh Mkdir /pnfs/<your domain>/data
[root] # echo "chimera" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/<your domain>
/data sGroup
[root] # echo "StoreName sql" | /opt/d-cache/libexec/chimera/chimera-cli.sh Writetag /pnfs/<your do
main>/data OSMTemplate

If you plan to use dCap with mounted file system instead of the URL-syntax (e.g. dccp /pnfs/desy.de/data/file1 /tmp/file1), we need to mount the root of Chimera locally (remote mounts are not allowed yet). This will allow us to establish wormhole files so dCap clients can discover the dCap doors.

[root] # mount localhost:/ /mnt
[root] # mkdir /mnt/admin/etc/config/dCache
[root] # touch /mnt/admin/etc/config/dCache/dcache.conf
[root] # touch /mnt/admin/etc/config/dCache/'.(fset)(dcache.conf)(io)(on)'
[root] # echo "<door host>:<port>" > /mnt/admin/etc/config/dCache/dcache.conf

The default values for ports can be found in Chapter 31, dCache default port values. They can be altered in /opt/d-cache/config/dCacheSetup

The configuration is done now, so unmount Chimera:

[root] # umount /mnt

Please note that whenever you need to change the configuration, you have to remount the root localhost:/ to a temporary location like /mnt.

The user's view of Chimera is automatically mounted by the dCache init script. You have to make sure that the mountpoint is created on the machine (/pnfs). Chimera can be mounted manually with:

[root] # mkdir /pnfs
[root] # mount localhost:/pnfs /pnfs

[return to top]

Creating users and databases for dCache

The dCache components will access the database server with the user srmdcache which can be created with the createuser; for example:

[root] # createuser -U postgres --no-superuser --no-createrole --createdb --pwprompt srmdcache

Several management components running on the head node as well as the SRM will use the database dcache for state information:

[root] # createdb -U srmdcache dcache

There might be several of these on several hosts. Each is used by the dCache components running on the respective host.

[root] # createdb -U srmdcache companion
[root] # psql -U srmdcache companion -f /opt/d-cache/etc/psql_install_companion.sql

If the resilience feature provided by the replica manager is used, the database replicas has to be prepared on the head node with the command:

[root] # createdb -U srmdcache replicas
[root] # psql -U srmdcache replicas -f /opt/d-cache/etc/psql_install_replicas.sql

Note

Note that the disk space will at least be cut in half if the replica manager is used.

If the billing information should also be stored in a database (in addition to files) the database billing has to be created:

[root] # createdb -U srmdcache billing

However, we strongly advise against using the same database server for Chimera and the billing information. For how to configure the billing to write into this database, see below.

[return to top]

Installing dCache Components

The main configuration file of a dCache instance is /opt/d-cache/config/dCacheSetup. Set the variable java to the binary of the Java VM and the variable serviceLocatorHost to the hostname of the single node running dCache.

Use the templates of the configuration files found in /opt/d-cache/etc/ to create the following files.

The installation and start-up scripts use the information in /opt/d-cache/etc/node_config. First copy it from the template. For a setup with a single node, set NODE_TYPE to admin. To enable doors on this node, add the respective doors to SERVICES, for instance dcap or gridftp. Set NAMESPACE to chimera.

For authorization of grid users the file /opt/d-cache/etc/dcache.kpwd is needed. You can simply copy the template that is in the same directory. Note that it may be generated from the standard /etc/grid-security/grid-mapfile with the tool grid-mapfile2dcache-kpwd which is distributed with the WLCG software.

We proceed by finalising the initial configuration by executing /opt/d-cache/install/install.sh, for example:

[root] # /opt/d-cache/install/install.sh
INFO:Skipping ssh key generation

 Checking MasterSetup  ./config/dCacheSetup O.k.

   Sanning dCache batch files

    Processing adminDoor
    Processing chimera
    Processing dCache
    Processing dir
    Processing door
    Processing gPlazma
    Processing gridftpdoor
    Processing gsidcapdoor
    Processing httpd
    Processing info
    Processing infoProvider
    Processing lm
    Processing maintenance
    Processing chimera
    Processing pool
    Processing replica
    Processing srm
    Processing statistics
    Processing utility
    Processing xrootdDoor


 Checking Users database .... Ok
 Checking Security       .... Ok
 Checking JVM ........ Ok
 Checking Cells ...... Ok
 dCacheVersion ....... Version production-1-9-3-1
        

No pools have been created on the node yet. Adding pools to a node is a two step process:

  1. The directory layout of the pool is created and filled with a skeleton configuration using dcache pool create <poolSize> <poolDirectory>, where <poolDirectory> is the full path to the directory which will contain the data files as well as some of the configuration of the pool, and <poolSize> is the size of the pool, specified in bytes or with a M, G, or T suffix (for mibibytes, gibibytes and tibibytes, respectively).

    Make sure that there is always enough space under <poolDirectory>. Be aware that only pure data content is counted by dCache. Leave enough room for configuration files and filesystem overhead.

    Creating a pool does not modify the dCache configuration.

  2. The pool is given a unique name and added to the dCache configuration using dcache pool add <poolName> <poolDirectory>, where <poolDirectory> is the directory in which the pool was created and <poolName> is a name for the pool. The name must be unique throughout the whole dCache installation, not just on the node.

    Adding a pool to a configuration does not modify the pool or the data in it and can thus safely be undone or repeated.

Note

The default gap for poolsizes is 4GiB. This means you should make a bigger pool than 4GiB otherwise you would have to change this gap in the dCache admin tool. See the example below. See also the section called “The Admin Interface”.

(local) admin > cd <poolName>
(<poolname>) admin > set gap 2G
(<poolname>) admin > save

Example:

An example may help to clarify the use of these commands:

[root] # /opt/d-cache/bin/dcache pool create 500G /q/pool1
Created a 500 GiB pool in /q/pool1. The pool cannot be used until it has
been added to a domain. Use 'pool add' to do so.

Please note that this script does not set the owner of the pool directory.
You may need to adjust it.
[root] # /opt/d-cache/bin/dcache pool add myFirstPool /q/pool1/

Added pool myFirstPool in /q/pool1 to dcache-vmDomain.

The pool will not be operational until the domain has been started. Use
'start dcache-vmDomain' to start the pool domain.
[user] $ /opt/d-cache/bin/dcache pool ls
Pool        Domain                       Size   Free Path
myFirstPool dcache-vmDomain               500    550 /q/pool1
Disk space is measured in GiB.
        

All configured components can now be starting with dcache start, for example:

[root] # /opt/d-cache/bin/dcache start
Starting lmDomain  Done (pid=7514)
Starting dCacheDomain  Done (pid=7574)
Starting pnfsDomain  Done (pid=7647)
Starting dirDomain  Done (pid=7709)
Starting adminDomain  Done (pid=7791)
Starting httpdDomain  Done (pid=7849)
Starting utilityDomain  Done (pid=7925)
Starting gPlazma-dcache-vmDomain  Done (pid=8002)
Starting infoProviderDomain  Done (pid=8081)
Starting dcap-dcache-vmDomain  Done (pid=8154)
Starting gridftp-dcache-vmDomain  Done (pid=8221)
Starting gsidcap-dcache-vmDomain  Done (pid=8296)
Starting dcache-vmDomain  Done (pid=8369)