Part II. Configuration of dCache

This part contains descriptions of the components of dCache, their role, functionality within the framework. In short, all information necessary for configuring them.

Table of Contents

1. ZooKeeper

   • Deployment scenarios
   • Configuration
   • Inspecting ZooKeeper through dCache

2. Chimera

   • Mounting Chimera through NFS
   • Using dCap with a mounted file system
   • Communicating with Chimera
   • IDs
   • Directory Tags
   • Create, List and Read Directory Tags if the Namespace is not Mounted
   • Create, List and Read Directory Tags if the Namespace is Mounted
   • Directory Tags and Command Files
   • Directory Tags for dCache
   • Storage Class and Directory Tags

3. The Cleaner Service

• Configuration
• Disk Cleaner
• HSM Cleaner

1. Cell Message Passing

2. The Resilience Service

   • Configuring the Resilience Service
   • Resilience Home
   • Admin Commands
   • Tuning
   • Resilience’s View of Pool Status
   • Automatic Staging of Missing CUSTODIAL Replicas
   • Some typical scenarios part 1: what happens when …?
   • Some typical scenarios part 2: how do I …?

3. The PoolManager Service

   • The Pool Selection Mechanism
   • Links
   • Examples
   • The Partition Manager
   • Overview
   • Managing Partitions
   • Using Partitions
   • Classic Partitions
   • Link Groups

4. The dCache Tertiary Storage System Interface

   • Introduction
   • Scope of this chapter
   • Requirements for a Tertiary Storage System
   • Migrating Tertiary Storage Systems with a file system interface
   • Tertiary Storage Systems with a minimalistic PUT, GET and REMOVE interface
   • How dCache interacts with a Tertiary Storage System
   • Details on the TSS-support executable
   • Summary of command line options
   • Summary of return codes
   • The executable and the STORE FILE operation
   • The executable and the FETCH FILE operation
   • The executable and the REMOVE FILE operation
   • Configuring pools to interact with a Tertiary Storage System
   • The dCache layout files
   • What happens next
   • How to Store-/Restore files via the Admin Interface
   • How to monitor what’s going on
   • Log Files
   • Obtain information via the dCache Command Line Admin Interface
   • Example of an executable to simulate a tape backend

5. The PinManager Service

• The (Un-)Pinning Concept
• The pin life cycle
• Configuring

1. File Hopping

   • File Hopping on arrival from outside dCache
   • File mode of replicated files
   • File Hopping managed by the PoolManager
   • File Hopping managed by the HoppingManager

2. Authorization in dCache

   • Basics
   • Configuration
   • Plug-ins
   • Using X.509 Certificates
   • CA Certificates
   • User Certificate
   • Host Certificate
   • VOMS Proxy Certificate
   • Configuration files
   • storage-authzdb
   • The gplazmalite-vorole-mapping plug-in
   • Authorizing a VO
   • The kpwd plug-in
   • The gridmap plug-in
   • gPlazma specific dCache configuration
   • Enabling Username/Password Access for WebDAV
   • gPlazma config example to work with authenticated webadmin

3. dCache as XRootd-Server

   • Setting up
   • Parameters
   • Quick tests
   • Copying files with xrdcp
   • Accessing files from within ROOT
   • xrootd security
   • Read-Write access
   • Permitting read/write access on selected directories
   • Token-based authorization
   • Strong authentication
   • Precedence of security mechanisms
   • Other configuration options
   • Third-party transfer

4. dCache as NFSv4.1 Server

   • Setting up
   • Configuring NFSv4.1 door with GSS-API support
   • Configuring principal-id mapping for NFS access

5. dCache Storage Resource Manager

   • Introduction
   • Configuring the srm service
   • The Basic Setup
   • Important srm configuration options
   • Utilization of Space Reservations for Data Storage
   • Properties of Space Reservation
   • dCache specific concepts
   • Activating SRM SpaceManager
   • Explicit and Implicit Space Reservations for Data Storage in dCache
   • SpaceManager configuration
   • SRM SpaceManager and Link Groups
   • Making a Space Reservation
   • SRM configuration for experts
   • Configuring the PostgreSQL Database
   • SRM or srm monitoring on a separate node
   • General SRM Concepts (for developers)
   • The SRM service
   • Space Management Functions
   • Data Transfer Functions
   • Request Status Functions
   • Directory Functions
   • Permission functions

6. The Statistics Service

   • The Basic Setup
   • The Statistics Web Page
   • Explanation of the File Format of the xxx.raw Files

7. The Billing Service

   • The billing log files
   • The billing database
   • Billing Histogram Data
   • Billing Records
   • Upgrading a Previous Installation

8. The Alarms Service

   • The Basic Setup
   • Configure where the alarms service is Running
   • Types of Alarms
   • Alarm Priority
   • Working with Alarms: Shell Commands
   • Working with Alarms: Admin Commands
   • Advanced Service Configuration: Enabling Automatic Cleanup
   • Advanced Service Configuration: Enabling Email Alerts
   • Miscellaneous Properties of the alarms Service
   • Alarms SPI-

9. dCache Frontend Service

10. dCache History Service

11. dCache Bulk Service

12. ACLs in dCache

    • Introduction
    • Configuring ACL support
    • Setting and getting ACLs
    • Accessing ACL over NFS mount

13. GLUE Info Provider

    • Internal collection of information
    • Configuring the info provider
    • Testing the info provider
    • Decommissioning the old info provider
    • Publishing dCache information
    • Troubleshooting BDII problems
    • Updating information

14. Stage Protection

    • Configuration of Stage Protection
    • Definition of the White List

• dCache as NFSv4.1 Server

  • Setting up
  • Configuring NFSv4.1 door with GSS-API support
  • Configuring principal-id mapping for NFS access
  • Managing group ids
  • Accessing directory tags

• The PinManager Service

  • The (Un-)Pinning Concept
  • The pin life cycle
  • Configuring

• The PoolManager Service

  • The Pool Selection Mechanism
  • Links
  • Examples
  • The Partition Manager
  • Overview
  • Managing Partitions
  • Using Partitions
  • Classic Partitions
  • Link Groups

• The Resilience Service

  • Configuring the Resilience Service
  • Resilience Home
  • Admin Commands
  • Tuning
  • Resilience’s View of Pool Status
  • Automatic Staging of Missing CUSTODIAL Replicas
  • Handling of QoS Transitions for Resilient Files
  • Some typical scenarios part - what happens when …?
  • Some typical scenarios part - how do I …?

• dCache Storage Resource Manager

  • Configuring the srm service
  • Utilization of Space Reservations for Data Storage
  • dCache specific concepts
  • SpaceManager configuration
  • SRM configuration for experts
  • Configuring the PostgreSQL Database
  • Tape Recall Scheduling
  • General SRM Concepts (for developers)

• The Statistics Service

  • The Basic Setup
  • The Statistics Web Page
  • Explanation of the File Format of the xxx.raw Files

• dCache as XRoot-Server

  • Setting up
  • Quick tests
  • Pool memory requirements
  • Xroot security
  • Third-party transfer

Overarching concepts:

• ACLs in dCache
• Introduction
• Configuring ACL support
• Setting and getting ACLs
• Accessing ACL over NFS mount

• Chimera

• Mounting Chimera through NFS
• Using dCap with a mounted file system
• Communicating with Chimera
• IDs
• Directory Tags
• Create, List and Read Directory Tags if the Namespace is not Mounted
• Create, List and Read Directory Tags if the Namespace is Mounted
• Directory Tags and Command Files
• Directory Tags for dCache
• Storage Class and Directory Tags

• Cell Message Passing

• Naming and addressing
• Routing
• Configuration

• The dCache Tertiary Storage System Interface

• Scope of this chapter
• Requirements for a tertiary storage system
• How dCache interacts with a Tertiary Storage System
• Details on the TSS-support executable
• Configuring pools to interact with a Tertiary Storage System
• How to Store-/Restore files via the Admin Interface
• How to monitor what’s going on

• File Hopping

• File Hopping on arrival from outside dCache
• File mode of replicated files
• File Hopping managed by the PoolManager
• File Hopping managed by the HoppingManager

• gPlazma: Authorization in dCache

• Basics
• Configuration
• Plug-ins
• Using X.509 Certificates
• CA Certificates
• User Certificate
• Host Certificate
• VOMS Proxy Certificate
• Configuration files
• storage-authzdb
• The gplazmalite-vorole-mapping plug-in
• Authorizing a VO
• The kpwd plug-in
• The gridmap plug-in
• gPlazma specific dCache configuration
• Enabling Username/Password Access for WebDAV
• Roles

• Kafka

• What do you need to download to enable Kafka in dCache?
• Apache Kafka Installation.

• User and Group Quota

• Stage Protection

• Configuration of Stage Protection
• Definition of the White List

• Using Space Reservations without SRM

  • The Space Reservation
  • The WriteToken tag
  • Copy a File into the WriteToken