release notes | Book: 2.10, 2.11, 2.12, 2.13, 2.14, 2.15, 2.16, 3.0, 3.1, 3.2 | Wiki | Q&A black_bg
Web: Multi-page, Single page | PDF: A4-size, Letter-size | eBook: epub black_bg
Preface
Prev   Next
The dCache Book > Preface

Preface

Table of Contents

Who should read this book?
Minimum System Requirements?
What is inside?
Looking for help?

Welcome to the dCache. dCache is a distributed storage solution for storing huge amounts of data without a hard limit, used to provide storage in the petabyte range. Therefore it qualifies as the storage system supporting data intensive experiments.

dCache is a joined effort between the Deutsches Elektronen-Synchrotron (DESY) in Hamburg, Nordic Data Grid Facility (NDGF based in Copenhagen), the Fermi National Accelerator Laboratory near Chicago with significant distributions and support from the University of California, San Diego, INFN, Bari as well as Rutherford Appleton Laboratory, UK and CERN in Geneva.

dCache can use hierarchical storage management (e.g., hard disk and tape), provides mechanisms to automatically increase performance and balance loads, increase resilience and availability. It also supplies advanced control systems to manage data as well as data flows. Normal filesystem (btrfs, ext4, XFS, ZFS) is used to store data on storage nodes. There are several ways of accessing data stored in dCache:

  • NFS 4.1 (Chimera)

  • HTTP and WebDAV

  • GridFTP (GSI-FTP)

  • xrootd

  • SRM (versions 1.1 and 2.2)

  • dCap and GSIdCap

dCache supports certificate based authentication through the Grid Security Infrastructure used in GSI-FTP, GSIdCap transfer protocols and the SRM management protocol. Certificate authentication is also available for HTTP and WebDAV. dCache also supports fine-grain authorization with support for POSIX file permissions and NFS-style access control lists. Other features of dCache are:

  • Resilience and high availability can be implemented in different ways by having multiple replicas of the same files.

  • Easy migration of data via the migration module.

  • A powerful cost calculation system that allows to control the data flow (reading and writing from/to pools, between pools and also between pools and tape).

  • Load balancing and performance tuning by hot pool replication (via cost calculation and replicas created by pool-to-pool-transfers).

  • Space management and support for space tokens.

  • Garbage collection of replicas, depending on their flags, age, et cetera.

  • Detailed logging and debugging as well as accounting and statistics.

  • XML information provider with detailed live information about the cluster.

  • Scriptable adminstration interface with a terminal-based front-end.

  • Web-interface with live information of the most important information.

  • Ensuring data integrity through checksumming.

dCache / SRM can transparently manage data distributed among dozens of disk storage nodes (sometimes distributed over several countries). The system has shown to significantly improve the efficiency of connected tape storage systems, by caching, gather and flush and scheduled staging techniques. Furthermore, it optimizes the throughput to and from data clients by dynamically replicating datasets on the detection of load hot spots. The system is tolerant against failures of its data servers, which allows administrators to deploy commodity disk storage components.

Access to the data is provided by various standard protocols. Furthermore the software comes with an implementation of the Storage Resource Manager protocol (SRM), which is an open standard for grid middleware to communicate with site specific storage fabrics.

Prev   Next
The dCache Book Home Who should read this book?