dCache 2.3 Release Notes

Highlights:

Table of contents

Upgrade Instructions

A long time ago the original dCache namespace PNFS has been replaced by the namespace called Chimera. Chimera has a lot of advantages against PNFS. The performance is better, the namespace transactions are more secure in terms of interruption and it is possible to do nice database queries for example for accounting info. Almost all sites have by now switched to Chimera so we decided to completely drop PNFS. If your site has not yet migrated from PNFS to Chimera you need to do so before updating to dCache 2.3.

Incompatibilities

Please consider the following changes when upgrading from a version before dCache 2.3.0:

PNFS no longer supported

The namespace PNFS is no longer supported. You need to migrate to chimera before upgrading to dCache 2.3.

Please note that:

Other important changes

Compatibility

All head nodes must be running dCache 2.3 and are compatible with pool nodes of 2.2 releases, but if NFSv4.1 or HTTP PUT redirect is used then the respective pools must be 2.3 pools. Components of different 2.3 releases can be mixed freely.

dCache 2.3.2

Service: pool

Fixed a race condition in pools affecting srmCopy transfers.

Fixed a bug which caused a NPE for pool to pool transfers when the pool setup file did not exist.

Fixed logging of erroneous context information in pools for HTTP and xrootd transfers. The problem was attempted fixed in an earlier release, however the prior fix did not work as intended.

Fixed xrootd vector read bug affecting reads of more than one file

Fixed formatting of date strings.

Fixed more potential race conditions.

Service: poolmanager

Fixed wrong conversion of network masks for converted IPv6 addresses.

Service: pnfsmanager

Fixed a null pointer exception that typically occurs when reading files migrated from PNFS.

Service: transfermanagers

Fixed bug that caused an error message, when srmDbLogEnabled was set to false.

Service: billing, httpd

Fixed formatting of date strings.

Fixed potential race conditions.

Fixed error reporting.

Service: replica manager

Fixed potential race conditions.

Service: gplazma

Fixed longest prefix match in vorolemap.

Service: srm

Improved error messages if the host certificate, the host key or the ca certificates are are owned by a wrong user.

Fixed formatting of date strings.

Fixed a potential race condition.

Service: xrootd

The properties xrootdAllowedPaths and xrootdIsReadOnly are forbidden now. The same functionality can be achieved by setting xrootdAllowedWritePaths and xrootdAllowedReadPaths. The configuration of existing deployments may have to be adjusted on upgrade.

Fixed a bug that prevented GSI xrootd and the gPlazma2 kpwd plugin from working together.

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Fixed a potential race condition.

Fixed root path interpretation to be the same for all users.

Service: ftp

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Fixed potential race conditions.

Service: webdav

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Fixed a potential race condition.

Service: nfs

Fixed hanging downloads during or shortly after a file is being uploaded.

Service: nfsv4.1

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Fixed a potential race condition.

Service: dcap

Fixed a potential race condition.

Miscellanious

Fixed messaging bug that potentially can affect all services.

Fix display of arrays for bean property and bean properties commands.

Reordered the classpath such that plugin jars are at the front of the classpath. This allows plugins to replace components already shipped with dCache.

Moved trace and events log files back to /tmp and set the default log level for those files to off.

Properties

Forbidden properties (dCache will refuse to start when these properties are used):

Changelog 2.3.1 to 2.3.2

dCache 2.3.1

Service: pool

Fixed a bug that would cause the wrong PNFS ID and other contextual information to be logged for xrootd and HTTP transfers.

Fixed a regression in the recovery logic executed during pool startup. The regression would in some cases prevent broken files from being recovered.

Modified the recovery logic such that storage info is always fetched from pnfs manager when a file is recovered. Earlier releases would only fetch the storage info from pnfs manager if the storage info was lost at the pool.

Service: xrootd

Fixed a regression bug that caused a NullPointerException when clients attempted to list directories. The regression was introduced in dCache 2.2.0.

Modified the ALICE authorization token plugin for xrootd to no longer require an authorization token for directory listing. Directory listing can thus be performed without explicit authorization. The change only affects users that use the ALICE authorization token plugin. The change was requested by the ALICE project.

Service: ftp

Fixed a regression that prevented read-only only anonymous users in the plain FTP door.

Service: gplazma

Fixed a regression in the XACML plugin that would cause it to fail with IllegalArgumentException.

All domains

Fixed a race condition in the route command of the System cell, that would cause a ConcurrentModificationException to be logged.

Fixed a race condition in the ls command of RoutingManager.

Fixed a bug that caused excessive logging in RoutingMgr when debug level logging was enabled. As a side effect the fix reduces CPU consumption of the RoutingMgr even when debug logging is not enabled.

Fixed a bug that caused options with a single dash to be ignored by the route command of System cell.

dcache script

dcache pool create

Fixed a bug in dcache pool create that prevented the script to change the owner of the pool directory.

dcache dump heap

Fixed a bug in dcache dump heap that prevented root from dumping the heap of non-root domains.

dcache kpwd

Fixed a bug in dcache kpwd that prevented UID and GID 0 to be used.

Changelog 2.3.0 to 2.3.1

dCache 2.3.0

Service: gplazma

Service: nfs

Switch to grizzly 2.2.9 with many improvements and bug fixes including internal IO buffers recycling.

Service: SRM

Chimera

Service: pnfsmanager, NFSv3, NFSv4.1

A potential problem, where Chimera can leak a database connection has been fixed. Affected sites will see "list full" in their log file.

Service: webdav

Service: webdav/pool

Added an option to redirect HTTP uploads. When enabled, data is transferred directly from the client to the pool without using the door as a proxy. 2.3 pools are required to use this new feature. Not all clients support this feature, but it should be safe to enable it even if the client does not support it. Set the property webdav.redirect.on-write=false to disable this feature.

The pool now provides the content-location header in the response to HTTP GET requests.

Service: billing

Modified billing, such that empty messages will be skipped.

Fixed typos in the sql scripts which are needed for migration of billing info from previous installations (migrate_from_messageinfo.sql and migrate_from_preexistent.sql).

Service: billing, httpd

Service: billing, pinmanager, transfermanagers

Updraded to version 3 of the DataNucleus library.

Service: Replica Manager

Service: webadmin/HTTPD

Service: pool

Fixed several HTTP mover bugs causing wrong, incomplete, or duplicated error responses to be generated. Symptoms would range from hanging clients, to failed transfers without any indication of error.

NFSv4.1 mover on pools may include all host interfaces into multi-path reply. This allows a client to choose the most appropriate network end point.

Fixed several xrootd releated race conditions and protocol violations in the pool.

Service: admin

The server ssh2 key is created during installation of the package for rpm and deb based systems if it does not already exist.

Service: dcap

Service: gplazma, gridftp, srm, xrootd, dcap

Introduced new properties to make the path to /etc/grid-security and subdirectories configurable. Some doors already had properties for setting this path. These properties, namely hostCertificateRefreshPeriod, trustAnchorRefreshPeriod and verifyHostCertificateChain, are replaced by the generic properties grid.hostcert.refresh, grid.ca.refresh and grid.hostcert.verify that apply to all of dCache.

bootloader

Modifying the test for the validity of the layout file, such that trailing white spaces are allowed.

dcache script

The robustness of starting and stopping dCache has been improved.

A bug that caused grep: /etc/grid-security/hostkey.pem: Permission denied to be printed has been fixed.

dcache services

The services broadcast, cns, hopping, topo, dir, transfermanagers, srm-loginbroker can now be listed by the dcache script using the command dcache services.

dcache database update

Chimera's DB schema can now be managed by dcache database commands. On update/install admins can use the dcache database update command to create a new schema or update existing one. Please note, that only the DB schema is created. The database still has to be created separately. During the schema update some error messages may be printed if your installation already has required changes applied. It is safe to run the dcache database update command multiple times as the DB keeps track of the applied changes.

dcache database ls

The dcache script lists the databases for billing, nfsv41, srm, transfermanagers using the command dcache database ls.

dcache ports

The command dcache ports shows the ports used by the NFSv4.1 service.

dcache pool ls

Fixed a problem with dcache pool ls that prevented it from listing pools without a setup file.

dcache pool create

Fixed an issue with the dcache pool create command that would cause it to leave behind an empty pool directory after having failed with an error.

dcache import hostcert

Fixed a problem with dcache import hostcert in which repeated import would fail with a permission error.

rpm

Properties

New properties: Deprecated properties (Properties which are not used anymore, but still work and will become obsolete or forbidden): Obsolete properties (Properties which are not used anymore and have no effect on dCache):

Miscellaneous

NFSv3 can only run as root. As dCache now runs by default as the user dcache we removed the NFSv3 service from the default single.conf layout. As the acl services is obsolete now it has been removed from the default head.conf layout.

Changelog 2.2.0 to 2.3.0

Crossed out entries have been merged into the 2.2 branch.