1.9.12 Release Notes

dCache 1.9.12 focuses on:

Table of contents

Upgrade Instructions

Please use The Ultimate Golden Release Upgrade Guide to upgrade from 1.9.5 to 1.9.12.

Incompatibilities

Please consider the following changes when upgrading from a version before 1.9.12-1:

Compatibility

It is safe to mix pools of releases 1.9.5, 1.9.6, 1.9.7, 1.9.8, 1.9.9, 1.9.10, 1.9.11 and 1.9.12. Head nodes and doors must be upgraded to 1.9.12 together and cannot be mixed with head nodes or doors of releases before 1.9.12. Components of different 1.9.12 releases can be mixed freely.

Compatibility Matrix

We distinguish between pool components and head nodes. Any component which is not a pool is considered a head node, including doors. The following table shows compatibility between different releases.

1.9.12 Head1.9.12 Pool
Head 1.9.5nono
1.9.7nono
1.9.8nono
1.9.9nono
1.9.10nono
1.9.11nono
1.9.12yesyes
Pool 1.9.5yesyes
1.9.7yesyes
1.9.8yesyes
1.9.9yesyes
1.9.10yesyes
1.9.11yesyes
1.9.12yesyes

dCache 1.9.12-26

Miscellaneous

Updated JGlobus libraries to support jumbo proxies.

Changelog 1.9.12-25 to 1.9.12-26

dCache 1.9.12-25

Service: pool

Fixed timeout bug in migration module

Service: httpd

Fixed a problem with usage charts on the pool and pool groups pages not being shown when using certain settings for system locals.

Service: srm

Fixed a bug that prevented clean shutdown of the SRM.

Changelog 1.9.12-24 to 1.9.12-25

dCache 1.9.12-24

Miscellanious

Fixed the storage-info check which is needed for the migration from PNFS to chimera.

Changelog 1.9.12-23 to 1.9.12-24

dCache 1.9.12-23

Chimera

Fixed incorrect reply on database errors which may cause data loss on flush. The situation may occur if chimera's postgresql database is not available during flush to the HSM.

Service: xrootd

Fixed an NPE in xrootd mover.

Changelog 1.9.12-22 to 1.9.12-23

dCache 1.9.12-22

Incompatibility

Please consider the following change when upgrading from a version before 1.9.12-22

Due to a change in the pnfsmanager the chimera stored procedures have to be recreated. Execute psql -U chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql and substitute <user> by the proper database role.

This change does also require a synchronized upgrade of

i.e., all services using Chimera directly.

Chimera

Added support to change the mode, the group and the owner of a directory tag if the tag permission allows this operation.

Service: pool

Fixed a race condition in pools affecting srmCopy transfers.

Fixed formatting of date strings.

Fixed logging of erroneous context information in pools for HTTP and xrootd transfers. The problem was attempted fixed in an earlier release, however the prior fix did not work as intended.

Fixed xrootd vector read bug affecting reads of more than one file

Service: pnfsmanager

Fixed a null pointer exception that typically occurs when reading files migrated from PNFS.

Added an option to verify lookup permissions of the entire path.
For performance reasons dCache with PNFS only verified the lookup permissions of the directory containing the file system entry corresponding to the path. I.e., only the lookup permissions for the last parent dirctory of the path were enforced. For compatibility reasons Chimera inherited these semantics.
When this option is set to true, Chimera will verify the lookup permissions of all directories of a path.
This change requires that chimera users recreate the stored procedures by psql -U chimera -f /opt/d-cache/libexec/chimera/sql/pgsql-procedures.sql and substitute <user> by the proper database role. Please note that this new funcitionality is disabled by default. You can enable it by setting pnfsVerifyAllLookups=true.

Service: billing

Fixed formatting of date strings.

Added a NULL check to avoid NPE.

Service: httpd

Fixed formatting of date strings.

Service: gplazma

Fixed longest prefix match in vorolemap.

Service: srm

Fixed a potential race condition.

Fixed formatting of date strings.

Improved error messages if the host certificate, the host key or the ca certificates are are owned by a wrong user.

Service: xrootd

Fixed a potential race condition.

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

The properties xrootdAllowedPaths and xrootdIsReadOnly are forbidden now. The same functionality can be achieved by setting xrootdAllowedWritePaths and xrootdAllowedReadPaths. The configuration of existing deployments may have to be adjusted on upgrade.

Fixed a bug that prevented GSI xrootd and the gPlazma2 kpwd plugin from working together.

Service: ftp

Fixed a potential race condition.

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Service: gridftp

Fixed the issue that mkdir did not work with UberFTP.

Service: webdav

Fixed a potential race condition.

Fixed hanging downloads during or shortly after a file is being uploaded.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Service: nfsv

Fixed hanging downloads during or shortly after a file is being uploaded.

Service: nfsv4.1

Fixed a potential race condition.

Fixed bug which produced wrong TTL values when sending requests to pool manager and other cells.

Service: dcap

Fixed a potential race condition.

Miscellanious

Fixed messaging bug that potentially can affect all services.

Fix display of arrays for bean property and bean properties commands.

Reordered the classpath such that plugin jars are at the front of the classpath. This allows plugins to replace components already shipped with dCache.

Properties

New property: Forbidden properties (dCache will refuse to start when these properties are used):

Changelog 1.9.12-21 to 1.9.12-22

dCache 1.9.12-21

Service: info

Modified the logging such that the IllegalArgumentException, which used to be logged if info checks for the current status of a poolgroup, link ... that does not exist anymore, is now logged at debug level.

Fixed a bug that caused the info service to report cannot find StateChangeSet for path object if object is a non-existent part of the info tree.

Service: webdav

Fixed bugs that prevented downloads from working correctly. These were the lack of proper escaping of the redirect URL, of proper UTF-8 encoding of the displayable file name, and of marking the output as UTF-8 encoded and proper encoding of the URLs for the links in the listing, which prevented correctly displaying non-ASCII file- and directory names.

Provided a work-around for a Java bug that would cause any attempt to view a directory to fail with a 500 Internal Error page being shown, if this directory or a file inside the directory triggers the bug.

Chimera

Improving the identification which dCache service is using a particular database connection.

Service: gplazma

Fixed a bug that occurred when two gplazma cells were configured in the same domain and dCache tries to terminate one during startup.

Fixed XACML plugin by upgrading to latest versions of EMI voms and trustmanager libraries. Symptoms of the bug were NoClassDefFoundError error messages in the log files.

Increased available space from 10 Petabyte to 1 Exabyte.

Service: xrootd

Removed a restriction that prevented an xrootd authentication plugin and an xrootd authorization plugin to be used at the same time. Removed a limitation on xrootd authentication bucket size. The limit would have resulted in error messages like Caused by: java.io.IOException: Encountered a bucket with length 11088. Maximum allowed bucket length 10000.

Modified the ALICE authorization token plugin for xrootd to no longer require an authorization token for directory listing. Directory listing can thus be performed without explicit authorization. The change only affects users that use the ALICE authorization token plugin. The change was requested by the ALICE project.

Service: ftp

The different kinds of ftp doors now all support an ftp.read-only property. Defaults are as before.

Fixed a regression that prevented read-only only anonymous users in the plain FTP door.

Fixed Kerberos FTP, which could not start.

Service: pool

Fixed performance issues that affected FTP downloads of files bigger than 2GB. These issues have been observed on Solaris, but may have affected other operating systems too.

Fixed several xrootd releated race conditions and protocol violations in the pool.

Upraded Berkeley DB Java Edition to 4.1.21. This library may be used by pools to store meta data. The upgrade fixes numerous bugs in the library.

Fixed a null pointer exception in the xrootd mover. The exception was triggered by clients contacting a pool directly without being redirected by an xrootd door first.

Fixed several race conditions in the xrootd mover that would be triggered when the mover is killed. The symptoms of the race condition were stack traces in the pool log.

Fixed erroneous context information in log messages generated by xrootd and http movers.

Service: srm

Fixed mistake in error message that could lead to misleading and ambiguous error messages being logged.

Service: spacemanager

Enabling the spacemanager service with srmSpaceManagerEnabled=true used to cause a misconfiguration of the space manager. Fixed a test such that it is now possible to choose one of the allowed values (yes, true, on and enabled) to enable the spacemanager service without misconfiguration.

All domains

Fixed a race condition in the route command of the System cell, that would cause a ConcurrentModificationException to be logged.

Fixed a race condition in the ls command of the RoutingManager.

dcache script

A check to all dcache commands has been added to check whether the hostkey.pem is in PKCS#8 format, if present. If so, a warning is emitted and the command returns with an error code. The warning includes instructions describing how to fix the problem. Unfortunately this new feature introduces a regression that causes grep: /etc/grid-security/hostkey.pem: Permission denied to be written to the console when the dcache script is used by a user without read access to the hostkey. The bug can safely be ignored and will be fixed in 1.9.12-22.

dcache pool create

Fixed a bug in dcache pool create that prevented the script to change the owner of the pool directory.

dcache pool ls

Fixed a bug that prevented dcache pool ls from listing pools without a setup file.

dcache dump heap

Fixed a bug in dcache dump heap that prevented root from dumping the heap of non-root domains.

Changelog 1.9.12-20 to 1.9.12-21

dCache 1.9.12-20

Java

Java 7 is allowed as runtime platform.

Chimera

A potential problem, where chimera can leak a database connection has been fixed. Affected sites will see "list full" in their log file.

Changelog 1.9.12-19 to 1.9.12-20

dCache 1.9.12-19

NFS

Fixed behaviour on bad RPC messages.

Chimera

Switched to EXISTS statement to avoid duplicate key error in Postgres log files.

Service: webdav

Fixes a problem in the WebDAV door in which proxied reads would pause 5 to 10 seconds after the transfer.

Code

Fixed compatibility with Java7

Changelog 1.9.12-18 to 1.9.12-19

dCache 1.9.12-18

Service: xrootd

Fix incorrect fd access on READV.

Service: pool

Checksum scanner should survive orphan files.

Scripts: hsmcp

Fix storage class parsing and escape unsafe characters in URI.

Changelog 1.9.12-17 to 1.9.12-18

dCache 1.9.12-17

Chimera

Fixed problem where user was not able to rename a file into an existing one if the existing file had some data.

Service: ftp

Improved RFC 959 compliance w.r.t. handling of data channels and closing data channels.

Improved security by only listening for data channel connections on one address rather than on the wildcard address.

Improved GFD 21 (gridftp 1) compliance by no longer enforcing the concurrent stream limit (controlled by the gsiftpMaxStreamPerClient setting) on passive mode transfers. The setting now only applies to active mode transfers (which is the only case allowed by the specification).

Changed the interpretation of GFD 47 (gridftp 2) w.r.t. to the scope of parameters to GET and PUT commands. To our knowledge this change is supported by all known GridFTP 2 clients.

Service: gplazma

A bug in gPlazma 1, which was triggered when a user authenticates with Kerberos (NFS 4.1, Kerberised FTP, Kerberised dcap), has been fixed.

Changelog 1.9.12-16 to 1.9.12-17

dCache 1.9.12-16

Service: webdav

Fix buffering issues in the door.

Service: xrootd

Allow read-only mode.

Service: broadcast

Messages are no longer sent to disabled SpaceManager.

Service: gPlazma

kpwd: Fix support for updating password files via the dcache script.

Suppress log4j initialization in XACML plugin.

Changelog 1.9.12-15 to 1.9.12-16

dCache 1.9.12-15

Service: srm

Fix problem with 3rd party transfers with SRM version 2.

Service: pool

Fix classpath for pool reconstruct command.

Service: nfsv41

Reactivated expired session.

Service: xrootd, webdav

Allow embedded gPlazma to be used and allow the gPlazma cell name to use to be configurable.

dCache Script

Fix dCache script for Solaris machines.

Changelog 1.9.12-14 to 1.9.12-15

dCache 1.9.12-14

Service: nfsv3

Reduce log level of client errors.

Service: nfsv41

Reduce memory consumption for directory cache.

Service: dcap

Fix harmless null pointer exception.

Service: webdav

Avoid creation of temporary buffer files in /tmp. These were created during directory listing or when redirect on read was enabled. Instead the door will now use HTTP chunked encoding.

Service: gridftp, kerberosftp

Fix regression in directory listing which resulted in the DN or Kerberos principal rather than the mapped user name being used. The regression was introduced in 1.9.12-8.

Service: httpd

Fix match feature of Pool Selection Configuration page.

Service: xrootd, webdav

Allow embedded gPlazma to be used and allow the gPlazma cell name to use to be configurable.

Init scripts

The return code of dcache status now reflects the status of the service.

The dcache status command now lists orphaned domains for which the wrapper process has died. Previously such domains were reported as stopped and the dCache script would happily create a second instance. Now such instances can be killed by dcache stop.

Logging

Fix time stamp formatting in some log files.

Changelog 1.9.12-13 to 1.9.12-14

dCache 1.9.12-13

Service: pool

Fix another HTTP compliance issues related to range request. This time it is about using the correct reply code (416) when the client requests a range beyond the end of the file. This fix resolves a compatibility issue with ARC.

Fix 'dcache pool ls' to show the pool size stored in the pool's setup file, in case such a pool size exists.

Service: xrootd, webdav, ftp, gridftp, kerberosftp

Fix a harmless but noisy null pointer exception. The problem could affect FTP and WebDAV doors too, but was first observed in the xrootd door.

Service: webdav

Fix a null pointer exception being thrown while generating the error page.

Service: xrootd

Fix regression in Alice token authorization plugin. The plugin failed to load in 1.9.12-12.

Changelog 1.9.12-12 to 1.9.12-13

dCache 1.9.12-12

Service: pool

Fix several race conditions that could lead to stale hidden movers, TCP connection leaks and link count inconsistencies.

Service: admin

Add detection of terminal size. Some ssh clients do not transmit terminal resize messages. For those clients resizing the terminal after login to the admin service will break line editing.

Service: ftp, gridftp, kerberosftp

Make timeout of write mover creation configurable. The timeout is shared with read mover creation. The timeout can be configured by adjusting the gsiftpPoolTimeout property. A side effect is that the default timeout for write mover creation increases from 10 seconds to 10 minutes.

Service: xrootd

Refactored authorization plugin support to make plugins truly pluggable. These are now loaded through the Java ServiceLoader facility. This allows third parties to develop authorization and path mapping plugins the dCache xrootd implementation. Existing third party xrootd authorization plugins will have to be updated.

Miscellaneous

Fix well known cell name resolving when using JMS messaging.

Added a plugin directory to dCache. The following directories are scanned for plugins: /usr/share/dcache/plugins, /usr/local/share/dcache/plugins, /opt/d-cache/share/plugins. Plugins are represented as sub-directories to any of these directories. jar files are included on the Java classpath, properties files are parsed and added as default configuration values, and batch files are made recognized as service definition scripts.

Changelog 1.9.12-11 to 1.9.12-12

dCache 1.9.12-11

Service: dcap, gsidcap, kerberosdcap

Resolve an issue when the dcap client is in active mode (ie when the -A option is used) and the server cannot resolve the host name sent by the client (eg when the client is behind NAT). Instead the door now uses the remote address of the control channel when selecting an appropriate pool.

Improve logging in case of errors during GSI or Kerberos sessions.

Fix race condition that could occur during shutdown.

Service: pool

Ensure forward compatibility of the HTTP mover.

Fix several HTTP compliance issues related to range request (ie partial reads).

Service: ftp, gridftp, kerberosftp

Add glob support to the NLST command. This allows mget to work in some clients.

Fix race condition that could occur during shutdown.

Service: webdav

Improve logging on fatal errors. This change may help debug the source of the errors.

Service: poolmanager

Fix a race condition.

Service: admin

Fix several bugs related to line editing. Add history search (press Ctrl-R).

An unfortunate side effect of these fixes is that line editing capabilities need to know the terminal size. The admin door currently does not support determining the terminal size and thus always assumes that the terminal is 80 characters wide. Line editing will misbehave with other terminal sizes.

Fix listing of admin service in dcache services.

Service: pnfsmanager, nfsv3, nvfsv41

Fix name space corruption when moving directories (chimera only).

Fix RFC 5661 compliance of ACL delete permission check. A file or directory can now be deleted if the subject has the DELETE_CHILD permission on the parent directory, or the DELETE permission on the entry being deleted, or in case neither are specified if the subject has the ADD_FILE permission on the parent directory.

Service: srm

Improve robustness against ill formed srmLs requests.

Service: info

Fix logging of several erroneous warning and error messages.

Info provider

Fix GLUE2 compliance. Be sure you have at least v2.0.8 of glue-schema RPM installed on the node running the info provider.

Miscellaneous

Improve documentation in configuration defaults.

Improve white space processing around property values.

Fix race condition in getcellinfos command.

Add service definition for hopping manager. The name of the new service is hopping.

Changelog 1.9.12-10 to 1.9.12-11

dCache 1.9.12-10

Service: pool

Fixed bug in which failure to stage would leave behind a repository entry in state D. The only way to get rid of the entry was by restarting the poo.

Service: pinmanager

Improved robustness of pinmanager. In particular an out-of-memory problem has been fixed.

Changelog 1.9.12-9 to 1.9.12-10

dCache 1.9.12-9

Service: poolmanager

Fixed a race condition that caused a null pointer exception in rebalancer.

Fixed persistence of the slope setting.

Updated format of PoolManager.conf such that settings are persistet using the partition manager commands introduced in dCache 1.7. The legacy commands are still supported, but will be removed in a future version. Starting with version 1.9.12-9 the pool manager save command save in a format that will be compatible with future versions.

Services: ftp,gridftp,kerberosftp,xrootd,webdav

Use correct transaction duration for billing records.

Services: pool

Fixed the 'dcache pool reconstruct' on Solaris. Without the fix the command would fail with the error "File name too long".

Fixed URI generation in hsmcp.rb. The pool now also logs incorrect URIs returned by the HSM script.

Fixed HTTP range support. Added support for multiple ranges. Fixed path check. These fixes resolve compatibility issues with ARC and ROOT.

Service: billing

Fixed a race condition related to formating of timestamps.

Service: pnfsmanager

Fixed a bug in the PNFS backend that prevented listing through SRM.

Service: xrootd

Improved robustness of transfer timeout logic.

Changelog 1.9.12-8 to 1.9.12-9

dCache 1.9.12-8

Service: pnfsmanager

Fixed service instantiation with PNFS backend.

Service: pool

Avoid verbose logging of NoSuchElementException when listing movers. The fix possible solves similar problems for other operations and other services.

Avoid verbose logging of non-fatal DCAP client failures.

Service: ftp, gridftp, kerberosftp

Fixed a race condition that would cause the door to become unresponsive to client commands.

Fixed a potential denial of service problem.

Fixed a null pointer exception triggered by running the info command before the client was authenticated.

Avoid verbose logging when clients do not disconnect cleanly.

Service: spacemanager

Fixed pool selection cost model for systems with space manager enabled. Without the fix a system with space manager would perform pool selection with outdated cost information.

dcache script

Fixed file system size check for the dcache pool create command. It didn't actually check the file system size and in some cases it caused the operation to fail, leaving an incomplete pool behind.

Changelog 1.9.12-7 to 1.9.12-8

dCache 1.9.12-7

Service: gplazma

The configuration property gPlazmaNumberOfSimutaneousRequests was deprecated and replaced by gPlazmaNumberOfSimultaneousRequests.

Service: webdav

Generate themable HTML for 404 errors.

Service: nfsv41

Improved compatibility with early versions of rhel5 and MacOS.

Service: transfermanagers

Fixed deletion of the target file in case of transfer failures. Previous releases would fail with a stack trace logged.

Service: dcap

Reduced logging noise triggerd by file upload.

Service: ftp, gridftp, kerberosftp

Added a workaround for a bug in Globus Online.

Service: srm, webdav, webadmin

Downgraded to Jetty 7.2-2. Jetty 7.4.2 introduced in dCache 1.9.12-6 was the source of a regression in socket error handling.

Init script

Ensure that dcache start starts all domains even when some are already running.

Changelog 1.9.12-6 to 1.9.12-7

dCache 1.9.12-6

Service: nfsv3

Fixed a regression introduced in 1.9.12-5.

Fixed portmap registeration: Previous releases registered TCP even when only UDP was enabled.

Fixed attribute caching.

Service: acl

Make acladmin cell a well known cell (like it was in 1.9.5).

Service: dcap

Enable TCP_NODELAY on the control and data channels. Should reduce latency.

Service: ftp

Enable TCP_NODELAY on the control channel. Should reduce latency.

Service: admin

Enable TCP_NODELAY. Should reduce latency.

Service: webdav

Upgraded to Jetty 7.4.2.

Fixed error reporting. Previous versions reported all errors as "server error". Now the proper HTTP error code is reported.

Added HTML formatting and CSS styling of error messages. The template file has been replaced. The property webdav.templates.list is deprecated and has been replaced by webdav.templates.html.

Service: webadmin

Upgraded to Jetty 7.4.2.

Service: gplazma (version 2)

The kpwd plugin now uses defaults from /opt/d-cache/share/defaults/gplazma.properties.

Now supports ${...} substitution for properties defined in dcache.conf or the layout file.

Fixed reloading of the kpwd file.

Service: xrootd

Fixed recovery logic from internal message loss. Ensures that transfers are removed from the active transfers page of the httpd service.

Info provider

Fixed path to XSLT file in FHS packages.

1.9.5 migration script

Updated list of properties to rewrite during upgrade.

Changelog 1.9.12-5 to 1.9.12-6

dCache 1.9.12-5

Service: nfsv41

Fix a null pointer exception.

Adds optional support for NFS 3 in the NFS 4 door. Eventually the nfsv3 and nfsv41 doors will be merged into one.

Service: ftp

Support UTF-8 encoded file names independent of the local of the host system.

Output permission information by default in MLSD and MLST commands.

Fixed encoding of the UNIX.mode fact introduced in 1.9.12-4. The mode field is now encoded as an octal. This ensure compatibility with UberFTP and other clients.

Added authorization checks for various attributes accessed with MLSD and MLST commands.

Surpress the size fact for MLSD and MLST commands with the entity is a directory. This improves RFC 3659 compliance.

Publish GridFTP protocol version to the "Active Transfers" monitoring page.

Service: admin

Fix null pointer exception.

Service: pool

Improve error handling in case file deletion fails.

Chimera client

Fix readtag and writetag commands and added writefile command.

Configuration

Flags the properties httpdEnablePoolCollector and removeUnexistingEntriesOnFlush as obsolete. The properties have not been used since dCache 1.9.5.

1.9.5 migration script

Fixed translation of the dCacheSetup file. In previous releases only the first line of dCacheSetup was translated.

Logging

dCache configuration properties are now available as ${...} placeholders in logback.xml.

The log files /tmp/events.out and /tmp/trace.out have been replaced by per domain log files. This solves a permission problem when domains are run as different users.

Changelog 1.9.12-4 to 1.9.12-5

dCache 1.9.12-4

Service: nfsv41

Fixed attribute caching.

Service: xrootd

Kill mover if client or door timed out before the mover was started. Should reduce the number of mover timeouts.

Service: ftp

Implemented support for RFC 3659 UNIX facts (these are not specified by the RFC, but several clients implement these facts).

Improved compatibility with the UberFTP client (recursive directory listing now works).

Service: srm

Fixed a bug that would cause the message "can not find handler with id=nnnn" to be logged by the transfer manager after srmCopy requests successfully completed.

Service: webadmin

Fixed authorization.

Service: pnfsmanager

Fixed batch file for use with PNFS.

Service: gplazma

Fixed banning in gplazmalite-vorole-mapping plugin. Adding the entry:

"DN" -

will now ban the DN no matter whether it is used in combination with an FQAN or not. Note that banning in the gplazmalite-vorole-mapping plugin doesn't prevent other plugins from authorizing the request.

Fixed a bug which would cause a missing entry in storage-authzdb to authorize a user rather than reject the request. The problem would only occur if grid-vorolemap mapped a user to a user name that isn't mapped in storage-authzdb.

Service: pool

Fixed a conflict between the HTTP and xrootd mover that caused them to use the same TCP port when they shouldn't.

Fixed logging context handling. The bug could cause session IDs from transfers to erroneously be assigned other transfers and could even bleed through to other components.

The pool now no longer updates the last access time of a file during checksum scanning.

Fixed TCP buffer size handling in DCAP movers. The bug in some cases caused the requested buffer size to be incorrectly applied.

Enabled TCP keep alive in DCAP movers. This will ensure that DCAP movers eventually fail if the client abruptly disappears (eg due to a operating system crash or a network failure).

Service: pinmanager

Avoid unintended pool manager cost adjustment when pinning file. Also requires update of poolmanager.

Service: poolmanager

Avoid unintended pool manager cost adjustment when pinning file. Also requires update of pinmanager.

Service: info

Add context information to log messages.

Fixed bug that triggered "StateUpdate with zero updates encountered" log messages.

Removed the GlueSAAccessControlBaseRule work-around that publishes legacy values for broken clients. Any problems with lcg-infosites should be solved by upgrading to the latest version of the UI.

Service: admin

Fixed a couple of error messages.

Fixed an issue that could cause socket to be leaked.

Packaging

Fixed a number of RPM packaging errors in the EMI packages.

Other

Fixed System cell CLI help output.

Changelog 1.9.12-3 to 1.9.12-4

dCache 1.9.12-3

Service: info

The info service lists the network interfaces of each door in dCache. The DNS name of the interface is used as a key in the list. This causes problems when several interfaces share the same name and would often result in the host name being absent in the data published by the info provider. This problem has now been fixed by giving each network interface a unique name.

Service: httpd

Fixed a number of image files that were corrupted in the 1.9.12-2 release.

Service: pool

The lifetime of sticky bits are now reported in human readable format. The output of the 'rep sticky ls' command is affected.

Service: dcap

Minor logging related cleanups.

Service: nfsv3

Improved server side readdir caching.

Service: srm

Fixed processing of escape codes in SURLs and TURLs.

Out of memory handling

dCache triggers an automatic restart when a domain runs out of memory. We have observed that he restart sometimes fails, presumably because further out of memory errors are triggered during shutdown. A number of changes have been made to make the shutdown sequence more robust.

Since 1.9.12-1 dCache automatically generates heap dump files upon out of memory errors. A check has now been added to the startup script to check for the existing of these files and issue a warning.

Libraries

Several third party libraries have been upgraded. The upgrades represent minor releases that fix bugs.

RPM packaging

Ownership of the billing directory is now preserved accross upgrades.

Changelog 1.9.12-2 to 1.9.12-3

dCache 1.9.12-2

Service: srm

The following properties can now be specified separately for ls, put, get, bringonline and reserve requests: srmDatabaseEnabled, srmCleanPendingRequestsOnRestart, srmKeepRequestHistoryPeriod, srmExpiredRequestRemovalPeriod, srmRequestHistoryDatabaseEnabled, and srmStoreCompletedRequestsOnly. These properties replace the now deprecated properties srmJdbcMonitoringLogEnabled, srmNumberOfDaysInDatabaseHistory, srmOldRequestRemovalPeriodSeconds, srmJdbcSaveCompletedRequestsOnly, srmJdbcEnabled, and srmCleanPendingRequestsOnRestart.

Service: pool

Fix the processing of the waitForFiles property. The property did not have the inteded effect.

Migration script

Fix a typo that caused the script to fail in 1.9.12-1.

Added translation of SRM properties deprecated in 1.9.12-2.

Added support for moving admin authorization files to new location.

dCache now refuses to start if it detects a 1.9.5 configuration. This prevents that dCache is accidentally started before the configuration has been migrated.

Service: gplazma

Propagate the read-only flag back to doors. In recent versions the flag was ignored.

Several gPlazma 2 related changes were made, but gPlazma 2 is still under development and should not be used in this version. We expect gPlazma 2 to be production ready in 1.9.12-3.

Service: pnfsmanager

Reintroduce check that prevents existing checksums from being altered. The check was present in 1.9.5 but was lost in later versions.

Service: admin

SSH keys have moved from /opt/d-cache/config/ to /opt/d-cache/etc/. The migrate-from-1.9.5.sh script has been updated to move the files. However when upgrading from a version newer than 1.9.5 the SSH keys have to be moved by hand.

Location manager

Fixed a problem in which dCacheDomain would fail to start if /opt/d-cache/etc/lm.config could not be created. This would typically happen if dCacheDomain is started as a user different from root.

The location of lm.config is now configurable.

Service: httpd

Static content used by the monitoring service has been moved to /opt/d-cache/share/httpd/static/.

Service: webdav

WebDAV doors now report the DN to billing when files are deleted.

Changelog 1.9.12-1 to 1.9.12-2

dCache 1.9.12-1

Service: pool

The /opt/d-cache/bin/dcache pool commands have been updated to provide an ls subcommand to list configured pools. The create subcommand has been extended to add the new pool directly to the layout file. The command no longer creates the pool setup file and instead adds the pool specific configuration to the layout file. The command thus provides the functionallity of both the create and add commands found in some earlier releases.

The migration module was extended to reduce interference between multiple migration jobs running on the same pool. The migration module now respects the pool mode on the target pool.

Service: srm

The SRM used to have the ability to periodically submit vacuum commands to the database. This command is PostgreSQL specific and thus tied the SRM to PostgreSQL. PostgreSQL has for quite a while supported auto vacuuming, and manually executing the vacuum command is no longer necessary. We have therfore removed the submission of vacuum commands from dCache. The configuration parameters srmVacuum and srmVacuumPeriod are obsolete.

Some SRM operations may be processed synchronously or asynchronously, at the server's discretion. Since 1.9.10 dCache can start to process such requests synchronously and, if this is taking too long, reply asynchronously and continue to work on the operation in the background. In dCache 1.9.12 this behaviour is enabled by default for get, put, bring online, and list operations. The default switch over time is 1 second. The old behaviour can be restored by setting the srm*SwitchToAsynchronousModeDelay parameters.

The RemoteHttpTransferManager cell has been removed. The functionality has been merged into the RemoteGsiftpTransferManager cell, which has in turn been renamed to RemoteTransferManager. The SRM was extended to support third party srmCopy transfers from HTTP resources. Third party upload to HTTP resources and support for HTTPS is not available yet in the SRM.

Service: dcap, gsidcap, kerberosdcap

DCAP supports prestaging from tape. The DCAP door used to rely on a stager service to perform the staging. In dCache 1.9.12 the DCAP door instead uses the PinManager to trigger staging. The dummy-prestager service is obsolete and can be deleted from the layout.

Service: pnfsmanager

The PNFS manager has been restructured. One consquence is that the output of the info command in the cell admin interface has changed slightly: It is now grouped into sections in line with how it is done in pool manager and pools. Third party scripts may need to be adjusted.

Service: webdav

WebDAV doors now submit remove entries to billing when files are deleted.

The HTML rendering of a directory listing now includes a column containing the file size. The HTML is customizable through a template file. As a consequence the properties webdav.images.logo, webdav.images.directory, webdav.images.file, webdav.style.css, webdavLogoPath, webdavDirIconPath, webdavFileIconPath, webdavCssPath, dirIconPath, fileIconPath, and cssPath are obsolete. Please read the document embedded in the default template file /opt/d-cache/share/webdav/templates/list.stg for details on customizing the look and feel.

The property webdavAnonymousListing was introduced to prevent unauthorized users from listing directories.

The property webdavProtocol was extended with a third option, https-jglobus, to enable HTTPS support based on the JGlobus library. The advantages compared to the https option are that host and CA certificates are read and refreshed directly from /etc/grid-security/ and that proxy certificates (including VOMS certificates) can be used for client authentication.

Info Provider

The info provider is an XSLT script that transforms the output of the info service into GLUE compliant LDIF. The XSLT script used to be stored in /opt/d-cache/etc/glue-1.3.xml and used to contain both the transformation logic and local configuration parameters.

In dCache 1.9.12 the info provider has been extended to support GLUE 2.0 in addition to GLUE 1.3, and it has been restructured to separate configuration parameters from the transformation logic.

The old info provider file, /opt/d-cache/etc/glue-1.3.xml, is obsolete. All site specific configuration is done within the file /opt/d-cache/etc/info-provider.xml. The configuration parameters httpHost, xsltProcessor, xylophoneConfigurationDir, xylophoneConfigurationFile, xylophoneXSLTDir, and saxonDir are deprecated and have been replaced by other parameters. The configuration parameter httpPort is obsolete. The parameter info-provider.publish specifies whether to publish GLUE 1.3, 2.0, or both. See /opt/d-cache/share/defaults/info-provider.properties for details.

Service: pinmanager

The pin manager is responsible for staging and pinning files on behalf of SRM and DCAP doors. In this release the pin manager has been reimplemented from scratch. The design has been significantly simplified to increase performance and robustness of this critical component.

The database schema is separate from the old implementation's schema. The new schema is created the first time the pin manager is started. Existing pins are automatically imported if the same database is used. We strongly recommend using the same database. The tables of the old pin manager are not deleted, however downgrading is not possible without loosing the pins that were created with the new pin manager.

Notice that the migration from the old pin manager requires at least PostgreSQL 8.3. The migration will fail when using older versions.

The command line interface of the pin manager has changed slightly and third party scripts may have to be updated.

Service: poolmanager

Pool manager has seen a fair amount of cleaning in this release, however user visible changes should be small. The most significant change is that pool manager no longer verifies the correctness of location information stored in the name space, that is, on which pools a file is stored. Instead pool manager expects doors to verify the information and deal with any stale locations. This change eliminates a round trip to the pool and thus reduces the latency of opening files.

Pool mode changes are now logged at warning level by pool manager.

Doors and DNS lookups

Most doors and movers have been updated to avoid DNS lookups and reverse lookups that slow down transfers. Instead IP addresses are passed around. This may break existing deployments on multi-homed hosts. When this happens doors may need to be bound explicitly to particular network interfaces.

DCAP, FTP, SSH and admin doors now support the listen property to bind the door to a particular IP address.

Service: gplazma

The get mapping cell command was added. The command was present in earlier releases, but was dropped when the gPlazma cell was reimplemented in 1.9.9.

Database Support

The /opt/d-cache/bin/dcache script has been updated with several new commands related to databases used by various services.

The subcommand database ls lists the databases used by services on this host and is supported by all services that use an external database.

Other database subcommands only apply to services using the LiquiBase schema management library. Currently pin manager is the only service that uses this library. Please consult the man page for details about these commands.

Scripts

Besides changes to the pool and database commands, /opt/d-cache/bin/dcache now supports the services command. The command is similar to the services command found in dCache 1.9.5: It lists all configured services and the domains they belong to. The status command was extended to provide additional information. The new check-config command provides diagnostic information about obsolete and deprecated properties used.

The migration script was moved and renamed to /opt/d-cache/libexec/migrate-from-1.9.5.sh. It was updated to follow best practice by using the host name as the name of the layout. The new version does its best to remove lines copied verbatime from the original template into dCacheSetup. Many deprecated properties are automatically renamed to the new names. At the end of the migration the script invokes the check-config command to report about any deprecated or obsolete properties being used.

The /opt/d-cache/install/install.sh script is no longer needed and has been removed. The remaining two scripts in /opt/d-cache/install/ have been moved to /opt/d-cache/libexec/.

Configuration Properties

Lots of minor issues have been fixed in the configuration system. Many properties that were already deprecated or obsoleted in earlier releases are now marked as such and generate errors or warnings. Other properties that were removed during the transition to the new configiration system in 1.9.7 have been reintroduced as deprecated properties. Default declarations of most services have been moved from batch files and into properties files in /opt/d-cache/share/defaults/.

Changelog 1.9.11-1 to 1.9.12-1

Crossed out entries have been merged into the 1.9.11 branch.