dCache 2.14 Release Notes

Executive summary

Highlights from this release:

Significant performance and storage space improvements in Chimera.
Almost all certificiate handling code has been ported to the EMI CANL library rather than JGlobus. Provides OCSP support and EUGRIDPMA name space support.
Concurrent request processing in gPlazma.
Pool group patterns in admin shell.
Improved UberFTP support.
WebDAV door can report file locality.
Fair share scheduling for SRM transfers.
SRM database managed by liquibase.
Configurable kXR_Qconfig support for xrootd.

Incompatibilities

Strict RFC 2818 host name checking in SRM/HTTP clients - this in particular affects server side srmCopy and WebDAV COPY. It is expected that this will break with server side copy to/from some WLCG sites. Globus Toolkit enables the same strict mode by default by the end of the year, so we expect the affected sites to get new host certificates soon. Additionally, server side copy is not used much, so it is likely not be a big issue for dCache.
Significant schema changes in Chimera. Upon upgrade the schema will be updated, which may take a while for large sites. To downgrade, the schema changes have to be rolled back with the appropriate command in dCache before downgrading dCache.
PnfsManager now uses a single request queue. This could possibly affected third-party monitoring scripts that scrape the info output.
All forbidden and obsolete properties in 2.13 have been removed and all deprecated properties in 2.13 have been marked obsolete.
RC4 ciphers are banned by default and the option to ban Diffie Hellman key exchange if broken has been removed.
gPlazma now instantiates plugins separately for every use in gplazma.conf. This means that the configuration has to be specified for all lines, not just the first. This provides the flexibility to use the same plugin several times with different configurations.
The WebDAV door enables BASIC authentication by default over HTTPS.
Pools always compute checksums on the fly and the ontransfer checksum policy has been dropped.
The SRM by default uses a fair share scheduler for scheduling transfers. Thus the observed behaviour under load may have changed.
SRM 1 support is deprecated and disabled by default.
SRM scheduler states have been simplified. Third party monitoring scripts that track request states may have to be updated.
SRM no longer retries SRM requests internally. Failures are propagated to the client to provide fail-fast behaviour.
SRM no longer resolves host names in SURLs when determining whether they are local to this dCache instance or not. In particular, this may affect sites that use DNS aliases.
The SRM database schema is now managed by liquibase and the schema is updated upon upgrade. To downgrade, the schema changes have to be rolled back with the appropriate command in dCache before downgrading dCache. As a consequence, user information for existing SRM requests will be lost from the database. Third party scripts that access the SRM database directly may have to be updated.

Release 2.14.51

admin

While migration move tasks on pools were working correctly, for migration info command an error occurred, that the current user (root) wasn’t allowed to execute anything (due to missing ACLs). This is now fixed.

Changelog 2.14.50..2.14.51

68975a9: [maven-release-plugin] prepare release 2.14.51
ee228bb: admin: Fix Inconsistent ACL enforcement, RT 9207
1421637: [maven-release-plugin] prepare for next development iteration

Release 2.14.50

httpd

The “Disk Space Usage” webpage (/usageInfo) contains a table showing information about each pool in the dCache cluster. The “Layout” column showed the capacity usage graphically, with different colours showing how much of that pool’s capacity is being used for different tasks. This release fixes the Layout heading to describe a previously undocumented colour.

Changelog 2.14.49..2.14.50

ad1ba42: [maven-release-plugin] prepare release 2.14.50
36e6e64: [maven-release-plugin] prepare for next development iteration
37a50d3: httpd: Fixed table headers in usageInfo

Release 2.14.49

pool

If the communication between a pool and PnfsManager times out, the error message is not well suited to diagnosing the problem: Failed to instantiate mover due to unsupported checksum type: Request to [>PnfsManager@local] timed out. The checksum type is not playing an important role here. Hence, this patch updates the error message.

Changelog 2.14.48..2.14.49

b30310c225: [maven-release-plugin] prepare release 2.14.49
1f225db503: pool: fix error message on timeout
4df7bf679d: [maven-release-plugin] prepare for next development iteration

Release 2.14.48

webdav

A recent update, commit 5abc0e1c, improved the behaviour of the Milton WebDAV libraries if an IOException occurs during an upload. That patch, unfortunately, did not address all issues, and when non-spec-conformant clients are used against dCache, stacktraces can be triggered.

This patch corrects that behaviour. Also, in case of errors, the error code returned in case of any problems was changed from 400 to 500, which should signal cliens that they are free to retry the transfer after a timeout.

Changelog 2.14.47..2.14.48

f35052b139: [maven-release-plugin] prepare release 2.14.48
4e71787f3e: webdav: make Milton work-around more robust
8207aefbff: [maven-release-plugin] prepare for next development iteration

Release 2.14.47

Changes affecting multiple services

The version of the PostgreSQL driver used by dCache internally was brought up to 9.4.1212. This fixes the issue described in liquibase bug 2939.

system-test

The system-test module, used for demonstration or testing purposes, comes with a built-in X.509 infrastructure. With this release, expired certificates are replaced by new ones.

Changelog 2.14.46..2.14.47

07e8ec485b: [maven-release-plugin] prepare release 2.14.47
77d71f072f: system-test: update disposable-CA generated credentials
a523fb322f: postgresql driver: update version to 9.4.1212
26e6beceb1: [maven-release-plugin] prepare for next development iteration

Release 2.14.46

chimera

There was an issue with a symbolic link to a directory where destination where destination contained trailing slash. This is now fixed.

Changelog 2.14.45..2.14.46

9fbbeb2: [maven-release-plugin] prepare release 2.14.46
0890408: [maven-release-plugin] prepare for next development iteration
c478efd: chimera : handle empty paths elements in path2inode stored procedure

Release 2.14.45

chimera

The current release fixed database query for storing multiple checksums for a file.

ftp

The Socket read method may return zero to indicate that no bytes were read. Although this is not an error, such occurances will result in a transfer failing.

This is now fixed.

Changelog 2.14.44..2.14.45

3e285f6: [maven-release-plugin] prepare release 2.14.45
7431759: ftp: prevent execution of most commands when unwrapped
5799780: chimera: fixed database query for storing multiple checksums for a file.
5116a7c: ftp: do not fail proxy transfer if read returns zero bytes
76250c3: [maven-release-plugin] prepare for next development iteration

Release 2.14.44

ftp

The current release added implementation of MLSC. As a result Globus is able to query the contents of dCache directories using FTP and without creating additional TCP connections.

Changelog 2.14.43..2.14.44

6e65fa8: [maven-release-plugin] prepare release 2.14.44
bc0bc02: ftp: implement the MLSC command
1943889: [maven-release-plugin] prepare for next development iteration

Release 2.14.43

ftp

The current release improves compatibility between dCache FTP client and Globus GridFTP server.

srm

During an ATLAS stress-test of tape recalls, it was discovered that various sites had relatively short request lifetimes. However, the SRM spec provides the opportunity for the server to inform the client (FTS, in this case) of what lifetime a request actually has. The current release includes the requests remaining lifetime in the response from the server.

The current release improves the documentation to help Admins to have a better understanding how to configure dCache correctly.

Changelog 2.14.42..2.14.43

810b42c: [maven-release-plugin] prepare release 2.14.43
f7cd009: ftp: add support for paths relative to home directory
8249c4c: ftp: Add support for SITE WHOAMI command
e6f1e5f: ftp: update parsing of CLIENTINFO command
d73e230: srm: include remaining request lifetime in various responses
de71502: srm: update srm request.*.lifetime configuration properties documentation
db2c488: ftp: modify facts describing namespace ownership
344c0b5: ftp: add support for SITE TASKID command
1d6fabb: ftp: add initial support for checksum performance markers
90880cb: ftp: show SIZE facts for directories
ca29a64: ftp: add support in OPTS RETR for specifying performance marker frequency
9f51f16: [maven-release-plugin] prepare for next development iteration

Release 2.14.42

xrootd

In https://github.com/xrootd/xrootd/issues/459, it became apparent that dCache could improve xrdcp compatibility by sending checksum information in lower case. This release contains this change, which should improve xrootd operations.

Changelog 2.14.41..2.14.42

5c18fda: [maven-release-plugin] prepare release 2.14.42
55f8cbd: xrootd : use lower case for checksum algorithm names when replying to checksum queries.
6e7c89c: [maven-release-plugin] prepare for next development iteration

Release 2.14.41

srm

The SRM code has been made more robust against races between file deletions and copies.

systemtest

The ‘system-test’ script was updated to ensure anonymous dcap tests succeed.

Changelog 2.14.40..2.14.41

4158477: [maven-release-plugin] prepare release 2.14.41
6c40d2f: systemtest: allow anonymous dcap activity
5a1279f: srm: fix recovery procedure in internal copy if source is deleted
02430b6: [maven-release-plugin] prepare for next development iteration

Release 2.14.40

cleaner

Users reported that they wanted to see space freed up by cleaner processes to be reported as free as soon as possible. This patch sends notifications about freed up space more often, resulting in quicker status updates.

pool

A problem was fixed that could cause the csm check to fail on pools containing broken files.

Changelog 2.14.39..2.14.40

4146bf6: [maven-release-plugin] prepare release 2.14.40
60764e5: cleaner: Send notification more often
02e9d86: pool: Fix csm check command in the pressence of broken files
124fb56: [maven-release-plugin] prepare for next development iteration

Release 2.14.39

ftpclient

The current release improves compatibility between dCache FTP client and Globus GridFTP server.

Changelog 2.14.38..2.14.39

f04f1e7: [maven-release-plugin] prepare release 2.14.39
be1932c: ftpclient: fix multiline ADAT reponses
fa93188: ftpclient: encrypt SITE CLIENTINFO command
6b2cf9b: [maven-release-plugin] prepare for next development iteration

Release 2.14.38

dcap

Connections of non-DCAP clients to a dCache no longer result in stack-traces in the logs.

poolmanager

PoolManager was updated to properly handle the dcache.authz.staging.pep and dcache.authz.staging parameters. This allows to enable stage protection properly.

Changelog 2.14.37..2.14.38

d1c87a9: [maven-release-plugin] prepare release 2.14.38
3aeb652: dcap: don’t create stack-trace if tunnel fails due to bad client
051f57f: PoolManager : stage protection, fix error in stage.fragment
fa39df6: [maven-release-plugin] prepare for next development iteration

Release 2.14.37

Changes affecting multiple services

Internal notification processing between cleaners and Pin Manager, Replica Manager and Space Manager was improved and runs quicker now.

admin

A bug in the Admin service that caused it to not detect certain error replies from other components was fixed.

billing

If a dCache instance was shut down while the billing service was in the middle of a refresh, an exception was logged and shutdown was delayed. A change in exception handling fixes this rare scenario, ensuring a quick shutdown and no unnecessary log entries.

dcap

This release makes it possible for admins to ban outdated, problematic versions of dcap clients. Some old client versions contain a bug that causes the client to make unsatisfiable requests to a pool with no way for dCache to reject the request: the client will simply retry.

The client version limits are exposed using the new configuration property dcap.limits.client-version. The default is to allow all dcap client versions, unchanged from the previous behaviour.

This release fixes a regression through which Kerberos dcap would not work for host principals containing a ‘-’ character. GSI dcap was not affected.

doors

Fixed a bug in the lb set tags command in doors that prevented setting an empty list of tags.

pool

A bug that caused non-critical stack traces to be logged on the pool after stage or deletion failure from nearline storage has been fixed.

Changelog 2.14.36..2.14.37

48ab090: [maven-release-plugin] prepare release 2.14.37
bbed67b: dcap: expose dcap client version limit
10e2317: dcap: fix Kerberos dcap if principal contains a ‘-’
3799546: dcap: fix regression in handling old version
1c2d2c5: pool: Suppress two stack traces in nearline storage handling
5dc425d: cleaner: Send notifications concurrently
fe023c9: billing: fix stacktrace and slow shutdown if in refresh
bd28892: doors: Allow setting an empty list of login broker tags
77cdddf: [maven-release-plugin] prepare for next development iteration
307e556: dcache: Fix detection of message errors in admin

Release 2.14.36

doors

Fixed a bug in which information on stage pool and number of attempts were lost when retrying pool selection requests.

Changelog 2.14.35..2.14.36

aa669de: [maven-release-plugin] prepare release 2.14.36
9a52fdc: doors: Ensure that pool selection context survives between retries
cfe45ad: [maven-release-plugin] prepare for next development iteration

Release 2.14.35

billing

Fixed a problem in the output of the dcache billing command using JSON or YAML when the billing format includes a custom date format.

dcap

Add support for the dcap client supplying additional version information.

gplazma

Fix explain login and test login commands so they are able to test logging in with username and password.

Add examples to explain login command help. Our thanks to Onno Zweers for this change.

httpd

Fix regression in the transfers.txt output format.

Update the transfers.html page so it no longer includes <unknown> for default/unknown protocols. With this version of dCache, these are represented by a ? character. The webadmin page is updated to give consistent output.

scripts

Fix the billing indexer to ignore the format string, if present.

Changelog 2.14.34..2.14.35

b513ad0: [maven-release-plugin] prepare release 2.14.35
96b772a: Active Transfers: substitute ? for <unknown> on html pages
ec8a1ee: common: add support for UserNamePrincipal as user:<name>
0a4af22: Added ‘explain login’ examples to help text in Gplazma2LoginStrategy.java
a869f1c: billing: Strip format string from attribute name
895b33e: transferObserverV1: replace Args with Joiner to construct transfers.txt linesMotivation:
a0928a3: billing: Make billing indexer work with custom format strings
9a3c997: [maven-release-plugin] prepare for next development iteration
f69b830: dcap: add support for clients presenting more version metadata

Release 2.14.34

commons

If dCache’s internal ShellApplication framework detects a critical behaviour that might indicate a bug in the application, error messages now include an explicit request to send a mail to the developers and more relevant information for assessing and reproducing the situation.

gplazma2

This release fixes a small bug in GPlazma which inappropriately tried to handle non-DN subjects in x509 certificates. These will usually fail in gPlazma anyway, but the reported error was confusing.

srm

Handling of DNS names without trailing dots in certificates has been made more robust and universal.

Changelog 2.14.33..2.14.34

fc15abe: [maven-release-plugin] prepare release 2.14.34
a70b77c: commons: log bugs with stack-trace and instructions
6f5294f: gplazma2-xacml: remove erroneous creation of placeholder extensions
014ce27: [maven-release-plugin] prepare for next development iteration
849c1b8: srm: remove trailing dot from reverse lookup result

Release 2.14.33

cells

In rare cases, an interrupt needed to cleanly shut down the location manager connector would not arrive. This issue was corrected, ensuring more reliable behaviour on cell shutdown.

Fixed a problem in which threads could inappropriately be created as daemon threads, causing problems in killing those threads when the cell shuts down.

Changelog 2.14.32..2.14.33

1d86940: [maven-release-plugin] prepare release 2.14.33
14d0735: cells: Fix lost interrupt exception
c045712: cells: Ensure that newly created threads are non-daemon normal priority threads
8513e45: [maven-release-plugin] prepare for next development iteration

Release 2.14.32

commons

The \s admin command uses the toString method to serialise the requested so that the remote cell may correctly parse it. This did not always work: ‘=’ characters in arguments were escaped but not unescaped; arguments that start with a ‘-’ character were not escaped; empty words were lost. In the current release this is fixed and \s command works as expected.

dcache

The current release fixes a regression in which the exit code of check-config would always be zero even when errors were detected.

Changelog 2.14.31..2.14.32

3f9b3d3: [maven-release-plugin] prepare release 2.14.32
b74342f: dcache: Generate proper exit code for check-config command
9336cda: commons: fix Args string parsing and toString method
f0a6386: [maven-release-plugin] prepare for next development iteration

Release 2.14.31

alarms

Alarm email notifications are now sent only on the first occurrence of given alarm (i.e., for that alarm instance’s unique ID).

If an alarm has been closed and not deleted, but then occurs again, the counter for receiving that alarm is now reset to 1, in order to treat this as a new (set of) occurrences, and to guarantee a new notification will be sent.

webadmin

Due to an implementation detail in a library used for the webadmin pages, filtering tables was a bit unintuitive until now: A filter that was set in a certain table column would reappear on tables in other browser windows if they had similar columns.

This behaviour was corrected, and tables on different pages exposed simultaneously in different browser tabs are now filtered independently. However, the fix also has the side effect that now with page reloads and form submissions the filters are cleared. Any commands, however, will always be issued correctly.

Filtering rows in webadmin tables could occasionally lead to unintuitive behaviour with regard to selections: Filtering a table and hiding rows may be included in a “select all” or “deselect all” operation. This was fixed, and selection/Deselection of hidden rows is now prevented.

Issues with filter boxes disappearing or filters resetting have been solved by disabling AJAX auto refresh for the affected pages.

Changelog 2.14.30..2.14.31

65df3fc: [maven-release-plugin] prepare release 2.14.31
1c40ccd: dcache-webadmin: synchronize client-side filtering with server-side selection of rows on pages using picnet table filters
e243006: alarms: reset count history on reopened alarm
bd728db: dcache-webadmin: disable saving table filter settings to browser cookies
3075b49: dcache-webadmin: disable AJAX autorefresh on pages using picnet table filter library
ba3523e: alarms: only send email on first alarm occurrence
37f6a13: [maven-release-plugin] prepare for next development iteration

Release 2.14.30

chimera

Chimera occasionally suffered from (operationally irrelevant) IllegalStateExceptions. Those are now avoided.

doors

Doors could get stuck temporarily if a file was deleted during pool selection. This has been fixed, and in such cases, transfers are now aborted properly.

Changelog 2.14.29..2.14.30

1aa2fcb: [maven-release-plugin] prepare release 2.14.30
15daf6d: doors: Abort transfer if file is deleted during pool selection
726b7a8: chimera: Fix IllegalStateException in inode cache
08823f2: [maven-release-plugin] prepare for next development iteration

Release 2.14.29

billing

When using the dcache billing command with a non-default date format in the billing file, an unneccessary stack trace was printed. This has been corrected.

srm

The SRM should periodically (by default every 10 minutes) delete obsolete historic data (older than 10 days by default) from the database. For cases where there are problems with that process, error logging and robustness against temporary database problems have been improved.

webadmin

The Wicket library used by dCache internally issued warnings about upcoming naming changes. Those cluttered the log files, and are silenced from the current version on.

Changelog 2.14.28..2.14.29

acc657b: [maven-release-plugin] prepare release 2.14.29
aafe0bd: srm: make out-of-date historic data deletion more robust
189dbd0: webadmin: silence warning about future change in wicket
745d1bc: [maven-release-plugin] prepare for next development iteration
fd1d0a8: billing: Removing erroneous stack trace output

Release 2.14.28

ftp

The Apache Commons FtpClient can issue the LIST command with the non-standard -a option. Which was causing dCache to switch output format from the long (ls -l-like) to the short (ls-like) response. This is fixed now and dCache is more compatible with Apache Commons FtpClient.

Changelog 2.14.27..2.14.28

dfd09d0: [maven-release-plugin] prepare release 2.14.28
9af35c6: ftp: improve compatibility with Apache Commons FtpClient
4c17cd6: [maven-release-plugin] prepare for next development iteration

Release 2.14.27

cells

If the create command in CellShell fails because of unreadable setup files, it throws an IOException. This was incorrectly reported as a bug. Reporting has been corrected now.

In rare cases, active transfers would show up with an incorrect state in the active transfers page of the admin backend. This was fixed, so that ransfers which are staging from non-DCAP doors are correctly indicated (in yellow) on the active transfers page, instead of showing up as “No Mover found” (in red).

common

If there is an IOException when trying to read a setup file, the corresponding file name is now listed in the error message.

poolmanager

An issue with PoolManager prevented it from delivering correct cost estimates. This was fixed, resulting in improved estimations of pool load.

Changelog 2.14.26..2.14.27

23263d1: [maven-release-plugin] prepare release 2.14.27
a61373e: cells: IOException is not a bug in create command
8bf50db: common: include filename in error message
f0d2757: poolmanager: Fix incorrect correction of pool cost
3e656bf: cells: handle empty string pool value on staging in TransferObserver
13bd117: [maven-release-plugin] prepare for next development iteration

Release 2.14.26

alarms

Log entries that were promoted to alarm status and that show up in the webadmin table can now contain more detailed information.

Changelog 2.14.25..2.14.26

05895da: [maven-release-plugin] prepare release 2.14.26
9cbee43: alarms: add ndc info to alarm info
9f2b5f0: [maven-release-plugin] prepare for next development iteration

Release 2.14.25

gplazma2-argus

Fixed a problem with the gPlazma argus plugin that caused it to fail with a ClassNotFoundException.

Changelog 2.14.24..2.14.25

bafb5a4: [maven-release-plugin] prepare release 2.14.25
03ec581: gplazma2-argus: Update to Argus client 2.2.0 to fix dependency on VOMS library
2478dd6: [maven-release-plugin] prepare for next development iteration

Release 2.14.24

alarms

A change to the alarms system improves handling of alarms with unset types.

nfs

A race condition in the NFS door that could result in the creation of multiple inconsistent copies of a file being uploaded has been fixed.

pool

A regression was fixed that caused the jtm go command to occasionally not work.

When a transfer’s status is queried before the transfer is initiated, which can occasionally happen for queued requests, Exceptions were logged. This behaviour has now been corrected, providing more robust operation.

Several race conditions in the pool’s migration module are fixed now.

A regression prevented queues to be set to not handle any jobs at all. The fix allows to pass a limit of 0 to mover set max active.

A regression was fixed that caused pools to “leak” movers if those were cancelled while still being queued.

spacemanager

Fixes a compatibility problem with NFS in which space manager would fail with a duplicate key error.

srm

A hint to describe the necessity to include escaping has been added.

The current release fixes issues in which the use of SRM third party copy operations could cause the SRM cell to become unresponsive, possibly even run out of memory.

Changelog 2.14.23..2.14.24

912669a: [maven-release-plugin] prepare release 2.14.24
a7c5866: srm: add hint to escape IDs
45c4db9: nfs: Fix race condition in transfer startup
17e0259: info: fix broken unit-test
15aa4b1: srm: Resolve message thead blocking issues with SRM third party copy
7d3d568: spacemanager: Work around for doors resubmitting PoolAcceptFileMessage
f67c8a0: pool: Fix several race conditions in migration module
5874750: pool: Fix regression in mover set max active command
79cfa23: pool: Fix mover leak
9042ba0: pool: Fix synchronization regression in jtm
e48472c: pool: avoid NPE when querying status of a 3rd-party HTTP transfer
2d97cdf: alarms: fix NPE in type setter
b7ebe35: [maven-release-plugin] prepare for next development iteration

Release 2.14.23

admin

The admin door now generates SSH keys to ensure compatibility with OpenSSH 7. Additionally, a new property admin.paths.host-keys was introduced in the admin.properties file, allowing to specify the location of keys.

script

When a pool’s metadata conversion operations would fail, an error caused a script to report successful conversions. This error has been fixed now.

Changelog 2.14.22..2.14.23

f308728: [maven-release-plugin] prepare release 2.14.23
9c37845: admin: Fix compatibility with OpenSSH 7
a422596: script: Do not claim success if meta data conversion failed
2008471: [maven-release-plugin] prepare for next development iteration

Release 2.14.22

admin

First observed on Ubuntu Xenial, dCache fails to install on modern Linux distributions due to the short key length of the SSH 1 keys generated in the post install script. This patch removes those keys and their generation code. dCache has been supporting only modern key formats for quite some time now, so this change should not have any impact on users.

pool

When creating movers, some error conditions are expected to occur and dCache is designed to transparently recover from these. Consequently, this patch lowers the log level for the related error messages to reflect that their causes are harmless.

Fixed a staging problem that would lead to failures in nearline COPY operations.

srm

Some race conditions during SRM startup were fixed. Those race conditions could potentially have lead to failures to expire jobs and to wrong job counts in the SRM schedulers.

webdav

Until now, trying to access a file for which the client was not authorized would generate a reply with a status code 200 OK, but an empty body, rather than an error page. This patch corrects that behaviour and also improves exception handling for that case.

Changelog 2.14.21..2.14.22

1472172: [maven-release-plugin] prepare release 2.14.22
ae318fc: admin: Drop old ssh 1 keys
4831b01: pool: Lower log level of certain failures to create mover
9eb8dd0: pool: fix staging for CopyNearlineStorage
a890967: webdav: Fix error reporting when client is unauthorized
e7135f5: srm: Fix job expiration during service startup
ab3a2dd: [maven-release-plugin] prepare for next development iteration

Release 2.14.21

billing

The data used to create the 24 hour billing overviews is aggregated in hourly intervals before creating the plots. However, if there is very high activity on the system during an entire 24 hour period, there have occasionally been timeouts when querying the database for this aggregate data. This patch makes the data aggregation more robust against such situations, resulting in lower latency for histogram generation and no more timeouts.

core

The Online Certificate Status Protocol (OCSP) is used to query status information about certificates during authentication. dCache supports this protocol, but relies on a functioning OCSP server for it to work properly. This patch changes the default OCSP mode for dCache to IGNORE, effectively disabling it, which is helpful for sites without a working OCSP server in place.

gplazma

Previously, attempts to authenticate users against an htpasswd entry that was malformed resulted in a stack trace. This patch modifies the error handling so that only a detailed error message (“Bad entry in file: hash does not start ‘$1$’ or ’$apr1$”) is logged.

Changelog 2.14.20..2.14.21

340638c: [maven-release-plugin] prepare release 2.14.21
907a59e: (2.14) billing: use in-memory buffer for hourly aggregate data
f25df85: Disable OCSP by default
53c3804: [maven-release-plugin] prepare for next development iteration
ea3120a: gplazma: don’t generate a stack-trace if htaccess is malformed

Release 2.14.20

Changes affecting multiple services

Fixed an issue with the dcache heap dump command when called with a simple file name as the output path. In this case the dump could in some cases be written to a different directory while the script claimed the dump had failed. The dcache dump heap command has a --force option for cases in which the JVM is unresponsive. This option was ignored for processes not running as root. This is fixed now.

cells

Fixed a problem causing FTP and DCAP per connection instances to subscribe to topics they should not subscribe to. This reduces overhead caused by routing updates.

Fixes a problem during shutdown in which communication tunnels between domains were shut down too early.

Fixed an issue that would cause log messages in which placeholders had not been replaced with actual values.

A bouncing message bug in System cell is fixed.

pnfsmanager

Setting atime-gap to –1 (default value) should disable file’s last access time updates. Nevertheless, this was not the case and atime update was always enabled. This is fixed now and file’s last access time can be disabled as described in the documentation.

pool

Fixes an issue with pools becoming unresponsive in case of slow DNS reverse lookups.

Fix race condition in request scheduler.

If FTP clients disconnect mid-transfer, pools log a DoorTransferFinished delivery failure as the door is gone. This is fixed now and log messages like Failed to deliver DoorTransferFinishedMessage message are suppressed in pools. When cancelling a job in a state that doesn’t allow cancellation, an illegal state exception is thrown. This was logged as a bug. This is now fixed.

Pool to pool transfers are supposed to be cancellable, but was not working as the HttpURLConnection does not appear to react to thread interrupt. This could lead to migration job and rebalance job cancellation appearing to hang. This is now fixed.

Fixes several performance regressions in pools that reduced mover creation rate and could cause pools to become unresponsive due to lock contention.

srm

Fixed a bug that caused delivery failures of credential service announcements to be logged. The ls -completed=n command has been observed to fail with SRMInvalidRequestException. This is fixed now and the output format of listing list requests has been changed to match that of other requests.

Changelog 2.14.19..2.14.20

84328f0: [maven-release-plugin] prepare release 2.14.20
f92cdcc: srm: Fix listing of completed list requests
2aeec7f: dcache: fix heap dump to simple file names
786ebc6: script: Make dump heap –force work for non-root processes
0f82952: script: Add missing she-bang
c22210d: pool: Fix p2p cancellation
dc08cb7: srm: Do not expose TURL before request is ready
0cebf53: pool: Don’t log illegal state exception on migration job cancellation as a bug
d37fde7: pool: Reduce lock contention on mover creation
435cc50: pool: Fix race condition in request scheduler
0d50cd3: pool: Avoid reverse DNS lookup in HTTP mover
159eea7: billing: additional fixes to insert triggers
ec3ac5d: pool: Suppress logging of delivery failure of DoorTransferFinished
e782584: system-test: update disposable-CA generated credentials
a34cfee: pnfsmanager: fix atime update regression
b7aef46: cells: Fix logging formatting string
91fc2e0: cells: Avoid bouncing message on no-route errors in System cell
ec75096: srm: Suppress message delivery failures for credential service announcements
3559ff9: cells: Fix tunnel shutdown order
07350e6: cells: Do not subscribe to topics in per-session door instances
b403160: [maven-release-plugin] prepare for next development iteration

Release 2.14.19

chimera

An internal database trigger was updated to insert data in the correct table, fixing a problem with the Enstore client.

When accessing a file for reading, the atime value must be updated. Previously, due to an error, the ctime (intended to reflect the time of changes to file attributes) was also changed. This update corrects that problem.

http

In order to increase the performance of the Billing system, reverse DNS lookups were removed from the code. While this will result in IP addresses representing hosts in the billing file, DNS performance no longer impacts overall system performance.

many

When representing checksums in the admin interface and configuration files, checksums are now presented in an improved format.

pool

The nearline storage subsystem uses thread pools to manage its workload. Since some tasks are blocking, very high activity can cause these thread pools to grow beyond effective sizes. This may even lead to the pool becoming unresponsive.

This change introduces a new configuration property, “pool.limits.nearline-threads”, which limits the thread pool size. The default value, 30, is chosen to be sufficient for almost all imaginable use cases while at the same time avoiding potential problems with resource exhaustion.

srm

Due to a timing issue, an initial service announcement in the SRM was sent before any listeners could register for those announcements. Thus, upon startup, a delivery error would be logged. With this patch, sending of the initial message is delayed until after the registration of listeners, and the irrelevant error messages are avoided.

Changelog 2.14.18..2.14.19

f27d953: [maven-release-plugin] prepare release 2.14.19
a10ce6c: chimera : fix trigger that populates data in t_locationinfo and t_inodes on insert or update of t_level_4
cc07d28: common: fix ChecksumType.toString()
c80ef7e: http: avoid dns reverse lookup on HttpProtocolInfo#toString()
3343fde: chimera: do not update ctime on atime only attribute update
e411aae: [maven-release-plugin] prepare for next development iteration
d101c0d: srm: Delay announcing credential service after cell start
d76de7f: pool: Improve scalability of nearline storage subsystem

Release 2.14.18

doors

Fixes a bug in which a host name set in *.net.listen properties was not preserved when publishing a door or generating SURLs.

Changes affecting multiple services

When building rpm files a package which is now explicitly required as a dependency.

cells

This change fixes a bug in routing manager that would leave orphaned topic routes in dCache domain.

pool

The nearline storage subsystem has a thread pool for various tasks. Some of these tasks are blocking. Since the thread pool is unlimited, a high inflow of new requests can cause the thread pool to grow rapidly and even exceed thread limitations. In that case the pool dies. In the current release a new pool.limits.nearline-threads property is introduced which allow to limit the number of threads used by the nearline storage subsystem. Note that, the default is 30 threads.

Changelog 2.14.17..2.14.18

e03110e: [maven-release-plugin] prepare release 2.14.18
00ce67f: pool: Fix regression breaking hopping mananger
b43a335: cells: Fix route removal in routing manager
488d51f: rpm: explicitly require which package
9cd14bc: doors: Preserve name when publishing the address of doors
0d73d58: [maven-release-plugin] prepare for next development iteration

Release 2.14.17

cells

LoginManager would occasionally generate error messages similar to “Discarding listening on $LOCATION 53684’ because its age of 18721640 ms exceeds its time to live of 4500 ms.”. This was due to erroneous reuse of old message envelopes in the internal messaging. This change fixes that problem.

This change addresses a potential problem in which messages sent between cells in the same domain could appear older than they are and thus would risk being discarded due to the time-to-live being expired.

Contains corrections to cells logging. The routing manager pinboard now shows information previously logged to various other cells.

nfs

This change avoids ERR_PERM errors on NFS writes in situations where NFS doors do not receive redirect messages within the allowed timeframe.

pool

This patch avoids errors occuring when mover doors would not receive a start mover request within the allowed timeframe, providing more robustness on heavily loaded instances.

srm

A Tier–1 site reported problems with a major WLCG VO’s read requests. Investigating the source of the problems showed that the srm_ifce library, used by the (outdated) GFAL v1 and the (supported) GFAL v2 SRM libraries, drastically limits the permitted lifetime of requests without providing admins any way to configure this.

For sites seeing errors related to desiredTotalRequestTime being exceeded, this change provides the new configuration option srm.request.maximum-client-assumed-bandwidth in srm.properties as a work-around.

Sites not observing such errors do not need to change anything with regard to this value.

Changelog 2.14.16..2.14.17

441214c: [maven-release-plugin] prepare release 2.14.17
b13fa36: srm: add short request lifetime work-around
99bb9c0: cells: Set correct logging context in cell callbacks
317c4af: cells: Improve robustness of message time to live
1284b71: cells: Fix erroneous reuse of message envelope in location manager registration
1827932: nfs: try to re-use transfer class on client retry
0f6d8a6: pool: handle duplicated start mover requests
a8d4ca0: [maven-release-plugin] prepare for next development iteration

Release 2.14.16

chimera

Fix a regression causing directories to inherit ACLs as if they were files rather than directories. Soon we will provide a procedure to clean-up already existing wrongly inherited directories.

Changelog 2.14.15..2.14.16

7b3dc90: [maven-release-plugin] prepare release 2.14.16
189fd27: chimera: Fix regression in inheriting ACLs on directory creation
75a76a4: [maven-release-plugin] prepare for next development iteration

Release 2.14.15

ftp

On very short transfers, two internal messages could occasionally arrive in the wrong order. This would cause clients to see the “226 Transfer complete.” message without the “150 Opening BINARY data connection for” immediate reply. This rare issue is now fixed.

pnfsmanager

Billing entries for SRM uploads recently lost the storage class part of the entry. This update fixes that issue. We observed an error caused by the parallel execution of two uploads, both trying to create the same (previously non-existing) directory). This modification fixes the underlying race condition, allowing such transfers to succeed.

pool

Fixed a regression introduced in 2.13 in which internal options like -c:puts were erroneously included to the call out to the HSM script.

poolmanager

This modification fixes a potential race condition in pool manager.

This modification fixes a race condition in pool manager that could theoretically have provided erroneous data to pin manager, space manager, srm, xrootd and webdav. This modification fixes a race condition in pool manager that could theoretically have provided erroneous data to pin manager, space manager, srm, xrootd and webdav.

scripts

The ctlcluster utility is now fully supported under Linux.

srm

When writing to a path /a/b/c, if b exists and is a file, SRM currently returns SRM_INTERNAL_ERROR. This modification changes that behaviour so that the more appropriate status SRM_INVALID_PATH is returned. During SRM-based operations, some sites reported problems with the delegation of user credentials. This modification remedies those problems, while at the same time reducing the CPU usage of establishing 3rd-party SRM connections.

webdav

This modification corrects the error reporting under WebDAV. When attempting to delete a non-existing file, unauthenticated users receive a 401 Unauthorized response, while authenticated users receive a 404 Not Found response.

Changelog 2.14.14..2.14.15

0e8915d: [maven-release-plugin] prepare release 2.14.15
3026bfd: pnfsmanager: Fix regression in SRM billing entries
c2166d9: pool: Fix filtering of options in script HSM driver
3919bb1: pnfsmanager: Fix race leading to transaction failures in Chimera
7340b3e: srm: Disable delegation on srmCopy to or from other SRMs
4eb54e9: srm: Return SRM_INVALID_PATH when target directory is a file
7e848e5: ftp: Fix race on short transfers
c4c7f53: [maven-release-plugin] prepare for next development iteration
ae8a0ea: Fix compatibility issues with ctlcluster
cc6aa1f: poolmanager: Acquire read lock when serializing cost module and partition manager
63f8fa5: poolmanager: Acquire read lock when serializing pool selection unit
e46f168: poolmanager: Fix race condition in pool selection unit
c90dcc1: webdav: fix 404 error if attempting to delete a nonexistent file

Release 2.14.14

doors

Fixes a race condition and a responsiveness issue in doors.

spacemanager

Fixes a race condition during space manager startup that could lead to log messages about failed link group updates and failed transfers.

Changelog 2.14.13..2.14.14

d32ea47: [maven-release-plugin] prepare release 2.14.14
ed94389: spacemanager: Fix pool monitor fetch race during startup
84fac07: doors: Fix contention point and race in login broker publishing
6537ac6: [maven-release-plugin] prepare for next development iteration

Release 2.14.13

pnfsmanager

With this release, PnfsManager adds safety checks rejecting invalid upload paths that SRM might erroneously supply. This release hardens an installation against possible bugs triggering data loss.

This release adds a check that detects failed or incomplete SRM uploads and prevents the file from being committed to its final path. Common symptoms of this bug were zero sized files that experiment catalogues registered as successfully uploaded.

pool

This patch improves support for HTTP Keep-Alive connections. On closing a connection, in accordance with RFC 2616, an HTTP header indicating closure of the connection is sent out.

On highly loaded systems, DNS slowdowns could lead to pools and/or domains becoming unresponsive. This fix increases resilience against DNS problems.

Increased performance and resilience in situations where NFS clients send many requests at once.

srm

This release adds a check to detect broken uploads using SRM during the final stage of file transmission. While it causes transfers to take a little more time, resilience against upload failures is increased.

webdav

dCache has supported 3rd-party HTTP push requests for WebDAV for some time. This release adds support for 3rd-party HTTP pull requests.

Changelog 2.14.12..2.14.13

c0a3ec5: [maven-release-plugin] prepare release 2.14.13
8762d5e: pool: send connection header when closing a connection.
410ef0f: pool: Avoid lock contention on DNS lookup in p2p component
7186416: srm: Check for broken files during srmPutDone
7a6f5a6: pnfsmanager: Check file size and upload completion when committing temporary upload paths
6084a86: pnfsmanager: Protect against erroneous upload paths
ab69eac: pool: use “same thread strategy” for nfs movers
d43f045: webdav: add support for 3rd-party HTTP pull
ec8ce92: [maven-release-plugin] prepare for next development iteration

Release 2.14.12

Changes affecting multiple services

Several cases of slow performance were reported while deleting directory in Chimera. This is now fixed.

pool

dCache pool yaml command dumps the meta data of a pool using the human an machine readable YAML format. A regression causing a null pointer exception has been fixed.

When command execution to migrate files between pools (e.g. migration concurrency or migration copy) is interrupted due to the failure to find migration job the returned error message is considered as a bug. This is now fixed so that a new message is returned indicating that the job being requested does not exist.

srm

When file upload is cancelled the value of temporary upload path tracked by SRM could be a value different from a regular path, either because it was changed outside of dCache, or it contains entries from a very old version of dCache. This could result in data loss while canceling upload. The current release fixed a potential data loss scenario.

statistics

The statistics service creates static HTML pages that describe dCache usage over time as simple files that the webadmin service can serve. This includes information about pools and store-units. The problem is that the statistics webpages do not show information about any pool or store-unit that contains a / in the name. This is now fixed. A side-effect is that the history of any pool or store-unit containing a ^ in the name is lost.

xrootd

In XRootd Protocol Specification kXR_dirlist request is used to list the contents of a directory. XRootd version 3.0.0 introduced the kXR_dstat option to the kXR_dirlist command which returns stat information for each entry. The new release adds support for kXR_dstat in dCache. Note that for VOs that rely on token authorization, this is a requirement for functional verbose listing.

Changelog 2.14.11..2.14.12

a5fc47c: [maven-release-plugin] prepare release 2.14.12
9a67073: xrootd: Add kXR_dstat support
c2e7620: chimera: Alter statistics target for t_tags(itagid)
f41c5c2: srm: Add safe-guard against invalid file ID in put requests
0a10e60: pool: Don’t consider failure to find migration job a bug
fe3bc12: statistics: encode ‘/’ in filenames
53d31a4: [maven-release-plugin] prepare for next development iteration
07d5b22: pool: Fix NPE in pool yaml tool

Release 2.14.11

alarms

On heavily loaded systems creating alarms at very high frequencies and with comparatively slow databases for alarm logging, components ran out of memory and blocked. This release introduces a mechanism that will drop alarm messages that would cause the system to fail, ensuring continued operation. Note that no messages that are critical to system operation will ever be dropped by this change.

nfs

Pinning files is now a non-blocking operation. For files stored on tape, this should result in a more responsive system behaviour, avoiding NFS blocking in situations with many concurrent pin requests.

Changelog 2.14.10..2.14.11

1e2eb2c: [maven-release-plugin] prepare release 2.14.11
cdee225: (2.14) alarms: change executor to have bounded queue and discard events on overrun
755f85c: nfs: use noitify instead of blocking sendAndWait when sending pin/unpin messages via touch “.(get)(<file_name>)(pin)” command
7868f81: [maven-release-plugin] prepare for next development iteration

Release 2.14.10

Changes affecting multiple services

Sometimes when a cell start up was interrupted an error message was logged as a bug. This is now fixed.

info-provider

The GLUE infomation provider supplies information about the dCache instance, which is important for the clients in WLCG area. Because in dCache different doors can have different roots, clients may need to adjust their path when accessing dCache through different doors. The info-provider is updated so that a new path root property is provided. This allows clients to modify paths, as necessary. Note that the SRM door already supports this translation when redirecting clients for transfers.

pool

Transfers into or out of dCache that do not go through the client (“3rd-party transfers”) may be initiated through srm and webdav doors. This release fixes allows https transfers without any X.509 credential.

Changelog 2.14.9..2.14.10

174e0bd: [maven-release-plugin] prepare release 2.14.10
d67f620: info-provider: publish door root path
84acc3c: cells: Suppress illegal state exception during initialization
eda4a70: pool: fix HTTPS third-party transfers without X.509 credential
34703bf: [maven-release-plugin] prepare for next development iteration

Release 2.14.9

nfs

This release includes the PNFS-ID in the door’s logging output when proxying data movement.

Fix infinite loop for a specific failure mode when writing data into dCache: with this release, dCache will indicate a permanent error.

spacemanager

Spacemanager backs off when it encounters a problem writing to the database. Previously, if the problem was due to deadlocks then the two tasks involved are delayed by the same amount, which means it is possible that subsequent attempt will also deadlock. This release randomises the delay to reduce the likelihood of this problem occuring.

Changelog 2.14.8..2.14.9

0fd5707: [maven-release-plugin] prepare release 2.14.9
1c93e26: spacemanager: Randomize backoff in case of transient errors
bb84805: nfs-proxy: include file’s pnfsid into debug context
70403d9: nfs: convert FILE_IN_CACHE into NFS_EIO
27d3f09: [maven-release-plugin] prepare for next development iteration

Release 2.14.8

pool

Fix buffer leak in the HTTP mover.

Improve scalability of pools when opening files or changing sticky flags.

Resolve problem where starting a migration job on a pool with many files can result in an unresponsive pool.

This release fixes a bug that caused the output of the rep ls -s admin command to contain negative values.

Improve pool scalability of pool, allowing it to remain responsive even when undertaking many pool-to-pool transfers.

xrootd

The prevelance of old versions (3.x) of xrootd client along with that version’s unfortunate behaviour when replying asynchronously has forced this release to revert the asychronous response to open requests.

Fix dCache handling of open requests where uploads were considered downloads.

Changelog 2.14.7..2.14.8

e7f2ff1: [maven-release-plugin] prepare release 2.14.8
d7abddf: pool: Fix lock contention during heavy p2p activity
aa3cad2: Revert “pool: Expose Berkeley DB configuration as dCache properties”
2e068a5: pool: Fix buffer leak in HTTP mover
c0f8ae6: xrootd: Fix classification of uploads
4f777e5: pool: Expose Berkeley DB configuration as dCache properties
966ca5a: xrootd: Roll back asynchronous reply on open
935649a: pool: Fix lock starvation in migration module
6ba57be: pool: Fix accounting error in repository statistics
0a48e12: pool: Avoid lock contention when opening files and setting sticky flags
063afcc: pool: Simplify synchronization during repository setup
51a05f6: [maven-release-plugin] prepare for next development iteration

Release 2.14.7

Changes affecting multiple services

Don’t log Error while reading from tunnel: java.nio.channels.AsynchronousCloseException when a domain shuts down.

cleaner

Fix cleaner so it no longer sends cleaning requests to pools it knows cannot enact these requests. This prevents the cleaner from spamming a disabled pool.

pool

No longer spam log file with 666:"Client disconnected without closing file." when an xrootd client closes a file it was reading. This is still logged if the client fails to close a file opened for writing.

Improve pool startup time.

This release provides more robust monitoring of a pool’s health by periodically testing the data directory in addition to testing the metadata.

With this release, the pool provides a more robust behaviour when the client requests a file without being redirected from a door. The pool will redirect the client back to its previously used door, if known. Additionally, the pool no longer logs when this happens in the pool log file, but only in the pinboard.

This release fixes buffer leaks in the xrootd mover. It also removes endsess spam from pool log files.

The pool’s info output now includes a progress report during initialisation.

Changelog 2.14.6..2.14.7

a3466a5: [maven-release-plugin] prepare release 2.14.7
ab3dab0: Fix broken pull-request PR:2096
9aedb72: pool: Upgrade xrootd4j to 3.0.2
aebd82c: pool: Redirect xrootd client to door on failure to open file
6039b7d: pool: Do not fail read if xrootd client doesn’t close file
849f59f: cleaner: Fix black listing in case of disabled pools
77ee337: cells: Don’t log AsynchronousCloseException when tunnel closes
923c13d: pool: Show progress during repository initialization
e7b1a00: pool: Fix health check of file store
8981994: pool: Use disk ordered cursor for listing meta data on pool startup
92d2531: [maven-release-plugin] prepare for next development iteration

Release 2.14.6

Changes affecting multiple services

Update the Spring, Milton, AspectJ, Jetty and DataNucleus-core libraries to latest version. All dCache services are affected.

gplazma

Fix debug logging of the voms gPlazma plugin so it shows a clear and complete description of which FQAN is mapped; previously the log would contain confusing entries if the first FQAN is not mapped exactly.

pool

If a 3rd-party transfer fails then the pool may log and report incomplete information on why this happened. This release fixes this problem.

On startup, a pool will scan its stored files and associated metadata to ensure consistent and accurate information. As this can take some time, with this release the pool will log how long these scans take to provide objective information when optimising this process.

srm

This release fixes the interpretation of srm.persistence.enable.store-transient-state when srm.persistence.enable.history is disabled. Sites that have history disabled may put additional load on the SRM database if store-transient-state is true; the old behaviour can be restored by setting srm.persistence.enable.store-transient-state to false.

Fix the report sent to the srm client so that the failure code and error message always match. Earlier releases of this branch sometimes returned an inconsistent combination of failure code and error message.

Fix potential for inconsistent information being logged in the SRM database, where the request state was different than the last logged transition.

The SRM provides legacy support for SRM clients that require the TCP connection to be closed without the orderly process that SSL/TLS clients normally expect. Unfortunately, this work-around contained a bug where a client interaction can trigger an thread being caught in a tight-loop, constantly consuming CPU. This release fixes that problem.

Changelog 2.14.5..2.14.6

bda53c5: [maven-release-plugin] prepare release 2.14.6
fef8106: pool: Log information on runtime for listing the repository
5d248cd: common-security,ftp-client: Allow compilation with Java 7
4397121: common: Move Checksums to fix compilation with Java 7
e445f6b: common: Move NetLoggerBuilder out of common
7283ebe: srm: Lock job while saving to create consistent persistent state
7b7ec53: srm: Fix saving of transient states to database
dbaa0ba: srm: Fix legacy close (again)
8dd8768: 2.14: upgrade third party dependencies
e80adbe: http–3rd-party: ensure IOException logged with toString
e9757a7: srm-client: Fix GSI delegation for old servers
744d25b: srm: Fix race in state reporting
81a0f65: gplazma: Improve logging in voms plugin
93e6e8e: info: fix test to be less critical on timing
93f193e: [maven-release-plugin] prepare for next development iteration

Release 2.14.5

Changes affecting multiple services

Enable SRM to discover FTP and dcap doors faster after restart. May also resolve messages like Failed to deliver LoginBrokerInfoRequest message <1449649391771:176> to [>LoginBrokerRequestTopic@local]: Route for >*@srm-bombayDomain< not found at >dCacheDomain<. All ftp and dcap doors should be updated.

Fix tab completion for non-default cells (messaging) topologies; all domains must be upgraded to deploy the fix. Sites with default topologies are unaffected.

The doors that use TLS (SRM, gsi-dcap, gsi-ftp, gsi-xrootd) must verify the user’s certificate. Previously this was both slow and could only take advantage of a single core. This release caches the result for 5 minutes, ameliorating both problems.

For non-default star topologies, there are intermediate domains that pass messages between dCacheDomain and end domains. This release fixes propagation of routing information by these intermediate domains. Sites with default topologies are unaffected; those with non-default topologies should upgrade all intermediate domains.

admin

When using the admin interface, there are three possible behaviours: interactive mode (new interface), no-terminal mode (new interface targeted for scripts) and legacy mode (old interface). The latter two could become stuck in an endless loop if the client disconnects before all data was sent. This is fixed with this release.

gplazma

Fix how X.509 Distinguish Names (DNs) are handled. This could cause the x509 plugin to fail, logged as a NoSuchMethodError message.

pool

Add new options to the rep set sticky command to allow filtering by access-latency, retention-policy, storage class and cache class.

Add an option to migration module to support filtering by cache class.

Add the -meta-only option to migration module. This limits the affected replicas to those where the file’s data is not transferred; i.e., the target is some existing replica of the file. Local replicas that do not exist on any other pool are skipped.

Extend the migration module’s -sticky option to allow negated selection. This is marked by prefixing the sticky owner with a -. For example, -sticky=-system selects replicas that do not have the system sticky.

Fix persistency of the sticky bits. Should a replica’s list of sticky bits be modified by either the rep set sticky command or the migration module modifying some existing replica then the sticky flags are held in memory but not written to the Berkeley DB. The next restart of the pool will loose that information, potentially resulting in data loss. Pools that store metadata as files are not affected.

poolmanager

Previous releases of dCache contained a bug where replicas generated by pool-to-pool copies failed to include the access latency and retention policy. While not directly affecting dCache operations, the result is that this information is no longer reliable.

spacemanager

Fix listing by PNFS-ID. Glob support is removed as it was non-functional.

srm

Allow SRM to start after it was shut down with IN_PROGRESS jobs. Solves Failed to restore job: Illegal state transition from InProgress to Queued.

Update migration procedure to remove obsolete states from the history table.

Changelog 2.14.4..2.14.5

2adfacd: [maven-release-plugin] prepare release 2.14.5
9be3fcc: pool: Do not output expired sticky flags
f7f7e0b: pool: Only save sticky bits if not already set
c74b297: poolmanager: Fix missing access latency and retention policy on pool to pool copy
01859f7: admin: Fix endless loop in non-interactive mode
fb94bc7: pool: Make bulk sticky bit operation robust against repository changes
5506ae8: pool: Throw IllegalArgumentException on rep set sticky errors
d7fce30: spacemanager: Fix listing by pnfs id
127c9fa: Add caching layer for TLS validation
227337e: pool: Extend migration module with -meta-only option
f23b270: pool: Add option to migration module to filter by cache class
5971062: pool: Add bulk mode for rep set sticky command
9268623: pool: Add migration option to filter by absense of sticky flags
3bd9ab2: pool: Fix persistence of sticky bits
a1cfc59: cells: Tab complete on distant downstream domains too
4f79b86: cells: Fix route propagation trigger on non-default topologies
f4cfd03: srm: Allow transition from InProgress to Queued
ad96e6c: srm: Map obsolete states in request history tables
07705d6: gplazma: Make GlobusPrincipal compatibly with JGlobus
756489e: [maven-release-plugin] prepare for next development iteration
6780e53: ftp,dcap: Fix LoginBrokerRequestTopic subscription

Release 2.14.4

nfs

A bug-fix introduced with 2.14.3 also introduced a regression that prevented directory listing over NFS protocol. This is fixed with this release.

Changelog 2.14.3..2.14.4

7c13aba: [maven-release-plugin] prepare release 2.14.4
bf173c2: chimera: fix regressin introduced by fa9a749c4
1af6698: [maven-release-plugin] prepare for next development iteration

Release 2.14.3

Changes affecting multiple services

The update to Chimera that came with 2.14 introduced a possible deadlock. These are clearly logged with ERROR: deadlock detected in the domain log file, and occur when a file is both deleted and its attributes are updated at nearly the same time. This problem is fixed with this release. Domains hosting either the nfs door or the pnfsmanager service should be upgraded.

Fix a problem in how cached values are updated within Chimera. Domains hosting either an nfs door or the pnfsmanager service should be upgraded.

A request to read a file stored only on tape when the user does not have permission to stage files will fail. Additionally, a user attempting to upload a file that would trigger creating a new space-reservation but the user is not authorised to create such a reservation will also fail. With earlier versions of dCache, in both these cases, the door would retry the request until the times out. This version fixes this issue. All doors are affected.

This release fixes a caching issue where changes to inode metadata (e.g., ownership or permissions) for / (the root directory of Chimera) are not visible until the service is restarted. This affects NFS doors and pnfsmanager service.

Update Chimera so it does not trigger DuplicateKeyException, so avoiding errors like current transaction is aborted, commands ignored until end of transaction block; nested exception is org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block. Both NFS doors and pnfsmanager are affected. Sites that use PostgreSQL v9.5 or later are encouraged to use the newer dCache driver: chimera.db.dialect = PgSQL95.

nfs

Fix the NFS door so it no longer logs Post-processing failed: No such file or directory if a file is deleted while still open.

pnfsmanager

Fix support for enstore.

Fix the Access Latency and Retention Policy of files when listing a directory.

Changelog 2.14.2..2.14.3

d62b652: [maven-release-plugin] prepare release 2.14.3
0289f60: transfers: fail request if pool/space manager reject request
ff64d88: chimera: Prevent filling of stat cache of root inode
c16c259: chimera: When updating stat cache, take the file type into account
b577b92: chimera: Read AL and RP on directory listing
566f6a5: Motivation: enstore based installations need a special trigger to keep t_inodes table in sync.
fc13ac4: nfs: ingore file_not_found on close
45dddd1: chimera: Make FsSqlDriver#addInodeLocation PostgreSQL compatible
49526f2: chimera: Do not ignore DuplicateKeyExceptions
4d89937: chimera: Fix deadlock
0184615: [maven-release-plugin] prepare for next development iteration

Release 2.14.2

pnfsmanager

Fix the error message (logged by the domain hosting pnfsmanager) if an attempt to finalise an SRM upload fails within pnfsmanager, or if an attempt to cancel an SRM upload fails within pnfsmanager.

pool

Fix the NFS mover so that, when the client is reading a file and reaches the end of file, the EOF flag is set correctly. This bugfix applies both to files read through pNFS and those read through the door.

When a pool is initialising FTP-based third-party transfers, load all trusted certificate authorities once rather than loading them on demand. This results in better performance over the lifetime of the pool when transferring many files to remote servers with certificates issued by different certificate authorities.

poolmanager

When reporting an alarm that a pool is declared DOWN, the message mistakenly omitted the name of the pool. This is now fixed.

webdav

Update to the latest version of milton.

xrootd

Update the alice-token plugin to allow the host name check to succeed on dual-stack (IPv4 and IPv6) machines.

Log initial response (usually that an asynchronous reply is forthcoming) when client attempts to open a file.

Fix door so that it no longer logs: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

Changelog 2.14.1..2.14.2

802c243: [maven-release-plugin] prepare release 2.14.2
197b757: xrootd: Update alice token plugin to fix IPv6 compatibility
8aabecb: xrootd: Update to xrootd4j 3.0.1
a049d16: system-test: Update ctlcluster command to configure CRL settings
af58f03: xrootd: Fix logging of Netty exceptions
85edc0a: webdav: update to latest milton
b3a9355: PnfsManager: remove copy-n-paste error in error message
595d885: (2.14) pool manager: add pool name to alarm
21b8272: pool: Make context factory non-lazy for remote gsiftp transfers
a022aa0: system-test: Fix grid-security settings
1f2c6b7: nfs: set EOF flag when channel indicated EOF
1040300: rpm: enforce SL5 compatibility when building RPM packages
afcd1b2: [maven-release-plugin] prepare for next development iteration

Release 2.14.1

admin

Fix potential IndexOutOfBoundsException should the response from the acm cell be malformed.

alarms

Update the text format of logged and emailed alarms so that messages include the type of alarm rather than simply the phrase ALARM [ ALARM_TYPE ]. Note that the alarms.email.encoding-pattern and alarms.history.encoding-pattern properties have been updated; sites that have modified either of these properties must review their configuration to take advantage of this update.

pool

Fix an intended pool-to-pool transfer optimisation: the receiving pool failed to reuse a delayed mover, should the pool-to-pool request timeout and be retried.

Changelog 2.14.0..2.14.1

9121417: [maven-release-plugin] prepare release 2.14.1
4edc896: srm-client, dcache: fixed passing incompatible arguments to functions
975e5ba: rpm: build SNAPSHOT RPM with filenames using only commit id
4d830fa: dcache-nfs, dcache: removed unecessary use of non-short-circuit logic
3a7b359: (2.14) alarms: modify logback encoding to show actual alarm type
70048cd: [maven-release-plugin] prepare for next development iteration

Release 2.14.0

Improved execution time statistics

Several cells provide information on request execution times. The code has been improved to reduce the effect of numerical instabilities. This improves the accuracy of the collected statistics. Rather than the population standard deviation we now report the sample standard deviation.

Use CANL for certificate verification

CANL is a library provided by EMI that provides support for OpenSSL style trust stores, PEM encoded certificates, proxy certificates, certificate path verification, CRL and OCSP validation, etc. In the past, dCache has relied upon the JGlobus library for such operations, but JGlobus has not seen much maintenance lately. We have switched to using CANL instead as we feel it is more up to date (an important aspect for security critical libraries), provides a cleaner API, has more features, and doesn’t lock us into old versions of other third party libraries. There may also be performance benefits - we know for certain that the startup time for client tools is improved, and time will tell if the server side is faster too.

The amount of changes required in dCache to implement this was quite large and touched most components, although the externally visible number of changes should be low. Still, with this amount of changes there is bound to be a regression or two.

One feature of JGlobus missing in CANL is an implementation of the GSI protocol. GSI is essentially TLS with an additional credential delegation step after the initial TLS handshake. In dCache 2.13 we introduced our own GSI implementation in dCache to provide support for non-blocking IO in the SRM door. In 2.14 this has been adopted by all doors providing GSI support, i.e. SRM, FTP and DCAP. Furthermore, the client side of GSI has been implemented and is used by our FTP and SRM clients, as well as our GridSite delegation shell.

As part of porting DCAP and FTP to our own GSI implementation, the SSL context is now cached between sessions, greatly reducing the per-connection overhead of these doors. Consequently these doors now expose settings for key pair cache lifetime:

#  ---- Delegation key pair reuse lifetime
#
#  X.509 clients may delegate credentials to dCache using GSI or the dedicated GridSite
#  delegation service. The delegation works by the server generating a certificate
#  signing request which the client signs. Since generating a key pair as part of the
#  signing request is expensive, dCache caches and reuses the key pair.
#
#  This setting controls for how long a key pair is reused. A compromised key could be used
#  for new signing requests for this amount of time. If the reuse time is too short, the
#  overhead of generating new key pairs increases.
#
dcache.authn.gsi.delegation.cache.lifetime = 30000
dcache.authn.gsi.delegation.cache.lifetime.unit = MILLISECONDS

The default host and CA certificate refresh periods have been significantly reduced to 60 seconds:

# ---- Host certificate refresh period
#
# This option influences in which intervals the host certificate will be
# reloaded on a running door.
#
dcache.authn.hostcert.refresh = 60
dcache.authn.hostcert.refresh.unit = SECONDS

# ---- CA certificates refresh period
#
# Grid-based authentication usually requires to load a set of
# certificates that are accepted as certificate authorities. This
# option influences in which interval these trust anchors are
# reloaded.
#
dcache.authn.capath.refresh = 60
dcache.authn.capath.refresh.unit = SECONDS

Since CANL provides configurable name space verification, CRL validation, and OCSP validation, all doors and pools relying on TLS now respect the following new configuration properties:

# ---- Certificate Authority Namespace usage mode
#
# A CA namespace restricts the certificates dCache accepts as issued by a given CA. The namespace
# is defined in .namespaces (EURGIDPMA) or .signing_policy (GLOBUS) files alongside the CA certificate.
#
# This setting controls the rules governing the verification of these namespaces. The following
# documentation is copied from the EMI CANL library used by dCache.
#
#       GLOBUS_EUGRIDPMA
#
#          A Globus EACL is checked first. If found for the issuing CA then it is used and enforced.
#             If not found then EuGridPMA namespaces definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA_GLOBUS
#
#          An EuGridPMA namespaces definition is checked first. If found for the issuing CA then it is enforced.
#             If not found then Globus EACL definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be passed.
#
#       GLOBUS
#
#          A Globus EACL is checked only. If found for the issuing CA then it is used and enforced.
#             If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA
#
#          An EuGridPMA namespaces definition is checked only. If found for the issuing CA then it is enforced.
#             If no definition is present then namespaces check is considered to be passed.
#
#       GLOBUS_EUGRIDPMA_REQUIRE
#
#          A Globus EACL is checked first. If found for the issuing CA then it is used and enforced.
#             If not found then EuGridPMA namespaces definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_GLOBUS_REQUIRE
#
#          An EuGridPMA namespaces definition is checked first. If found for the issuing CA then it is enforced.
#             If not found then Globus EACL definition is searched. If found for the issuing CA
#                then it is enforced.
#                   If no definition is present then namespaces check is considered to be failed.
#
#       GLOBUS_REQUIRE
#
#          A Globus EACL is checked only. If found for the issuing CA then it is used and enforced.
#             If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_REQUIRE
#
#          An EuGridPMA namespaces definition is checked only. If found for the issuing CA then it is enforced.
#             If no definition is present then namespaces check is considered to be failed.
#
#       EUGRIDPMA_AND_GLOBUS
#
#          Both EuGridPMA namespaces definition and Globus EACL are enforced for the issuer.
#             If no definition is present then namespaces check is considered to be passed.
#
#       EUGRIDPMA_AND_GLOBUS_REQUIRE
#
#          Both EuGridPMA namespaces definition and Globus EACL are enforced for the issuer.
#             If no definition is present then namespaces check is considered to be failed.
#
#       IGNORE
#
#          CA namespaces are fully ignored, even if present.
#
dcache.authn.namespace-mode=EUGRIDPMA_AND_GLOBUS_REQUIRE

# ---- Certificate Revocation List usage mode
#
# CAs regularly publish certificate revocation lists (CRLs) containing the serial id of
# certificates that have been revoked. Such certificates should not be accepted by dCache.
#
# Such CRLs are stored in .r? files alongside the CA certificate. It is outside the scope
# of dCache to refresh the CRLs.
#
# This setting controls how dCache makes use of such CRLs. The following documentation is copied
# from the EMI CANL library used by dCache.
#
#       REQUIRE
#
#          A CRL for CA which issued a certificate being validated
#             must be present and valid and the certificate must not be on the list.
#
#       IF_VALID
#
#          If a CRL for CA which issued a certificate being validated
#             is present and valid then the certificate must not be listed on the CRL.
#                If the CRL is present but it is outdated (or anyhow else corrupted) then the validation fails.
#                   If CRL is missing then validation is successful.
#
#       IGNORE
#
#          CRL is not checked even if it exists.
#
dcache.authn.crl-mode=REQUIRE

# ---- On-line Certificate Status Protocol usage mode
#
# On-line Certificate Status Protocol (OCSP) is an alternative to CRLs in which
# dCache consults an external service to check the validity of a particular certificate.
#
# This setting controls how dCache makes use of this protocol. The following documentation is copied
# from the EMI CANL library used by dCache.
#
#       REQUIRE
#
#          Require, for each checked certificate, that at least one valid OCSP responder is defined and
#             that at least one responder of those defined returns a correct certificate status.
#                If all OCSP responders return error or unknown status, the last one received is treated as a
#                   critical validation error.
#                      Not suggested, unless it is guaranteed that well configured responder(s) is(are) defined
#                         and can handle all queries without timeouts.
#
#       IF_AVAILABLE
#
#          Use OCSP for each certificate if a responder is available. OCSP 'unknown' status and
#             query errors (as timeout) do not cause the validation to fail.
#                Also a lack of defined responder doesn't cause the validation to fail.
#
#       IGNORE
#
#          Do not use OCSP.
#
dcache.authn.ocsp-mode=IF_AVAILABLE

Finally, logging from CANL is different. Our initial impression is also that it is significantly better.

FTP Client

Another area in which dCache relied on the JGlobus client was for the FTP client. An FTP client is needed as part of our SRM client as well as for SRM and WebDAV third party copy. In the quest to drop the JGlobus dependency, we have forked the JGlobus FTP client and ported it to CANL. There should be minimal user visible changes from this. The most interesting may be that our client now identifies itself to the server as being dCache and using the dCache version number.

SRM client

As mentioned above, the SRM client has been updated to use our own GSI implementation on top of CANL for certificate handling. On a host with a large number of CA certificates, the startup time is significantly reduced.

The SRM protocol is implemented using SOAP over HTTP over GSI. Due to the old SOAP version used we are bound to the Axis 1 library. In the past we relied upon the minimalistic HTTP client integrated into Axis 1. In dCache 2.14 we added an Axis handler to use the Apache HTTP components client. The primary benefits are:

Proper keep-alive support, meaning that the client no longer needs to reestablish the connection to the server for every low level SRM request. This both reduces the latency experienced by the client and reduces load on the server.
Strict RFC 2818 host name verification.

In partiular the latter point has the potential to break compatibility with some sites. This is the case if the host certificate used by the site does not comply with RFC 2818. Without strict RFC 2818 support, TLS/GSI connections are susceptible to man in the middle attacks. The Globus Toolkit (the C library, not the JGlobus Java version) will switch to strict RFC 2818 support by default at the end of 2015 and we expect that most sites will quickly request certificates with correct subject alternative names.

If compatibility with non-compliant sites is required, we recommend using a version of the SRM client prior to 2.14. On the server side, strict RFC 2818 host name verification is only used for server side srmCopy or WEBDAV copy. If these are needed with non-compilant sites, we suggest using dCache 2.13.

Updated help pages for many admin shell commands

dCache has for several years been in a transition between two infrastructures for providing admin shell commands. The newer of these provides much better help pages. In dCache 2.14 many commands have been ported from old to the new scheme.

Admin shell gains pool group globs

The new admin shell introduced in dCache 2.13 provides bulk commands with cell name globs. In dCache 2.14 pool group globs have been added to these commands. A pool group glob follows the pattern pool/poolgroup, with either side accepting ? and * wildcards. It expands to all pools that match the left side of the pattern and which are within a pool group matching the right side. Either side may be empty and is equivalent to *.

Thus / will match any pool that appears in any pool group, /atlas_disk matches any pool in atlas_disk, and *_rack1/atlas_buffer matches any pool with a name ending in _rack1 in the atlas_buffer pool group. These could for instance be used to perform bulk commands over a set of pool:

\s /atlas_tape flush set interval 600
\s /atlas_tape save

Significant schema changes in Chimera

The Chimera database schema has been updated to improve throughput, reduce latency and reduce disk space. It will be updated automatically the first time the PnfsManager is started, but is is advisable to apply the schema changes ahead of time by using the dcache database update command. It is also advisable to make a backup of the database prior to upgrading as well as run a test migration on a clone of the database to know how long it will take as well discover any incompatibilities with local schema modification.

Once upgraded to dCache 2.14, one cannot downgrade to 2.13 without rolling back the schema changes. This must be done before downgrading using the dcach database rollbackToDate command.

Several schema changes are applied durin upgrade:

The . and .. directory entries are no longer stored in the database. Since Chimera is stored in a relational database, it can find parrent directories without these backpointers. Where applicable, these directories entries are created on the fly in dCache.
Directory tags are no longer created by a trigger. In the past the trigger reacted upon the insertion of the .. entry, but since we no longer insert this entry we cannot rely on this trigger. Furthermore, for the temporary directories created for every SRM upload the tags should not be copied and the trigger needlessly slowed down creating such directories.
Inodes are now created using the new f_create_inode stored procedure. This eliminates several round trips between the client and PostgreSQL.
The type of the itagid column of the t_tags_inodes table has been changed to a 64 bit auto sequence field. This reduces storage space as well as CPU load for these tables.
The t_access_latency and t_retention_policy tables have been merged into the t_inodes table. The former two tables are dropped. This reduces storage space as well as latency in creating and reading inodes.
The address mask is removed from the t_acl. Chimera ACLs no longer support access control by client IP.

Chimera PostgreSQL 9.5 driver

Chimera has pluggable RDBMS drivers. dCache 2.14 adds a driver specific for PostgreSQL 9.5 and newer. This driver avoids the excessive logging caused by primary key uniqueness violations that are common with Chimera.

To activate, set chimera.db.dialect to PgSQL95.

Chimera race conditions and performance improvements

Many rare races and sublte bugs have been fixed, and lots of small performance improvements have been made.

PnfsManager uses single request queue

Due to the extensive changes in Chimera, PnfsManager can now use a single request queue shared between all threads. Thus the info output no longer shows a queue per thread and it is easier to keep all threads busy.

Technically, there is a queue per thread-group, but since thread-groups are not currently used by Chimera, for all intends and purposes there is only one general request queue.

Forbidden and obsolete properties have been dropped

Properties marked as forbidden or obsolete in 2.13 have been removed from dCache 2.14. Properties marked deprecated in 2.13 have been marked obsolete. Please verify and fix any warnings produced by dcache check-config before upgrading.

Cryptographic cipher selection

dCache now bans RC4 ciphers by default. The ability to disable Diffie Hellman key exchange on JVMs in which this is broken has been removed since Diffie Hellman now works on all supported JVMs.

FTP gains new SITE commands

The FTP door now supports the SITE CHGRP, SITE SYMLINKFROM and SITE SYMLINKTO non-standard commands to change the file group and create symbolic links. These commands are supported by the UberFTP client.

FTP proxy limits internal network interface

FTP doors may act as proxies for the data connection. In this case the pool establishes a TCP connection to the door. Previously the door would listen to the wildcard address (i.e. all interfaces) even though only one specific address was sent to the pool to connect to. In dCache 2.14 the door now limits the server socket to this one address.

gPlazma drops support for plugin caching

gPlazma used to only instantiate a plugin once even if it was mentioned several times in gplazma.conf. This meant that only a single configuration could be applied, thus making several usecases impossible to handle.

dCache now instantiates every plugin every time it is referenced in gplazma.conf and each instance can have its own configuration. The options for forcing enabling the old behaviour have been removed.

gPlazma supports concurrent request processing

In the past gPlazma was single threaded, which in particular was a problem with plugins that called out to external services. gPlazma is now multi-threaded and will happily call several plugins concurrently. Even the same plugin may be called concurrently, which means that all plugins - including those by third-parties - must be thread safe.

gPlazma XACML plugin update for improved spec compliance

The XACML plugin has been ported to CANL too and in the process it was updated to improved compliance with the specification. The client library used by the plugin unfortunately depends on JGlobus and thus the current version of dCache still ships with JGlobus.

DCAP delegates X.509 and VOMS proxy processing to gPlazma

Previously DCAP would validate the VOMS signatures of a proxy in the door, thus requiring a vomsdir to be set up on the door. gPlazma now delegates this to gPlazma and the gPlazma voms plugin. Thus it is no longer necessary to keep a vomsdir setup on GSIDCAP doors.

NFS door publish exported paths login broker information

The NFS door now include the exported paths as part of the login broker information. This information is available to the info service and the srm door.

NFS door sees large number of fixes and improvements

As always, the NFS door has received a fair share of fixes and improvements. If you rely heavily on the NFS door, using the latest version of dCache is recommended.

Pool manager generates sorted configuration files

Previously entries where output in hash order, which meant that minor changes in configuration could cause drastic changes to the order.

WebDAV door links to https version of dCache home page

Thanks to a contribution by Christoph Anton Mitterer, the footer on WebDAV door directory listings now point to the https version of the dCache home page.

WebDAV door can report file locality

SRM has the concept of file locality, i.e. whether a file is on disk, on tape, both, offline or lost. The WebDAV door now exposes this information too:

The HTML rendering of directory listings uses the icon to indicate the file locality.
For programatic access, the WebDAV PROPFIND method.

An example of the latter follows:

$ curl -X PROPFIND -H Depth:0 http://localhost:2880/disk/test-1447231167-1 \
  --data '<?xml version="1.0" encoding="utf-8"?>
          <D:propfind xmlns:D="DAV:">
              <D:prop xmlns:R="http://www.dcache.org/2013/webdav"
                      xmlns:S="http://srm.lbl.gov/StorageResourceManager">
                  <R:Checksums/>
                  <S:AccessLatency/>
                  <S:RetentionPolicy/><S:FileLocality/>
              </D:prop>
          </D:propfind>'

<?xml version="1.0" encoding="utf-8" ?>
<d:multistatus xmlns:cal="urn:ietf:params:xml:ns:caldav"
               xmlns:cs="http://calendarserver.org/ns/"
               xmlns:card="urn:ietf:params:xml:ns:carddav"
               xmlns:ns2="http://www.dcache.org/2013/webdav"
               xmlns:ns1="http://srm.lbl.gov/StorageResourceManager"
               xmlns:d="DAV:">
    <d:response>
        <d:href>/disk/test-1447231167-1</d:href>
        <d:propstat>
            <d:prop>
                <ns1:FileLocality>ONLINE</ns1:FileLocality>
                <ns1:RetentionPolicy>REPLICA</ns1:RetentionPolicy>
                <ns2:Checksums>adler32=3b8711d6</ns2:Checksums>
                <ns1:AccessLatency>ONLINE</ns1:AccessLatency>
            </d:prop>
            <d:status>HTTP/1.1 200 OK</d:status>
        </d:propstat>
    </d:response>
</d:multistatus>

WebDAV door provides human-friendly file sizes

An abbreviated file size is shown as a hint in the WebDAV door HTML rendering. To see, however the mouse over the file entry.

WebDAV door enables BASIC authentication by default over https connections

This provides a means for clients without a certificate to authenticate with the server. Note that BASIC authentication should not be used over unencrypted connections.

Pools now always compute checksums on the fly

Recently we changes the default checksum policy for new pools from onwrite to ontransfer. The latter requires less resources as it computes the checksum during the upload rather than reading the file back from disk after the transfer.

In dCache 2.14 the ontransfer checksum calculation is no longer optional - it is always computed for streaming uploads. The onwrite policy can still be enabled, but it would calculate the checksum a second time. A future update of dCache will remove the onwrite policy.

SRM database schema changes

The SRM database is now managed by the liquibase schema management library. An existing schema from a previous installation is automatically detected, but we recommend doing a test upgrade on a clone of the database to check compatibility with any local schema modifications.

Several changes are made to the schema of the SRM database:

Missing indexes are added.
Unused indexes are removed.
The count column of the lsrequests table is renamed to cnt to avoid conflicts with the like named SQL keyword.
The encoding of user information is redone entirely and user information is now stored in a binary format.

In particular the latter change is worth paying attention to. Due to this change, existing requsts will fail during upgrade as the existing user information is deleted upon upgrade. Already finished requests will be kept in the database until they are garbage collected, but they too loose all information about who created it. Furthermore, the binary encoding in the new user record means that it is no longer easy to access this information in local queries. This change was necessary to work around limitations in the previous serialization - in short, we cannot serialize X.500 names in string form without loosing information.

SRM gains a pluggable transfer strategy mechanism

Plugins may decide in which order to serve transfer requests by clients. Two strategies are shipped with dCache with the default implementing a simple fair share strategy:

# ---- Request transfer strategy
#
# srmPrepareToPut and srmPrepareToGet requests enter the READY state when the TURL is handed
# out to the client. If the request is processed asynchronously (client is polling for the result),
# the request stays in the RQUEUED state until the client queries the result.
#
# A transfer strategy plugin has the ability to prevent the transition from the RQUEUED to READY
# state. A plugin may allow the number of concurrent transfers to be limited, or may provide
# some fair-share between parties.
#
# Two plugins ship with dCache:
#
#     first-come-first-served
#
#         The maximum number of requests in the READY state is limited by
#         srm.request.*.max-transfers, but otherwise TURLs are handed out to
#         whomever comes first.
#
#     fair-share
#
#         The maximum number of requests in the READY state is limited by
#         srm.request.*.max-transfers, but slots are kept free such that each party with
#         requests in RQUEUED will receive its fair share.
#
#           A configurable discriminator defines the grouping between which to provide a
#           fair share.
#
srm.plugins.transfer = fair-share

# Discriminator for the fair share transfer strategy.
srm.transfer.fair-share!discriminator = ${srm.plugins.discriminator}

Deprecation of SRM 1 support

The SRM 1 protocol has long been superseeded by the SRM 2.2 protocol. dCache 2.14 adds options to disable SRM 1 support and disables it by default:

#
# SRM versions to support.
#
# Comma separated list of SRM versions to support.
#
(any-of?1|2)srm.version=2

You can reenable SRM 1 support by adding it to the above configuration option. Please contact us if you do so we can ascertain when to drop SRM 1 entirely.

This change also removes support for third party srmCopy to/from SRM 1 servers, no matter whether SRM 1 support in the server is enabled or not.

SRM request scheduler state simplification

SRM scheduler states have been simplified by collapsing several states into the new InProgress state. Besides changing the output of the info command, this will change the state values observed in the database. Sites that monitor these states will have to update their monitoring scripts. The job state mappings can be viewed here: https://github.com/dCache/dcache/blob/master/modules/srm-server/src/main/resources/org/dcache/srm/request/sql/srmjobstate–214.csv

SRM no longer retries requests

SRM used to have support to retry requests internally upon errors. The implementation of this functionality was problematic since

many errors were not retried even when they should, and worse
many errors that should not be retried were.

In the interest of fail fast behaviour and simpler code, support for internally retrying requests has been dropped. Failures are propagated to the client and it is up to the client to determine whether and how to retry. This also allows client to more quickly fall back to other sites in case of problems.

SRM changes algorithm to detect a local SURL

The SRM protocol has a concept of site URLs and classifies these into those local to the storage system and those belonging to other storage systems. This classification is important when the SRM is to determine which file is local and which is remote in a third party SRM copy operation, or just to determine if the SURL is indeed local in put or get operations.

The most important changes are:

srmCopy is now limited to having a local source or local destination SURL. It no longer allows a transfer between a local non-SRM door and a remote system.
no DNS lookup is performed in determining whether the SURL is local. Previously the host name would be resolved to an IP addressed and the IP addressed would be compared to local addresses. Now the list of local host names is determined during startup and the host name in the SURL is compared to the local names directly. An administrator still has control over which names to consider local by setting the srm.net.local-hosts property.

Several minor SRM fixes

Several corner cases and race conditions have been fixed, as well as better failure handling to prevent leaking pins or upload directories.

Configurable kXR_Qconfig replies for xrootd

The xrootd protocol has a mechanism to query the server configuration using a kXR_Qconfig request. dCache provided a few hard-coded properties, but the xrootd protocol specifies an open list of properties.

In dCache 2.14 the admin may inject additional properties to be returned to the client. These are configured through the dCache configuration system:

#  ----- Custom kXR_Qconfig responses
#
#   xrootd clients may query the server configuration using a kXR_Qconfig request.
#   These key value pairs can be queried. Additional key value pairs may be added as
#   needed, see the xrootd protocol specification at http://xrootd.org/ for details.
#
(prefix)xrootd.query-config = kXR_Qconfig responses
xrootd.query-config!version = dCache ${dcache.version}
xrootd.query-config!sitename = ${dcache.description}
xrootd.query-config!role = none

#  ----- Custom kXR_Qconfig responses
#
#   xrootd clients may query the server configuration using a kXR_Qconfig request.
#   These key value pairs can be queried. Additional key value pairs may be added as
#   needed, see the xrootd protocol specification at http://xrootd.org/ for details.
#
(prefix)pool.mover.xrootd.query-config = kXR_Qconfig responses
pool.mover.xrootd.query-config!version = dCache ${dcache.version}
pool.mover.xrootd.query-config!sitename = ${dcache.description}
pool.mover.xrootd.query-config!role = none

Many third party libraries have been updated

Most notably the new version of the PostgreSQL JDBC driver has some nice performance improvements.

Changelog from 2.13.0 to 2.14.0

388ee5d: chimera: Propagate failures to read or write in-db data
7f38e2a: srm: Rename count column
f6aa2a3: common-security: Do not log stack traces for CANL notifications
4d2ce2d: srm: Use correct logging context when saving jobs
984c385: common-security: Lower log messages for CANL notifications
dc54153: srm-client: Add initialization of Axis handler in SRM 1 client
88be0f1: srm: Fix NPE when saving requests with an unknown user
90689dd: srm: Fix race in user persistence
c1a2040: gplazma: fix erroneous logging in x509 plugin
ab6c401: pom: generate rpm friendly dirty versions number
9c851b0: gplazma: remove support for plugin caching
1c53eea: gplazma: make x509 generation of LoA principals optional
bac28ce: [maven-release-plugin] prepare branch 2.14
886fe81: webdav: enable BASIC authn for https by default
99e5ec1: Logging CANL notifications to correct cells context
9a2689e: gplazma2-voms: Fix error message
dba3da8: pool: Lazily initialize trust manager for remote https and gsiftp movers
d58e8a0: pom: Downgrade liquibase to 3.3.2
6d6676e: cells: Fix NPE during shutdown
c58fc42: srm: Reimplement legacy close
ed89d6b: dcap,ftp: Add remote address to logging context while creating child cell
06cc7d1: gsidcap: Avoid including payload data in the log
4ffef54: gplazma2: Call plugins concurrently
50581b4: Downgrade PostgreSQL jdbc driver
3586071: liquibase: Downgrade to 3.3.3
e604095: srm: Do not attempt to drop sequence objects on upgrade
9060d2d: srm: Add missing rollback
55b603e: srm: Fix quoting for deleting old authorization tables
cd565a6: srm: Fix renaming of count column
3196ffe: gplazma2-xacml: Add JGlobus dependency
a6674ba: Update third party dependencies
a3b2467: srm: Add missing index on bringonlinerequests_protocols foreign key
0baeb6a: srm: Drop unused indexes on transitiontime columns
242ed7b: srm: Drop indexes on nextjobid column
90cd0b2: srm: Rename count column of lsrequest table
480c8f5: srm: Replace AuthorizationRecord with new user manager
1f86515: x509: add Principals that describe LoA and Entity-Definition
c3984dc: pool: Do not announce cell until the cell is running
d08db11: pool: Refine Berkeley DB failure handling
f95ef8f: srm: Report correct estimated wait time for SRM 2.2 copy requests
b8359f0: common-security: Renamed javatunnel package
9b77b04: Revert “Update third party dependencies”
e60ea77: Update third party dependencies
da7693d: voms: Fix resource leak regression
69eac64: srm: Fix regression in database schema for ls requests
9f7311c: srm: Fix regression in extracting paths from local SURLs
977ddd8: srm: Port request id table schema management to liquibase
82183bf: ScriptNearlineStorage: Fix missing -uri parameter for fetch
26f7ca1: srm: Port schema management to liquibase
ee59461: doors: Add CANL listeners for logging errors and warnings
a7556e5: Reduce duplication between context factories
e84a24d: system-test: apply IF_VALID crl policy to all services
b1767a2: pool: java8 into IoQueueManager
9e7687c: pool: simplify pool queues
f10ee1f: build: Fix build performance regression caused by git describe
9d512b0: Revert “build: Fix build performance regression caused by git describe”
59c54ef: build: Fix build performance regression caused by git describe
1b1793f: srm: Fix regression in detection of local SURLs
47b11bf: Drop JGlobus
a84b439: xrootd: Upgrade to xrootd4j
85b5949: webdav: update file icon(s) in directory HTML page so they show file locality
d4ec10f: ftp-client: Resolving merge conflict
4fe019b: webdav: Respect webdav.static-content.uri property
49fcd3e: doors: Do not log failure to delete absent files on upload failures:
79a3ce7: ftp-client: Port to our own GSI implementation
d1aaa05: javatunnel: Add client side GSI implementation
173e7df: ftp-client: Rip out DCAU support
bdb4e83: javatunnel: Decouple from Jetty
bfd3f90: ftp-client: Push X509Credential to GSSCredential conversion into the client
d3b2477: ftp-client: Refactor control channel implementation
f262180: ftp-client: Port TCP port range and buffer size handling to dCache
b5d649b: ftp-client: Use Unix style newline encoding
90df88f: ftp-client: Automatic IntelliJ fixes
4ec8439: ftp-client: Add missing @Override annotations
c1d39dc: ftp-client: Use our own networking and version information
aa28fbd: common: Refactor PortRange and move to common module
7b9d121: ftp-client: Automatic code reformatting
a0b56d6: ftp-client: Delete dead code
b4a31b2: ftp-client: Move code to org.dcache.ftp.client
330792d: ftp-client: Import JGlobus gridftp client code
22ab9db: srm-client: Port SRM client to Apache HTTP components and CANL
3780943: pool: Turn remote HTTP movers into dedicates transfer service and port to CANL
cecc4c1: nfs4: fix race in request processing
bc168bf: libs: update to nfs4j–0.11.2
bd5c32a: webdav: Add robots.txt
501ed18: srm-client: Get rid of GlobusURL
28d23b7: Stop using JGlobus for delegation CSR
68b18ba: srm: Port most credential handling code from JGlobus to CANL
cbe6985: pool: implementing annotated command syntax for pools (3)
099bc30: pool: implementing annotated command syntax for admin interface commands (4)
76d2ac5: pool: implementing annotated command syntax for admin interface commands (SchedulerEntry)
d4613a5: hsmcp: update to match new HSM interface.
0d6b1c8: cells: clean up
1af9745: webdav: Port COPY credentials to CANL
eaf91db: gplazma: Port to CANL
9be9dfb: common: Copy GlobusPrincipal to dCache code base
5778614: ftp,dcap,srm,httpd,webdav: Switch to CANL for certificate handling
f311c37: ftp: Refactor FTP door to separate settings from the interpreter
5d4fcda: ftp: Reuse DssContxtFactory
66845ec: webadmin: improve user-friendliness of alarms page on load
9165b28: old replica manager: prevent pool being listed as offline when there are files with corrupt metadata
bad630a: webdav: add support for querying file locality.
99e4a00: Avoid deprecated sameThreadExecutor
fc1fc7c: webdav: Kill abandoned movers
57ee102: xrootd: Kill mover on aborted write
719a1c7: pool: Fix transfer prioritization
5c3800d: pool: Add nearline storage default timeouts
b2372cb: ftp: Refactor interpreter instantiation
5329756: xrootd: Fix race condition in request cancelation
0644d66: pool: Let script nearline storage provider scale down when lowering limits
223f02c: pom: remove dependency on grizzly
10a20e4: srm-client: Delete dead code
d465784: Delete dead test code
85c4086: srm: Delete dead test code
251b67b: dcap: Fix broken argument parsing (for krb5)
634c712: cells: Refactor option parsing
92dd85f: system-test: Restrict port range to avoid conflicts with services
506f4ab: dcap: Fix broken argument parsing
708fdd4: dcap: Fix broken argument parsing
f0eb468: xacml: fix voms attribute validation
576848f: ftp: Limit proxy to the interface that the pool connects to
55c5c5d: Fix parsing of single port port range
6b57edd: cells,javatunnel: Revert dependencies
e0ddf83: dcap: Fix socket factory argument parsing
1428da0: system-test: Restrict more sockets to loopback
ec6d153: Fix build and startup regression
1323016: srm: Disable SRM 1 by default
1a8b2a1: srm: Drop dead configuration code
a94ce34: srm: Remove standalone server
adbf507: srm: Drop generic support for retrying requests
e9542f7: srm: Delete upload directory if SRM request gets garbage collected
4fc93f0: srm: Move final state check out of Scheduler
78c2996: srm,ftp,dcap: Respect banned ciphers
431e801: pool: Minor refactoring of remote HTTP mover
0168ba9: pool: Fix error reporting in remote HTTP mover
b746c36: pool: Update HTTP client and avoid deprecated calls
1472ae2: libs: update to nfs4j–0.11.1
59a67f4: ftp: Don’t hold locking during proxy shut down
9a424a8: ftp: Fix deadlock triggered by failures
9a5c117: HSM Script: pass arguments in array
1d3bf97: srm: Unpin files when aborting prepareToGet
99bdf4b: srm: Refactor Scheduler#schedule method
5783a1f: srm: Add in-memory history logging
8c35457: srm: Simplify scheduler states
06377ce: srm: Refactor exception handling in Scheduler
13daf0e: srm: Refactor CopyRequest to make more fields final
7edc287: srm: Don’t use SRM 1 for srmCopy
fa95576: srm: Remove third party copy from SRM front-end code
ea23dc8: srm: Avoid URL copying in copy requests
e0c4af0: srm: Make list of file requests immutable
5cf98e4: srm: Drop dead code
e87fa4a: common: update NetLoggerBuilder to use Logger during building
4f69954: chimera: Another attempt at fixing auto-generated key extraction
95be135: ftp: Fix reporting of CRL expiration
e709eac: chimera: Fix Postgresql compatibility
b43de19: admin: Update help strings
d01c45c: admin: JavaDoc and some renaming
9abbbc7: admin: Add pool group patterns
71d2e00: libs: Update to Wicket 7
e124392: libs: Upgrade Jetty
ccfe23d: libs: Update to Spring Framework 4.2
ce37d36: libs: Update jline
b5fbb05: libs: Update dependencies
4848a7f: chimera: Use auto-increment field as itagid
83e2d8b: Port more Guava code to Java 8
30b1b44: chimera: Refactor transaction template use
d298cbb: pnfsmanager: Fix regression in cancelling uploads
3a94450: chimera: fix object type check on create
718212c: nfs: fix behavior on parentOf
5ad1fd5: pools: documentation and implementation of command annotation for pool sweeper commands
3c995b9: topology: Improve documentation of topo cell commands
b1ac329: docs: Point README at BUILDING
849a960: cells: xgetcellinfo and info commands documentation
e92b91c: cells: admin interface commands conversion with implementation of the command annotation for better documentation
54ee545: pool: removal of crash command in PoolV4
72c2765: chimera: Fix incomplete stat bug in listing
87254c4: nfs: fix lookup of ‘..’ after 2677f7ac
9699094: nfs: do not check for proxyIO factory
ec3d568: javatunnel: Fix build failure
81d4076: srm: return invalid request when the result of ls request is unknown
49428fb: Relocate GSI related classes to javatunnel
7f314ff: dcap,ftp: Wrap GsiEngine to provide GSI support
b6c66a4: chimera: Resolve performance regression in directory deletion
25e5832: chimera: Optimize inode creation with stored procedure
0d19395: chimera: Populate stat cache on inode creation
66e47ae: pnfsmanager: Improve caching in Chimera inode wrapper
2677f7a: chimera: Do not store ‘.’ and ‘..’ entries in the database
2228bb8: chimera: Fix hard link creation in the pressence of ACLs
7074fe1: chimera: Drop trigger to copy tags
2c4042a: pnfsmanager: Fix races in file deletion
8ca7b9b: pnfsmanager: Fix races and hard link issues in move
f9a2473: chimera: Fix races in move/rename logic
04b1887: chimera: Fix two race conditions in file deletion
6aaf08b: chimera: Fix remove by id
c299bb8: pnfsmanager: Use a single shared request queue for threads
ed7b90e: chimera: Port Chimera to Spring JDBC
6f6d48a: nfs: cleanup exception handling/conversion
787724b: chimera: Close datasource on shutdown
afc5e02: libs: switch to nfs4j–0.11
254a0ce: nfs: integrate ChimeraVFS into dCache code
10cfa65: chimera: merge access_latency and retention_policy into t_inodes
9525a3d: chimera: Fix performance issue in directory deletion for HSQLDB
10d7343: Improve execution time statistics
6f7adf0: ftp: Refactor GSS FTP door on top of DssApi
6f8d7a2: gsidcap: Let GSI DCAP use gPlazma for certificate chain processing
666b447: dcap: Refactor javatunnel
14f23b2: chimera: Refactor and fix performance test tool
73b2d94: doors: Fix timeout handling during retries
1ec6400: Fix timeout math to avoid overflow
604a0ba: srm: Fix credential delegation
517055b: doors: Fix string parsing for door tags and paths
b4fd156: chimera: Invalidate stat cache
5a1399b: pnfsmanager: Refactor inode wrapper
1cc90bd: dcap: Respect dcap.authz.anonymous-operations when publishing login information
56edf34: srm: Cache the vomsdir and ca stores
0f77a7a: srm,dcap: Respect cadir and vomsdir settings
143652b: cells: conversion and improve documentation of get hostname command Motivation:
8a0915d: cells: conversion and improve documentation of get hostname command
ecbb816: CellGlue: Bug fix in CellGlue
3711b70: rpm: remove “commented out” macros lines from spec file
decafd2: cells: Fix dumpster and default route removal
c6eeea9: cellshell: admin interface commands (method -> class conversion)
6e2f642: edited
2622c8e: edited
d00b041: chimera: fix nameof and pathof for paths containing unicode
a20f6d1: pool: include client’s InetAddress when discovering corresponding mover
749e39e: poolmanager: Make psu output deterministic
9ba6599: httpd: Fix transfer collection for duplicate movers
9dae274: pnfsmanager: Remove uid,gid and mode from upload path creation
40a9516: chimera: Let SRM respect setgid on upload
e9355db: srm: Add authorization to put done and abort requests
bac6cea: pnfsmanager: Minor refactoring of access mask check for delete
e2c25b1: pnfsmanager: Move PnfsID verification on delete into name space provider
848a10f: pnfsmanager: Push file type check for file deletion to name space provider
08bbd57: pnfsmanager: Deprecate path field in PnfsDeleteEntryMessage
8dab192: module: cells
2784404: nfs4: more java8 cleanups
c67b553: nfs: publish exports paths in login broker
c280f98: nfs: add an utility class to convert CacheExceptions into NFS Exceptions
a71eac1: dcache-webadmin: add TimeoutCacheException to catch clause in billing service
b270a63: Fix broken commit 3bfcc2da
3bfcc2d: pool: fix ftp mover to provide better logging when failing to connect
108e86b: crypto: remove banning of broken DH ciphers
7205931: crypto: ban RC4 by default
7de9b7b: PoolManager: batch file typo
9806a32: PoolManager: batch file typo
09db1ed: properties: update description of dcache.net.listen
3890809: ftp: consolidate transaction-log creation
e8aa7d1: crypto: refine handling of broken ciphers
26182b7: libs: update to nfs4j–0.10.6
21cc1d3: improve to the webdav.templates.config!footer URL
2643c63: ftp: minor refactor
ad7214f: chimera: make discovery of directory’s parent more efficient
571f65d: pool: always use on-transfer checksum
abc4583: doors: Include root path in login broker info output
4e1f316: admin: Fix cell name tab completion
e23a630: cell: Don’t quote string command in event logger
24b08f9: srm: Fix read path check for srmPrepareToGet
0235ddb: pool: Avoid rereading cache entry when enqueing file for flush
9f5871d: pool: Explicitly enable compressed oops to calculate correct cache size
4ec8d0c: pool: Fix locking bug causing high memory usage during pool initialization
3e3800c: chimera: fix FsInode_CONST#stat()
a1bc2be: chimera: add PostgreSQL 9.5 specific driver
b37e0eb: chimera: Fix path resolution on delete
8337f1f: gplazma: Remove synchronization making gPlazma single threaded
80b613a: xrootd: Refactor xrootd redirect handler concurrency
ce113f3: xrootd: Fix ‘xrootd logs stack-trace on malformed request’
ca0b0a5: move execution of the superclass method before any concrete class initializations
8b4d397: gplazma: Fix VOMS plugin regressions
5e19ef8: pool: do not list a repository during initialization
73441f8: xrootd: Add session to access log
cecd533: xrootd: Add support for custom kXR_Qconfig replies
7e25b6b: pool: update abstractMover to use ChecksumChannel on write
cd7637d: pool: Fix race condition in migration module
095d689: xrootd: Fix GSI authentication
14ee077: pool: Disable pool on meta data failures
1d7f0b4: srm: fix race condition in ls response
3c70fa2: chimera: fix acl triggers and changeset rollback procedure
a90d59b: srm: Add transfer strategy plugin mechanism
a34fe50: srm: Fix wrong Spring factory bean type
a16d63a: srm: Fix illegal state transition from Failed to RetryWait
f72fa50: acl: remove origin evaluation
e6847b4: PoolStatisticsV0: refactoring
df69b4a: PoolStatisticsV0: more whitespace changes
831a988: PoolStatisticsV0: fix indentation
91e1084: gplazma-xacml: add GlobusPrincipal (DN) to identified principals
c345ed2: systemtest: fix install command in credentials command
0dc4063: chimera: throw FileExistChimeraException if tag already exists
75749f3: srm: Fix merge failure in ‘Make supported SRM protocol versions configurable’
ad21ac8: srm: Make supported SRM protocol versions configurable
36fc0b5: webadmin: do not display numerical value for max restores or stores
e1d998f: p2p: include source pool into thread name
0263743: src: add README.md to make Github page more friendly
f22243f: Revert “webadmin: do not display numerical value for max restores or stores”
fcd2542: webadmin: do not display numerical value for max restores or stores
44b3ed9: srm: Fix illegal state transition from Canceled to Failed
10e2f67: all: Clean up FQAN extraction from Subject
0762be1: ftp: add support for SITE SYMLINKFROM / SYMLINKTO commands
7b3fbd5: boot: Don’t fail on missing layout file
5279408: all: Replace direct Predicate instantiation with lambdas
a1f50f6: all: Replace direct Function instantiation with lambdas
63ad6e8: srm: Use Java 8 constructs for SchedulingStrategy implementations
edfa418: srm: Refactor scheduler to avoid unused queues and per user counters
214963f: srm: Reimplement scheduling policy to resolve scalability issues
6302038: webdav: provide a human-friendly size as file-size hint
3a67610: ftp: add support for SITE CHGRP command
38ec2c7: httpd: Do now show maximum for restore and flush queues
53f637e: deb: Downgrade recommended dependencies and add to description
ec1c6bb: poolmanager: Reduce log level of p2p denial
0da7ac6: pool: remove redundant code in AbstractMover
1823449: pool: do not use RandomAccessFile in FileRepositoryChannel
99b1d03: libs: update to nfs4j–0.10.5
f1a0f51: infoprovider: remove single SRM instance limitation
edffb45: pool: simplify duplicate request handling
57f5e2c: pool: do not create new stateid in NfsMover#getStateId()
890fd80: pool: reduce load on back-end file system
2cda82d: pool: introduce unique port number for nfs mover
0b8d50d: pool: Store nearline storage timeouts to pool setup file
fae7341: pnfsmanager: Create base upload directories without tags and acls
9feb7c3: pnfsmanager: Inherit ACLs on upload with SRM
d38b70a: chimera: Add ACL insert triggers for HSQLDB
73fc966: Fix limited class path generation
f39aa93: pool: Let random pool selection select pools with removable files
8b72498: crypto: allow banning of RC4 cipher suites
dfafdf6: spacemanager: Allow unowned files in reservations
7090629: srm: Don’t log erroneous stack trace
c7581cf: Delete deprecated classes
bc29c80: configuration: Mark deprecated properties as obsolete
d8244d4: configuration: Delete obsolete properties
5400ea8: configuration: Delete forbidden properties
20cce56: nfs: include read/write information to transfers
59bf7e4: replicamanager: use Collections.emptyIterator() when needed
6a2d877: releases: update dCache version to v2.14

What’s new in dCache 2.14 The release notes

Executive summary

Incompatibilities

Release 2.14.51

admin

Changelog 2.14.50..2.14.51

Release 2.14.50

httpd

Changelog 2.14.49..2.14.50

Release 2.14.49

pool

Changelog 2.14.48..2.14.49

Release 2.14.48

webdav

Changelog 2.14.47..2.14.48

Release 2.14.47

Changes affecting multiple services

system-test

Changelog 2.14.46..2.14.47

Release 2.14.46

chimera

Changelog 2.14.45..2.14.46

Release 2.14.45

chimera

ftp

Changelog 2.14.44..2.14.45

Release 2.14.44

ftp

Changelog 2.14.43..2.14.44

Release 2.14.43

ftp

srm

Changelog 2.14.42..2.14.43

Release 2.14.42

xrootd

Changelog 2.14.41..2.14.42

Release 2.14.41

srm

systemtest

Changelog 2.14.40..2.14.41

Release 2.14.40

cleaner

pool

Changelog 2.14.39..2.14.40

Release 2.14.39

ftpclient

Changelog 2.14.38..2.14.39

Release 2.14.38

dcap

poolmanager

Changelog 2.14.37..2.14.38

Release 2.14.37

Changes affecting multiple services

admin

billing

dcap

doors

pool

Changelog 2.14.36..2.14.37

Release 2.14.36

doors

Changelog 2.14.35..2.14.36

Release 2.14.35

billing

dcap

gplazma

httpd

scripts

Changelog 2.14.34..2.14.35

Release 2.14.34

commons

gplazma2

srm

Changelog 2.14.33..2.14.34

Release 2.14.33

cells

Changelog 2.14.32..2.14.33

Release 2.14.32

commons

dcache

What’s new in dCache 2.14
The release notes