Executive summary

• Reduces chimera overhead on file creation and file open.

• New database connection pool for more robust database interaction.

• Supports NFS 4 (without parallel NFS)

• DCAP clients can now be forced to do I/O through a mounted NFS 4 file system.

• Much improved performance and standard compliance for NFS 4.

• Improves support for meta data in FTP.

• Improves FTP client compatibility.

• Implements FTP delayed passive, allowing IPv6 FTP transfers without proxying.

• New pool manager partition for transfer buffers (e.g. import and export pools).

• Improves ROOT compatibility for HTTP access.

• GridSite delegation support for SRM.

• Unique TURLs for SRM upload for improved error handling.

• Improves transfer protocol negotiation for SRM.

• Reduces impact of SRM restart.

• Improves protocol support for xrootd.

• More flexible tape integration.

Incompatibilities

• The spacemanager database schema has changed. Changes are automatically applied after upgrade. Downgrade is not possible without rolling back the schema changes. See the section on the spacemanager service for details.

• The chimera database schema has changed. Changes are automatically applied during upgrade. Downgrade is not possible without rolling back the schema changes. See the section on chimera for details.

• The new property dcache.upload-directory may have to be defined on upgrade. Read the section on the srm service for further details.

• Third party plugins that provide custom service definitions may have to be updated to be compatible with dCache 2.9.

• SRM uploads that are not finalized by submitting a srmPutDone will be deleted. The srmPutDone message is required by the SRM standard and standard compliant clients will not have a problem with this. The file is not available for reading until srmPutDone has been issued.

• IPv6 support is enabled by default.

• Tape integration was rewritten. Classic HSM scripts continue to be supported, however the pool setup file should be regenerated after upgrade. See the section on the pool service for details.

• The HSM cleaner is enabled by default.

Release 2.9.27

Changes affecting multiple services

dCache uses a standard format to monitor the performance of various components: in the srm door to record how quickly SRM requests are processed (print srm counters command), the generic cell message monitoring (monitoring info command), and pnfsmanager service (the “Statistics” section in info). This release fixes a rounding error that prevents these statistics from including long-lived requests.

pinmanager

The pinmanager service has the ls command that allows the admin to limit the results to a specific pin or all pins against some PNFS-ID. This release fixes listing by pin id.

srm

This release fixes the srm service so it cancels corresponding pinmanager requests when an SRM client aborts a bring-online request.

Changelog 2.9.26..2.9.27

cf4affe

[maven-release-plugin] prepare release 2.9.27

3e12a3b

common: Fix division by zero regression in gauges

71adc02

common: Fix rounding error in request gauge

7bfd5c7

srm: Abort pinning when cancelling bring-online requests

7d8dff4

pinmanager: Fix listing by id

ccaba7e

system-test: add missing dCache disposible CA certificate

0098f1d

system-test: add regenerated host and user credentials

bfef06f

[maven-release-plugin] prepare for next development iteration

Release 2.9.26

dcap

Update dcap door to avoid possible NullPointerException when describing an error to the client.

Changelog 2.9.25..2.9.26

047ce71

[maven-release-plugin] prepare release 2.9.26

a397d8a

dcap: do not call toString() on error object

250326f

[maven-release-plugin] prepare for next development iteration

Release 2.9.25

pool

When a client reads a file, the pool reads blocks of data from the local filesystem. When reading such a block, the pool could receive fewer bytes than requested. Previously, the pool assumed that this only happens when the end-of-file is reached; however, this is not guaranteed. Should this assumption be violated then the data sent to the client will be corrupt. In practise, the pool’s assumption is true for Linux and local filesystems; however, the code has been updated to remove this theoretical cause of corruption.

Changelog 2.9.24..2.9.25

38602db

[maven-release-plugin] prepare release 2.9.25

65fe614

pool: Fix read corruption in HTTP mover

5cdae6e

[maven-release-plugin] prepare for next development iteration

Release 2.9.24

Changes affecting multiple services

If a bug is found in dCache then it should be logged. For the older admin commands, an uninformative message was logged.

Previously, if a bug was discovered when starting up a service dCache would abort starting up with a non-informative log message. Now, the stack-trace is logged.

pnfsmanager

When writing into dCache with SRM, the Access Latency (AL) and Retential Policy (RP) may be specified or omitted. Additionally, the client may specify a space reservation into which the file should be written. If the client specifies both, they must match.

In dCache, there are three mechanisms to support a client that specifies neither AL/RP nor space reservation: the directory can have AccessLatency and RetentialPolicy tags, the directory can have the WriteToken tag, there are a system-wide default AL/RP values.

Previously, dCache would reject uploads where the user-supplied AL/RP information does not match the AccessLatency/RetentialPolicy tags, despite the latter being intended as default values.

With this release, if the client specifies neither space token, AL or RP then the directory tags will be used. If a directory specifies both WriteToken and AccessLatency/RetentionPolicy tags, then these have to be consistent. If the directory conains a WriteToken tag and the client specifies AL/RP, then the client specified values have to be consistent with the WriteToken tag.

pool

When attempting to upgrade non-precious and non-cached files (e.g. a file marked broken), the receiving end of the migration module would answer twice: first (correctly) with a failure and then (incorrectly) with a success. This is now fixed.

Changelog 2.9.23..2.9.24

a51dd3a

[maven-release-plugin] prepare release 2.9.24

5d6fd19

cells: log bugs found by CellShell

c02aeaa

cells: fix how bugs are reported from ac_ command.

255fa85

pool: Fix bug in migration module upgrade logic

e51c5b2

pnfsmanager: Fix upload to space token that conflicts with AL and RP tags

7d30b6a

[maven-release-plugin] prepare for next development iteration

Release 2.9.23

Changes affecting multiple services

Various scripts, including the dcache command, invoke the java command with a list of directories in which Java should look for support libraries. Previously, the current working directory was (mistakenly) included in that list. This could lead to odd behaviour; one particular example is running a dcache database command from the /etc/dcache directory. This release fixes this problem by excluding the current directory.

When there is some problem in the communication between domains and error message is logged. Previously the explanation for the problem was logged as “null”. With this release, a more descriptive explanation is provided.

spacemanager

Fixed the ls spaces and ls files commands so they do not fail if there is a reservation without an owner.

Changelog 2.9.22..2.9.23

e02cb1c

[maven-release-plugin] prepare release 2.9.23

d776ce6

spacemanager: Fix NPE in listing space reservations

0af6f6e

tunnel: use toString if IOException#getMessage returns null

0256405

Exclude cwd from classpath

7af5f04

[maven-release-plugin] prepare for next development iteration

Release 2.9.22

Changes affecting multiple services

This patch fixes a race condition in Chimera that affects the nfs door and the pnfsmanager service. The effect is that, if two clients attempt to delete the same target (a file, link or directory) at the same time then the nlink count for the parent directory is decreased twice. “At the same time” means within the time taken to process the deletion; this is instance-specific but should be much less than 1 ms for well-configured systems. Sites can repare any incorrect nlinks with the following SQL:

UPDATE t_inodes SET inlink = (
    SELECT COUNT(*) FROM t_dirs  WHERE t_inodes.ipnfsid = t_dirs.iparent
) WHERE itype = 16384;

This is safe to run on a running production instance, but may take some time and will affect dCache’s responsiveness while running.

gplazma

The description for how to migrate away from using the forbidden useGPlazmaAuthorizationModule and useGPlazmaAuthorizationCell properties had caused confusion. The description has now been updated to be more explicit.

httpd

Fix filtering boxes and sorting on Pool Admin, Pool Usage, Poolgroups, Space Tokens and Tape Transfer Queue.

srm

Uploading a file with SRM involves three steps: preparing for the upload (srmPrepareToPut), uploading the file, marking the upload finished (srmPutDone). The third step can fail but previously the response from dCache is always Upload failed.. With this release, a meaningful error message is returned.

Changelog 2.9.21..2.9.22

273a6bd

[maven-release-plugin] prepare release 2.9.22

2dfe39a

gplazma: update error message for forbidden properties

fd03121

(2.9) webadmin: make jquery selector specific to individual tables

d826d12

(2.9) webadmin: restore missing components to respect jquery script options

cc7d855

srm: include the reason why upload failed

4ee0db1

chimera: fix race condition on remove

344adbd

webadmin: ensure unique id attributes for all (currently) tested UI elements

159a2dc

[maven-release-plugin] prepare for next development iteration

Release 2.9.21

ftp

Fix default value for ftp.authz.readonly for plain (unencrypted) doors. This restores the default value to the dCache v2.6 default value of true.

The response from the plain (unencrypted) ftp door if the user specifies the wrong password is badly formed. Althogh it is possible that some clients are robust against such incorrect responses, with this release the ftp door responds correctly.

webdav

Fixes a bug where, if a double-slash is present, all parts of the path leading up to the double-slash are ignored; for example, with the bug, a path like /a/b//c/d is handled as if /c/d was specified. With this release, double-slashes are treated like single slashes; the above example is handled as if /a/b/c/d was specified.

Fix the NullPointerException triggered if client attempts to upload a file as a child of some existing file.

Changelog 2.9.20..2.9.21

9a9624d

[maven-release-plugin] prepare release 2.9.21

c258357

ftp: fix response if user fails to authenticate to weak FTP door

6ae35be

ftp: fix invalid default for ftp.authz.readonly property

ac12004

webdav: fix double-slash bug by upgrading to patched milton

1add5a4

webdav: fix NullPointerException when PUT as a child of a file

050ff78

[maven-release-plugin] prepare for next development iteration

Release 2.9.20

Changes affecting multiple services

Previously releases of dCache claimed to support Java–8; however, this is not true. For Java–8 support, sites must upgrade to dCache v2.10 or newer.

httpd

Fix two minor issues when authenticating with the webadmin interface: “unauthorised access” and being redirected to the home page. The unauthorised access error can occur when selecting “Login” under the bird logo (top right corner); this is now fixed. The redirection problem occurs when selecting a tab that requires administrative privileges while not logged in; this redirects the browser to the login page. Previously, after a successful login, the browser was redirected to the home page. Now the browser is redirected to the selected tab.

webdav

A previous bug-fix release fixed how dCache responds when the client attempts to DELETE a non-existent file. Unfortunately, this triggered a different problem where such activity results in a stack-trace that starts java.lang.ClassCastException: java.lang.String cannot be cast to javax.security.auth.Subject. This second problem is now fixed.

Changelog 2.9.19..2.9.20

6639259

[maven-release-plugin] prepare release 2.9.20

d29b079

Disallow Java 8

e4428e0

webdav: Alternative to fixing return code of DELETE of absent file

5b1a81e

(2.9) webadmin: fix login redirect bug

7fe25d4

[maven-release-plugin] prepare for next development iteration

Release 2.9.19

Changes affecting multiple services

It is possible that a sudden burst of activity from many clients exceeds dCache capacity to queue such requests. Although dCache is designed to degrade gracefully under such circumstances, there existed the possibility of certain requests becoming stuck or memory leaking. This is now fixed.

info-provider

In previous releases, the info-provider assumed the broker domain is dCacheDomain. This assumption has been removed.

nfs

Upgrade to nfs4j v0.8.5. This fixes export file parsing if a host is mentioned multiple times; note localhost must now have an explicit entry in the exports file. This also fixes a deadlock when closing a file on a busy dCache instance.

pnfsmanager

Fix NullPointerException when a file is stored in a directory with an empty tag.

pool

Upon reloading the pool configuration an error is produced when an nearline storage was already defined in the existing configuration. This is now fixed.

A recent release fixed a bug that caused pool.mover.ftp.allow-incoming-connections to be ignored. Fixing that bug revealed another that caused the property to have the opposite effect. This is now fixed.

dCache pool configuration allows passing (fixed) arguments to the HSM script. In earlier releases, these arguments were supplied to the script in an arbitrary order. While HSM scripts should not depend on the order, they might; so, with this release, the order is preserved.

webdav

Although dCache behaves correctly if the client interrupts a proxied transfer; however, this is logged as a bug. This is now fixed.

Changelog 2.9.18..2.9.19

16a165b

[maven-release-plugin] prepare release 2.9.19

ab8e906

webdav: don’t log a stack-track when proxy transfer is interrupted

fc0dd3c

info-provider: remove dCacheDomain assumption.

9d5724f

Fix compilation with Java 7

b5f3b1c

cells: Fix message timeout in case of thread pool overflows

048f4da

libs: pull grizzly-framework–2.3.12 for nfs4j

bbccfd9

chimera: handle NULL field of directory tags

98a595b

pool: Fix regression preventing configuration to be reloaded

fd20bb6

(2.9) gitignore additions for IntelliJ

8c33655

libs: update to nfs4j–0.8.5

3c778f1

pool: Fix regression causing FTP movers to default to proxy mode

1392a58

pool: Preserve HSM option ordering

c91e5f3

[maven-release-plugin] prepare for next development iteration

Release 2.9.18

Changes affecting multiple services

Although dCache system configuration property names do not contain spaces, it is possible to define such properties. Previously, doing so breaks the dcache command. This is now fixed.

If the number of concurrent connections is set to -1 then the ftp and dcap doors will leak memory, eventually triggering an out-of-memory error that will restart the domain. This is now fixed.

admin

Fix erronous reporting of bugs when the user supplies incorrect arguments to an admin command.

dcap

Fix regression against v2.6 and earlier dCache in how gsidcap doors are known to SRM and how they are published in BDII/GLUE.

ftp

The ftp door supports the HELP command as some clients use this commands output to discover if certain optional functionality is available. This release fixes the output from this command.

pool

Improve which interface the ftp mover selects when the client is redirected to the pool. In particular, the door will only redirect the client if the IP protocol matches (IPv4 vs IPv6); for example, if a client connects with IPv6 and the pool has no IPv6 address then the door will now proxy the data connection.

The pool.mover.ftp.allow-incoming-connections property had no effect. This is now fixed.

Improve shutdown of nearline storage subsystem; now dCache will attempt to cancel all ongoing activity before shutting down the pool. The previous asynchronous approach could lead to IllegalStateException being logged if the pool was busy at that time.

spacemanager

Adjust spacemanager schema migration to be aware of earlier dCache bugs and to work-around site-local indexes that clash with new indexes that dCache needs.

srm

Previously, if the SRM client requests listing a directory, specifies a non-zero offset and does not limit the response size then dCache would fail this request with an IllegalArgumentException. This is now fixed.

webdav

With the recent upgrade of the Milton library some new behaviour was introduced. One example is that, under certain circumstances (and to support certain clients) the Milton library returns a 401 (not authorised) when attempting to delete a non-existing file. Unfortunately, this change then broke ATLAS clients. This patch updates dCache so it returns 404 (not found) under these circumstances.

Fix the response when a client requests a byte-range beyond the end of a file. This is necessary for compatibly with ARC clients.

Changelog 2.9.17 to 2.9.18

4dc77ac

[maven-release-plugin] prepare release 2.9.18

5c91187

webdav: Fix reported content length for partial GETs

34efc53

webdav: Fix return code on DELETE of absent file

97622d9

srm: fix semi-infinite ls range with non-zero offset

9c416b2

pool: Partially fix interface selection for FTP mover

5ff1529

pool: Fix typo that breaks pool.mover.ftp.allow-incoming-connections

12a1086

shell: fix shell oracle for configuration keys with a space

8f2a131

ftp: fix help output

a20e635

dcap: Fix regression in published protocol family

6b27f82

loginmanager: Fix leak caused by absent child limit

c904227

pool: Fix shutdown of nearline storage subsystem

a9a4700

spacemanager: Fix null constraints and other schema migration issues

4830544

admin: Fix error reporting

20dbb7a

[maven-release-plugin] prepare for next development iteration

Release 2.9.17

pool

The NFS specification allows the server to specify multiple addresses when telling the client where to connect; for example, specifying both an IPv4 and an IPv6 address, or both addresses for multi-homed machines. This requires the client to choose the appropriate interface. For Scientific Linux 6, the kernel client will always use the first supplied address in the list and fail if it cannot access the pool with that address. With this release, pools will order the list, using heuristics to select which IPv4 address is “correct” and list it first.

poolmanager

With previous dCache versions, the WAAS selection algorithm had a bug where it could (mistakenly) consider pools full if all pools had very fresh files. This is fixed with this release.

spacemanager

Fix various issues to improve the robustness of spacemanager: handle failed uploads correctly, handle files deleted during upload correctly, increase robustness against dCache loosing (internal) messages.

webdav

Upgrade to Milton v2.6. This fixes the buffering problem where a proxied vector read request results in the entire file being written to a tmp directory and not deleted. With this release, requests for 100 kiB or less data result in no data being written to disk; requests for more than 100 kiB are still written to disk, but only the data needed to satisfy the request is stored and the file is deleted once the response has been sent. Some issues persist: data isn’t deleted if there is a failure sending it to the client and the whole file is requested from the pool.

Changelog 2.9.16 to 2.9.17

1477be1

[maven-release-plugin] prepare release 2.9.17

c3d30aa

poolmanager: Fix full pool detection for WASS

04841d4

Upgrade to Milton 2.6

f400a57

pool: reorder ip addresses returned to NFS client

1d27710

spacemanager: Fix various error recovery scenarios

f9b7139

[maven-release-plugin] prepare for next development iteration

Release 2.9.16

Changes affecting multiple services

Restarting a domain within an active dCache instance can lead to a domain receiving messages for a cell as it is starting. Strict control is require to avoid the cell attempting to process messages before it is ready. This release fixes one place where this control was missed, which could lead to a NullPointerException. While this problem can affect any core dCache service, it was noticed with the spacemanager service.

httpd

dCache versions including and after 2.11.0, 2.10.9, 2.9.12, 2.8.16, 2.7.21 and 2.6.36 required sites to delete existing RRD files when upgrading; i.e., run the command rm -f /var/lib/dcache/plots/*.rrd when the domain hosting the httpd service is stopped. This release reverts that change, but requires sites that have already upgraded to repeat the rm command. Sites upgrading from an earlier dCache version do not need to delete anything.

Fix a potential NullPointerException in poolCollector.

nfs

Protect against a NullPointerException if the client attempts to read the contents of a file’s level where that level exists in the database but contains a Nil value. This does not happen under normal circumstances.

pool

The Berkeley DB, which may be used to store file metadata on the pool, does not like being interrupted. The pool tries hard to avoid interrupting reading or writing; this release fixes one place that slipped through.

Fix regression in the output from the info command: it did not include statistics about the number of HSM requests and HSM timeouts. Also fixes how active HSM jobs are counted: cancelled jobs are still active until the underlying job as ended.

Changelog 2.9.15 to 2.9.16

ad2863a

[maven-release-plugin] prepare release 2.9.16

2eb29b8

imera: protect against NPE in FsSqlDriver#read

4fca3c4

pool: Restore flush and stage stats in info

12496d6

Fix NPE in cell initialization

99a8479

Fix NPE in httpd service

fdda8bf

pool: Avoid interrupting Berkeley DB in migration module server

47a5c36

dcache-webadmin: revert rrd data source names

17b04cc

[maven-release-plugin] prepare for next development iteration

Release 2.9.15

alarms

Fix minor corruption at beginning of the alarms defaults file.

httpd

In previous versions of dCache, the webadmin war file is automatically unpacked. This has been problematic as a dCache upgrade did not always trigger updating the unpacked webadmin, resulting in dCache running the older webadmin. With this release, dCache no longer unpacks the war file; the webadminWarunpackdir and httpd.container.webapps.tmp-dir properties are now obsolete.

Changelog from 2.9.14 to 2.9.15

1354cc5

[maven-release-plugin] prepare release 2.9.15

6e0bee1

alarms.properties fix accidental header corruption

256b40d

[maven-release-plugin] prepare for next development iteration

daf71ba

(2.9) dcache-webadmin: change Jetty setting so .war is not unpacked

Release 2.9.14

Changes affecting multiple services

A bug was discovered that resulted in the nlink count becoming negative. While wrong, this had a knock-on effect that prevented pnfsmanager and nfs services from listing the affected directory. With this release, directory listing are robust against such problems.

alarms

Fix alarm definition that triggers when the pool discovers a file with the wrong checksum. Previously it mistakenly triggered if an upload was incomplete.

nfs

Update nfs4j to v0.8.3. This brings some small performance benefits when listing a directory and for reading data through NFS v3.

Fix bug where, when a client read activity is proxied, the corresponding dcap mover wasn’t removed if client didn’t wait for the queued mover to start. Previously, such movers would accumulate on the pool.

pnfsmanager

Prevent attempts to remove the . and .. directories.

webadmin

Display the command’s output in the cell admin page to use a monospace font.

Remove a javascript error due to missing clojure dependency.

Sending commands in cell admin page is fixed.

Changelog from 2.9.13 to 2.9.14

2f4d1e4

[maven-release-plugin] prepare release 2.9.14

9dc9a7e

webadmin: Fix ClassCastException in cell admin

09888d6

alarms: fix regex for checksum alarm

2329345

webadmin: remove redundant head element in alarms panel html

045091a

dcache-webadmin: change output field of cell admin page to monospace font

29251e9

(2.11) dcache-webadmin: eliminate clojure dependency

eec425b

nfs-proxy: kill mover if we get a timeout on redirect

a2c27f1

libs: update to nfs4j–0.8.3

b44bc21

chimera: protect list initialization from FS inconsistencies

b773683

webadmin: tidy up unavailable page slightly

2cdcfc4

chimera: prevent attempts to remove ‘.’ and ‘..’

0db8034

[maven-release-plugin] prepare for next development iteration

Release 2.9.13

Fixes affecting multiple services

Update dcache check-config to print an error if site-local configuration contains scoped properties: one that contain a ‘/’ character.

httpd

Fix pool queue plots in webadmin.

nfs

Performance improvements.

pnfsmanager

Fix the message reported when moving (or renaming) a file or directory and certain problems were found: the destination directory isn’t a directory, the source doesn’t exist, or overwriting with different types (e.g., overwriting a directory with a file).

Fail requests earlier that plan to upload a file if that file’s path already exists as a directory. Previously, dCache would fail such uploads after the file was upload (during srmPutDone). With this release, dCache will fail during srmPrepareToPut instead.

poolmanager

If a site using WAAS specifies too large a Space Cost Factor then the algorithm used for write pool selection breaks down, logging “Unreachable statement.” Starting with this release, a warning is logged indicating the cause of the problem.

srm

The srm.enable.space-reservation.implicit property is now marked forbidden. This means a dCache instance with this property configured will not start. Consider the spacemanager.enable.unreserved-uploads-to-linkgroups property instead.

Changelog from 2.9.12 to 2.9.13

f1f9b72

[maven-release-plugin] prepare release 2.9.13

34a56eb

chimera: Add messages to JdbcFs#move exceptions

1a93191

poolmanager: Warn when spacecostfactor is too big

d265156

pnfsmanager: Check file type before overwrite

d2fe485

check-config: Produce error when using scoped properties

635f30a

srm: Mark srm.enable.space-reservation.implicit forbidden

709a549

(2.9) pool queue plots - fix partial package refactoring and neglected removal of jndi arguments

b4ab18b

pom: update to nfs4j–0.8.2

bd8c1be

[maven-release-plugin] prepare for next development iteration

Release 2.9.12

Fixes affecting multiple services

Running the eval admin command so it returns a non-zero return-code would respond indicating that a bug had been found. This has been fixed.

An named environment may be executed and can contain a reference to itself. If this recursion is unchecked, eventually memory is exhausted and the domain would restart. Now an error is logged and the domain is not killed.

Previously, some exceptions were logged with the wrong cell context or without any context. This has been fixed.

billing

Billing records issued by doors never included the file’s size. This is now fixed for all doors except NFS.

gplazma

The xacml plugin now supports the gums server returning GID values.

pool

Fix internal copying of files (triggered by the SRM copy command) so they respect the LAN port range.

Fix migration module’s random pool selection so that it does not select pools that are full.

Fix NullPointerException triggered when using migration module’s proportional pool selection and all pools are full.

The rep ls command can calculate per-storage-class statistics. This release fixes a problem where files without a storage-info would trigger an IllegalStateException when calculating these statistics.

Fix sweeping of files without a storage-info. Previously an IllegalStateException was thrown; this could be triggered by the admin interface or by the sweeper.

Fix a problem where the pool would register free space before actually deleting the file from the file system; for a brief moment, the pool would appear to have more free space that is actually available.

Periodically, pools check that dCache’s internal accounting of total and free capacity does not exceed the OS supplied values for the partition (i.e., the output from the df command); if they do then the dCache internal accounting is adjusted to match. The dCache.org team have observed that, after a file is deleted, some seconds may elapse before the corresponding extra free capacity is reported by the OS. To protect pools from this effect, the total and free capacity check is suppressed for 60 seconds after a file was deleted.

spacemanager

If a transient deadlock resolution error occurs, unconditionally retry the operation and do not log it. Such errors are part of RDBMS design when resolving concurrent updates.

srm

Fix the srm service’s admin interface ls command so that listed jobs can optionally include only those that have failed, have completed or were cancelled.

webadmin

Webadmin periodically creates billing graphs by querying the billing service for the information it needs. Previously, if this query failed (e.g., the billing service was restarted or failed to start first) then no further billing graphs are generated and a domain restart is needed. As a result of this work, the properties poolqplots.refresh-interval, poolqplots.refresh-interval-unit, httpd.plots.pool-queue.refresh and httpd.plots.pool-queue.refresh.unit are no longer supported. The httpd.plots.pool-queue.min-time-step and httpd.plots.pool-queue.min-time-step.unit properties now also cover this configuration.

Changelog from 2.9.11 to 2.9.12

f83516c

[maven-release-plugin] prepare release 2.9.12

9414bdf

gplazma-xacml: Add handling of GID returned by GUMS in gPlazma2 XACML plugin

26d72d7

chimera: fix unit tests

671c821

pool: Fix pool size health check in case of asynchronous release of space

c4323d1

spacemanager: Suppress transient deadlock resolution errors

556ee8f

pool: Fix race leading to false positices in pool size health checks

43fc3b4

pool: Fix ISE in CacheEntryImpl#toString

ab3e5b8

pool: Fix ISE in ‘rep ls’

35c97d2

Made PoolQueuePlotData enum compatible with java 8.

ffaddad

(2.9) webadmin: fix exit login in billing refresh loop

d64ce31

Marked refresh properties obsolete.

001429e

cell: Log exceptions within the correct cell context

985dd21

pool: Fix pool selection bugs in migration module

85cb091

billing: Add file size to request records

8850768

(2.9) webadmin: minor improvements to rrd4j-based pool-queue plots

2a880f7

pool: Respect LAN port range for internal srmcp transfers

7959ee7

srm: Fix listing of failed, done and cancelled jobs

f35d56c

cell: Prevent interpreter stack overflow from killing the domain

30d4605

cell: Declassify eval failure as a bug

d325f02

[maven-release-plugin] prepare for next development iteration

Release 2.9.11

Fixes affecting multiple services

The nfs and srm both cache replies from gPlazma to increase the speed of authenticating and identifying the user. Previously transient errors when communicating with gPlazma were also cached, delaying the recovery time from such errors. Now, such errors are not cached.

Update the comments within the configuration property files to reflect the new configuration property names.

Fix OpenMQ communications to use new configuration property names.

dcap

Fix opening a file when space-manager enabled.

nfs

No longer trigger an NullPointerException when opening the .(parent)() dot-command file in Chimera’s root directory.

The dcache ports command now shows the NFS port based on the new configuration properties.

pool

Fix a race-condition when a user starts uploading a file that should go to tape, deletes the file before the write is completed, then cancels the file upload. Previously, there was a risk that the pool fails to delete the uploaded data.

Fix Berkley DB usage when an HSM operation is cancelled to avoid an error logged as:

InterruptedException may cause incorrect internal state, unable to
continue. Environment is invalid and must be closed.

Fix xrootd support for vector read. The problem was discovered with ROOT v6, which reports errors like Single readv transfer is too large.

Fix the dcap mover to use the new pool.mover.dcap.port configuration property.

srm

Fix cleaning of upload directories for expired uploads after service restart.

xrootd

Fix problems when calculating a response to the kXR_set request. If encountered, the problem results in the following being logged, where ‘nnn’ in ‘xrootd-disk-nnn’ is some integer number:

Uncaught exception in thread xrootd-disk-nnn java.lang.IllegalStateException: null

Changelog from 2.9.10 to 2.9.11

550ec26

[maven-release-plugin] prepare release 2.9.11

d7f2d5c

ftp: Fix root path validation for upload directory

04c9e10

dcap: fix interaction with Spacemanager

b378eb9

xrootd: Upgrade to xrootd4j 1.3.5

fc0562e

pool: Prevent interruption of replica deletion during HSM flush

9a11bcb

pool: Fix HSM cancellation

0efcf8f

The gPlazma cache used by NFS and SRM caches both positive and negative replies. Unfortunately it also caches failures to communicate with gPlazma. This means that a transient timeout would be cached too, thus increasing the effect of the transient error.

7e1a676

pool: Fix xrootd vector read limits

3eb760b

chimera: fix NPE on ‘.(parent)()’ for root inode

83dabd5

srm-client: Fix logback configuration

24eadff

srm: Fix asynchroneous job storage leak

15348a0

configuration: adjust references to deprecated properties

52d4116

unittests: increase timeouts and fix race in DiskSpaceAllocatorTest

5464c99

[maven-release-plugin] prepare for next development iteration

Release 2.9.10

Fixes affecting multiple services

In some cases, dCache would log the wrong cause of an internal failure. This has been fixed.

Two problems are fixed that affected those services that make use of the grid trust store (/etc/grid-security/certificates): gplazma, srm, gsiftp, webdav and xrootd:

• When a CA is found not to have a signing-policy file, subsequent attempts to discover a signing-policy file will automatically fail for one hour. Previously, if any CA was removed since the service started then certain CAs (those using UTF–8 encoded subject RDNs) would be placed in this not-found cache by mistake. This resulted users with certificates from such CAs would succeed the first time they used dCache but all subsequent attempts would fail until the cache expires or the trust-store is refreshed.

• A minor memory leak is fixed. The amount of memory leaked was proportial to the number of different CAs in use.

Two problems are fixed that affected services providing GSI or plain SSL authentication using the jGlobus library:

• dCache now supports TLS v1.2 clients when using GSI or plain SSL using the jGlobus library.

• Signal end-of-stream when remote sends a CLOSE notification. The remote party normally sends a CLOSE notification before terminating the TCP connection, typically send from server to client. Previously, receiving such a notification was treated as an error.

Fixed thread safetly of timestamp formatting, principally used in logging and generating the output of admin commands.

billing

Fixes a bug where the bytes read graph can show inflated values.

nfs

Fix permission check when client requests a list of some directory’s contents.

Add support for mounting when the server-side mount-point is a sym-link.

srm

Fix the lifetime of the response when client makes a successful request for a reservation with infinite lifetime.

Fix incomplete restore of “ready queued” jobs. After restarting the srm service, any jobs in this state RQUEUED would not be handled correctly. This has been fixed.

Allow srm to start if there were active bring-online requests when the service was stopped.

Fix the set max ready get command in the admin interface.

Fix a problem where, if a client aborts an upload shortly after initiating the upload then the server blocks all activity for a timeout period. The SRM will eventually recover but, during this period, it appears deadlocked.

chimera

The chimera shell failed to interpret the creation time flag when choosing which time should be displayed in directory listing.

Changelog from 2.9.9 to 2.9.10

676caee

[maven-release-plugin] prepare release 2.9.10

9c9a614

common: Fix race condition in AtomicCounter unit test

a663f44

Add missing import

d442963

srm: Fix deadlock like bug

655f568

billing plots: use ‘transferred’ rather than ‘size’ for bytes

b9ddd09

srm: Fix reporting of infinite space lifetime

9a59574

chimera: Fix creation time flag in ls command

c0960b0

Fix CacheException references in error messages

58902cd

libs: update jglobus to 2.0.6-rc8.d

c00f2ec

srm: Fix incomplete restore of “ready queued” jobs

0ff73d0

Avoid thread-unsafe use of SimpleDateFormatter

7bc2fc3

srm: Fix set max ready get command

1ad3bd1

libs: update to jglobus–2.0.6-rc7.d

01d61f2

libs: update to nfs4j–0.8.1

65a248a

[maven-release-plugin] prepare for next development iteration

c60ec10

build: Update to findbugs 3

Release 2.9.9

pnfsmanager

Previously, the automatic schema management for Chimera failed to create an index on the iparent column of t_dirs table. The lack of this index results in deletions becoming progressively slower as dCache stores more files. If your dCache instance was created with dCache v2.3.0 or newer then it is likely affected by this problem. Instances created with an earlier version are not affected. The pnfsmanager service will check for the index when it starts and add the index if it is missing. Alternatively, the check may be performed manually using the dcache database update command.

pool

This release fixes several issues with the sweeper:

• Previously, the sweeper consumed considerable amount of memory while freeing up space; in the worse case, this could result in the domain running out of memory and restarting; this has been fixed.

• The sweeper ls command would make the pool unresponsive while the output is generated; now, running this command no longer prevents the pool from taking on more work.

• As the sweeper purge, sweeper free and rep rmclass commands can take a long time to finish, the admin interface does not wait for them to finish; instead, a quick response is returned and the command continues in the background. Previously there was no indication when such commands had finished; now, the pool will log when the command has completed.

• The pool logs regular sweeper operations at lower priority; this makes the pool logs less noisy when operating under normal conditions.

poolmanager

In previous versions, poolmanager would log a stack-trace if a user attempts to open a file for reading and no read pool could be found because a unit failed to match. Now the failure is logged as a normal message.

xrootd

The xrootd door now informs billing when a user uses the xrootd protocol to delete a file.

gplazma

The ldap plugin allows a door to request converting a uid or gid back to a username; currently, only the nfs door makes use of this functionality, when generating a directory list. In previous dCache releases, the plugin behaved incorrectly if the uid or gid is unknown; this is now fixed.

Changelog from 2.9.8 to 2.9.9

965cf4f

[maven-release-plugin] prepare release 2.9.9

d1c4a76

chimera: mark as-run if i_dirs_ipnfsid exists

1943248

solaris: fix solaris package script and add work-around for pkgmk bug

29a8de4

chimera: create missing index i_dirs_iparent

8aaa6da

pool: Resolve high memory usage and other issues in sweeper

03a161d

info: Fix HashMap ordering assumption in unit test

c69cb9d

Make JDK byte code verification bug workaround Java 8 compatible

47b6e56

gplazma: Fix JVM implementation dependency in unit test

fc992b9

poolmanager: Suppress stack trace in case of unmatch units

5b25913

xrootd: Add billing entry on delete

b2ea7ff

bugfix: fix reverseMap not throwing NoSuchPrincipalExceptions

07f3b1f

[maven-release-plugin] prepare for next development iteration

Release 2.9.8

Cell startup

Fix potential dead-lock when a dCache domain is starting. The problem is triggered when a cell receives a message while initialising. If dCache is started “from cold” then this cannot happen. However, if a domain that contains well-known cell restarts while the rest of dCache is operational then the problem could be triggered. For example, if a door domain ran out of memory then there is a small risk that the domain would not restart correctly. The problem was present since dCache v2.8.0 and is fixed with this release.

Changelog from 2.9.7 to 2.9.8

8a0d7e8

[maven-release-plugin] prepare release 2.9.8

03204ef

cells: Fix deadlock during startup

0f49bf4

[maven-release-plugin] prepare for next development iteration

Release 2.9.7

Packaging

Add python-psycopg2 as recommended dependency to debian package; it’s needed by the dcache-star command.

alarms

The alert should be generated if a pool’s background checksum scanning (“scrubbing”) discovers a broken file. With this release, such alerts are now generated.

dcap

When the client authenticates via GSI, the dcap door will check the user certificate (generally, a voms-proxy certificate) to extract the DN and primary FQAN. To do this, it needs to know where the trusted CAs and trusted VOMS server identities are stored; these are typically /etc/grid-security/certificates and /etc/grid-security/vomsdir respectively. Previously, dcap ignored the dcap.authn.capath and dcap.authn.vomsdir properties. With this release, these properties are honoured.

pool

If a stage requests has been forwarded to the nearline subsystem and the file is deleted then the stage request is now cancelled.

Nearline storage configuration is now applied atomically rather than one attribute at a time.

The maximum number of concurrent nearline operations may be adjusted using the hsm set admin command. If the concurrency is increased then previous versions would only start additional activity as more requests are received. This has been fixed so increasing the concurrency limit will immediately start more activity if there is any backlog of requests.

poolmanager

Stage and pool-to-pool transfers, triggered by poolmanager, are monitored by poolmanager querying the source pool to check if the request is still active. In previous versions of dCache, for n transfers from a pool, this monitoring generated O(n*n) of network traffic. In particular, poolmanager’s monitoring of bulk staging requests can result in sufficient network traffic between the pool and poolmanager that other dCache activity is impacted. With this release, the monitoring traffic is reduced in general, most prominently for large concurrent activity, such as bulk stage requests.

The rc destroy command has been removed. The implementation was incomplete and the operation was dangerous.

If the door resubmits a request to poolmanager that triggers staging of a file, poolmanager uses additional memory while waiting for the stage to complete. For bulk staging, this additional memory usage can be significant, potentially resulting in poolmanager exhausting the available memory. This release updates poolmanager so a door resubmitting a stage request does not increase the memory footprint of poolmanager.

The poolmanager has an incomplete feature called ‘clumping’. This is where poolmanager can handle multiple select-pool-for-read requests for the same file as a single request. There is a hard-coded clumping limit that prevents too many files from being handled together; this allows poolmanager to choose an alternative pool when staging or replicating to make the file available. Once this limit is reached, subsequent open requests for the same file will fail in poolmanager. Doors handle this failure differently; the xrootd and webdav doors propagates this failure back to the client, while the dcap and ftp doors will resubmit the request to poolmanager. This release increases the clumping limit from 1 to 20; it also changes the behaviour of the doors so all will retry the pool-selection if the clumping limit is reached. This fixes the ‘request clumping limit reached’ failures.

This release fixes a race-condition between two clients opening the same file. Previously, if poolmanager starts processing the second open request as it is finishing processing the first then there is a tiny chance that the second request is lost.

srm

The srm service will check the user certificate (generally a voms-proxy certificate) to extract the DN and primary FQAN. To do this, the srm service needs to know where the trusted CAs and trusted VOMS server identities are stored; these are typically /etc/grid-security/certificates and /etc/grid-security/vomsdir respectively. Previously the SRM ignored the srm.authn.capath property and there was no way to configure the trusted VOMS server identity path. With this release, the srm.authn.capath is honoured and the srm.authn.vomsdir property is introduced.

When the SRM server sends an asychronous response to the client, the client must make subsequent queries to check if the request has completed. Part of each response is a hint suggesting for how long the client should wait before querying for a progress update. Prior to dCache v2.9, the srm service provided this hint where subsequent responses suggested a longer wait time. With v2.9, this feature was lost, resulting in the SRM client hammering the srm service. This has been fixed.

Changelog from 2.9.6 to 2.9.7

f870af1

[maven-release-plugin] prepare release 2.9.7

4b41502

build: add work-around for JDK bug and PowerMock

d1af046

pool: Fix concurrency settings for script nearline storage

406da03

poolmanager: Fix ‘request clumping limit reached’ failures

c48cf23

poolmanager: Avoid request leak

ce74569

poolmanager: Fix excessive p2p and stage alive checks

8dcf7f5

srm: Restore estimated wait time update

82e885c

pool: Fix several minor issues in nearline storage subsystem

e526b93

srm,dcap: Use configured vomsdir and capath

f9df2b1

srm-client: by default only delegate for srmcp with 3rd-party copies

ae6d7a5

alarms: Fix checksum alarm filter.

d07d31d

deb: Add python-psycopg2 as a recommended dependency

fae3266

delegation-shell: generate delegated credential of correct type

dfd8b1e

[maven-release-plugin] prepare for next development iteration

Release 2.9.6

webadmin

Fixes a regression that prevented the admin from deleting alerts in the alarm system.

Improve esthetics of the pool queue plot grid.

pool

Fixes a problem where, under specific circumstances, a pool with a file’s data will erroneously claim it does not have that file’s data. This problem occurs only if an HTTP or xrootd client attempts to read a file immediately after uploading it; it does not affect other protocols. If a client triggers the problem then PnfsManager will remove the pool as the location of the file’s data with two consequences: first, the file cannot be read; second, the orphaned entry will not be removed if the file is deleted. With this release, the problem is fixed; we recommend running the ‘pnfs register’ command on all pools as a precaution against orphaned entries.

info-provider

Suppress publishing NFS door endpoints. Publishing these endpoints can result in sites failing functional tests since dCache running an NFS endpoint does not mean that files are accessible from the test machine.

Changelog from 2.9.5 to 2.9.6

01ed787

[maven-release-plugin] prepare release 2.9.6

4a767f6

Suppress nfs’ file protocol from being published

b8feb9f

pool: Fix race condition that leads to orphaned files

794d35e

dcache-webadmin: fix pool queue plot grid placeholder image

93027a4

(2.9) webadmin: fix regression in alarm deletion filter

e1d7f61

cells: remove System time dependency in unit-test

06ac76f

[maven-release-plugin] prepare for next development iteration

Release 2.9.5

ftp

Fix globus-url-copy hanging on proxied passive-mode connections when the mover is killed. In addition to reported the error on the control channel, the ftp door now also closes the data channel to force globus-url-copy to fail such transfers.

Fix the control channel responses for LIST, NLST and MLSD commands; the broken response prevented Globus (formally GlobusOnline) from listing directories.

pool

The background checksum scanner now treats files that cannot be accessed as an error rather than skipping them.

Update the compatibility script-based HSM module to be more compatible with the HSM interface in earlier versions of dCache.

Fix the response headers when HTTP clients (such as Davix) make a request for multiple fragments of a file.

srm

Remove a race-condition where aborting an upload at the wrong moment resulted in dCache failing to remove the temporary upload directory.

Fix issue where pools can appear to be offline on lightly loaded systems.

Increase scalability of SRM by allowing work to spread over more CPU cores during file upload and download.

pnfsmanager

For systems using enstore, fix restoring of files with filenames that contain a ‘+’ in the filename.

xrootd

Honour the WriteToken directory tag and implicit reservation when writing data.

Fix issue where pools can appear to be offline on lightly loaded systems.

nfs

Honour the WriteToken directory tag and implicit reservation when writing data.

pinmanager

Fix issue where pools can appear to be offline on lightly loaded systems.

spacemanager

Fix issue where pools can appear to be offline on lightly loaded systems.

Changelog from 2.9.4 to 2.9.5

c64705d

pool: log with error if checksum scanner gets an IO error

9457a17

ftp: Close passive port on failure

6dc0d49

srm: Cancel upload path if request was aborted

e70b06c

ftp: fix LIST, NLST and MLSD cmd-channel output

b4c17f3

pool: Fix regressions in new HSM subsystem

4d34c55

Fix ArrayIndexOutOfBoundsException when processing multiple surls invoking srmget.

66839ee

Fix issue with incorrect determination of whether or not a string is encoded by replacing URLDecoder with import org.springframework.web.util.UriUtils as the fo

bfc8a79

http: fix multipart response size

8d08231

xrootd,nfs: Add support for space management

310268b

pinmanager,srm,spacemanager,xrootd: Fix pool monitor expiration

f407ca3

srm: Fix lock congestion

16a6444

[maven-release-plugin] prepare for next development iteration

Release 2.9.4

xrootd

Fix redirection to IPv6 pools.

Report read requests that suffer a permission denied failure as “Read permission denied” rather than “Write permission denied”.

To allow xrootd plugins to intercept and translate errors.

Update dCache responses to avoid intermittent failures with xrdfs.

webadmin

Fix login bug that results in Access Denied page.

dcap

Provide more robust cleanup if there is a problem after the client finishes reading or writing data. The symptom is an increasing number of “active transfers” reported in the web monitoring pages. Eventually, the httpd and webadmin services consume all available memory and the domain(s) hosting them will die with an OutOfMemoryError.

Scripts

Update dCache scripts to allow running with Java 8 runtime.

Changelog from 2.9.3 to 2.9.4

c63e827

[maven-release-plugin] prepare release 2.9.4

cc2eb38

xrootd: Quote IPv6 addresses in kXR_redirect replies

356d360

xrootd: Fix error message for read patch check

4c961c6

dcache-webadmin: fix login redirect bug which misleadingly results in Access Denied Page

5621b2e

xrootd: Upgrade to xrootd4j 1.3.3

7ab97da

scripts: allow java8 runtime

8772b25

dcap: ensure that we remove session on DoorTransferFinishedMessage

5ac7a33

[maven-release-plugin] prepare for next development iteration

Release 2.9.3

spacemanager

Fix the column formatting for various admin commands should one of the numerical values be negative.

srm

Fix a null-pointer exception triggered by a client querying a request that survived an SRM restart.

Update the SRM to use the SRM-specific configuration property (‘srm.enable.space-reservation’) rather than the generic property (‘dcache.enable.space-reservation’). As the default value for srm.enable.space-reservation is the value of dcache.enable.space-reservation, this change should not affect sites.

cleaner

Fix the shutdown sequence of cleaner to ensure quick and clean shutdown.

ftp

Don’t export child doors. When a client connects, the ftp door creates a child cell that handles the client requests. Changes in 2.7 meant that these child cells mistakenly became well-known cells. This is now fixed.

dcap

Don’t export child doors. When a client connects, the dcap door creates a child cell that handles the client requests. Changes in 2.7 meant that these child cells mistakenly became well-known cells. This is now fixed.

info-provider

Update the use of xmllint to include the ‘–noent’ option; mitigation against CVE–2014–0191 resulted in xmllint behaving differently when ‘–noent’ is omitted.

libraries

Update jGlobus version. This results in better error reporting for FTP transfers and X.509 certificates with a comma in the DN are handled correctly.

Changelog from 2.9.2 to 2.9.3

208eb38

[maven-release-plugin] prepare release 2.9.3

dd94fc0

info-provider: fix works-by-accent bug when importing dCache config

7f5e292

pom: update jglobus library

d8bd7f2

srm: Use srm.enable.space-reservation rather than dcache.enable.space-reservation

01da41f

Fix rendering of negative numbers in columm layout

1340c0d

srm: Fix NPE

efb1cb9

ftp,dcap: Don’t export child doors

945dc8d

cleaner: Shut down timer thread

5618794

[maven-release-plugin] prepare for next development iteration

Release 2.9.2

httpd

Prevent reading of configuration files.

srm

Fix how the status of requests that can contain multiple files (Get, Put, Ls, BringOnline, Copy) are calculated and avoid “illegal state transition” errors that are sometimes logged.

Clean up correctly when a client deletes a file that is being uploaded (thereby cancelling any concurrent uploads) by deleting the temporary upload directory.

Fix how a request is reported to the client after it times out, so the value follows the SRM specification. This also fixes some of the logged “illegal state transition” errors.

Ensure that request history information is listed chronologically.

Avoid deadlocking should either a client tries to pin files or if the SRM attempts to retry pinning a file, while the SRM is updating its knowledge of the pools.

xrootd

When logging a transfer, place a colon between the version and the client IP address.

pnfsmanager

Don’t log “duplicate key” errors if the error is understood.

messaging

Report correct statistics about messages even after a cell’s message queue overflows.

pool

Make pools more robust against the unexpected. Files that appear unexpectedly (e.g., an HSM script delivers a file after dCache has cancelled the request) do not trigger the pool from disabling itself but are deleted; requests to create a file that already exists trigger re-registration of that file in pnfsmanager.

Log a stack-trace for pool-to-pool transfers that encounter an extreme problem with the virtual machine.

Fix the ordering of arguments when invoking an HSM script to be compatible with pre–2.9 versions of dCache.

webdav

Work-around bug in StringTemplate library that can result in NullPointerException being logged when generating an error response for the client.

pnfsmanager

Fix uploads through SRM if the client supplies the file’s expected size and fix an internal cache that allows database-overhead to be avoided.

Changelog from 2.9.1 to 2.9.2

68836e1

[maven-release-plugin] prepare release 2.9.2

fec4353

httpd: Fix a couple of bugs

caf0a5f

srm: Avoid duplicate status code update on status query

4862f03

xrootd: split protocol name form the client IP in XrootdProtocolInfo

e982836

chimera: Fix duplicate key supression

4420364

Revert “srm: Drop fail fast for protocol checks”

e8dae00

srm: Fix request cancellation due to SURL deletion

f3bddc5

srm: Fix request expiration

197b22a

cells: Fix message counting in case of queue overflow

aed3ae7

pool: Improve how we deal with unexpected files in pools

4af53d0

pool: Fix error handling in staging code

7582b00

pool: Log P2P fatal errors with stack trace

dc9384e

srm: Fix transition ordering for restored requests

f4467a7

hsm scripts: fix command and options order

7631420

webdav: work-around race-condition in StringTemplate library

50708a4

systemtest: regenerate host credentials, add user credentials

249ac36

srm: Fix deadlock in pinning

2eb3a5e

srm: Schedule resubmit of pin request to avoid deadlock

7d73630

srm: Drop fail fast for protocol checks

a5e3e46

pnfsmanager: Fix expected size extraction and tag caching

5fa84b0

fhs: Use gzip compression for deb package

1e67db1

[maven-release-plugin] prepare for next development iteration

Release 2.9.1

alarms

Add missing configuration elements and documentation to support sending an email when an alert is received.

statistics

Fix NullPointerException in statistics service

Changelog from 2.9.0 to 2.9.1

f9a18f0

[maven-release-plugin] prepare release 2.9.1

cdd8be0

dcache alarms: add missing configuration elements and documentation for SMTP appender

a6a0d97

statistics: Fix NPE

3e85c8f

[maven-release-plugin] prepare for next development iteration

Release 2.9.0

Changes affecting several services

Libraries

Removed the saxon library from the dCache classpath. Removed the xalan XSLT library from dCache as the library is included in all Java 7 JVMs.

Upgraded Netty from version 3.6.5 to 3.9.0. This I/O library is used by xrootd doors and to support xrootd and http data transfers in pools. Consult the Netty news archive for information about Netty bug fixes.

Upgraded Jetty from version 8.1.13 to 8.1.14. This embedded webserver is used by the webdav and srm doors, and the httpd service.

Upgraded the PostgreSQL JDBC driver from version 9.3–1100 to 9.3–1101.

Upgraded the Berkeley DB library from version 5.0.97 to 5.0.103. This embedded database library is used by pools to store meta data. Please consult the change log for details about bugs fixed in this release (you can ignore the information about disk format changes - those only apply when upgrading from older versions of the library).

Upgraded the Bouncy Castle Security Provider from version 1.43 to 1.45. This library is used by most certificate related code in dCache.

chimera

A schema change to the t_inodes table will cause the first start of pnfsmanager to take a while. Large sites are encouraged to time the schema change on a clone of the database before upgrading the production system. Due to these changes, dCache cannot be downgraded without rolling back the schema changes using one of the dcache database rollback commands with a 2.9 dCache and only then downgrading dCache. The schema change was necessary to fix issues with inode cache invalidation.

File creation and file open overhead has been reduced.

Fixed error reporting for hard link creation.

Changed registration of file locations to avoid integrity constraint warnings in the PostgreSQL log file.

Fixed a bug that would cause sockets to be interpreted as directories.

The chimera command line interface was extended with a mv command and new time stamp options for the ls command.

Messaging

The cell message passing system has been refactored. This is an internal change and mostly invisible. If messages are monitored using the events log files, you will notice that the blocking mode is no longer used.

Plugin compatibility

An internal change to how we implement dCache services in the Spring framework breaks compatibility with third party plugins providing custom services implemented with Spring. Plugin providers should update their plugin to match the changes in b7234ee.

Databases

The BoneCP database connection pool library has been replaced with HikariCP. HikariCP is more maintained, smaller and faster. We also hope it is more robust.

HikariCP has slightly different controls for adjusting the database connection pool. While BoneCP partitioned connections, HikariCP has no such concept. HikariCP also maintains a minimum number of idle connections; this is in contrast to the minimum number of connections that BoneCP maintains. Sites that have tuned their data connections should note these differences and adjust their configuration accordingly.

The *.db.driver configuration properties have been marked obsolete. The proper driver is now automatically discovered from the JDBC URI. Only JDBC 4 compliant database drivers are supported (this is only relevant if you install a custom JDBC driver).

The liquibase database schema management library has been updated to version 3.1.1. Version 3.1 adds support for generating SQL scripts without a live database connection. Details about this feature can be found in the Liquibase documentation. There is no wrapper for this functionality in dCache, so this is to be considered for expert use only.

Added the dcache database showUpdateSQL command to output the SQL that would be executed if dcache database update was executed. The emitted SQL may be applied manually to the database, so updating the schema.

Packaging

Tarball packages no longer have a trailing -1 in the file name or base directory name.

Command line tools

Startup speed of several command line tools has been improved.

IPv6

dCache now supports IPv6 by default. It still prefers using IPv4 addresses on dual-stack machines (those with both IPv4 and IPv6 addresses) so the impact should be minor. However, the change does mean that services will now also listen on IPv6 interfaces. To disable this, define

dcache.java.options.extra=-Djava.net.preferIPv4Stack=true

in your configuration.

Changes to services

loginbroker

Loginbroker now supports registering the root directory of doors. As a consequence SRM no longer needs to be configured with the root paths of each door and the properties srm.service.webdav.root and srm.service.xrootd.root are obsolete.

billing

Billing information as published by several services has been completed to reduce the number of values marked as Unknown in the billing log files. The fix was made in the services publishing billing information. Thus only services updated to 2.9 publish complete information.

The billing indexer was extended with the -f option to the find command. It allows search strings to be read from a file, enabling efficient billing entry lookup for a collection of files.

info

Add additional attributes to the domain.<domain-name>.static node, including information on the OS and the JVM.

nfs

Added new export options dcap and no_dcap. The former is the default while the latter hides the .(get)(cursor) dot command that DCAP clients use to detect an NFS-mounted dCache. Using no_dcap will force DCAP clients to read data through NFS instead of through DCAP.

A new command has been added to the NFS dot-command set to allow users to pin and unpin files. Files which are not on disk will be brought on line when pinned. The command takes the following form:

touch ".(fset)(filename)(pin|stage|bringonline)(duration)([SECONDS|MINUTES|HOURS|DAYS])"

The variants of the third argument are equivalent in effect. The last argument is optional and defaults to SECONDS. A duration value of 0 will unpin the file. The command does not allow the user to pin the file indefinitely as the duration value must be a positive integer.

Three new properties associated with the configuration of the pinmanager have been added to nfs.properties: nfs.service.pinmanager, nfs.service.pinmanager.timeout and nfs.service.pinmanager.timeout.unit.

Server side attribute caching drastically improves performance for some workloads. Three new attributes have been added to configure the cache: nfs.namespace-cache.time, nfs.namespace-cache.time.unit and nfs.namespace-cache.size.

A large number of fixes to improve logging, protocol compliance, error handling and performance have been implemented.

Adds support for NFS v4.0. Note that there is no parallel NFS (pNFS) extension for NFS v4.0; therefore, the NFS door will act as a proxy. This means that all data will flow through the door for clients using NFS v4.0.

alarms

Improved logging and handling of configuration errors.

ftp

Added support for the RFC 3659 CREATE fact to report file creation time in directory listings.

Added support for the MFMT, MFCT and MFF commands as described in this internet draft.

Added support for the UNIX.ctime and UNIX.atime facts.

Improved compatibility with the ncftp client by adding a leading zero to the value of the mode fact.

Added support for the HELP and SITE HELP commands as defined in RFC 959. This improves compatibility with some clients.

dCache user accounts can be configured with a personal root and home directory. How these are interpreted depends on the door and protocol used. For FTP, the account root directory forms the root of the name space and the home directory is the initial directory of the FTP session. For other doors, a configuration property usually defines a directory to export as the root of that door, and the account root directory is used as an additional authorization check to prevent users from accessing any directory outside their root. This behaviour is now configurable for the FTP door using the new ftp.root property. If left empty, the per account root is used as before. If set to a path, that path is exported as the root of the door.

Added FTP protocol request logging to the access log files. The access logs were introduced in dCache 2.8, but initial support was limited to SRM. In dCache 2.9 the FTP door logs every command interaction to these log files. The access logs are automatically compressed each night and deleted after one month.

Added a workaround that selects the MD5 checksum algorithm when a file is written by Globus Online. This allows such transfers to succeed with integrety checking.

Added support for Globus delayed passive. This is an alternative to using GridFTP v2 GETPUT. It allows the data channel to be created directly between the client and the pool. In contrast to GridFTP v2, delayed passive supports IPv6. Note that current clients often default to disabling delayed passive, even when delayed passive is implemented. If the client connects to the door using IPv4 and the pool does not support IPv4, then the data connection is proxied through the door. Similarly, if the client connects to the door using IPv6 and the pool does not support IPv6, then the data connection is proxied through the door. This is the case even when the client is dual stacked. Like with GridFTP v2, use of direct data channels is subject to the ftp.proxy.on-passive and ftp.proxy.on-active properties.

poolmanager

A new partition type, buffer, was added that is intended for pools used as transfer buffers. This new partition type chooses a pool based on a weighted random selection, with weights derived from the pools’ load only. Free space has no effect as all pools with sufficient space to store the file are considered. This partition type is suitable for pools which do not hold files permanently, such as import and export pools.

The wass partition type is not suited for this use case, as wass prioritizes balancing free space over load.

The buffer partition type respects the mover cost factor and performance cost factor settings, using an interpretation similar to wass.

pool

Disabled statistics files for Berkeley DB meta data backend. These statistics files consumed disk space, which would periodically prompt the pool to generate a warning as it adjusted for the unexpected loss of storage capacity.

The time format of the sweeper ls -l command has been changed to conform to ISO 8601.

Improved thread termination during pool shutdown.

Modified permanent migration jobs to react to access time and sticky flag changes. Thus migration jobs that filter by access time or sticky flags may now pick up files when access time or sticky flags change.

HTTP mover

Implemented limited support for the HEAD HTTP method.

FTP mover

The listening socket of a passive FTP data channel is now bound to a single address rather than the wildcard address. The FTP protocol only allows a single address to be returned to the client, so there is no point in binding to the wildcard address. Binding to the wildcard address had the problem that it could succeed even if the port on the specific address returned to the client was already open. The change does have observable effects in that previously the pool would submit a hostname back to the door and the door would do a DNS lookup to find the address to send to the client. Now the pool sends an IP address.

The TCP accept backlog for FTP passive data channels has been increased. This reduces the risk of rejected connections during a surge of activity.

Nearline storage

The nearline storage support has been rewritten. Nearline storages are supported through drivers. These drivers can be packaged and shipped as dCache plugins. A driver that supports the classic HSM scripts is shipped with dCache and will be used by default for existing setups.

Nearline storage drivers solve many issues of the classic HSM scripts:

• A callout to an external script for every invocation was a performance issue, as such a callout to an external interpreter was associated with considerable overhead.

• A callout to an external script was a tape efficiency issue, as it would limit the number of concurrent callouts due to allocating a process and four Java threads for every file. Thus one had to balance how many processes and threads the host could handle with how large batches one could submit to the tape system.

• The stage queue was limited to available disk capacity, as the pool had to reserve space for the file before calling the script. Thus the tape reodering queue was limited by disk space.

A nearline storage driver is written in JVM-supported language (for example, Java, Scala or Groovy) and is executed within dCache. This avoids the overhead from calling out to an external script. Request queuing is dealt with by the driver, which allows the driver to reorder requests in a way that optimizes tape access.

Since nearline storage request queuing is now no longer part of dCache, the maximum active requests limits are no longer reported to poolmanager. For some drivers, such a limit may not even exist. Pool selection has been updated to no longer rely on these values.

The hsm family of commands in the pool service have been updated to allow the nearline driver to be specified when defining a new nearline storage. The new hsm create command is used to define the nearline storage, while the hsm set command has been updated to allow the driver to be configured.

dCache ships with a number of nearline storage drivers. These are explained in the following sections.

script driver

dCache ships with a driver called script. This driver performs a callout to the classic HSM integration script. The driver accepts a number of arguments, settable through the hsm set command:

-command

The path to the script.

-c:gets

Maximum number of concurrent reads.

-c:puts

Maximum number of concurrent writes.

-c:removes

Maximum number of concurrent removes.

To provide backwards compatibility with existing setups, the script driver is the default. During intialization, the rh set max active, st set max active and rm set max active commands are translated to configuration parameters for instances of the script driver. Support for these commands is however deprecated and it is recommended that the pool setup file is regenerated using the save command.

copy driver

The copy driver is similar to the classic hsmcp script. It copies files between the pool and some other directory. It accepts a single configuration property:

-directory

The path of the directory into which files are copied.

link driver

Similar to the copy driver, but creates hard links rather than copying files. The directory in which hard links are created must be on the same file system as the pool’s data directory. Like the copy driver, the link driver only accepts a single configuration property:

-directory

The path of the directory within which hard links are created.

tar driver

This is an experimental driver, provided mostly as a demonstration right now. The driver bundles files into tarballs and stores those in a configurable directory. The driver does currently not support removing files.

-directory

The path of the directory within which tarball files are created.

srm

Miscellaneous

BringOnline requests, as well as get and put requests for which a transfer URL (TURL) was already prepared, now survive an srm restart.

The database code was updated to avoid warnings about duplicate key violations in the PostgreSQL log file during startup.

The SRM list response has been fixed such that the status of failed or unprocessed SURLs is included.

Failure reporting for SRM v1 requests has been improved.

srm shutdown was improved to avoid errors about unterminated threads in log files. Additionally, the service now waits for up to three seconds for database updates to finish before shutting down.

Culling of jobs in the database is now limited to a single thread to reduce the impact on regular operation.

The ThreadManager cell has been eliminated. The following new properties have been introduced instead: srm.cell.limits.message.threads.min, srm.cell.limits.message.threads.max, srm.cell.limits.message.threads.max-idle-time, srm.cell.limits.message.threads.max-idle-time.unit, srm.cell.limits.message.queue.max. The defaults will suffice for most deployments.

The output of the ls command in the srm admin interface has been cleaned up.

Absolute FTP TURLs

The SRM protocol allows the client to negotiate a transfer URL (TURL) for uploading or downloading a file. TURLs for FTP transfers are defined in RFC 1738. According to the standard, they are to be interpreted relative to the user’s login directory (home directory) and not relative to the root directory.

The ‘srm’ service in previous versions of dCache generated TURLs relative to the root directory. This will not work as expected for RFC 1738-compliant clients if the user has a home directory other than ‘/’.

This problem has now been fixed and the returned TURL will work for users with a non-root home directory. Unfortuantely neither JGlobus nor Globus interpret RFC 1738 correctly. As a workaround we now generate TURLs with an extra leading / in the path. This provides the correct behaviour with FTP clients typically used for SRM, but would fail if a standard compliant client would be used.

Root path aware transfer protocol negotiation

When negotiating a TURL, the srm service must choose from a list of protocols provided by the client. dCache will use the list of doors registered with the loginbroker to determine supported protocols. If doors are configured to use different root directories, not all doors may be able to serve a file. This is now taken into account when choosing the transfer protocol to use.

GridSite compatible delegation

Support for GridSite delegation v2.0.0 was added. The endpoint is run as part of the SRM service and has the path /srm/delegation. The GridSite delegation endpoint allows a user to delegate a credential, discover the remaining lifetime of any delegated credential and destroy delegated credentials. Those credentials delegated via GSI (typically as part of an SRM operation) will have the delegated ID of gsi. Using this, a client may query the lifetime of any already delegated credential and delete credentials delegated via SRM operations.

A delegation client that allows scripted and console-based interaction with GridSite endpoints has been implemented. The client is released independently, as part of the dCache srm client package.

Unique write TURLs

Until version 2.9, the dCache srm service has acted as a redirector for regular dCache doors. That means that the TURLs generated by the srm service have been the same URLs you would have used to access those files without SRM.

With dCache 2.9 this has changed and, for uploads, each TURL is now unique. The uploaded file remains invisible to users using non-SRM protocols until the upload is committed by the SRM client calling srmPutDone.

SRM upload requests have a finite lifetime; if the SRM client does not call srmPutDone in time, the request will timeout. Should this happen, any file uploaded using the corresponding TURL will be deleted and any subsequent attempt to upload to the TURL will fail.

There are many reasons for making this change. To name a few:

• The expected file size, access latency and retention policy provided by the SRM client could not be supported without enabling the spacemanager service. These parameters are now respected even without space management.

• Failure recovery was difficult, as dCache could never be certain whether a file uploaded through a door happened as part of the SRM upload or was done by another non-SRM client that incidentally uploaded a file to the same path at the same time.

• If an SRM upload was aborted and retried, we could not prevent the first client from completing the second upload as we always generated the same TURL. Thus we couldn’t properly isolate the two uploads from each other.

• Since the only way to bind file size, space token, access latency and retention policy to a file before upload was through the spacemanager service, space reservations were bound to paths. Such reservations would prevent the srm from creating concurrent uploads to the same SURL, but to work correctly, the two services had to agree on the state. Wherever there is distributed state, there is a risk that the state is inconsistent. This problem was at the heart of the “Already have 1 record(s)” errors.

To generate unique write TURLs, the srm creates a unique upload directory for each upload. After the transfer completes, the file is moved to the target path and the upload directory is deleted. Parameters like expected file size, access latency, retention policy and space token are bound to the directory using Chimera directory tags.

By default, these temporary upload directories are created under the /upload directory. Chimera automatically creates this directory the first time it is needed. Ownership and permissions are set such that nobody can list the directory and thus nobody can discover the temporary upload directories.

IMPORTANT: If the transfer doors used for srm do no export the /upload directory, the dcache.upload-directory property has to be altered to a path that is exported by the doors. A typical site value would be /pnfs/example.org/data/upload. This property has to be set for pnfsmanger and all doors used for SRM transfers. In the unlikely event that you cannot find a directory visible through all doors used by the srm, you have to introduce special door instances only used for SRM transfers. Such doors can be configured to only export the upload base directory.

The srm.enable.space-reservation.implicit and srmImplicitSpaceManagerEnabled properties are obsolete. To allow uploads to links in link groups outside a space reservation, enable spacemanager.enable.unreserved-uploads-to-linkgroups in spacemanager. That property deprecates spacemanager.enable.reserve-space-for-non-srm-transfers. The property srm.enable.space-reservation.strict is obsolete too as it did not have any effect.

spacemanager

Due to the changes to the srm service, several features of spacemanager are no longer needed and have been removed. The relevant options have been removed from the admin shell interface.

Upon upgrade, the space manager database schema is changed: The pnfspath, expirationtime and deleted columns are dropped from the srmspacefile table. State value 0 is no longer used in the srmspacefile table. The freespaceinbytes column is replaced by availablespaceinbytes in the srmlinkgroup table.

Due to these changes, dCache cannot be downgraded without rolling back the schema changes using one of the dcache database rollback commands with a 2.9 dCache and only then downgrading dCache.

pnfsmanager

The service was extended with a feature to generate temporary upload paths. This is an internal change that is used by the srm service to generate unique write TURLs.

cleaner

The HSM cleaner is enabled by default. It used to be disabled by default because traditionally PNFS setups all had custom solutions for how to remove files from tape. Now that cleaning from tape is an integral part of dCache and PNFS is no longer supported, having the HSM cleaner work out of the box is a sensible default. It can be disabled by setting:

cleaner.enable.hsm = false

admin

dCache is in the middle of a transition between two frameworks for implementing the command line commands available through the admin service shell.

The newer of these frameworks provides much nicer help output. This output has been improved to include a description of the arguments accepted by a command and the default values for arguments and options. This information is only available for commands implemented using the new command framework.

Many of the commands shared between services have been ported to the new command framework.

The say and exec context commands are no longer available from cells other than the System cell.

Several admin commands primarily intended for internal use by dCache return array data. Executing these commands through the admin interface resulted in only an internal object reference. Now the array content is output.

xrootd

Added suport for the kXR_locate, kXR_query, and kXR_set xrootd commands.

Improved thread pool termination on door shutdown.

transfermanager

There is no longer a requirement that this service is hosted in the same domain as the srm service.

Error log messages have been simplified.

gplazma

LDAP

The LDAP plugin now allows the connection URL to defined by setting gplazma.ldap.url. The properties gplazma.ldap.server and gplazma.ldap.port are deprecated.

Changelog from 2.8.0 to 2.9.0

8e7eba6

pool: Report correct client IP to billing for passive FTP transfers

5dd65b3

srm: Fix race condition in listing

1b0f1a0

poolmanager: Fix infinite loop in lru partition type

24ab8b6

gplazma-nis: pulgin must fail if no matching principal found

16b2edd

nfs: do not use CDC as try-with-resource

99d3d1e

databases: update hikari minimum connection configuration

1a6ae5b

nfs4: do not log stacktrace on IO errors

9eb2d0e

poolmanager,pool: Fix precision underflow in WASS

318eb9c

scripts: fix host-credential generation

2f0997e

pool: Rewrite of HSM interface

820d1e9

Generify CDC executor decorators

7fba48d

pool: Let HsmStorageHandler2#store accept more than one file

004d4d0

Update libraries to the latest releases

b72f2b5

ftp: Implement delayed passive and enable redirect FTP for IPv6

db0e2f3

Enable IPv6 stack by default

9f0a1ab

TransferManager: fix error-handling

702feeb

nfs: use vfs cache

0e2fbbe

libs: use nfs 0.8.0

768470d

chimera: fix behavior on create of ‘.(fset)’ files

0c96cca

srm: tidy up toString methods

4fd65b8

info-provider: fix capitalisation of SRM in ControlProtocol

a506324

libs: update to nfs4j–0.7.5

8b79285

scripts: fix shell-specific expansion

49cb42c

srm/transfermanager: make errors less noisy

092c797

srm/transfermanager: make errors less noisy

000773d

srm: tidy up SRM CopyRequest and CopyFileRequest classes

5c82e13

ftp: fix unit tests so they run on IPv4-only machine

e740767

Kill workers when calling BoundedExecutor#shutdownNow

155c918

Merge remote-tracking branch ‘upstream/master’ into ldap-use-url/master

e9b7e51

srm: fix stored diff so that the Solaris ‘patch’ command works

5882be5

gplazma2-ldap: use url rather than host+port

c04dea8

chimera: introduce inode generation number

c0f05a8

scripts: add database updateSQL command

f145450

admin: Fix help on error

3283fb0

ftp: add work-around for GlobusOnline uploads

5ccbc24

ftp: fix CKSM command for files with white-space

debd98f

info: add potentially useful static information

8e84d93

info: remove emi.version from non-EMI dCache releases

bfd7687

pool: Let permanent migration jobs react to access and sticky changes

f66ae63

admin: Add array output

1987725

cells: Add help command to System cell

fa19746

transfermanagers: Fix multi-domain issue

990f351

gplazma-nis: suppress annoying error message

13f3750

ftp: add logging when client disconnects

9a20adc

cells: Fix message timeout

c09d854

admin: Fix cd to System cell

4602713

pkg: fix solaris packaging

eb5600d

ftp: add support for updating atime

75682ae

Fix CellStub callback.

e5a79ab

pnfsmanager: Fix tag handling in upload path creation

5911c11

billing: Add -f option to dcache billing command

2cf7540

chimera: do not explicitly update parent’s mtime on create/remove

d7c72d4

spacemanager: Another hsqldb schema management fix

9ff6642

Fix executor flaws

6b45326

Make RemotePoolMonitor asynchronous

cabc9b7

srm: Avoid loading the same job multiple times

834a73f

spacemanager: Fix DB trigger migration

7b0b285

nfs4: add support for WRITE with proxy IO

060f6c9

nfs4: refactor proxy io adapter

468ecc6

nfs4: implement write for DcapChannel

05adb25

nfs4: reuse existing mover if client comes with the same state id

780ae6e

srm: fix bring-online rescheduling on restart

268de60

srm: fix unit-tests after recent update

49d96a5

srm: fix restore so jobs for other schedulers are not restored

6b2f769

srm: Added missing import

233a8e7

srm: Break cyclic dependencies

72b3d4d

xrootd: Shutdown channel factory on shutdown

8d4b4bd

srm: Fix scheduling period for expiration task

da0fa23

srm: allow jobs in RQUEUED and READY states to survive restart

079b681

chimera: fix NPE if mkdir fails with permission denied

e14f2c2

srm: add description of job states

a6a5407

srm: fix restore from database

5905060

srm: Improve database initialization and shutdown

8743668

srm: Add error message to an IllegalArgumentException

5c94086

pool: Controlled termination of thread pools during pool shutdown

5d49667

poolmanager: Get rid of ThreadPool interface

51561af

Get rid of callback executor in CellNucleus

896a7ad

srm: Get rid of ThreadManager

f570ca7

nfs4: update stateid on read with proxy adapter

848ff79

Revert “poolmanager: Get rid of ThreadPool interface”

d06eb68

poolmanager: Get rid of ThreadPool interface

9e92fd4

srm: Terminate db threads on shutdown

ca3f244

billing: Make indexer case insensitive

d0e45fa

srm: Restore RESTORED state

7e6df20

pool: Avoid memory exhaustion problems on write with xrootd and http

a5b01e4

pool: Fix lock problems in xrootd and http movers

86f3486

ftp: add read-only support for ctime and atime

5d6ae37

logback: add comment about bugs in 1.1.1 to poms

9c19cca

pnfsmanager: Clean up cleaner by making use of new CellStub features

09a2ba9

Mock CellEndpoint rather than CellAdapter in HsmRestoreTest

40ef881

Get rid of support for synchronous messages in CellNucleus

6cbd994

loginbroker: Fix login broker registration retry

c1de9d8

chimera: allow updating the creation time

669659b

remote logging: add empty string defense to remote server wrapper

58fe22e

logback: correct MDC dependency in appender

5211277

srm: remove unused state

5bd41ca

srm: refactor SchedulerFactory

3744980

delegation: add support for GridSite

f35bf9b

libraries: update bouncy-castle

522e734

srm: fix race condition if stop is called too soon after start

bf1bb73

Get rid of more uses of CellNucleus#send methods

5ed882c

Avoid direct use of CellAdapter#sendMessage

1e70679

Remove direct calls of CellEndpoint#sendAndWait outside cells module

54e1d97

Layer synchronous CellStub use on top of async code

b45667f

Delete obsolete code

0e81660

Add transparent retry support to async use of CellStub

ef9ebc5

Eliminate some direct use of AbstractCellComponent#send

60ec6f7

Refactor CellStub

6fe60cc

Rename one-way CellStuf#send to notify

b3a7f60

srm: Replace pin and unpin callbacks with ListenableFuture

8040e1e

http: add minimal HEAD support in the mover

618a4d6

spacemanager: Remove obsolete features

7fb85dc

srm: Generate unique write TURLs

58e3a3f

Auto indented file to clean up inconsistencies

0d9b0fc

pnfsmanager: Add support for temporary upload paths

a000a73

http: adjust unit test to multirage changes

ef2091a

nfs: return NFSERR_IO is we detect broken file on tape

5c9b6b8

ftp: add support for MFMT, MFCT and MFF commands

f8d6408

alarms: add logging and peaceful exit for logback startup errors

702c68b

http: fix multipart reply

1d711d3

srm: Fix compatibility with PostgreSQL 8.4

89d0fdf

ftp: fix printing of UNIX.Mode fact

ac9a2b9

ftp: add logging to access log file

94afd50

ftp: add support for HELP and SITE HELP commands

206160a

logback: revert to 1.0.12

0efb799

http: fix mocking of Netty in unit-tests

a6c8a3a

nfs: do not label pnfsid and client ip in NDC

cead0f5

core: update PoolPassiveIoFileMessage to provide boot verifier

0d31355

Upgrade third party libraries

19bdad9

xrootd,httpd,pool: Upgrade to Netty 3.9.0

bbd7877

system-test: Explicitly disable GridFTP 2 in tests

baff93a

pool: Fix pf command regression

e3ac8f6

spacemanager: Fix database sequence initialization

0b8f0cd

srm: Make FTP TURLs absolute

844ee81

ftp: Make root path behaviour configurable

ef8ef96

Allow retry on transient transfer failures

a816ba1

srm-client: Do not retry on permanent FTP failures

3158ca2

pnfsmanager: Fix compatibility with 2.6 pools

93fe751

ftp: Fix race condition in command dispatching

cbadbf7

ftp: Give io thread pool a name

76e9ae4

chimera: Avoid integrity constraint violations on registering locations

71d15f9

srm: Avoid duplicate key violations on startup

2b6a29f

ssh1: Fix session instantiation

465705b

ssh1: Fix typo in service batch file

1393be6

ftp: add support for the CREATE fact

db108eb

system-test: Enabled tiered compilation

832b1d1

Improve startup speed of tools

76223fb

srm-client: Improve startup time

ad823f4

Convert tabs to spaces

a31b513

billing: Initialize all fields in billing records

4f79cb3

pool: Fix logging context for thread pool

f437cac

pool: Use FireAndForgetTask instead of custom class

b8108fe

chimera: fix inode type detection

773611e

srm: Include failed or unprocessed surls in list response

1987331

Add missing newline when quitting shell applications

e01bb75

Upgrade to HikariCP 1.3.2

401321e

xrootd: Upgrade to xrootd4j 1.3.1

eb58014

srm: Add SRM status code to SRMException

de267dc

srm: Various minor refactorings

d321353

srm: Take root path into account when selecting protocol

4e82995

chimera: fix create link on existing object

2775f0c

packaging: remove ‘–1’ from tarball

50440d2

srm: simply job.setState

9f18e80

gss: tidy up FQAN-extraction

e4f8fc5

nfs: fix NPE on door reboot

53a6ccb

nfs: remove unused exception declaration

e55e234

Convert tabs to space

af49653

chimera: Minor refactoring of tag reading code

5d30def

loginbroker: Register root path of doors

29e6e8d

Update to liquibase 3.1.1

227a73c

chimera: Add mv to chimera shell

7e516df

srm,srm-client: Report failure for failed SRM v1 transfers

77a0166

poolmanager: Add partition for transfer buffers

880920e

use ISO 8601 time format in sweeper ls -l output

051be99

srm-client: fix adler32

ce18915

pool: fix lost space messages (and cause)

ed34406

srmclient: fix support for URLs with IPv6 address

15a5d33

chimera: Optimize read of file and directory attributes

7eec2be

system-test: Fix wrong path in hostCredentials script

2dd9d0e

pool: Bind FTP data channel to single interface

f66153d

spacemanager: Replace freespaceinbytes with availablespaceinbytes in link group table

758f19e

spacemanager: Set file to TRANSFERRING on accept file request message

6b07486

src: split nfs code info a dedicated module

546074d

chimera: change FileSystemProvider#move() return value meaning

1287a28

chimera: fix transaction leak

fb72047

libs: update nfs to 0.7.4

119ecfe

chimera: fix rename into a file

6250598

cells: move command support into commons-cli and create skeleton CLI client

67eaf90

libraries: update Hikari to v1.3.0

3fee37e

nfs4: remove dead class

6e7943f

nfs-proxy: fix logging CDC

9f78b39

nfs4: clean failed request from pending queue

2a0723b

nfs-proxy: use sessionId provided by Transfer class

3aadb92

nfs-proxy: fix logging

8fe71fa

system-test: Add support for compatibility testing releases

dfa9c12

pool: Increase TCP backlog for FTP mover

6589114

ftp: Auto indent data channel code

f29613b

nfs-proxy: add support for open-less read requests

eaf12dd

Revert “nfs-proxy: add support for open-less read requests”

20d0df6

nfs-proxy: add support for open-less read requests

9841b1a

Log fatal errors when fetching pool monitor

18e90f3

gplazma-xacml: remove dependency on org.apache.xalan

0350958

Revert “Revert ”packages, fhs: fix saxon includes/excludes expression“”

71de63c

gplazma2-xacml: revert version of opensaml

09108b7

xrootd: Add support for kXR_locate, kXR_query and kXR_set

ca34bf8

dcache-webadmin, alarms: remove required annotations from datastore setters

2661256

dcache-core, alarms: restore xml connector to initialization properties

358648b

Revert “packages, fhs: fix saxon includes/excludes expression”

9ddc482

packages, fhs: fix saxon includes/excludes expression

05cb3fc

pool: Fix access time update

68055d4

xrootd: Upgrade to xrootd4j v1.2.4

1c05dce

chimera: Add time options to ls command

77c17d3

chimera: Add help command in shell

d50ddc2

chimera: Fix option parsing in FsFactory

ef153ef

core, billing: remove unit test

2d14212

libs: update to bugfix nfs4j–0.7.3

363cca6

billing,webadmin,alarms: Use HikariCP

772b9d9

Use HikariCP connection pool in most services

b84b3fa

dcache-core, alarms: remove postgresql precondition on schema

9ea839f

webadmin: Inject service configuration into Spring environment

b7234ee

Use Spring’s Environment and PropertySource feature to inject configuration

47b841f

Drop JDBC driver properties

0c41bc1

Add alarms and transfermanager database to system-test setup

a6bb4b0

libs: update nfs code to 0.7.2

3dee982

httpd: revert datanucleus-xml version (pom.xml)

79d9b47

chimera-nfs: add bringonline command to “.(fset)”

739be39

system-test: Add utility to start HSQLDB Swing Console

514cc1d

Use annotated command syntax for most shared commands

92645b9

Include arguments and default values in admin command help output

b25f8df

chimera: Optimized creation time column changeset

ed919cf

dcache-webadmin: fix ArrayIndexOutOfBounds exception in ListView (Tape Transfers page)

e0c5433

ftp: Reenable support for RFC 2428

cfe60f0

cells: remove conflict between CellAdapter and CellShell

0920e5a

srm-client: switch to common Args

2bd7d37

utils: move Args class into commons

acdad0f

Update branch ready for the 2.9 branch