What’s new in dCache 3.2
Release notes
Highlights
In terms of security and access management, dCache 3.2.0 offers several significant new features. This release introduces TLS encryption for domain communication, which will greatly facilitate setting up large distributed instances with WAN interconnections. On the client-visible side, macaroons can now be used as a means for detailed access control. And on a more admin-oriented level, the newly-introduced concept of roles makes privilege management easier and facilitates the delegation of tasks to trusted users.
From an administrative perspective, perhaps the most outstanding change is the switch to systemd-compatible scripts on Debian systems.
Incompatibilities
- Classic replica manager is no longer supported.
- Uses systemd only on Debian-derived systems if available.
- The admin door limits key-based login to the usernames listed in the
authorized_keys2
file. - The output of the
ps
admin command has changed. External scripts must be updated. - The
dcache.broker.port
property is deprecated. - The
srmmanager.net.port
andsrmmanager.net.local-hosts
are no longer used. - Upgrading nodes running
frontend
,webdav
orhttpd
to version 3.2.7 (or newer) requires upgrading nodes runningpoolmanager
at least to 3.2.7 (or newer) version.
Acknowledgments
Once again, we are pleased to have received contributions by several people who are not members of our core team.
We would like to thank Ivan Kadochnikov for his patches to xrootd.
For the first time, dCache was used in teaching a software development course at a university. We would like to thank the students of HTW Berlin who participated, and especially Fritz Heiden, Vuong Luu Minh, Stefan Moll, Lotta Rüger, Robin Wenzel, Yannick Vahldieck, Alena Schemmert, Marisa Nest, Martin Bürger, Sarah Schulte, Hasan Jahid and Max Patzelt, whose code made it into 3.2.0.
Differences from dCache v3.1
The notes for release 3.2.0 detail the differences from dCache 3.1. Please read this section very carefully when upgrading from this version.
Release 3.2.56
pool
Attempting to start a full checksum scan (with csm check *
) while an
existing scan is still running is no longer reported as a bug.
Changelog 3.2.55..3.2.56
- c3ad410dca
- [maven-release-plugin] prepare release 3.2.56
- 2fc1418c94
- pool: avoid IllegalStateException in ‘csm check *’ command
- 5de570d4cc
- [maven-release-plugin] prepare for next development iteration
Release 3.2.55
alarms
To ease troubleshooting, the POOL_DEAD alarm message now includes the pool name.
ftp
This release fixes an issue where a NullPointerException was thrown by FTP doors. The error occured only in very specific circumstances: a client would need to either download a file or issue a LIST, NLST or MLSD command after uploading a file through a passive door using proxied transfers.
pinmanager
A bug was fixed where PinManager’s bulk ls
admin command yielded a
NullPointerException if the optional argument was omitted.
A typo prevented the error message “Remote connection failure while unpinning…” from appearing completely and correctly in the logs. The error message string now contains the message string of the underlying Exception, hopefully providing helpful details for troubleshooting.
pool
A regression that prevented a replica’s position in the LRU queue for garbage collection from being updated was fixed.
webdav
Users asserting the “admin” role would occasionally receive NullPointerExceptions when trying to transfer files through WebDAV. This release fixes that issue.
Changelog 3.2.54..3.2.55
- ad31b5e459
- [maven-release-plugin] prepare release 3.2.55
- e13874a7f1
- UnpinProcessor: fix assumed typo
{)
- f3bace2ec8
- webdav: allow transfers as user with role ‘admin’
- cc2fc40a50
- pinmanager: avoid NPE if no argument given for ‘bulk ls’ command
- 082ae84015
- alarms: add pool name to POOL_DEAD alarm
- 9b894d4c4c
- ftp: avoid NullPointerException after passive upload
- b3fc6e3780
- pool: fix reordering of removable replicas on access
- 255241e81f
- [maven-release-plugin] prepare for next development iteration
Release 3.2.54
Changes affecting multiple services
Stage request from unknown locations resulted in NPE in dcap and pinmanager services.
this is now fixed and using dccp
to stage a file should work even if the location is unknown. ‘–’
resilience
The current release fixed race condition on replica state and no inaccessible file errors
occures for a newly written file.
Changelog 3.2.53..3.2.54
- c39f701
- [maven-release-plugin] prepare release 3.2.54
- 25d4a66
- dcap/pinmanager: stage request for unknown location results in NPE
- 102ceeb
- dcache-resilience (stable branches): fix race condition on replica state
- 1e0ba11
- [maven-release-plugin] prepare for next development iteration
- 829a090
- resilience: adjust synchronization of file operation removal from map
Release 3.2.53
pool
An unhelpful error message “Parameter directory
is not a directory” is
replaced with one that provides information on which directory is
missing.
Update error messages (previously “Could not create mover”) to provide more information about why the mover could not be created.
The pool no longer logs configuration or deployment problems that prevent the pool from creating a mover as if that problem was a bug.
The current release fixed certain error cases, where a pool is unable to create a mover are no longer logged as a bug in dCache.
transfermanager
The current release fixed a NPE if transfer was cancelled.
Changelog 3.2.52..3.2.53
- 2470947
- [maven-release-plugin] prepare release 3.2.53
- 6403603
- pool: avoid throwing a RuntimeException for non-bugs
- ad82d27
- pool: avoid log-and-throw anti-pattern
- caf8d9c
- transfermanager: avoid NPE on shutdown
- 27a281a
- pool: throw exception with meaningful error message
- c9d1f88
- pool: update error messages to make them distinct
- 9afcf32
- [maven-release-plugin] prepare for next development iteration
Release 3.2.52
resilience
In rare circumstances dark removes can result in data loss by removing of all replicas for a given file. The current release fixed the issue.
The admin command pool ctrl
provides for start
and shutdown
options which stop the processing of all pool operations. This is now fixed and pool operations can successfully be restarted from the
command line after they have been shutdown, without
restarting resilience.
Changelog 3.2.51..3.2.52
- 298adb4
- [maven-release-plugin] prepare release 3.2.52
- 66f9203
- resilience: update state on pool operations when restarted from admin command
- 33ff6b8
- resilience: do simple existence check of replica on pool to avoid dark removes
- 20bf397
- [maven-release-plugin] prepare for next development iteration
Release 3.2.51
dcache
Jetty version is updated to 9.4.12.v20180830.
Changelog 3.2.50..3.2.51
- c6504b8
- [maven-release-plugin] prepare release 3.2.51
- b4676ac
- libs: update jetty version to 9.4.12.v20180830
- 9b8db34
- [maven-release-plugin] prepare for next development iteration
Release 3.2.50
webdav
The current release fixed resource name for door root error.
Changelog 3.2.49..3.2.50
- d1ebac9
- [maven-release-plugin] prepare release 3.2.50
- af29d90
- webdav: fix resource name for door root
- b0953dd
- [maven-release-plugin] prepare for next development iteration
Release 3.2.49
ftp
Now clients can request the checksum value of a file not owned by that user and where dCache does not already know the checksum value.
pool
The current release fixed some logging on the pool where messages were recorded against an arbitrary context (i.e., the bit in square brackets), resulting in misleading information.
Changelog 3.2.48..3.2.49
- 4e7509c
- [maven-release-plugin] prepare release 3.2.49
- ad0e509
- pool: fix CDC for repository listener notification
- 4c4a603
- ftp: store calculated checksum using root privileges
- 46df236
- [maven-release-plugin] prepare for next development iteration
Release 3.2.48
webdav
When users request a macaroon via an HTTP POST request targeting a specific path, a caveat is created that restricts the macaroon to that path (requests to / result in a non-limited macaroon).
Commit 99c726e3 resulted in users getting back a non-limited macaroon for every request. This issue was fixed with this release.
Changelog 3.2.47..3.2.48
- de788e78a4
- [maven-release-plugin] prepare release 3.2.48
- 5102908077
- systemtest: fix OpenSSL DN format change
- c01e841292
- webdav: fix path-to-caveat for macaroon minting endpoint
- 4071b1f305
- [maven-release-plugin] prepare for next development iteration
Release 3.2.47
alarms
Pool errors involving a fatal repository fault, for instance, can be sent now as an email alarm without having to send all pool disabled alarms.
gplazma
The JAAS gplazma plugin no longer logs a stacktrace on bad configuration.
pool
The current release improved error messages making them clearer by avoid using the same error message in multiple places.
srm
A stack-traces fixed for concurrent updates in pin-manager and similar (expected) failures.
transfermanager
Now Third-party transfers fail if the client is requesting to copy a file from dCache that has not fully been uploaded.
webdav
Disabling basic authn should not now disable macaroons. The current release fixed webdav.authn.basic
and frontend.authn.basic
so that setting
these configuration properties to false
no longer blocks macaroons from being accepted in the HTTP Authorization header.
The current release impoved error messaging for unauthenticated request.
IllegalArgumentException exception is fixed now and attempts by a client to copy a file that has not fully been uploaded results in a clear error response.
Changelog 3.2.46..3.2.47
- 845c821
- [maven-release-plugin] prepare release 3.2.47
- 042dd88
- webdav/frontend: disabling basic authn should not disable macaroons
- c180cb9
- srm: do not log a stack-trace on expected Exception errors
- 53e91a3
- transfermanager: fail third-party copy if the file is still being uploaded
- c8aca24
- webdav: fail COPY early if file is currently being uploaded
- a229a2b
- transfermanager: abort transfer if there is a bug
- 9f4f57e
- gplazma: JAAS plugin logs a stack-trace on misconfiguration
- f51bdf5
- pool: avoid using the same error message in multiple places
- 136c71b
- alarms: add pool dead alarm
- d73afdd
- [maven-release-plugin] prepare for next development iteration
- 9ae8e3d
- pool: don’t update atime on flush
- e797a8d
- scripts: fix ‘dcache pool yaml’ command
- 9e1c8d5
- webdav: 401 for unauthenticated requests; message in status line
Release 3.2.47
alarms
Pool errors involving a fatal repository fault, for instance, can be sent now as an email alarm without having to send all pool disabled alarms.
gplazma
The JAAS gplazma plugin no longer logs a stacktrace on bad configuration.
pool
The current release improved error messages making them clearer by avoid using the same error message in multiple places.
srm
A stack-traces fixed for concurrent updates in pin-manager and similar (expected) failures.
transfermanager
Now Third-party transfers fail if the client is requesting to copy a file from dCache that has not fully been uploaded.
webdav
Disabling basic authn should not now disable macaroons. The current release fixed webdav.authn.basic
and frontend.authn.basic
so that setting
these configuration properties to false
no longer blocks macaroons from being accepted in the HTTP Authorization header.
The current release impoved error messaging for unauthenticated request.
IllegalArgumentException exception is fixed now and attempts by a client to copy a file that has not fully been uploaded results in a clear error response.
Changelog 3.2.46..3.2.47
- 845c821
- [maven-release-plugin] prepare release 3.2.47
- 042dd88
- webdav/frontend: disabling basic authn should not disable macaroons
- c180cb9
- srm: do not log a stack-trace on expected Exception errors
- 53e91a3
- transfermanager: fail third-party copy if the file is still being uploaded
- c8aca24
- webdav: fail COPY early if file is currently being uploaded
- a229a2b
- transfermanager: abort transfer if there is a bug
- 9f4f57e
- gplazma: JAAS plugin logs a stack-trace on misconfiguration
- f51bdf5
- pool: avoid using the same error message in multiple places
- 136c71b
- alarms: add pool dead alarm
- d73afdd
- [maven-release-plugin] prepare for next development iteration
- 9ae8e3d
- pool: don’t update atime on flush
- e797a8d
- scripts: fix ‘dcache pool yaml’ command
- 9e1c8d5
- webdav: 401 for unauthenticated requests; message in status line
Release 3.2.46
ftp
The current release fixed MLSC command for non-small directories and Globus is now able to list directories with > 100 directories.
Changelog 3.2.45..3.2.46
- aceef3b
- [maven-release-plugin] prepare release 3.2.46
- 446a8fd
- ftp: fix MLSC command for non-small directories
- aedc96b
- [maven-release-plugin] prepare for next development iteration
Release 3.2.45
billing
Database connection loss now is reported for billing.
webdav
The current release fixed the problem where all but one requests fail, if multiple concurrent PUT requests have directories in the path that do not already exist.
Changelog 3.2.44..3.2.45
- 2686bd9
- [maven-release-plugin] prepare release 3.2.45
- bdcbef7
- dcache: wrap billing data source with AlarmEnabledDataSource
- b28891d
- common: fix random data generation in TimeseriesHistogram unit test
- 03b7c02
- webdav: work-around Milton racy API for creating collections
- 31c4ea8
- webdav: fix name of root
- c201313
- [maven-release-plugin] prepare for next development iteration
Release 3.2.44
gplazma
Since update to newer BC and voms-java-api libraries sites report VOMS certificate validation errors like This is now fixed.
srm
The dcache ports
command now includes the srm’s TLS/SSL interface.
Changelog 3.2.43..3.2.44
- 7864f0e
- [maven-release-plugin] prepare release 3.2.44
- 4ba7a54
- gplazma voms plugin: add trust anchor refresh paramater
- 2ab9153
- srm: include TLS/SSL port in ‘dcache ports’ command
- 4d5e0eb
- [maven-release-plugin] prepare for next development iteration
Release 3.2.43
Changes affecting multiple services
The current release corrected the properties for access-log.
Changelog 3.2.42..3.2.43
- 53942b4
- [maven-release-plugin] prepare release 3.2.43
- 59f4163
- correct the properties for access-log
- 4b98a10
- [maven-release-plugin] prepare for next development iteration
Release 3.2.42
Changes affecting multiple services
If a client specifies a checksum value with either a WebDAV or FTP upload,
a Restriction check by-passed due to missing path
warning was logged occasionally.
This was fixed now, ensuring that restrictions are always applied.
pool
Space reservations on pools that are connected to tape showed a problem with failing restore requests: If a restore failed, the space that was reserved to hold the file that was supposed to come in from tape was not freed again but kept in the ‘sticky’ state. This resulted in lots of unusable space on pools that could only be reclaimed through a restart.
With the current release, this issue is fixed and space is freed as soon as possible after a failed restore request.
resilience
A very rare race-condition is fixed where a failed upload results in resilience recording a stack-trace.
webdav
An issue with the Milton WebDAV library prevented Partial (or vector-read) GET requests from succeeding. This was fixed now through both an update of the dependency and a local patch while we wait for the proposed fix to be included upstream.
Changelog 3.2.41..3.2.42
- e35f2a0252
- [maven-release-plugin] prepare release 3.2.42
- ecbb4843f9
- fix compilation
- 0c4c34bc5a
- webdav: fix proxied partial (vector-read) GET requests
- f9b7c732de
- pool: fix pool space accounting on failed restores
- e77e0164c0
- resilience: fix NPE if file unlinked when resilience processes a broken file
- 9b707a36a2
- ftp/webdav: fix bypass of restrictions
- 5e98600a74
- [maven-release-plugin] prepare for next development iteration
Release 3.2.41
alarms
An internal issue with the alarms configuration was fixed, which should prevent a rare NullPointerException from occuring.
dcap
Creating a file or directory using the DCAP protocol with a URL as parameter, the file permissions were not set correctly.
With the current release, this was corrected, and such files use the client-supplied file permissions. If none are provided, the default modes 0700 (for directories) and 0600 (for files) are used.
xrootd
An uncaught exception in xrootd doors was fixed.
Changelog 3.2.40..3.2.41
- 33df8badde
- [maven-release-plugin] prepare release 3.2.41
- ca929f7d3c
- alarms: fix persistence.xml configuration
- e44c049717
- dcap: fix permission propagation with DCAP
- 4783b17d3e
- dcache-xrootd: handle possible race condition in directory listing
- a84feebbc9
- [maven-release-plugin] prepare for next development iteration
Release 3.2.40
statistics
Metadata merge was using max when it should had used min, this is now fixed.
Changelog 3.2.39..3.2.40
- 61d4b76
- [maven-release-plugin] prepare release 3.2.40
- 4e1c2aa
- common: fix histogram metadata merge
- 788b788
- [maven-release-plugin] prepare for next development iteration
Release 3.2.39
ftp
The current release provides better protection against leaking proxy/data TCP sockets if client aborts a proxied transfer.
srm
Clients that use the gridsite protocol, such as davix, can now delegate their credential.
Changelog 3.2.38..3.2.39
- db287b9
- [maven-release-plugin] prepare release 3.2.39
- 06d5bfe
- ftp: make shutdown more robust
- 1f83d3b
- common: fix bug in CountingHistogram index computation
- 3dc7e59
- [maven-release-plugin] prepare for next development iteration
- e4d22d3
- srm: gridsite fix querying validity of delegated credential
Release 3.2.38
ftp
The performance markers that dCache sends back to the client in FTP transfers are now more robust against bugs.
nfs
When transient errors in pools cause NFS transfers to have to wait and retry, the system’s behaviour is now more robust and no StackOverflowErrors should be logged any more.
scripts
Maven’s findbugs plugin is now granted more working memory in order to make builds, especially on our continuous integration system, more robust.
srm
Certificate lifetime considerations for VOMS proxy certificates are improved in this release: if a client delegates a credential where the VOMS AC expires before the X.509 proxies, dCache now will not use the credential beyond the AC expiry time. This avoids unnecessary authentication errors.
webdav
When the WebDAV door is considering an HTTP third-party-copy request that uses grid-site delegation, there is a minimum 20 minute validity that any existing delegated credential must satisfy. If this is not satisfied then dCache will request a fresh delegated credential.
Until now, if the client failed to delegate a fresh certificate then the subsequent COPY request was rejected. This release changes that behaviour and enables such transfers.
Changelog 3.2.37..3.2.38
- 8d0835b43e
- [maven-release-plugin] prepare release 3.2.38
- b64009c878
- scripts: Avoid findbugs memory errors
- 8cd739b36d
- nfs: increase request retry delay when selecting/starting pool or mover
- 8e4c745633
- webdav: adjust minimum validity after requesting delegation
- 6a026804e2
- srmmanager/webdav: consider VOMS AC validity of delegated credential
- 8c15015f63
- ftp: make performance marker task robust.
- 8d877aec3b
- [maven-release-plugin] prepare for next development iteration
Release 3.2.37
pool
Diagnostic logging for failed HTTP third-party transfers was improved.
Billing records for failed transfers now show more detailed information.
The handling of cancelled flush requests for nearline media was rewritten to be more efficient. This resolves issues where pools report “Flush of 0000… failed with: CacheException” followed by “Pool restart required: Internal repository error”.
Compatibility with DPM was improved by increasing HTTP GET requests’ timeouts. This should allow more transfers to succeed.
poolmanager
Supplying poolmanager with an unresolvable hostname as the target will now result in an UnknownHostException instead of the previous behaviour where an (unnecessary) NullPointerException was thrown.
srm
Logging of errors in the SRM credential store was improved.
webdav
If a non-resolvable host name is given as the source or destination of a third-party copy request, WebDAV will now fail the transfer immediately instead of waiting for a Poolmanager timeout.
Diagnostic logging for failed HTTP third-party transfers was improved.
xrootd
dCache allows xrootd clients to specify a query/opaque string in a kXR_mv request’s source path.
Changelog 3.2.36..3.2.37
- 2015e44697
- [maven-release-plugin] prepare release 3.2.37
- da040a2395
- pool: HTTP TPC rework exception logging
- f2f359d250
- pool: increase TPC socket timeout for GET requests
- a3995fe5b7
- srm: fix credential store logging
- 1a474ac8f7
- pool: update log status using exception class name if no message
- 08e79346ef
- xrootd: strip off query part from kXR_mv source
- fd0187db6c
- webdav: fail TPC request early on unknown hostname
- 1986a52bfe
- nearline-provider: do not propagate thread interrupt flag
- f1cde32f35
- poolmanager: fix NPE on unknown host
- 632bd934dd
- webdav: improve logging of TPC requests
- 4c835638fa
- [maven-release-plugin] prepare for next development iteration
Release 3.2.36
Changes affecting multiple services
In order to more easily identify a rejected macaroon in the logs, its ID is now included in the log message.
An irrelevant stacktrace was logged on unexpected CacheExceptions. This was removed, leading to less clutter in the logs.
Different macaroons that were issued against the same secret are now discernible in the logs.
Users now get more information about the reasons why an invalid macaroon was rejected: HTTP requests that are made with an invalid macaroon have a 401 HTTP response with the status-line explanation phrase that describes why the macaroon is invalid.
The access log file also logs why a macaroon was rejected.
core
A library dependency was updated to avoid CVE–2018–11771. This patch introduces no user-visible changes.
gplazma
Invalid macaroon logins no longer “spam” gPlazma.
pnfsmanager
When creating a macaroon to allow uploading of data, the desired path may not already exist. Without restrictions, WebDAV will auto-create parent directory items that are missing, or the client can create these directory elements explicitly with MKCOL.
With restrictions (such as from a macaroon) such directory creation currently requires the MANAGE activity, which allows other actions beyond the scope of this scenario. With this release, the behaviour was changed so that a user with a macaroon that authorises them to upload data into a particular directory will be able to create parent directories to achieve uploading the data.
pool
A regression caused pools that had their size only specified in a layout file to report a size of 8 Exabytes. This issue was fixed.
dCache now supports a DPM-specific HTTP extension that indicates the checksum calculation is not yet complete, avoiding potential data corruption with third-party copies: If DPM is calculating a checksum, then any RFC 3230 (i.e., with a ‘Want-Digest’ header) GET or HEAD request returns ‘202 Accepted’ respond status line and an HTML page as the response entity. Since dCache considers any 2xx response as success, the HTML page was previously accepted as the file’s contents, resulting in data corruption.
dCache pools no longer log a stack-trace for non-bug P2P failures.
srm
The domain ‘.access’ log file now contains log information for grid-site delegation activity, which facilitates debugging of http third-party-copying issues.
transfermanagers
The “restriction check by-passed” warning for each WebDAV-initiated third-party transfer is fixed.
webdav
A user may request a macaroon by making an HTTP POST request to the WebDAV door. This log entry was augmented by the ID and type of macaroon used.
A previous patch needed a bit of an update to ensure that X.509-with-FQAN authenticated third-party transfers with macaroons work under all circumstances. This is now ensured.
xrootd
The --zip
option of xrootd clients is now supported.
Changelog 3.2.35..3.2.36
- 4166b4b29b
- [maven-release-plugin] prepare release 3.2.36
- acd769fba0
- xrootd: add support for kXR_stat on open files
- 9556512fa0
- pool: P2P failures trigger stack-trace
- 80dfa61625
- webdav: obtain FQAN from X.509 credential for gridsite
- c1b19f6443
- core: avoid sending bad macaroons to gplazma
- 4b26086b58
- webdav: update access log to record macaroon request details
- 5cea320aff
- transfermanager: fix missing path
- 7ff01148cc
- libs: update to commons-compress–1.18
- 1e84c57b12
- macaroons: include macaroon id in error message
- 635dee7b22
- pool: fix pool’s runtime configured size regression (b70b0d9)
- e3e03a50dd
- core: provide better feedback and logging if a macaroon is rejected
- eba086bd1a
- pool: update HTTP TPC to support retrying GET and HEAD requests for DPM
- 1ddd05ec66
- srm: add gridsite delegation interface access-log
- 06f235a099
- macaroons: fix logged id
- 5bcd4d03a8
- core: avoid stacktrace on arbitrary CacheException
- c5830a6522
- [maven-release-plugin] prepare for next development iteration
- cdc0d5eb18
- pnfsmanager: allow restricted user with UPLOAD to create parent directories
Release 3.2.35
poolmanager
This release increases responsiveness for users that are not allowed to stage files, and for NFS users who access offline files. In cases where such a user issued a read request at the same time that Pool Manager handled a staging request, the first request would block for the duration of the staging – potentially quite a while. From now on, users that are not allowed to stage receive appropriate error messages as soon as possible, without having to wait for anyone else.
xrootd
Support for xrootd mkdir
was improved.
Changelog 3.2.34..3.2.35
- 154c5445b1
- [maven-release-plugin] prepare release 3.2.35
- 22aaf03765
- xrootd: update to xrootd4j dependency to 3.2.3
- 1a1691e3fa
- poolmanager: do not squash request if state is not allowed
- 25815d60f0
- [maven-release-plugin] prepare for next development iteration
Release 3.2.34
sysytemd
Systemd did not inherite the system-wide limits and was completely ignoring /etc/security/limits.d/92-dcache.conf
.
This is now fixed and the limits successfully loaded and enabled as expected.
Changelog 3.2.33..3.2.34
- fa184d2
- [maven-release-plugin] prepare release 3.2.34
- e465689
- [maven-release-plugin] prepare for next development iteration
- 529a5a2
- systemd: Add
/etc/security/limits.d/92-dcache.conf
in the dcache systemd unit and generator.
Release 3.2.33
Changes affecting multiple services
This rlease fixes an issue with WebDAV 3rd-party-copy requests that are authorized using a macaroon that is only valid for writing a specific file.
NOTE: both the webdav door and transfermanagers must be updated before the fix is effective.
pool
In order to help with debugging issues with partial FTP transfers, dCache pools now are able to log considerable information about failed FTP transfers.
This is controlled by the new property pool.mover.ftp.enable.log-aborted-transfers
.
webdav
dCache can now transfer data with a remote site, authenticating with that remote site using a delegated X.509 credential, but authenticating locally with a macaroon.
xrootd
This release updates xrootd4j, which should help fix occasional “pad block corrupted” issues with older clients.
Changelog 3.2.32..3.2.33
- d390a16982
- [maven-release-plugin] prepare release 3.2.33
- f26c3650c1
- pom.xml: update xrootd4j dependency to 3.2.3
- 0c6f51c5e4
- webdav: use TLS credential directly for gridsite
- bd13c21bc2
- pool: instrument ftp mover to show partial transfers
- 9663d40ae5
- webdav+transfermanagers: support TPC pull with targeted macaroons
- c5a6d0af64
- [maven-release-plugin] prepare for next development iteration
Release 3.2.32
frontend
The current release fixed broken directory QoS reporting and now frontend now more accurately describes the QoS of directories; i.e., the QoS that newly written files will receive when written into this directory, assuming none of the targeted pools are volatile.
webdav
the macaroon creation with multiple path restrictions failed with a http error 500 and the error message. This is now fixed and the macaroon creation succeeds when multiple path restrictions are defined.
The current release improved error handling for PROPFIND request.
Changelog 3.2.31..3.2.32
- 117b68a
- [maven-release-plugin] prepare release 3.2.32
- 40e9387
- frontend: fix broken directory qos reporting
- c34cba3
- webdav: avoid throwing any exception when listing a directory for PROPFIND
- e370a41
- webdav/macaroon: Fix macaroon creation with multiple path restrictions.
- 07924ed
- [maven-release-plugin] prepare for next development iteration
Release 3.2.31
ftp
dCache now has the ability to log the current status of a transfer at the point the client decided to abort an FTP transfer. This should support a post mortem investigation on why a transfer was cancelled.
nfs
With the current release the timeout of pnfshandler is configurable and nfs door quicker recovers from situations, when a PnfsManager is not available.
Changelog 3.2.30..3.2.31
- ae85848
- [maven-release-plugin] prepare release 3.2.31
- bd2e07b
- ftp: add ability to log client-aborted transfers
- 8e491e4
- nfs: make timeout of pnfshandler configurable
- 1dffc9c
- dcache: release dcache-view version 1.3.3
- c65318a
- [maven-release-plugin] prepare for next development iteration
Release 3.2.30
NFS
When two clients A and B operate on a file in quick succession, A opening the file and B deleting it before LAYOUTGET is called, dCache puts the transfer into the list of active transfers and returned NFS4ERR_NOENT. If a client tries to optimize the corresponding CLOSE call away, as some do, the entries are never removed from the list, effectively creating a leak.
This problem was fixed. Clients now receive an NFS4ERR_STALE message in those cases.
core
Certain transfer failures, such as attempting to use a space-reservation that has insufficient capacity, resulted in the door eventually reporting a time-out problem to the client.
A typical error message would resemble
Request to [>SpaceManager@local ... ] timed out.
This problem was traced to an internal misconfiguration of a messaging component and is fixed from this release onwards.
frontend
The reporting of a file’s QoS status in frontend was improved. Files that are being scheduled for moving to tape are now reported as ‘tape’ instead of ‘disk’.
pool
A bug was fixed that occasionally caused problems with the pools’ Berkeley DB. This could, for example, be triggered by removing files which were in a flush queue.
A typical error message was, e.g.
27 Aug 2018 12:09:33 (cat2_lhcbtape) [Frontend-dcacheview PoolDataRequest] Fault occurred in repository: Internal repository error. Pool restart required: : CacheExcept
ion(rc=204;msg=Meta data lookup failed and a pool restart is required: (JE 7.3.7) Environment must be closed, caused by: com.sleepycat.je.ThreadInterruptedException: En
vironment invalid because of previous exception: (JE 7.3.7) /space/lhcb/tape/pool/meta java.lang.InterruptedException THREAD_INTERRUPTED: InterruptedException may cause
incorrect internal state, unable to continue. Environment is invalid and must be closed.)
27 Aug 2018 12:09:33 (cat2_lhcbtape) [Frontend-dcacheview PoolDataRequest] Pool mode changed to disabled(fetch,store,stage,p2p-client,p2p-server,dead): Pool restart req
uired: Internal repository error
webdav
Web clients (such as web-browsers) make OPTIONS pre-flight requests to discover what they are allowed to do, according to the CORS standard.
Unfortunately, some web-browsers make the OPTIONS request without presenting any credentials. If the resource is within a protected directory then dCache currently fails the OPTIONS request.
This release introduces a new behaviour where such requests will always succeed, so that browser pre-flight requests are not hampered.
Changelog 3.2.29..3.2.30
- a47eea0e8f
- [maven-release-plugin] prepare release 3.2.30
- 9bc218ab2a
- nearline-provides: do not interrupt processing thread on cancel
- 303de641f9
- nfs41: invalidate open-state on layoutget if file is removed
- 73a5f72db9
- webdav: always respond to OPTIONS request
- 129188e8ff
- core: ensure pool/poolmanager communication receives errors
- bb53e518f3
- frontend: add targetQoS for not-yet-flushed tape files
- 10d95ca99a
- [maven-release-plugin] prepare for next development iteration
- 4cbee39946
- dcache: release dcache-view version 1.3.2
Release 3.2.29
gplazma
The OidcAuthPlugin plugin was updated so that users whos op does not claim
name
, and does not claim given_name
nor
family_name
can use dCache.
pool
This release fixed the log stack-trace for queue
admin commands and now bad admin input for the following admin commands no longer results in a stack-trace being logged:
- queue activate
- queue activate class
- queue remove class
- queue suspend class
- queue resume class
- queue remove pnfsid
poolmanager
NPE is fixed when staging files back from tape and
poolmanager.enable.cache-hit-message
is true.
webdav
The current release updated default credential delegation for third-party copy so that now requesting a third-party copy using a macaroon does not trigger a failed attempt to OpenID-Connect delegation.
Changelog 3.2.28..3.2.29
- 608c97c
- [maven-release-plugin] prepare release 3.2.29
- 63e6a6b
- poolmanager: fix NullPointerException when staging files and reporting hits
- 65a8d62
- gplazma: oidc fix FullNamePrincipal creation
- 7fb4034
- libs: update jetty to version 9.4.11
- 013c846
- pool: ‘queue’ admin commands not the log stack-trace on bad arguments
- 7711b70
- webdav: update default credential delegation for third-party copy
- 28b9c34
- [maven-release-plugin] prepare for next development iteration
Release 3.2.28
history
This release fixes a bug that could cause startup errors in the history service in the face of network errors.
many
Remote pool monitor would occasionally log stack traces from exceptions
when a domain shut down due to an interrupt. This has been fixed, reducing
the number of irrelevant log entries in such situations.
Changelog 3.2.27..3.2.28
- ba9e256593
- [maven-release-plugin] prepare release 3.2.28
- 11c703f059
- dcache-history: handle Gson syntax errors explicitly
- 5088923788
- cells: add handling of RemoteProxyFailureException nested InterruptedException to UncaughtException handler
- 7e1e2a4bf9
- [maven-release-plugin] prepare for next development iteration
Release 3.2.27
nfs
dCache 4.0 and 3.2 now use nfs4j version 0.15.4, which includes bugfixes for rarely observed deadlocks and incomplete directory listing over nfs.
Changelog 3.2.26..3.2.27
- 406a375f8f
- [maven-release-plugin] prepare release 3.2.27
- a638309921
- pom: update nfs4j–0.15.4 bugfix version
- 79a88a12e4
- [maven-release-plugin] prepare for next development iteration
Release 3.2.26
pool
HTTP responses now contain more meaningful messages along with the HTTP response codes, instead of only just showing stock messages like “400 Bad request”.
Changelog 3.2.25..3.2.26
- 86f0c693ff
- [maven-release-plugin] prepare release 3.2.26
- d77154fa4a
- pool: update HTTP mover to report errors as HTTP status message phrase
- 6c809c0f4f
- [maven-release-plugin] prepare for next development iteration
Release 3.2.25
resilience
Resilience suffered from a bug that would lead to a NoSuchElementException when a pool name no longer mapped to a location known to the Resilience service. This issue has been fixed.
When multiple pools go offline it is possible that all replicas for a given resilient file become unreadable. If the file is not CUSTODIAL, and thus cannot be restored from tape, the discovery of such a file during scanning will generate an error in the ‘history errors’ listing, in the resilience domain .resilience log, and will also raise a general alarm concerning the pool.
There currently exists a command, ‘inaccessible’, which generates a listing of the pnfsids on a given pool which in the current state of dCache have no readable replicas. However, this command takes a while to complete (asynchronously), and the output is written to a file which must be viewed by logging in.
This release introduces ‘refering pool’ information to the error output so that grepping the resilience log for a given pool becomes easier, and adds options to the command to check further details.
Changelog 3.2.24..3.2.25
- 6f85108e66
- [maven-release-plugin] prepare release 3.2.25
- 5419c0a8a3
- dcache-resilience: improve inaccessible file accounting
- bc803be5d0
- dcache-resilience: skip invalid cancel filters
- 50ce7254b7
- [maven-release-plugin] prepare for next development iteration
Release 3.2.24
scripts
A regression in the dcache pool convert
command was fixed; the command works again.
scripts
The instructions that are printed out once dcache pool convert
completes successfully now correctly point to the
property that needs to be updated, namely pool.plugins.meta
.
Changelog 3.2.23..3.2.24
- 427f08e306
- [maven-release-plugin] prepare release 3.2.24
- c06f09b136
- pool: fix ‘dcache pool convert’ command
- 90a41ca479
- scripts: update reference to configuration property
- ce0430201e
- pool: fix metadata migration tool to use Path
- c7772ea933
- [maven-release-plugin] prepare for next development iteration
Release 3.2.23
pool
This release improves dCache’s robustness against network errors: In case registering a file with PNFS manager fails due to a timeout, the request is retried transparently.
Changelog 3.2.22..3.2.23
- bb4d51dd22
- [maven-release-plugin] prepare release 3.2.23
- a7739d65b9
- vehicles: fail-fast on invalid path
- 11ed4c7cea
- pool: retry request to pnfs manager if timed out
- b76775f728
- [maven-release-plugin] prepare for next development iteration
Release 3.2.22
door
The current release added support for a door advertising multiple hostnames or IP addresses. dCache doors can now advertise multiple interfaces, including DNS aliases.
webdav
Milton’s OPTIONS handler was returning a 404 error if an OPTIONS request targets an entity that did not exist. This behavior deviated from Apache httpd server and was resulting in failed uploads for dcache-view. The current release fixed.
Changelog 3.2.21..3.2.22
- 5b8e32b
- [maven-release-plugin] prepare release 3.2.22
- 2aafba0
- gplazma.properties: hint to enable roles
- 7fe48f1
- doors: support advertising multiple addresses in LoginBroker
- 7ded7cf
- webdav: do not return 404 for OPTIONS request targeting absent entity
- 64cf055
- [maven-release-plugin] prepare for next development iteration
Release 3.2.21
dcache-resilience
There was a small regression in the way resilience computes the number of operations necessary to adjust copies when a storage unit definition changes.
The current rellease fixed computation of operation count when storage requirements change.
ftp
In order to aid diagnosing problems when FTP response being lost, now dcache logs failures to wrap/encrypt responses.
webdav
The current release improved error handling for client authentication with OpenID-Connect. A more complete set of information is now logged if OIDC delegation fails, supporting the ability to discover why the delegation failed.
Changelog 3.2.20..3.2.21
- e10182e
- [maven-release-plugin] prepare release 3.2.21
- 163d4bb
- scripts: add support for parsing ZooKeeper transaction logs
- 38ae2b7
- ftp: log failures to wrap/encrypt responses
- 5b2ba4b
- dcache-resilience: fix computation of operation count when storage requirements change
- 4ef97b8
- webdav: log errors if OIDC delegation fails
- 7e86358
- [maven-release-plugin] prepare for next development iteration
Release 3.2.20
ftp
Error reporting in the FTP service has been improved: in some mixed IPv4 / IPv6 scenarios, only unclear error messages were reported.
Changelog 3.2.19..3.2.20
- ecd42ce507
- [maven-release-plugin] prepare release 3.2.20
- 7827470210
- ftp: returned error is too vague for meaningful investigation
- 77c9337778
- [maven-release-plugin] prepare for next development iteration
Release 3.2.19
nfs
Situations, where selection process was incomplete could not be manually recovered
When selection process incomplete, due to PoolManager
restart, there was no way to trigger a new selection.
The current release added two new commands to nfs door
: transfer retry
and transfer forget
.
The first command manually re-activates existing transfer by re-trying selection process. The second one should be used to completely ‘forget’ the stale transfer and let client to trigger a fresh selection process.
Changelog 3.2.18..3.2.19
- d580e01
- [maven-release-plugin] prepare release 3.2.19
- df58f42
- nfs: add commands to reactivate stale transfers
- 69d822a
- [maven-release-plugin] prepare for next development iteration
Release 3.2.18
gplazma
gplazma now supports a Fermilab-specific authorization data source in JSON format.
pool
During active ftp transfers, connection problems would lead to the rather unhelpful error message “451 General problem”. This error reporting was refactored, so that diagnosis of the cause is now greatly facilitated.
poolmanager
A potential NullPointerException (that was not observed in real-world usage until now) was fixed in Pool Manager.
resilience
A correction to resilience’s error handling results in no more reports on non-resilient (but corrupted) files.
spacemanager
In order to facilitate debugging, Space Manager now logs link-group related content in greater detail.
Changelog 3.2.17..3.2.18
- 3914a3e46b
- [maven-release-plugin] prepare release 3.2.18
- 9d01c5a804
- poolmanager: fix migration command if named pool is removed
- ed541dbd38
- dcache-resilience: repair over-aggressive handling of broken file messages
- bce6a346d6
- pool: fix error message for failed active FTP transfers
- 1e73d5f6a4
- fix the project version in pom.xml
- e58062e696
- gplazma-fermi: fix last modified check in junit test
- fef97c5dd1
- spacemanager: add remote pool monitor debug logging
- abf7548dd2
- gplazma-fermi: add mapping plugin to support VO group and username from file
- 8b228e192a
- [maven-release-plugin] prepare for next development iteration
Release 3.2.17
Changes affecting multiple services
This version removes the (by now unused) directory /var/lib/cell-info from a default installation. The directory was previously used to store cell info data. With the introduction of the history service in dCache 3.2.1, this became obsolete.
A small bug-fix addresses wrong directory permissions on tar or Debian packages, where the directory /var/lib/dcache/pool-history had the wrong permissions.
The deprecated properties for configuring alarms have now finally been made unavailable.
chimera
An internal update enables chimera to use PostgreSQL 10.
nfs
A modification in IP address handling greatly increases the speed of NFS client access for Linux clients in mixed IPv4/IPv6 environments.
Prior to Linux 4.12, Linux clients with only an IPv4 address would wait for (timeout * retry) seconds when connecting to pools with both a v4 and v6 address. This was fixed upstream in Linux 4.12, but that fix was not backported to e.g. RHEL 7 yet.
Changelog 3.2.16..3.2.17
- 3c3d887c68
- [maven-release-plugin] prepare release 3.2.17
- ab5d6e6e97
- nfs: filter out IPv6 DS addresses if client connected with v4
- 278d7e5a33
- chimera: adjust postgres driver provider to new version schema
- 1f18b011be
- skel: remove extraneous cell-info dir
- 2872ce17ba
- packaging: add missing chown and chmod on pool-history
- 742a3666fc
- skel: make deprecated alarms properties forbidden
- 24939dc5ac
- [maven-release-plugin] prepare for next development iteration
Release 3.2.16
resilience
Logging for cases where file replication was fatally aborted was improved. Previously, alarms messages pertained to the PNFSID of the affected files. In rare cases, like when facing network congestion, many hundreds of alarms could be created. With this change, alarms messages refer to the storage unit, and a suffix based on an hourly timestamp is added to the alarm message. The alarm will thus be incremented during the hour but a new alarm will be created (only) hourly; in this way, those receiving email alerts will receive them once an hour.
Changelog 3.2.15..3.2.16
- b2195c4e2f
- [maven-release-plugin] prepare release 3.2.16
- 6c98c2b867
- alarms: fix broken path
- 5ec525255d
- [maven-release-plugin] prepare for next development iteration
- 84f6af6218
- substituted Calendar for Instance which was failing.
- 529c10c14d
- dcache-resilience: avoid spamming alarms with abort messages
Release 3.2.15
ftp
A bug (that was not observed in real-world settings yet) that might have caused NullPointerExceptions was fixed in the ftp service.
httpd
dCache will no longer log a stack-trace if HTTP requests are made asking for information from the info service when the info service is not running.
poolmanager
An earlier change in PoolManager introduced regular broadcasting of the stage request queue to various internal listeners. In some cases, this could lead to erroneous NoRouteToHost error messages being logged. These error messages are now being suppressed.
dCache 3.0 introduced a regression where a dCache domain does not start up if it hosts a poolmanager with poolmanager.conf containing either the “rc set sameHostCopy” or the “rc set sameHostRetry” command. This regression was fixed.
resilience
Error handling within the Resilience service was improved.
Changelog 3.2.14..3.2.15
- 6e95e936af
- [maven-release-plugin] prepare release 3.2.15
- 525705459f
- dcache-resilience: handle properly RuntimeExceptions from tasks
- be6abaf5c7
- ftp: ensure adapter is closed
- 465d2c04b1
- ftp: remove rare NullPointerException when proxying data
- 40b57484c1
- httpd: do not log an exception if info cell not running
- 8cea3b4568
- poolmanager: silence NoRouteToCell for stage queue topic
- 71a1728ed2
- poolmanager: fix poolmanager startup with certain poolmanager.conf content
- 741864a1f6
- [maven-release-plugin] prepare for next development iteration
Release 3.2.14
dcache-resilience
When a checksum or broken file message/error is generated, Resilience makes a best effort to (a) remove the broken copy and (b) make another replica. This, of course, is not always possible, particularly if the broken file is the only accessible copy. This resulted in faulty behavior particularly the thrashing noted in the case of a restaging operation which results in a checksum error. This is now fixed.
The current release improved error handling for resilience. It fixed unnecessary Migration Task exceptions resulting from source pools with no replica in the repository.
Now it should be possible for Resilience to use pools blocked only for writes from doors.
packaging
Upgrading to dCache v3.2 (or newer) was resulting in a broken dCache
installation due to a missing services.sh
file. This is now fixed and
upgrade to dCache v3.2 (or newer) from dCache v3.1 (or older) no longer
breaks dCache by removing /usr/share/dcache/lib/services.sh.
Changelog 3.2.13..3.2.14
- a7c71df
- [maven-release-plugin] prepare release 3.2.14
- d455f9d
- bad commit put DOWN twice
- d848910
- dcache-resilience: define non-writable pool to mean p2p-client is disabled
- 8ce3c06
- dcache: fix remote pool monitor wait bug
- dac5e17
- dcache-resilience: repair handling of broken files*
- f2cb660
- packaging: check ‘services.sh’ after old rpm removed
- 0876aa8
- [maven-release-plugin] prepare for next development iteration
- e7d5e16
- dcache-resilience: fix bug in source handling with Clear Cache Location messages
Release 3.2.13
cells
The current release added explicit ZooKeeper/Curator monitoring. Events generated by ZooKeeper and Curator are now logged in a new, which may help diagnose problems that are suspected to come from bad ZooKeeper interaction.
frontend
The current release improved the error handling to work with Jackson exceptions.
Changelog 3.2.12..3.2.13
- 353bd42
- [maven-release-plugin] prepare release 3.2.13
- 80d60ae
- dcache-resilience: fix wrong assumption about error type in Message
- 673c067
- cells: add explicit ZooKeeper/Curator monitoring
- 1fa42cd
- frontend: Map requests with bad JSON to HTTP 400 Bad Request status code
- 8b24220
- [maven-release-plugin] prepare for next development iteration
Release 3.2.12
nfs
NFS door has been updated to return NFS4ERR_LAYOUTUNAVAILABLE for DOT files.
star
The current release improved documentation to help dCache admins to have a better understanding of how to generate StAR record.
The current release fixed fix printing exception error message for dcache-star script if there’s a problem when run with newer versions of Python.
Changelog 3.2.11..3.2.12
- 5ea66f8
- [maven-release-plugin] prepare release 3.2.12
- 8c1e09d
- nfs: return LAYOUTUNAVAILABLE for DOT files
- 78c57d8
- star: fix printing exception error message
- 4f62d5a
- star: update documentation to provide better description of script
- 554bcc0
- [maven-release-plugin] prepare for next development iteration
Release 3.2.11
info
The info service collects information about who is allowed to reserve space.
Since some of this information, like VOs, usernames and gids, may be considered
sensitive information, this update allows admins to control whether or not
to publish them. The default behaviour is unchanged from the previous behaviour,
i.e. info publishes everything. If a site admin wants to change this,
the info.limits.show-only-vo-authz
property can be set to true
.
nfs
Accessing a nonexisting file on recent NFS implementations could cause a FileNotFoundChimeraFsException, which is now caught and properly handled.
pool
In rare circumstances, running info
on a pool could cause a
NullPointerException. This issue has been fixed.
scripts
The dcache script and manpage still refered explicitely to Java 6. This patch changes the phrasing of the respective text.
Changelog 3.2.10..3.2.11
- 4791b79689
- [maven-release-plugin] prepare release 3.2.11
- 0dae9f19f4
- nfs: fix ServerFault on FileNotFoundHimeraFsException
- 952fd1d2c0
- [maven-release-plugin] prepare for next development iteration
- b68757dd48
- scripts: update reference to JDK to avoid mentioning specific java version
- f1c860a74b
- info: allow admin to control whether non-VO / non-FQAN identities are shown
- 64ac107458
- pools: fix NPE from info command at startup
Release 3.2.10
chimera
A database deadlock was observed in some rare situations with the latest 3.2 releases. This patch resolves the issue, ensuring trouble-free chimera operation.
Changelog 3.2.9..3.2.10
- f50d7d22c6
- [maven-release-plugin] prepare release 3.2.10
- 70c1ba902f
- chimera: fix deadlock in Postgres driver
- 89bb1f959d
- [maven-release-plugin] prepare for next development iteration
Release 3.2.9
chimera
The current release fixed previously introduced issues for lost+found
directory permissions.
Now, the lost+found
directory permissions is updated without causing
problems if that directory has been removed or permissions have been
modified.
Changelog 3.2.8..3.2.9
- 44a6ec8
- [maven-release-plugin] prepare release 3.2.9
- 91c30e6
- chimera: correct previous attempt to fix ‘lost+found’ directory permission
- 0c61067
- [maven-release-plugin] prepare for next development iteration
Release 3.2.8
nfs
The current release fixes transfer leak, if the door failed to start a mover.
pnfsmanager
The current release improves documentation for set log slow threshold
admin command help.
spacemanager
dCache now allows an SRM client to specifying from which linkgroup a reservation should be made.
When trying to upload into dCache using a space-token where there is no selectable link for this operation then the user was presented with a generic error message; for example,
No write links configured for [net=131.169.71.98,protocol=GFtp/2,store=dot:user@osm,cache=,linkgroup=]
.
This behavior is changed now and an improved error message is returned to the user if they attempt an upload data into dCache using a space-reservation in a way where
poolmanager
configuration prevents the upload.
srmclient
The srm-reserve-space
command now supports a user choosing from which
linkgroup a reservation should be made, provided the corresponding
dCache also supports this.
webdav
The current release improved error handling when dCache is full.
Changelog 3.2.7..3.2.8
- 558f21e
- [maven-release-plugin] prepare release 3.2.8
- 1781cee
- systemtest: work with new OpenSSL DN format
- 815a561
- spacemanager: allow SRM clients to specify linkgroup in reserve requests
- 086d559
- srmclient: add support for specifying linkgroup when reserving space
- 636fdce
- spacemanager: provide space-specific error message on bad upload
- f758653
- webdav: return 507 Insufficient Storage when dCache is full
- 606d7ed
- pnfsmanager: update slow logging admin command help
- bc15f25
- nfs: fix transfer leak, if the door failed to start a mover
- 2490982
- [maven-release-plugin] prepare for next development iteration
Release 3.2.7
cells
dCache no longer logs stack-traces when running multiple cells with the same name.
frontend, webdav, httpd
With high availability, it is now possible to run redundant
services. In the case of Pool Manager
, restore requests
are distributed to the separate instances, so as to avoid staging the same file twice.
This means, however, that the full list of current restore requests
is partitioned among the pool manager instances. To receive a full
listing, it is no longer possible to query for them on the named
PoolManager
queue, since this means the response will be from
the first responder only. The current issue fixed this issue and all current http services report all restore requests.
The current release fixed this issue.
It is important to note that, upgrading nodes running frontend
, webdav
or httpd
to version 3.2.7 (or newer) requires upgrading
nodes running poolmanager
at least to 3.2.7 (or newer) version.
pool
For certain failures,the pool was logging transfer failures twice. This is now fixed.
rpm
dCache ensures now that user ‘dcache’ is a member of group ‘dcache’.
srmclient
The same error has been logged multiple times resulting in stack-trace. This current release fixed both issues.
srmmanager
Support tickets indicated that for some services it was unclear how to fix a configuration that still
has assignments for either srmmanager.net.port
or srmmanager.net.local-hosts
.
The current release updated the documentation describing how to fix their configuration after upgrade.
star
The current release Introduced new property star.db.*
, which makes possible now to run PostgreSQL on non-standard ports can use STAR.
Changelog 3.2.6..3.2.7
- ff339ca
- [maven-release-plugin] prepare release 3.2.7
- 93cbaf0
- httpd, dcache-frontend: support requests for restore listing when there are multiple pool managers
- 58b0338
- [maven-release-plugin] prepare for next development iteration
- 8fd4ba4
- pool: fix double logging on remote FTP transfer error
- a1c9b6e
- srmclient: avoid stack-trace and repeated logging
- 7deae9d
- srmmanager: provide better hints on obsolete properties
- 29394bd
- pool: Fix how certain bugs are logged
- 3d8e83d
- star: support PostgreSQL running on non-standard TCP ports
- 64b97ef
- rpm: don’t assume existing dcache user is member of dcache group
- bbf75cb
- cells: don’t log stack-trace on starting cell with same name as running cell
Release 3.2.6
admin
The admin interface reported an attempt to connect to an absent cell as a bug. The current release fixed the issue.
httpd
Requests to httpd targeting an unknown resource was returning 200 OK
response code. Nevertheless the 404 NOT FOUND
response would be closer fit. This is now fixed.
maven
The global
dcache.service
was missing from the built packages.
The current release fixed this problem and dcache.service
is now included in Debian packages.
nfs
The current release corrects inaccurate documentation of nfs.enable.pnfsmanager-query-on-move
.
pool
Closing dcap mover connection no longer logs a stack trace.
statistics
Timeout in contacting PoolManager
no longer results in a stack-trace being logged.
Changelog 3.2.5..3.2.6
- 6956c9b
- [maven-release-plugin] prepare release 3.2.6
- b695c88
- statistics: avoid stack-trace on internal timeout
- b1201b6
- nfs: fix documentation of nfs.enable.pnfsmanager-query-on-move
- d327b87
- admin: do not report attempts to connect to missing cell as a bug
- 556e2d4
- maven: include dcache.service in Debian packages
- d830799
- pool: fix stack-trace when closing dcap mover connection
- 44b5541
- httpd: return 404 status code on an unknown page
- 995b5bf
- [maven-release-plugin] prepare for next development iteration
Release 3.2.5
Changes affecting multiple services
This release addresses several issues with systemd support and packaging on Debian systems.
The rsyslog configuration has been updated from using language version 7 to version 8.
There is a new systemd service unit dcache.service
that can be used to have all dCache domains started with only one startup call.
An installation bug with the Debian package was fixed that prevented a successful installation because of a missing cell-info
directory.
Under systemd, log files are again back in their usual location under /var/log/dcache/$DOMAIN.log
.
Changelog 3.2.4..3.2.5
- bc7d6088db
- [maven-release-plugin] prepare release 3.2.5
- 9ce9bc9ad4
- switch to rsyslog v8 configuration language
- 83a779d06f
- move logfiles back to /var/log/dcache
- d19373ae34
- systemd: adding a global dcache.service which pulls in dcache@*.service
- 0dff5cd0e0
- packaging: include empty var directory: ‘cell-info’
- d6696bcbd1
- [maven-release-plugin] prepare for next development iteration
Release 3.2.4
Changes affecting multiple services
dCache no longer logs stack-traces if a Java VirtualMachineError occurs. This is unnecessary as dCache was (presumably) working fine until Java discovered a problem.
chimera
Sites updating to dCache 2.15 or later might observe that a lost+found directory with incorrect permissions was created during the update. This patch ensures correct permissions. Since we cannot know if the current permissions in lost+found are intended, this patch does not modify any existing lost+found directory permissions.
frontend
An irrelevant stack trace could occasionally be logged by the frontend. This patch corrects that issue.
During service interruptions, timeouts have been logged at WARN level until now. The logging level has been changed to INFO with this release.
An irrelevant stack trace was occasionally logged by the frontend service. This release corrects that.
history
Currently, the history service will block dCache startup for history.service.poolmanager.timeout (2 minutes) if the service is started while PoolManager is not running. This blocking was removed, so that system starts are quicker and more reliable.
pool
The sweeper free
command no longer logs a stack trace if it is started with incorrect input information.
An irrelevant stack trace was logged by the pool. This release corrects that.
Changelog 3.2.3..3.2.4
- 4f25e874fe
- [maven-release-plugin] prepare release 3.2.4
- a760f18d72
- chimera: update schema migration when creating ‘lost+found’ directory.
- fc9f218337
- pool: fix stack-trace on bad command input
- e17c58e654
- pool: fix stacktrace on FaultEvent logging
- 3926ead39c
- system: Don’t log stack-trace on fatal JVM error
- a924c2175f
- dcache-frontend: fix ConcurrentModificationException in ReadWriteData
- 37e229cffe
- srmclient: refactor ‘srm’ helper script, enforcing environment variables
- bfda91c5cc
- history: do not block on startup if PoolManager is not running
- 732c5e2b9c
- dcache-frontend: adjust level of timeout logging
- 91b3d01342
- dcache-frontend: adjust REST API for Pool Info Resources
- 50652e65ea
- [maven-release-plugin] prepare for next development iteration
Release 3.2.3
alarms
Until now, the sorting order of alarms did not provide a correct ordering for all types of alarms. With this release, alarms are now implicitly ordered by at least their latest modification timestamp.
frontend
The cause of a stack-trace during system shutdown has been fixed.
nfs
The handling of directories with hard links in them has been improved, providing NFS clients with a way to correctly list them in all cases.
resilience
One of the features of resilience is the enforcement of file partioning on pools according to pool tags. The pool tag restrictions are observed whenever a file is copied. In addition, it is rechecked when a storage unit is updated, in order to make sure the files are distributed correctly according to the new requirements. This is done by removing the offending copies and recopying them in a new location.
Should files get redistributed, however, by rebalancing or a migration job, it is possible that the partitioning will be violated, since only resilience observes it.
The resilience service now verifies that files are distributed according to the requirements specified by pool tags while doing periodic scans (or scans initiated through the admin command).
statistics
A possible race condition was removed from the implementation of the create stat
admin command.
Changelog 3.2.2..3.2.3
- f4a96d52d6
- [maven-release-plugin] prepare release 3.2.3
- a95f3a56d2
- statistics: fix race in “create stat” admin command
- 17d15a2bf9
- srm: fix stacktrace on database failure
- d89fd3086c
- dcache-frontend: fix shutdown not to cause stack trace in collection services
- 245e40df10
- alarms: revert LogEntry.compareTo() to throw NPE on null object
- 5bec23331b
- nfs: change the way how directory cookies are generated
- ed2b1a7138
- resilience: force tag partition checking on scans from admin command and periodic checks
- fcfe6f0437
- alarms: fix natural order comparator to use timestamp first
- c33a097997
- [maven-release-plugin] prepare for next development iteration
Release 3.2.2
alarms
Shutting down dCache using dcache stop
is now faster.
dcap
A pool that has gone offline and comes back up again may become very slow to respond due to a large amount of superfluous error messages to dcap clients that disconnected in the meantime. This patch ensures a more responsive reaction to these cases by introducing a time-to-live value for such messages.
frontend
In cases where very many alarms need to be processed by the frontend, they are now fetched in batches, ensuring better responsiveness of the system.
The way in which services send large amounts of data to the frontend, as for example the active transfers or staging requests lists, has been made far more efficient and performant.
pool
Error reporting was improved for cases of IO errors in pools.
Changelog 3.2.1..3.2.2
- 983e3644bb
- [maven-release-plugin] prepare release 3.2.2
- 132d8fa428
- alarms: fix shutdown timeout
- c1d1324c07
- pool: avoid ‘null’ and other nondescript error messages
- 520bcdf633
- dcache-restful-api: change the way in which transfers and restores support paging
- a775365ef1
- [maven-release-plugin] prepare for next development iteration
- 3236047d4d
- dcap: add TTL information to dcap messages
- 0e9dd30126
- alarms: remove default value for LogEntry received
- b4676a9295
- dcache-restful-api: add offset and limit to fetch of alarms
Release 3.2.1
Changes affecting multiple services
Many dCache components use RemotePoolMonitor to provide fast access to the information that PoolManager has about pools. In order to facilitate system diagnosis, the ‘info’ admin command was augmented by information about the current status of the RemotePoolMonitor.
frontend
Nearline movers now display the elapsed time they ran for in the dCache frontend.
The RESTful API has received support for disabling and enabling of pools as well as killing movers.
Minor bugs in the history service were fixed.
The timeouts for the collectors run by the frontend service were reduced to 1 minute (2 for the history service, which needs to ping all pools in an instance) in order to provide more immediate change information in the frontend.
The RESTful API now offers access to the data about PoolSelectionUnit that was previously available on the webadmin pages.
history
In making the pool info service stateless, collection of timeseries information for queue status and file lifetime on the pools was moved into a separate service (history).
When pools are unavailable to the history service, as is possible during restarts, it is possible that history data is lost. This patch corrects that issue, so that history is preserved under all circumstances.
pool
A bug in gfal2 results in FTP transfers being aborted some 50 ms after being initiated. This results in the door killing the mover shortly after the pool received the PoolDeliverFile message. If the mover is not queued, but not yet fully started, this may lead to the pool disabling itself. This patch corrects that problem, ensuring that the pool continues to run despite any aborted transfers.
system-test
The system-test admin interface configuration was updated to allow admin logins after ssh user restrictions are now significant.
xrootd
In the 4.7 releases, the xrootd client started enforcing protocol requirements for kXR_login which, unfortunately, broke access to dCache. The xrootd client expects an answer with a 16-character session ID from the door and then the pool after the redirection. Without this ID, the client would retry (without success) repeatedly and appear to hang.
dCache’s xrootd implementation has been augmented with the session ID, enabling it to work with xrootd clients of version 4.7 and up.
Changelog 3.2.0..3.2.1
- bd6004d84f
- [maven-release-plugin] prepare release 3.2.1
- caeda8c243
- many: add diagnostic information about remote pool monitor
- 702caa2af1
- pool: dont disable pool if mover cancelled before open
- 4b58778caa
- systemtest: authorise developer to login as ‘admin’ via ssh
- 7a81aa71f0
- dcache-history: fix bug which forces overwrite of data when pool unavailable
- f34e87ccd9
- dcache-restful-api: fix aggregation issues for pool info and move aggregation to history service
- 8a012e6576
- dcache: add elapsed time to NearlineData JSON
- 78493a3d86
- dcache-restful-api: fix configuration bug in billing collection utils
- 996a1e2310
- [maven-release-plugin] prepare for next development iteration
- 871182acec
- dcache: fix NPE bug in TransferInfo.toFormattedString
- 614a805535
- dcache-restful-api: avoid NPE in PoolDataRequestProcessor
- 41767ab043
- dcache-restful-api: lower collector timeout defaults
- 32336bacbb
- dcache: fix bug in PoolSelectionUnitV2 match()
- 4e8edebf38
- dcache-xrootd: Fix login handshake to support xrootd clients (> 4.7.0)
- 52eaf075c5
- dcache-restful-api: Add POST for enable/disable or kill mover to pool info service
- 34720e0c43
- dcache-restful-api: add selection resource and providers
- 31930df123
- dcache-xrootd: (WIP) Add support to the xrootd (kxr)posc flag in (kXR)open.
Release 3.2.0
Authentication
We’ve added support for macaroons in the webdav door.
Macaroons are a new idea that comes from research by Google. They are bearer tokens that allow the bearer to do something, without requiring the users to identify who they are.
A macaroon may be limited by caveats which only allow the bearer to do certain things. These limitations could be nearly anything. A caveat could make the macaroon time-limited (e.g., only good for five minutes), location-limited (e.g., only from a particular IP address), or anything else. You can learn more about macaroons from an Air Mozilla talk called Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud.
Macaroons are a building-block technology, so have many potential uses. They may be used by a web-portal to allow its clients (of which dCache knows nothing) direct access to files stored in dCache. They may be used to authorise third-party transfers without requiring X.509 credential delegation. They may be used to create a link that allows sharing of resources. There are likely many other uses for them.
Requesting a macaroon is as simple as making a POST request with a
Content-Type of application/macaroon-request
. This returns a
time-limited (but otherwise unlimited) macaroon. Additional caveats
may be added by dCache (included as JSON in the POST request) or
subsequently by anyone else. dCache supports a rich set of possible
restrictions, including namespace (e.g., only a particular file or
directory), time (e.g., only the next 5 minutes), location (e.g., only
from a particular IP address or subnet), and activity (e.g., only
downloads).
Creating a macaroon takes very little CPU and no storage in dCache. Therefore it is safe to request many macaroons; for example, a web portal could create a seperate macaroon for every file its user wishes to view.
A macaroon is a bearer token. To use a macaroon in an HTTP request,
just add it to the Authorization header value with a prefix bearer
,
just like with an OpenID-Connect access-token.
Unfortunately, not all HTTP clients allow a user to add a bearer token
to the Authorization HTTP header; for example, a user cannot easily
include a bearer token in their web-browser. To support these
clients, dCache accepts URLs that include the ‘authz’ query
parameter, with the value used as a bearer token (e.g.,
https://dcache.example.org/path/to/file?authz=<token>
). This works
for both OpenID-Connect access-token or macaroon.
The webdav directory browsing has been updated so that any query
parameters in the original request are included in all the generated
page’s navigation and download links. This makes it easy to navigate
through the webdav interface using a macaroon or OpenID-Connect
access-token: just add ?authz=<token>
to the URL.
Roles
It may be desirable for dCache to behave differently for the same user between different login sessions. For example, an administrative user may wish sometimes to interact with dCache as a normal user (for testing purposes or to reduce the risk of mistakes), and other times to interact with dCache with effectively root-like privileges.
This functionality has been enabled by introducing the concept of user roles in dCache. A user may log in requesting zero or more roles and the login process chooses which roles (if any) to enact.
For the HTTP-based services (webdav and RESTful), roles can be requested by appending to the username a ‘#’ followed by a comma-separated list; for example, user ‘fred’ wishing to adopt ‘role-a’ and ‘role-b’ would authenticate with the username ‘fred#role-a,role-b’.
This release introduces limited support for the ‘admin’ role.
The ‘admin’ role is intended to give users root-like permissions without the user knowing the root credentials and with the user’s identity still being bound to that user. The result is that the user with the ‘admin’ role is able to do everything, but files created by that user are still owned by that user and log entries show which user made any changes.
An ‘admin’ role user has a root directory of /
and no restrictions.
The functionality is enabled by adding a new gPlazma session plugin, ‘roles’, to
the gplazma.conf
file, like this:
session required roles
A new property
gplazma.roles.admin-gid = 0
has been added to gplazma.properties
. A user must be a member of the above specified
group to be authorized to obtain the admin role.
External networking
The handling of external connections has changed for some protocols.
The SRM network configuration was simplified, and dCache can now work with multiple SRM services on different ports.
The deprecated configuration properties srmmanager.net.port
and
srmmanager.net.local-hosts
are no longer needed.
admin ssh public keys
In previous versions of dCache, any user who has their ssh public key
in the admin.paths.authorized-keys
file
(/etc/dcache/admin/authorized_keys2
by default) can log into the
admin interface as any user.
With this version of dCache, a public key is limited to authenticating as a single user. The standard ssh public key format does not have any way to describe for which user the line allows authentication; therefore, dCache uses the comment field to provide this information.
The comment field appears after the base64-encoded text of the public
key. dCache requires the comment contains the desired username
followed by an @
symbol; without this, the line is ignored. The
text after the @
does not matter. The username describes for which
user the corresponding private key may be used to authenticate.
The following (truncated) line shows an example authorized_keys2
file that allows the use of a public key to authentication as user
admin
:
ssh-rsa AAAAB...fPQ== admin@localhost
Internal networking
Network communication between core and satellite domains has, until now, been unencrypted. dCache 3.2 adds the possibility to use TLS encryption between domains, which should be extremely helpful for building larger, distributed setups.
A number of new configuration properties control encrypted cell communication.
Set dcache.broker.core.channel.security
to none
in order to
listen only for plain-text communication on dcache.broker.plain.port
,
to tls
in order to listen only for TLS encrypted communication
on dcache.broker.tls.port
or to none,tls
in order to listen
for both plain and encrypted communication.
dcache.broker.core.client.channel.security
and dcache.broker.satellite.channel.security
can be used to enable encryption between core and satellite domains.
In order to ease managing encrypted communications, the LocationManager
cell, lm
, now has three new commands available: ls
, get core-config
and set core-config
, which can be used to query and set the
operating mode for domains.
Regarding backwards compatibility, the introduction of TLS encryption
deprecates the dcache.broker.port
property. Satellite domains which
do not receive a configuration update can still connect to updated
core domains which listen to plain communication, and core domains
which are not updated can still connect to updated core domains.
Admin improvements
The ps
command of the System cell has been overhauled. In particular,
the output of the -f
option has been cleaned up and extended.
The admin cell now generates access log events containing connection and disconnection, as well as authentication, events.
Key-based authentication has been improved by enforcing that a
particular key can only log in with the user name specified in the
authorized_keys2
file. Upon upgrade, admins should review this file
and ensure that keys are mapped to the correct user name.
Namespace
Tags are now reference-counted like any other filesystem object. This avoids an expensive database query upon file deletion that in some cases has led to bottlenecks, in particular when PostgreSQL statistics were inaccurate. Upon upgrade, the reference count for existing tags has to be populated, which may take a little while on large databases.
Checksum handling
Two improvements have been made. The first has to do with what happens when a client provides a checksum. In this case, dCache now verifies and stores the provided checksum as well as computes and stores the checksum of the type configured by the admin in the pool setup.
The second has to do with interoperability between clients requesting different checksum types (this mainly derives from the Globus MD5 requirement). The FTP door has been modified to compute on the fly a missing checksum required by the client. This way, files written by a protocol requiring ADLER32 can be read by clients requiring MD5, and vice-versa. This fix requires updating only the FTP door.
Finally, an admin command
get file checksums <pnfsid>
has been added to PnfsManager to display multiple checksums.
RESTful api
With this release, information that has traditionally been made available via the legacy httpd and webadmin pages will also be accessible via REST apis.
A full description of the paths and parameters for each RESTful service, along with example JSON output, will be published separately on the dCache GitHub Wiki.
Included in this release are the apis and supporting services for alarms, billing, cell info, pool and pool group info, active transfers and tape restores/stages.
An illustrative example:
curl -k -u arossi#admin:xxxxx 'https://fndcatemp1.fnal.gov:3880/api/v1/pools/dmsdca22-7?info=true'
The request is for the basic information concerning a pool; the JSON object returned includes cell information, pool configuration/setup information, pool cost and request statistics, space statistics, sweeper statistics, etc. There are additional parameters for requesting histogram data on pool requests, space usage, and file lifetime on the pool, as well as listings of movers, stores and restores.
Notice the user name in the curl command. Use of these services requires admin privileges. Here, it is presumed that the user has been accorded the admin role; the ‘#’ after the login name indicates a request to express that role for this session. See the section on Roles above for a fuller explanation.
In addition to the new admin services, a small bug has been fixed in the handling of QoS updates on the namespace service, so that the request correctly considers the current locality of the file.
gPlazma
The X.509 plugin has been updated to extract email addresses from the Subject Alternative Name. This extracted information is now part of the principals identifying the user, which is available to all dCache components like webdav, frontend etc.
Frontend and WebDAV
dCache-view, which is part of Frontend services, has the following new features and fixes:
- display of user profile information
- added support for user roles
- drag and drop for moving files and directories
- user login with open-ID connect
- upload of files and directories using drag and drop
- a customised context menu replacing the hover-context
- users stay in their current path after successful authentication.
Since the gPlazma plugin has been updated to extract the user’s email, Frontend clients can now discover a user’s email addresses, if any are known.
Unnecessary Frontend backward-compatible configuration data, consumed mainly by dCache-view, were dropped.
For WebDAV and Frontend doors non-/ root has been fixed, ensuring that users with
non-/ root
directory will see the same files and directories under WebDAV/Frontend
as with other protocols. Also, Frontend ‘mkdir’ and ‘mv’ operations are updated
to honour door and user roots.
Obsolete services
dCache 2.16 introduced the next generation replication service called the resilience manager. At that time, we announced that the replica manager would eventually be removed. We try to keep what we promise, so now the old service is gone. If you haven’t migrated yet, you should do so before upgrading to dCache 3.2.
systemd support on Debian-based systems
Traditionally, we have shipped our own scripts to daemonize dCache. This made it easier to support many different distributions as well as the multi-process architecture of dCache (in other words, it was more fun to implement our own than to study how each distribution did it). For better or worse, the mainstream Linux distributions have all moved to systemd, so it now becomes hard to justify why dCache shouldn’t make use of the functionality offered.
As a first step towards systemd, this release recognizes systemd on Debian based systems. If detected during installation, part of the dCache runtime management scripts are replaced by callouts to systemd. A systemd generator scans the dCache configuration and creates a systemd unit for every dCache domain. systemd fully and directly manages each dCache process. This means:
No custom wrapper scripts; there is now only one process per dCache domain.
systemd monitors the Java process and restarts it if it quits directly. There is no separate ‘restart file’ to suppress the dCache auto-restart mechanism.
No PID files, as systemd tracks the dCache processes directly.
systemd captures the stdout/stderr output of the process and directs it to journald. The default journald setup passes the log on to syslog.
systemd drops privileges of the Java process during startup. systemd mounts
/etc
,/usr
,/boot
, and/home
read-only for the dCache process, preventing dCache from writing to any of these directories. Note: This is important to remember if you happen to use/home
for pool data or tape integration.
Starting and stopping dCache domains
Whenever the list of dCache domains changes, the dCache units need to
be regenerated. dCache will do this automatically whenever its
dcache
script is invoked as root, but one may also do this manually
by running systemctl daemon-reload
. This also fixes a long standing
issue with dCache loosing track of running domains that are removed
from its configuration.
The dcache
script has been updated to call out to systemctl
to
start and stop domains. One can continue to use these commands to
manually start and stop domains. Since the systemd support replaces
the classic SysV init script, auto-startup during boot has changed:
the generated systemd units are not flagged as enabled
automatically. Although one can start and stop these domains, they do
not start automatically. Use the systemd enable
command on every
domain that should start automatically. In contrast to before, one can
select exactly which domains should start automatically.
Logging
Since dCache by default logs to stdout, and since systemd redirects
stdout to journald and thus to syslog, dCache logs now end up in
syslog. An rsyslog configuration is included to seperate the dCache
log output from other syslog messages. Each domain logs to
/var/log/dcache@DOMAIN.log
where DOMAIN is the dCache domain. The
file cannot be placed in /var/log/dcache
due to permission
requirements enforced by rsyslog. The default logrotate setup is
adjusted to rotate the new files. It is no longer necessary to use the
copytruncate
option, which makes logrotation more efficient, uses
less disk space, and avoids the risk of losing log entries.
Since syslog includes timestamps automatically, the default log format is modified upon upgrade to not include the timestamp in the dCache output. If the log format has been customized, it must be adjusted. Since the log format changes anyway, we also adjusted how the NDC is logged.
If you really liked the old log format and placement, you may
reconfigure the logging in /etc/dcache/logback.xml
to log directly
to /var/log/dcache
rather than stdout. If you do, you should ideally
use the logback logrotation rather than rely on logrotated.
RedHat
Nobody has bothered to upgrade the RedHat packaging with systemd support yet. Volunteers are welcome.
FTP
The FTP door was updated to allow pipelining of commands. Since the FTP protocol expects some commands to have immediate effect – i.e., before the previous commands have finished – this is not as trivial as it sounds. We believe we have nailed it, both fixing bugs in the existing implementation and avoiding the bugs other services have in their attempt to support pipelining.
The immediate effect is better compatibility with Globus Online.
Space usage
Continuing the WLCG quest to reimplement the features of SRM in other protocols, dCache now exposes space reservation stats through both FTP and WebDAV.
For FTP, the SITE USAGE
command is implemented, supplying information
from SpaceManager using reservations with a description that matches
the TOKEN argument (if a TOKEN was given) or that are bound to the
supplied path (if TOKEN was omitted).
For WebDAV, reservations are exposed as RFC 4331 quotas and can be queried as such.
NFS
Updated handling of directory listings. This should avoid situations when the client receives a BAD_COOKIE error caused by server-side cache invalidation. The stage and p2p operations are handled the same way and are more client-friendly. The ‘show transfer’ admin command supports filtering based on client ip, pnfsid and pool name.
Xrootd
The xrootdfs FUSE driver immediately closes a file on creation, then reopens it to write. The behavior of the xrootd door in dCache has been modified to allow new empty files to be overwritten, thus enabling file copies from xrootdfs FUSE into a dCache mounted filesystem.
Pools
Pools can use mongoDB to store metadata. This can be enabled by the
pool.plugins.meta=org.dcache.pool.repository.meta.mongo.MongoDbMetadataRepository
configuration option. Additional properties control mongoDB server location, database name and collecton name:
pool.plugins.meta.mongo.url=mongodb://localhost:27017
pool.plugins.meta.mongo.db=pdm
pool.plugins.meta.mongo.collection=poolMetadata
A single shared mongoDB instance can be used for all pools.
NOTICE: in production, mongodb must run in a cluster in order to provide high performance and availability.
srmmanager
Most SRM operations are only allowed on local SURLs. Only third-party copying allows non-local SURLs; however, there at least one of source/destination SURL-pairs to be local.
In previous versions of dCache, the properties srmmanager.net.port
and srmmanager.net.local-hosts
allow the srmmanager to decide which
SURLs are local. In many cases, this information is redundant, as srm
services already publish this information within dCache. Therefore,
with this version of dCache, the srmmanager will consider SURLs local
if there is an srm door that advertises it listens on that host and
port.
Sites may have a DNS alias or have some proxy service to which SRM
clients connect. Under these circumstances, the client will not
connect to the FQDN of the machine hosting the srm service, but some
other address (that of the DNS alias or the proxy service). To
support this, the srm.loginbroker.address
and srm.loginbroker.port
properties must be configured correctly so that at least one srm
service advertises the hostname clients use when connecting to dCache.
Changelog from 3.1.1 to 3.2
- 636b169abb
- webdav: fix regression in OPTIONS response
- 60ddde1934
- dcache (collection service): handle execution exceptions correctly
- ad7fdddfc0
- libs: update nfs4j to version 0.15.2
- 129398d24e
- Update FileOperationHandler.java
- 8951fd2b77
- pool: fix data integrity regression for 3rd-party GridFTP pull transfers
- c205175281
- pool: fix regression in GridFTP OPTS CKSM command
- d7c5d2d63b
- common: fix time computation in TimeseriesHistogramTest
- c6bba6d32d
- resilience: handle file deletion during scan correctly
- 7826205a32
- pool: ceph: ignore file-not-found on remove
- 16f408d5a4
- dcache-restful-api: return incomplete info instead of throwing NoSuchElementException
- 17721ed564
- resilience: add pool operation logging
- 2351b6779d
- resilience: handle storage unit NoSuchElement failure
- fd502cd927
- srmclient: parameterise shell path of srmclient utilities
- 3a4c2ead2d
- nfs: shutdown callback ScheduledExecutorService on shutdown
- 7ee44eea7e
- libs: update to nfs4j–0.15.1
- 53068de346
- webdav: adjust header parsing to be case insensitive
- 1d8239fade
- cells,dcap,ftp: Support for accepting connections from an allowed list of subnets and IP addresses
- dd671b22b9
- resilience: handle all cases where no locations for file may be discovered
- 04e32327d7
- resilience: distinguish correctly between file not in repository and file not found
- ffb85de53c
- resilience: fix bug in formatting and handling of cache exception types
- 26d939d611
- dcache-restful-api: extract the locations from storage info and add to JSON attributes
- c824efb49d
- common: fix bug in histogram max index computation
- 12b9a8ada0
- srmclient: remove non-functioning script with BASH dependencies
- c972cdcc12
- [maven-release-plugin] prepare branch 3.2
- 366fb071da
- dcache-restful-api: add missing aggregated cost data to JSON
- d59342ab7e
- srm-server: refactoring slf4j logging messages
- 39f10cda45
- srm-common: refactoring slf4j logging messages
- 345dbfbbd5
- nfsv41door: add filter method for show transfers command
- db17e99d25
- cells: refactoring slf4j logging messages
- 46b799062e
- dcache-chimera: refactoring slf4j logging messages
- a7497f3b57
- ssh: add username check to pubkey authentication
- 266caa6228
- ftp: update exception logging to include context
- 728e32a9d8
- gplazma2-grid: refactoring slf4j logging messages
- 2626ed9619
- gplazma2: refactoring slf4j logging messages
- aaa7f3cc40
- dcache-xrootd: refactoring slf4j logging messages
- a705d04296
- dcache-webdav: refactoring slf4j logging messages
- 7bca95e137
- dcache-webadmin: refactoring slf4j logging messages
- 6fe5ff78b5
- corrected requested typo
- 4972298c18
- dcache-spacemanager: refactoring slf4j logging messages
- 21a077d341
- dcache-info: refactoring slf4j logging messages
- a337212e36
- dcache-ftp: refactoring slf4j logging messages
- c68428c074
- dcache-dcap: refactoring slf4j logging messages
- e32bf805f4
- dcache: update Subnet utility class to have a isValid method
- 7f5fe17d6c
- srmmanager: reduce network configuration
- e0867e15f0
- pool: fix minor errors in PoolInfoRequestHandler
- 7fb9b61d44
- fix RPM package building and dangling reference in text
- b2cbf3016b
- core: Add Remote-Host-Restriction capability to Ssh2Admin PublicKeyAu… (#3431)
- a768abfae7
- dcache-core: refactoring slf4j logging messages
- 17154424c2
- dcache: added a decorator for RepositoryChannels to get IO statistics (#3430)
- 9eb63b13b6
- acl: refactoring slf4j logging messages
- 2761aca19d
- skel: remove legacy Berkeley DB jar and corresponding preupgrade-script
- 536dbdf652
- inserted again requested changes
- 154709991e
- Motivation: Prior to Java 1.5 enums did not exist, thus integer constants were used. These constants can now be replaced by enums.
- 38c1144452
- ssh: add logging to domain access log file (#3427)
- 5b1f65b0ce
- dcache: introduce history service with pool timeseries component
- 0f2188aab9
- dcache-restful-api: pool info service implementation
- fa3d4eb0f7
- dcache-restful-api: move admin collector service abstractions to dcache module
- 213893df48
- dcache-common: refactoring slf4j logging messages and logger variable name
- 311f09acfa
- chimera: fix broken commit e064f5577b
- e064f5577b
- libs: switch to nfs4j–0.15
- e88c1e31c0
- acl: refactoring slf4j logging messages Motivation: with normal string concatenations in log-messages strings are always build, regardless if log-level is activated or not. with parameterized log-messages the strings only become build, when the log-level is activated;
- 625ff42cfb
- acl-vehicles: refactoring slf4j logging messages
- a10786c84a
- chimerashell: use chimera.db.* options as defaults
- cf9154131b
- cells: better handling of rogue domains with badly formatted dCache versions
- 79161ee475
- configuration: update zookeeper configuration with hints
- a1a2e50d4b
- pool: suppress unecessary ‘jtm set timeout’ in setup
- d5adeb363e
- pool: fix loading ‘setup’ that requires queues created by ‘pool.queues’
- 38d4249f98
- pool: consolidate error-handling in pool IoQueueManager admin commands
- 13074ca921
- fix minor tyo
- df1bc6d682
- various: miscellaneous minor adjustments from restful commits
- 858fbfa6c1
- webdav: avoid stack-trace on bad user requests
- f106f7a8df
- systemd: start service after ZooKeeper
- 3f30893ac2
- Martin (#3403)
- e7946a7130
- Vuong-Test (#3401)
- 7e29c99ec6
- Signed-off-by: local local@lp-hrz-d209-linux.wh.f4.htw-berlin.de
- 506e612582
- dcache : added reedme.md
- 27d0d5a837
- gPlazma2-voms: Add README for Module Info
- bd8265e0bf
- webdav: add README file to gplazma-nis module
- d5897c1cae
- srm-common: add README file
- feb41dec5c
- Motivation: Test-Readme for gplazma2-roles
- c83e3dd7d4
- cells: readme file for testing
- 6d1833b204
- Signed-off-by: Lotta Rüger l.r.@Lottas-MacBook-Pro.local
- df1ac51260
- common: HTW-Berlin Big Data Test Commit
- 2a8da23880
- alarms: guard against NPEs on LogEntry getters
- af3d12aa4d
- admin: Fix Inconsistent ACL enforcement, RT 9207
- 28e093eec0
- nfs: add a possibility to specify offered layout types
- 68f49618a9
- system-test: add list of allowed client origins
- 770082b527
- systemtest: update systemtest to point to a reasonable WebDAV door
- 6ee63306d6
- pools: add support for requesting live data for histories
- f75794c68c
- common: fixes potential NPEs in histogram metadata
- 56bc72919e
- webdav: fix more regressions with CredentialSource.NONE
- 05026a1511
- pool: fix regression in HTTP third-party transfer with redirection
- 355ff88135
- systemtest: fix populate script for when systemtest already exists
- 13bb5878c5
- webdav: fix error recovery for macaroon users without DELETE
- aabac213aa
- macaroons: add implicit authorisation of READ_METADATA
- fbae70d684
- dcache-restful-api: simplify alarms api and service
- 7e5e81e173
- webdav: fix regression in third-party copy with no delegation
- 52a409c136
- dcache: release dcache-view version 1.3.1
- b70b0d946a
- dcache (pools): Add messaging support for frontend/restful pool info service
- 891060e863
- dcache-restful-api: removing disk-based caching from CellInfoService
- 27841575f4
- configuration: update description for replicable
- 0b21d9d5e4
- dcache-restful-api: fix handling of no route to cell in alarms collector
- c5199298a6
- debian: Adjust how NDC gets logged
- c668b85cbc
- gplazma-oidc: improve code-style for oidc plugin in accordance with the dCache code style guideline
- 96e14cece5
- srm/srmmanager: fix srmPing confusion
- 56fd242379
- dcache-restful-api: add api and implementation for alarms service
- bdd188b536
- dcache-restful-api: fix misnamed restores command
- dfe1b89c38
- dcache-restful-api: fix restore service initialization
- 306c0d06ab
- dcache-restful-api: add service implementation for collecting staging/restore info
- af7136e45d
- ftp: convert timestamps to GMT (to follow RFC 3659)
- 7d5e0fef1e
- Revert “pool: handle initial space allocation for existing files”
- 22365dcfb6
- pool: fix regression in accounting during file upload
- feddfcc542
- pool: use Throwables.getRootCause(e) inspecting RuntimeException
- 691ad40190
- packaging: tighten permissions in var directories
- f23954a64a
- billing: update documentation to describe CellAddress
- c5db7fa1cf
- srm/srmmanager: update documentation about root path
- 4ca756f32a
- srm,srmmanager: add configuration property to allow easy modification of srm root
- ade17bb00e
- pool: handle initial space allocation for existing files
- f7dae69511
- gplazma: extract email from x.509 certificates
- e5e80f491e
- frontend: expose users email address
- 85d4801aae
- logback: make socket appender construction depend on log level
- 4139d89bcb
- frontend: fix NPE when billing is disabled.
- 3e596ed3ba
- resilience: remove reference to pnfsmanager property
- 3b7013bfd6
- frontend: fix regression in configuration properties
- d60f09b0c9
- config: improved description of port numbers
- 6ea03d7155
- config: add obsolete|forbidden annotation for dropped properties
- 79fde8079e
- resilience: remove stray conflict marker
- bce7615108
- resilience: make namespace provider properties immutable
- 3b372e2b51
- dcache: release dcache-view 1.3.0 for dcache current master
- ba946f112e
- frontend: remove backwards compatible config data
- 2a53e1f2e3
- config: add obsolete|forbidden annotation for dropped properties
- 745149fb99
- config: add obsolete|forbidden annotation for dropped properties
- 728db53ef9
- pnfsmanager: remove obsolete comments from properties file
- c9f38a3b40
- clarified documentation of gplazma.authz.upload-directory
- 6b64e86ab0
- improved description of upload-directories
- 7a9dd29913
- added hint that pnfsmanagers must use the same DB
- 670584f7a0
- fixed several typos in the documentation
- 400b4fc8be
- use correct terminology
- b0abb286b6
- cells: fix TLS support to work with embedded zookeeper
- 47afd23844
- systemtest: fix regression introduced with systemd integration
- 5b081219eb
- srmclient: give version of srmclient
- e86a92a893
- packages: Fix build of RPM
- 43253ec27e
- debian: Add rsyslog config for dCache
- 6374ddb965
- debian: Systemd integration
- 7b7369a60a
- frontend: avoid sending messages before cell is registered
- c977392bb0
- cells: add tls based encrypted channels for cell communication
- 4d27373d98
- zookeeper: work-around race-condition in zookeeper server shutdown
- 387fbbcef4
- common,system-test: fix minor annoyances with roles in system-test
- dd5fa1ccc3
- webadmin: use new role-based login and support active/deactive roles
- e29757959f
- authentication: add support for macaroons
- a18ba8f063
- libs: update jetty to latest 9.4.6.v20170531 release
- 3392b5dc0a
- nfs: merge p2p and stage handling
- 4d1e5e66ae
- dcache-restful-api: restore creationTime on JSONAttributes
- 732e938ca4
- packages: missing rpm server spec line for cell-info dir
- 815cf82abe
- ssh: fix handling of ssh idle timeout
- 96188c7a14
- dcache-restful-api: add cell service API and implementation
- 11392b9ae8
- dcache-restful-api: add RESTful billing service and support
- 50f32f5768
- dcache (alarms): add support for RESTful frontend messages to alarms service
- ced8d9fe75
- authz: return a list of allowed but unasserted roles
- 3ada58ab04
- common: add javadoc to AccessLatency and RetentionPolicy
- 4ccd181edc
- common: make AccessLatency and RetentionPolicy more enum like
- 4f8b39f7e4
- system-test: add series of functional test for frontend service
- 8d9c3bf7e3
- authz: add initial support for Roles
- 3cbc1f11da
- cells: improve System’s ps output
- f31bb95a16
- chimera: remove redundant code
- 1fc6082dc6
- security: drop dead code
- 789dbcaabc
- webdav: add support for RFC 4331 for reporting space usage
- 32db31a0bd
- chimera: keep track of tags usage
- 7fc9aaf7d0
- common: repair some minor issues with HistogramModel
- 5e286d5382
- common: use null/Optional instead of Double.MAX_VALUE and Double.MIN_VALUE for histogram stats
- 4761c1c73a
- frontend: refactor static (configuration) data
- f264c0fb96
- spacemanager: dont try to release expired spaces
- c94fc2e3c6
- srmmanager: use path to support srmSetPermission operations
- 4694030f9d
- common: use round instead of floor to bin counting histogram values
- 893846e231
- dcache-restful-api: fix pnsfHandler reference in IdResource
- 9b9879837e
- Revert “admin: Fix Inconsistent ACL enforcement, RT 9207”
- 4032dfbf1a
- frontend: fix Restriction usage
- 5b96a948ce
- pool: do not throw InterruptedException on Repository#openEntry
- 1668b1a66d
- dcache-restful-api: add API to get file attributes from pnfsid
- 8f8c79b3f9
- dcache-restful-api: add pnfsid and nlink to all attribute requests
- ad720c2725
- admin: Fix Inconsistent ACL enforcement, RT 9207
- c42a9dd8ef
- Fix typo in broken commit
- 30c3ec4e28
- poolmanager: fix unit-test to avoid race-condition
- 6f8da800c6
- zookeeper: work-around SessionTracker racy initialisation on startup
- ce61e28cf8
- srmmanager: fix NPE when querying spaces
- 80c082c258
- httpd: Fixed table headers in usageInfo
- 45a40c3e20
- ftp: add support for SITE USAGE command
- b864b313ff
- pool: use StndardOpenOption.CREATE instead of OpenFlags.CREATEFILE
- 273c83e048
- pool: use Set<OpenOption> instead of Set<Repository.OpenFlags>
- dca973f5b1
- pool: update Repository.OpenFlags to implement OpenOption
- b40a81e622
- pool: use OpenOption instead of StandardOpenOption
- 554b91b4b8
- zookeeper: work-around racy startup
- 4003ec7b35
- dache-restful-api: add log.debug to trace Qos transitions
- a7b3d077ad
- zookeeper: silence ZK server errors
- 7e210d8d98
- dependencies: remove log4j jar
- 0e07ff7a77
- replica: remove final references to replica manager
- 268af787ea
- pool: refactor pool to use StandardOpenOption instead of IoMode
- 28c15abb5a
- ssh: do not cast timeout to int
- c727a0628f
- frontend: fix “Attribute is not defined: SIZE” bug
- 315c25d312
- pool: fix error message on timeout
- 130e160f5c
- pool: add support for mongodb as a backend for medatada
- a912e8393b
- info: avoid sending messages too early.
- 4ac977ab43
- frontend,webdav: add supress-wwwauthenticate to allow headers
- f723e96b0c
- dcache: remove old Replica Manager
- 840ba3fb35
- authentication: suppress WWW-Authenticate if requested
- 6c37ffbc51
- pool: log why a transfer was forcefully aborted
- 5a369e8455
- webdav: improve logging on transfer failures
- cddc697a38
- webdav: make Milton work-around more robust
- 773bc0df99
- script-nearline-spi: fix space leak when polling script is used
- 0bc04f6b71
- http: fix non-/ root for WebDAV and frontend doors
- 554fd2a969
- authentication: support embedding a bearer token in HTTP URLs
- aa9cdbca04
- webdav: Fix restriction check when downloading a file
- 13b2d7716c
- common: factor out PathMapper as common feature
- d7db259922
- httpd: Fix incomplete restore list in HA setup
- f2392e4cdd
- httpd, admin: Fix some hard-coded cell names
- cec2f1fd4b
- frontend: expose open-id connect to dcache-view
- 83ba36f475
- rest-api: include username to the user attributes
- a35a988042
- systemtest: update Globus clients to use generated trust store
- 21c0b25341
- systemtest: add transfer tests for UberFTP
- 437f24c85c
- frontend: fix problem introduce by jetty update
- db29ebc60c
- Add cascadeConstraints=“true” to liquibase dropTable action on pinsv3 table
- 3961e557c6
- nfs: bind vfs cache invalidation with file’s layout
- 04f929cf1c
- ceph: map RadosException to corresponding IOException
- a10376ae03
- dache-restful-api: add transition state from DISK to TAPE to QoS description
- a899b42c34
- dcache-restful-api:bug fix: current locality of the file should be considered
- f6175a325d
- dcache: fix NPEs in TransferInfo time string methods
- 0e573b9685
- system-test: update test script so curl uses system-test’s trust-store
- f77ebaa6c6
- nfs: show transfer status when displayed
- a19f27589e
- system-test: update credential-generating script
- ce1deb63a9
- system-test: update disposable-CA generated credentials
- 0ebcceaff4
- nfs: fix loosing movers due to short timeout
- 03436a0da5
- pinmanager: fix query for getting the number of pins for a file.
- 976b783d46
- Fix class path generation
- ff52847c4f
- Add workaround for Liquibase bug CORE–3001
- 9ff9eb91d0
- Update various dependencies
- 3cabb610cb
- Upgrade base libraries to Java 8
- caccb7719d
- dcache: add timeElapsedSinceWaiting formatted string to TransferInfo
- 3d67d4c2ed
- xrootd: allow xrootd to overwrite new empty files. Motivation: Xrootdfs FUSE driver immediately closes a file on creation, then reopens it to write. The file cannot be opened as it already exists. This patch allows the dcache xrootd door to tolerate this behavior.
- 5c17c8f6f0
- ftp: fix (unreleased) regressions in certain FTP commands
- 73f60deb2a
- ceph: log repository IO error
- 6c9e3a1a2a
- writing to existing file is kXR_NotAuthorized, not kXR_Unsupported
- faf5e142df
- srm: remove file-level timeout
- 90f38a4cec
- ftp: ensure server replies even if there are bugs in dCache
- b97bd275a3
- Update code to use ByteUnit
- 19ec5b3078
- chimera: do not update inode generation on atime only update
- c960d46a47
- ftp: refactor error handling
- 2af2ae8c01
- nfs: use v4.1 for flex file layout
- 7dbbe2b0a2
- nfs: do not add Origin to read-only subject
- 23ab37a11e
- common: add general-purpose histogram and histogram models, with unit tests
- 502e09e561
- dcache-restful-api: migrating file to a an appropriate pool
- 68fc74c810
- common: move billing TimeFrame class to the common histograms package
- 7b0567fb46
- parent/dcache: add mail jar to deployment
- 23edcbd45e
- movers: fix NPE caused on upload of a zero length file
- 0b5141e849
- dcache-restful-api: add unit tests for transfer service
- d7440e22aa
- dcache-restful-api: add transfer service implementation
- 26104eac59
- dcache-restful-api: add transfer collection utilities
- 317030ed1f
- dcache-restful-api: Add RESTful resource for active transfers
- 03d529cd68
- dcache-restful-api: Add common abstractions for services supporting RESTful admin resources
- a06c2ec57e
- restful-api: add file mime-type to file attribute
- 4da5635cac
- dcache: added command to display multiple checksums for a file
- ebceda3297
- dcache,movers: Storing multiple checksums for a file when the client provided checksum is of different type than that of the pool setup.
- dd1b563498
- pool: handle CEPH exceptions
- da55011f64
- pool: fix double close on p2p
- 39fbb93c9e
- srm-client: improve handling of checksum options
- ee81623519
- nfs: recall file layout on pool disable
- de31f61417
- libs: update to bug fix release nfs4j–0.14.2
- 086c1e0136
- common, ftp-client, srm-client: remove 1.7 source and target restrictions on common, ftp and srm client modules
- 6c75e48889
- chimera: fixed database query for storing multiple checksums for a file.
- dd2b4bf1ea
- srmclient: fix handling of checksum options
- 75358b3283
- ftp: fix broken commit cfa765bb
- 7e2fa7bb9c
- ftp: add support for dynamic checksum calculation
- cfa765bb17
- ftp: add support for command pipelining
- 3d21e6e1b6
- ftp: prevent execution of most commands when unwrapped
- f49a1455c2
- srmclient: fix compatibility with castor
- 60b4224372
- chimera : handle empty paths elements path2inumber stored procedure
- 641dc750ac
- pom: Fix dCache version ready for 3.2.0
- 62be2a65c7
- [maven-release-plugin] prepare branch 3.1
- 5a333bcd3e
- [maven-release-plugin] prepare for next development iteration
- 3d91506a31
- xrootd : use lower case for checksum algorithm names when replying to checksum queries.