What’s new in dCache 2.16
The release notes
Highlights
New experimental RESTful API.
New experimental end user web interface called dCacheView.
New experimental support for redundant cell communication using multiple message brokers.
New resilience service for managing file replicas. Will eventually replace the replica manager.
New experimental support for replicable services.
Scalable SRM frontend service.
Limited OpenID Connect support.
Incompatibilities
Apache ZooKeeper is now a required dependency.
Several database schemas will be updated upon upgrade. This includes billing and pinmanager. Upon downgrade, these schemas must be rolled back using the
dcache database rollbackToDate
command prior to downgrading dCache. Third party code that uses these database will possibly have to be modified.The dCache location manager can no longer be configured to establish arbitrary topologies.
The
srm
service was split into two services - a frontend service calledsrm
and a backend service calledsrmmanager
. Both are required to operate an SRM service in dCache.Support for the SRM 1 protocol has been dropped.
The FTP door’s default root path behavior now matches that of other doors. That is, it no longer uses the per user root setting as the root directory. The original behavior can be restored by appropriate configuration.
The
RemoteTransferManager
andCopyManager
cells of thetransfermanagers
service have been merged.Sites with a custom
httpd.conf
configuration will have to adjust this upon upgrade to inject the cell address of theinfo
service.
Differences from dCache v2.15
The notes for release 2.16.0 detail the differences from dCache 2.15. Please read this section very carefully when upgrading from this version.
Release 2.16.66
scripts
The instructions that are printed out once dcache pool convert
completes successfully now correctly point to the
property that needs to be updated, namely pool.plugins.meta
.
Changelog 2.16.65..2.16.66
- f1b41a2393
- [maven-release-plugin] prepare release 2.16.66
- ee58228c14
- scripts: update reference to configuration property
- d0b33fce9b
- [maven-release-plugin] prepare for next development iteration
Release 2.16.65
dcache-resilience
There was a small regression in the way resilience computes the number of operations necessary to adjust copies when a storage unit definition changes.
The current rellease fixed computation of operation count when storage requirements change.
ftp
In order to aid diagnosing problems when FTP response being lost, now dcache logs failures to wrap/encrypt responses.
Changelog 2.16.64..2.16.65
- 22b932c
- [maven-release-plugin] prepare release 2.16.65
- a1e22fe
- scripts: add support for parsing ZooKeeper transaction logs
- 8b7354a
- ftp: log failures to wrap/encrypt responses
- bab0c9c
- dcache-resilience: fix computation of operation count when storage requirements change
- f415195
- [maven-release-plugin] prepare for next development iteration
Release 2.16.64
ftp
Error reporting in the FTP service has been improved: in some mixed IPv4 / IPv6 scenarios, only unclear error messages were reported.
Changelog 2.16.63..2.16.64
- 5c05ba0cf5
- [maven-release-plugin] prepare release 2.16.64
- dde92b3fae
- ftp: returned error is too vague for meaningful investigation
- aa68760e98
- [maven-release-plugin] prepare for next development iteration
Release 2.16.63
gplazma
gplazma now supports a Fermilab-specific authorization data source in JSON format.
pool
During active ftp transfers, connection problems would lead to the rather unhelpful error message “451 General problem”. This error reporting was refactored, so that diagnosis of the cause is now greatly facilitated.
poolmanager
A potential NullPointerException (that was not observed in real-world usage until now) was fixed in Pool Manager.
resilience
A correction to resilience’s error handling results in no more reports on non-resilient (but corrupted) files.
spacemanager
In order to facilitate debugging, Space Manager now logs link-group related content in greater detail.
Changelog 2.16.62..2.16.63
- 99ff6e5b70
- [maven-release-plugin] prepare release 2.16.63
- bf97198b25
- poolmanager: fix migration command if named pool is removed
- 1656cb63d5
- dcache-resilience: repair over-aggressive handling of broken file messages
- ac2bc68cf7
- pool: fix error message for failed active FTP transfers
- 5cc6fd8750
- correct the project version in pom.xml
- 525ba8f80b
- set project version
- 6ea80f7843
- gplazma-fermi: fix last modified check in junit test
- 9ddefe4776
- spacemanager: add remote pool monitor debug logging
- 6c2e08afaa
- gplazma-fermi: add mapping plugin to support VO group and username from file
- 3f2dbbfdb6
- [maven-release-plugin] prepare for next development iteration
Release 2.16.62
resilience
Logging for cases where file replication was fatally aborted was improved. Previously, alarms messages pertained to the PNFSID of the affected files. In rare cases, like when facing network congestion, many hundreds of alarms could be created. With this change, alarms messages refer to the storage unit, and a suffix based on an hourly timestamp is added to the alarm message. The alarm will thus be incremented during the hour but a new alarm will be created (only) hourly; in this way, those receiving email alerts will receive them once an hour.
Changelog 2.16.61..2.16.62
- ca403db91f
- [maven-release-plugin] prepare release 2.16.62
- da3ba6f48f
- [maven-release-plugin] prepare for next development iteration
- 5dab91daa9
- substituted Calendar for Instance which was failing.
- 58004eb269
- dcache-resilience: avoid spamming alarms with abort messages
Release 2.16.61
ftp
A bug (that was not observed in real-world settings yet) that might have caused NullPointerExceptions was fixed in the ftp service.
resilience
Error handling within the Resilience service was improved.
Changelog 2.16.60..2.16.61
- 787fd2b84a
- [maven-release-plugin] prepare release 2.16.61
- 155ac94dd8
- removed stray import
- d3ce544154
- dcache-resilience: handle properly RuntimeExceptions from tasks
- 83435d3188
- ftp: ensure adapter is closed
- f51e086f80
- ftp: remove rare NullPointerException when proxying data
- ac26d22940
- [maven-release-plugin] prepare for next development iteration
Release 2.16.60
dcache-resilience
When a checksum or broken file message/error is generated, Resilience makes a best effort to (a) remove the broken copy and (b) make another replica. This, of course, is not always possible, particularly if the broken file is the only accessible copy. This resulted in faulty behavior particularly the thrashing noted in the case of a restaging operation which results in a checksum error. This is now fixed.
The current release improved error handling for resilience. It fixed unnecessary Migration Task exceptions resulting from source pools with no replica in the repository.
Now it should be possible for Resilience to use pools blocked only for writes from doors.
Changelog 2.16.59..2.16.60
- c94e97d
- [maven-release-plugin] prepare release 2.16.60
- a2208da
- bad commit put DOWN twice
- 3019c7e
- dcache-resilience: define non-writable pool to mean p2p-client is disabled
- 5296408
- dcache: fix remote pool monitor wait bug
- c1a3389
- dcache-resilience: repair handling of broken files*
- 0005651
- [maven-release-plugin] prepare for next development iteration
- 1fce35f
- dcache-resilience: fix bug in source handling with Clear Cache Location messages
Release 2.16.59
cells
The current release added explicit ZooKeeper/Curator monitoring. Events generated by ZooKeeper and Curator are now logged in a new, which may help diagnose problems that are suspected to come from bad ZooKeeper interaction.
Changelog 2.16.58..2.16.59
- e6fe17c
- [maven-release-plugin] prepare release 2.16.59
- c5916dd
- dcache-resilience: fix wrong assumption about error type in Message
- 71c377f
- cells: add explicit ZooKeeper/Curator monitoring
- 6d78493
- [maven-release-plugin] prepare for next development iteration
Release 2.16.58
star
The current release improved documentation to help dCache admins to have a better understanding of how to generate StAR record.
The current release fixed fix printing exception error message for dcache-star script if there’s a problem when run with newer versions of Python.
Changelog 2.16.57..2.16.58
- e4c262a
- [maven-release-plugin] prepare release 2.16.58
- afcabff
- star: fix printing exception error message
- 7982d56
- star: update documentation to provide better description of script
- 8582150
- [maven-release-plugin] prepare for next development iteration
Release 2.16.57
info
The info service collects information about who is allowed to reserve space.
Since some of this information, like VOs, usernames and gids, may be considered
sensitive information, this update allows admins to control whether or not
to publish them. The default behaviour is unchanged from the previous behaviour,
i.e. info publishes everything. If a site admin wants to change this,
the info.limits.show-only-vo-authz
property can be set to true
.
scripts
The dcache script and manpage still refered explicitely to Java 6. This patch changes the phrasing of the respective text.
Changelog 2.16.56..2.16.57
- b1e67a165e
- [maven-release-plugin] prepare release 2.16.57
- ad748c8b9c
- scripts: update reference to JDK to avoid mentioning specific java version
- 264901a803
- info: allow admin to control whether non-VO / non-FQAN identities are shown
- a4c2d5e22c
- [maven-release-plugin] prepare for next development iteration
Release 2.16.56
chimera
The current release fixed previously introduced issues for lost+found
directory permissions.
Now, the lost+found
directory permissions is updated without causing
problems if that directory has been removed or permissions have been
modified.
Changelog 2.16.55..2.16.56
- cd08165
- [maven-release-plugin] prepare release 2.16.56
- 03a7300
- chimera: correct previous attempt to fix ‘lost+found’ directory permission
- 742ec49
- [maven-release-plugin] prepare for next development iteration
Release 2.16.55
pnfsmanager
The current release improves documentation for set log slow threshold
admin command help.
spacemanager
When trying to upload into dCache using a space-token where there is no selectable link for this operation then the user was presented with a generic error message; for example,
No write links configured for [net=131.169.71.98,protocol=GFtp/2,store=dot:user@osm,cache=,linkgroup=]
.
This behavior is changed now and an improved error message is returned to the user if they attempt an upload data into dCache using a space-reservation in a way where
poolmanager
configuration prevents the upload.
webdav
The current release improved error handling when dCache is full.
Changelog 2.16.54..2.16.55
- 6f50a50
- [maven-release-plugin] prepare release 2.16.55
- dd97a05
- systemtest: work with new OpenSSL DN format
- b61aab6
- spacemanager: provide space-specific error message on bad upload
- fb2285e
- webdav: return 507 Insufficient Storage when dCache is full
- 3123a7e
- pnfsmanager: update slow logging admin command help
- b95d321
- [maven-release-plugin] prepare for next development iteration
Release 2.16.54
cells
dCache no longer logs stack-traces when running multiple cells with the same name.
pool
For certain failures,the pool was logging transfer failures twice. This is now fixed.
rpm
dCache ensures now that user ‘dcache’ is a member of group ‘dcache’.
star
The current release Introduced new property star.db.*
, which makes possible now to run PostgreSQL on non-standard ports can use STAR.
statistics
Timeout in contacting PoolManager no longer results in a stack-trace being logged.
Changelog 2.16.53..2.16.54
- 928eedf
- [maven-release-plugin] prepare release 2.16.54
- c0ddc8b
- [maven-release-plugin] prepare for next development iteration
- 737c0c2
- pool: fix double logging on remote FTP transfer error
- 2d66570
- cells: don’t log stack-trace on starting cell with same name as running cell
- 76a0285
- pool: Fix how certain bugs are logged
- f9c3b4e
- star: support PostgreSQL running on non-standard TCP ports
- eb1e8c1
- rpm: don’t assume existing dcache user is member of dcache group
- 7d26897
- statistics: avoid stack-trace on internal timeout
Release 2.16.53
admin
The admin interface reported an attempt to connect to an absent cell as a bug. The current release fixed the issue.
httpd
Requests to httpd targeting an unknown resource was returning 200 OK
response code. Nevertheless the 404 NOT FOUND
response would be closer fit. This is now fixed.
Changelog 2.16.52..2.16.53
- d3251cd
- [maven-release-plugin] prepare release 2.16.53
- f9fa835
- admin: do not report attempts to connect to missing cell as a bug
- cb1dbde
- httpd: return 404 status code on an unknown page
- 6ce3535
- [maven-release-plugin] prepare for next development iteration
Release 2.16.52
Changes affecting multiple services
dCache no longer logs stack-traces if a Java VirtualMachineError occurs. This is unnecessary as dCache was (presumably) working fine until Java discovered a problem.
chimera
Sites updating to dCache 2.15 or later might observe that a lost+found directory with incorrect permissions was created during the update. This patch ensures correct permissions. Since we cannot know if the current permissions in lost+found are intended, this patch does not modify any existing lost+found directory permissions.
pool
The sweeper free
command no longer logs a stack trace if it is started with incorrect input information.
An irrelevant stack trace was logged by the pool. This release corrects that.
Changelog 2.16.51..2.16.52
- 35ee85d55b
- [maven-release-plugin] prepare release 2.16.52
- 33b71595ad
- chimera: update schema migration when creating ‘lost+found’ directory.
- f099997ebf
- pool: fix stack-trace on bad command input
- 0f767a691b
- pool: fix stacktrace on FaultEvent logging
- 9831422ad2
- system: Don’t log stack-trace on fatal JVM error
- f510231470
- [maven-release-plugin] prepare for next development iteration
Release 2.16.51
alarms
Until now, the sorting order of alarms did not provide a correct ordering for all types of alarms. With this release, alarms are now implicitly ordered by at least their latest modification timestamp.
resilience
One of the features of resilience is the enforcement of file partioning on pools according to pool tags. The pool tag restrictions are observed whenever a file is copied. In addition, it is rechecked when a storage unit is updated, in order to make sure the files are distributed correctly according to the new requirements. This is done by removing the offending copies and recopying them in a new location.
Should files get redistributed, however, by rebalancing or a migration job, it is possible that the partitioning will be violated, since only resilience observes it.
The resilience service now verifies that files are distributed according to the requirements specified by pool tags while doing periodic scans (or scans initiated through the admin command).
statistics
A possible race condition was removed from the implementation of the create stat
admin command.
Changelog 2.16.50..2.16.51
- 474cea883c
- [maven-release-plugin] prepare release 2.16.51
- 848ccaa4f8
- statistics: fix race in “create stat” admin command
- 6b91bf85e0
- srm: fix stacktrace on database failure
- 20e98664bc
- alarms: revert LogEntry.compareTo() to throw NPE on null object
- 06ed380733
- resilience: force tag partition checking on scans from admin command and periodic checks
- 9565b03a7a
- alarms: fix natural order comparator to use timestamp first
- 805892ebf6
- [maven-release-plugin] prepare for next development iteration
Release 2.16.50
dcap
A pool that has gone offline and comes back up again may become very slow to respond due to a large amount of superfluous error messages to dcap clients that disconnected in the meantime. This patch ensures a more responsive reaction to these cases by introducing a time-to-live value for such messages.
pool
Error reporting was improved for cases of IO errors in pools.
Changelog 2.16.49..2.16.50
- f8ef2c239f
- [maven-release-plugin] prepare release 2.16.50
- abfdee972d
- pool: avoid ‘null’ and other nondescript error messages
- 5a75c9b609
- [maven-release-plugin] prepare for next development iteration
- 9d55193337
- dcap: add TTL information to dcap messages
Release 2.16.49
Changes affecting multiple services
Many dCache components use RemotePoolMonitor to provide fast access to the information that PoolManager has about pools. In order to facilitate system diagnosis, the ‘info’ admin command was augmented by information about the current status of the RemotePoolMonitor.
admin
The Apache sshd library used by the admin interface was updated. Although we believe that dCache is not affected, some security vulnerability scanners alert users to a potentially vulnerable library version. The update serves mostly to avoid those false alarms.
dcap
In order to better handle large READV requests, the buffer size for dcap commands was changed to automatically grow to the required size, up to a maximum buffer size of 8 MB.
pool
A bug in gfal2 results in FTP transfers being aborted some 50 ms after being initiated. This results in the door killing the mover shortly after the pool received the PoolDeliverFile message. If the mover is not queued, but not yet fully started, this may lead to the pool disabling itself. This patch corrects that problem, ensuring that the pool continues to run despite any aborted transfers.
xrootd
In the 4.7 releases, the xrootd client started enforcing protocol requirements for kXR_login which, unfortunately, broke access to dCache. The xrootd client expects an answer with a 16-character session ID from the door and then the pool after the redirection. Without this ID, the client would retry (without success) repeatedly and appear to hang.
dCache’s xrootd implementation has been augmented with the session ID, enabling it to work with xrootd clients of version 4.7 and up.
Changelog 2.16.48..2.16.49
- 4c08188f48
- [maven-release-plugin] prepare release 2.16.49
- c89c0d0ffd
- many: add diagnostic information about remote pool monitor
- 5fd8a663e9
- pool: dont disable pool if mover cancelled before open
- efdcd16213
- Manual correction of missing mvn-release-plugin commit: Preparation for next development iteration
- 6185ebf498
- apache sshd: upgrade to 1.3.0 due to security vulnerability
- 92cadf866b
- dcache: fix bug in PoolSelectionUnitV2 match()
- 1e2adb151e
- dcache-xrootd: Fix login handshake to support xrootd clients (> 4.7.0)
- 089a4474a6
- dcap: bump max command size sent by client to 8MB
Release 2.16.48
cells
The current release improved some problematic error messages has better error reporting to ensure bugs are understood.
httpd
The curren release fixed the issue of change of format in transfers.txt after upgrading from 2.13 to 2.16.
pool
The current release fixed log and alarms duplication when rebuilding broken entry.
Changelog 2.16.47..2.16.48
- ec1553b
- [maven-release-plugin] prepare release 2.16.48
- 1b5d4b2
- httpd: restore millis to transfer time for transfers.txt
- eb6eacb
- pool: fix log and alarms duplication when rebuilding broken entry
- f8377dd
- cells: better error reporting to ensure bugs are understood
- ed9054b
- [maven-release-plugin] prepare for next development iteration
Release 2.16.47
pool
The current release fixed regression in GridFTP OPTS CKSM command and GridFTP OPTS CKSM
command is supported again.
The current release fixe regression with third-party pull transfers using GridFTP, with dCache acting as the GridFTP client that prevented data integrity checks.
resilience
When resilience tries to copy or remove a file, and it is no longer in the namespace, this should be a fatal error, differently from the discovery that a replica is no longer in cache. These two errors, however, were treated the same way. This is now fixed and the correct behavior is immediate abort of operation when file is not in namespace.
The current release improved error handling for file deletion during scan correctly it fixed bug in formatting and handling of cache exception types.
When doing a pool scan, the storage unit information must be recovered for each file. This is done from the chimera attributes. The code checks an internal map for an index number corresponding to the unit it knows about from the PoolMonitor.
However, if the pool selection unit configuration changes such that a storage unit is eliminated, that mapping will also be deleted from resilience. In the case that the attributes stored in Chimera still have the older storage class information, there will be a NoSuchElementException thrown.
This is fixed now and Pool scans that encounter this situation do not get stuck forever in the queue.
webdav
In RFC 7230, section 3.2, it states that HTTP header names should be case insensitive. Despite this, the standard milton mechanism for acquiring header information is case sensitive, which (in turn) means that several HTTP header values in dCache are similarly case sensitive. There were reports that this case sensitivity has caused problems with certain clients (e.g., go). The current release fixed this issue and headers that control third-party transfers are now case insensitive, so should work with more clients.
Changelog 2.16.46..2.16.47
- d13e0ce
- [maven-release-plugin] prepare release 2.16.47
- b018bd8
- Update FileOperationHandler.java
- 0bd79ab
- pool: fix data integrity regression for 3rd-party GridFTP pull transfers
- fda0922
- pool: fix regression in GridFTP OPTS CKSM command
- 937e483
- resilience: handle file deletion during scan correctly
- 3ce133b
- resilience: add pool operation logging
- 5a7bdfe
- resilience: handle storage unit NoSuchElement failure
- 0a9d62e
- webdav: adjust header parsing to be case insensitive
- d423c5e
- resilience: handle all cases where no locations for file may be discovered
- 3b390ba
- resilience: distinguish correctly between file not in repository and file not found
- 24f999b
- resilience: fix bug in formatting and handling of cache exception types
- 41985bd
- PoolManager : set return code to CacheException.PERMISSION_DENIED if staging is not allowed due to stage protection.
- 91ebad3
- [maven-release-plugin] prepare for next development iteration
Release 2.16.46
configuration
There are configuration options in zookeeper that may affect how well the zookeeper cluster will work with dCache. The lack of documentation of how dCache uses zookeeper prevents admins from tuning their zookeeper instance. This is now fixed and zookeeper configuration is updated with hints on how dCache uses zookeeper, along with the corresponding zookeeper configuration properties.
webdav
The current release fixed the stack-traces on bad client input.
Changelog 2.16.45..2.16.46
- c3a8fe8
- [maven-release-plugin] prepare release 2.16.46
- 704501a
- configuration: update zookeeper configuration with hints
- bf7f2c8
- webdav: avoid stack-trace on bad user requests
- 7663c8c
- alarms: guard against NPEs on LogEntry getters
- fa3f0a8
- [maven-release-plugin] prepare for next development iteration
Release 2.16.45
admin
While migration move
tasks on pools were working correctly, for migration info
command an error occurred,
that the current user (root) wasn’t allowed to execute anything (due to missing ACLs). This is now fixed.
Changelog 2.16.44..2.16.45
- dc7ea00
- [maven-release-plugin] prepare release 2.16.45
- 4a46bde
- resilience: restore extractor class value for extractor property
- 4f510f3
- admin: Fix Inconsistent ACL enforcement, RT 9207
- d4abc07
- [maven-release-plugin] prepare for next development iteration
Release 2.16.44
configuration
For some services it was unclear what were the admin’s responsibilities in terms of consistent configuration. The current release updated the documentation describing what is needed to deploy redundant services.
srm
The current release improved configuration for srmPing
responses.
Changelog 2.16.43..2.16.44
- 305fd82
- [maven-release-plugin] prepare release 2.16.44
- e914aad
- configuration: update description for replicable
- f2dd50e
- srm/srmmanager: fix srmPing confusion
- d2ff9fa
- [maven-release-plugin] prepare for next development iteration
- f99dd3c
- resilience: make namespace provider properties immutable
Release 2.16.43
billing
The current release added documentation that describes how a CellAddress
and its fields
expand.
config
The current release improves documentation for dCache admins by adding obsolete|forbidden
annotation for dropped properties.
This helps admins to discover changes that might affect their dCache
instance. This is both through manual inspection and through the
dcache check-config
command.
ftp
It has been discovered that timestamp facts reported by dCache
GFTP server are expressed in local (to server) time.
This is now fixed. Timestamp facts are reported in GMT.
Clients like globus-url-copty -sync
work properly and ubefrtp -ls
returns correct timestamp.
srm
The current release added new configuration properties to allow easy modification.
It upstated as well srm/manager.properties
documentation to describe the relationship between srmmanager.root
and srm.loginbroker.root
properties.
Changelog 2.16.42..2.16.43
- 5615481
- [maven-release-plugin] prepare release 2.16.43
- 7ee2066
- srm/srmmanager: update documentation about root path
- 3583ffd
- ftp: convert timestamps to GMT (to follow RFC 3659)
- 304dfb4
- config: use consistent terminology
- 9088d12
- billing: update documentation to describe CellAddress
- 5d2ae3f
- srm,srmmanager: add configuration property to allow easy modification of srm root
- 55139c0
- logback: make socket appender construction depend on log level
- 6f0c3a2
- dcache: release dcache-view 1.0.7 for dcache 2.16
- dbf4b49
- config: add obsolete|forbidden annotation for dropped properties
- f5397c6
- [maven-release-plugin] prepare for next development iteration
- ce9017a
- pnfsmanager: remove obsolete comments from properties file
Release 2.16.42
Changes affecting multiple services
Christoph Anton Mitterer submitted several corrections to the comment string in properties files.
zookeeper
A race condition in zookeeper caused irrelevant stacktraces to be logged on shutdowns. Since this problem is not easily solvable with Zookeeper 3.4, we decided to change the log-level threshold for NIOServerCnxnFactory. Note that the suppressed class is only used by ZooKeeper server (i.e., not the client). Therefore, only the ‘zookeeper’ service is affected.
Changelog 2.16.41..2.16.42
- e6347169b5
- [maven-release-plugin] prepare release 2.16.42
- 68239553d6
- use correct terminology
- 81ba905238
- fixed several typos in the documentation
- 6ee1514785
- added hint that pnfsmanagers must use the same DB
- 2e0c852009
- zookeeper: work-around race-condition in zookeeper server shutdown
- c1df82c29d
- [maven-release-plugin] prepare for next development iteration
Release 2.16.41
httpd
The “Disk Space Usage” webpage (/usageInfo
) contains a table showing
information about each pool in the dCache cluster. The “Layout”
column showed the capacity usage graphically, with different colours
showing how much of that pool’s capacity is being used for different
tasks. This release fixes the Layout heading to describe a previously
undocumented colour.
Changelog 2.16.40..2.16.41
- b7c9f6a
- [maven-release-plugin] prepare release 2.16.41
- 1c5af55
- [maven-release-plugin] prepare for next development iteration
- a234b9f
- httpd: Fixed table headers in usageInfo
Release 2.16.40
frontend
Several improvements are added to frontend such as restriction usage is improved and easily configurable JSON is now available via frontend.
srm
The current release enforced restrictions on srmSetPermission
operations.
zookeeper
The current release fixed race-condition when ZooKeeper server accepts requests before the server is fully initialised.
Changelog 2.16.39..2.16.40
- 9ffc7f8
- [maven-release-plugin] prepare release 2.16.40
- 496ac93
- frontend: refactor static (configuration) data
- c73b93d
- spacemanager: dont try to release expired spaces
- a0b8f09
- srmmanager: use path to support srmSetPermission operations
- 014924e
- frontend: fix Restriction usage
- 58451b7
- zookeeper: work-around SessionTracker racy initialisation on startup
- 8e35378
- [maven-release-plugin] prepare for next development iteration
Release 2.16.39
Changes affecting multiple services
dCache previously shipped both log4j and log4j-over-slf4j libraries. This patch simplifies deployment and prevents possible problems due to non-deterministic library loading sequences by removing log4j dependencies.
frontend
If frontend was asked to create a directory listing while a file was being uploaded into the directory being listed, previously, an error would occur. This patch addresses that issue.
This patch removes the error message No Server set for org.dcache.restful.errorHandling.RestAPIExceptionHandler
that is erroneously logged on dCache startup.
This release includes a new dCacheView version.
info
The info cell can occasionally send messages before it is properly registered. This can lead to Cannot send message with callback in state...
messages that
do not correspond to any valid error condition. This patch ensures that messages are only sent as soon as the cell is properly registered.
pool
If the communication between a pool and PnfsManager times out, the error message is not well suited to diagnosing the problem:
Failed to instantiate mover due to unsupported checksum type: Request to [>PnfsManager@local] timed out.
The checksum type is
not playing an important role here. Hence, this patch updates the error message.
zookeeper
Embedded Zookeeper instances would occasionally log stack-traces on startup. This patch adds a work-around.
Zookeeper logs a stack-trace in cases where the client disconnects unexpectedly. This is unnecessary and potentially confusing, so this patch changes Zookeeper’s behaviour to just logging the incident.
Changelog 2.16.38..2.16.39
- ce79d12a9e
- [maven-release-plugin] prepare release 2.16.39
- d828a57199
- Changelog from v1.0.5 to v1.0.6 (for dcache 2.16) 38304f8 dcache-view: log errors during vulcanisation 2a2589f dcache-view: provide more user information 73cb03b dcache-view: hide main menu button base on page width
- bfac33ad69
- zookeeper: work-around racy startup
- 3142d3d46a
- zookeeper: silence ZK server errors
- 2a4bd4c7f0
- dependencies: remove log4j jar
- 157b71171a
- frontend: fix “Attribute is not defined: SIZE” bug
- 63e0e90086
- frontend: avoid error No Server set for [..].RestAPIExceptionHandler@[..]
- e753e9bcbe
- pool: fix error message on timeout
- 131d94efa3
- info: avoid sending messages too early.
- 2097cbdfc4
- [maven-release-plugin] prepare for next development iteration
Release 2.16.38
webdav
A recent update, commit 5abc0e1c, improved the behaviour of the Milton WebDAV libraries if an IOException occurs during an upload. That patch, unfortunately, did not address all issues, and when non-spec-conformant clients are used against dCache, stacktraces can be triggered.
This patch corrects that behaviour. Also, in case of errors, the error code returned in case of any problems was changed from 400 to 500, which should signal cliens that they are free to retry the transfer after a timeout.
Changelog 2.16.37..2.16.38
- 6336fe961f
- [maven-release-plugin] prepare release 2.16.38
- 4954fae39b
- webdav: make Milton work-around more robust
- be034e047b
- [maven-release-plugin] prepare for next development iteration
Release 2.16.37
webdav
File transfers through WebDAV doors could potentially bypass any Restrictions checks in PnfsManager. This patch ensures that Restrictions are always checked and observed, and improves PnfsManager’s logging to give information in case a Restrictions check is not posssible.
Changelog 2.16.36..2.16.37
- 812935ab46
- [maven-release-plugin] prepare release 2.16.37
- fd8dec2f91
- webdav: Fix restriction check when downloading a file
- c93926e136
- [maven-release-plugin] prepare for next development iteration
Release 2.16.36
pinmanager
Upgrades from dCache 2.13 could occasionally fail during the Liquibase update stage.
The database table pinsv3
can not be dropped if it is still referenced by foreign
keys of other tables. This modification adds a cascadeConstraints=true
modifier to
the dropTable command used during the conversion process. Thereby, updates are possible
without errors and PinManager starts without issues after an upgrade.
Changelog 2.16.35..2.16.36
- a31859b134
- [maven-release-plugin] prepare release 2.16.36
- 1e518499c2
- Add cascadeConstraints=“true” to liquibase dropTable action on pinsv3 table
- 464e5a6e3c
- [maven-release-plugin] prepare for next development iteration
Release 2.16.35
nfs
Invalidation of VFS caches has been improved, so that there is no stale information left for dot files.
Changelog 2.16.34..2.16.35
- 44c617e1af
- [maven-release-plugin] prepare release 2.16.35
- ff0acf4a38
- nfs: bind vfs cache invalidation with file’s layout
- 8341e04819
- [maven-release-plugin] prepare for next development iteration
Release 2.16.34
nfs
Debugging of stuck transfers has been facilitated by adding the status of transfers to the output of the show transfers
command.
system-test
The system-test module, used for demonstration or testing purposes, comes with a built-in X.509 infrastructure. With this release, expired certificates are replaced by new ones.
Changelog 2.16.33..2.16.34
- 0f6615c72c
- [maven-release-plugin] prepare release 2.16.34
- f53c5a1952
- nfs: show transfer status when displayed
- d1191a524f
- system-test: update disposable-CA generated credentials
- ead58c7498
- [maven-release-plugin] prepare for next development iteration
Release 2.16.33
Changes affecting multiple services
The version of the PostgreSQL driver used by dCache internally was brought up to 9.4.1212. This fixes the issue described in liquibase bug 2939.
chimera
In order to improve IO performance, the way dCache updates atime
values was modified.
nfs
An internal change in the NFS door code helps reducing irrelevant exceptions being logged.
srm
SRM no longer times out file requests. This alleviates a rare race condition where timeouts for the file request and file access competed. Requests that time out now always return SRM_REQUEST_TIMED_OUT at request level and SRM_FAILURE for the SURLs in that request.
Changelog 2.16.32..2.16.33
- 1ae63cfdcf
- [maven-release-plugin] prepare release 2.16.33
- 689172ef74
- postgresql driver: update version to 9.4.1212
- 31f7a9e683
- srm: remove file-level timeout
- 7e2aa01927
- chimera: do not update inode generation on atime only update
- a9bcd86e2e
- nfs: do not add Origin to read-only subject
- d512dcd933
- Eliminated stray dependencies in the dcache/pom.xml accidentally carried over from the cherry-pick.
- 03dc2f1c5e
- [maven-release-plugin] prepare for next development iteration
- 5a4b4b89ba
- parent/dcache: add mail jar to deployment
Release 2.16.32
nfs
Change proxy-io client initialization to limit number of threads.
Bugfix update with better compatibility with REHL7 clients.
Changelog 2.16.31..2.16.32
- 186ab64
- [maven-release-plugin] prepare release 2.16.32
- f12b0c2
- nfs: change proxyio client initialization to limit number of threads
- 923ce9f
- libs: use nfs4j–0.12.4
- 4e192b1
- [maven-release-plugin] prepare for next development iteration
Release 2.16.31
chimera
There was an issue with a symbolic link to a directory where destination contained trailing slash. This is now fixed.
Changelog 2.16.30..2.16.31
- a37c3c7
- [maven-release-plugin] prepare release 2.16.31
- d74ab56
- [maven-release-plugin] prepare for next development iteration
- a92cefd
- chimera : handle empty paths elements path2inumber stored procedure
Release 2.16.30
chimera
The current release fixed database query for storing multiple checksums for a file.
ftp
The Socket read
method may return zero to indicate that no bytes were
read. Although this is not an error, such occurances will result
in a transfer failing.
This is now fixed.
Changelog 2.16.29..2.16.30
- e723896
- [maven-release-plugin] prepare release 2.16.30
- 9b50f3e
- ftp: prevent execution of most commands when unwrapped
- c092703
- chimera: fixed database query for storing multiple checksums for a file.
- 3a54ba5
- ftp: do not fail proxy transfer if read returns zero bytes
- 204b53d
- [maven-release-plugin] prepare for next development iteration
Release 2.16.29
ftp
The current release added implementation of MLSC. As a result Globus is able to query the contents of dCache directories using FTP and without creating additional TCP connections.
lm
The current release fixed the issue with duplicate location manager connectors.
Changelog 2.16.28..2.16.29
- 65fa709
- [maven-release-plugin] prepare release 2.16.29
- bd84187
- lm: Do not rely on thread interrupt for shutdown
- 775eda2
- ftp: implement the MLSC command
- f92152d
- [maven-release-plugin] prepare for next development iteration
Release 2.16.28
ftp
The current release improves compatibility between dCache FTP client and Globus GridFTP server.
resilience
When a file has no readable replicas, the default behavior of resilience
is to raise an alarm. Files marked CUSTODIAL
were excluded from this.
The current release fixed this in a way that an alarm is raised for all files without readable disk copies, whether the file is CUSTODIAL
or not.
srm
During an ATLAS stress-test of tape recalls, it was discovered that various sites had relatively short request lifetimes. However, the SRM spec provides the opportunity for the server to inform the client (FTS, in this case) of what lifetime a request actually has. The current release includes the requests remaining lifetime in the response from the server.
The current release improves the documentation to help Admins to have a better understanding how to configure dCache correctly.
Changelog 2.16.27..2.16.28
- 6f292e3
- [maven-release-plugin] prepare release 2.16.28
- 96fbfb3
- ftp: add support for paths relative to home directory
- 431ee9c
- ftp: Add support for SITE WHOAMI command
- 657346a
- resilience: remove check on CUSTODIAL for inaccessible files
- a3b4b91
- ftp: update parsing of CLIENTINFO command
- 030f3e1
- srm: include remaining request lifetime in various responses
- d976a1d
- srm: update srm request.*.lifetime configuration properties documentation
- bead579
- ftp: modify facts describing namespace ownership
- 52d5400
- ftp: add support for SITE TASKID command
- 8129a45
- ftp: add initial support for checksum performance markers
- 4ec6a45
- [maven-release-plugin] prepare for next development iteration
- da8c3ea
- ftp: add support in OPTS RETR for specifying performance marker frequency
- f90912b
- ftp: show SIZE facts for directories
Release 2.16.27
frontend
This release of dCache ships with an updated dCache View version.
xrootd
In https://github.com/xrootd/xrootd/issues/459, it became apparent that dCache could improve xrdcp compatibility by sending checksum information in lower case. This release contains this change, which should improve xrootd operations.
Changelog 2.16.26..2.16.27
- 2df1660
- [maven-release-plugin] prepare release 2.16.27
- 6adf977
- xrootd : use lower case for checksum algorithm names when replying to checksum queries.
- d398d53
- [maven-release-plugin] prepare for next development iteration
- e7ffba9
- dcache: update dcache-view version to 1.0.5
Release 2.16.26
cleaner
The list of Delete Notification targets that the Cleaner will inform when a file’s content has been cleaned is admin-configurable. In order to facilitate this configuration, the ‘show info’ command has been modified to also show Delete Notification targets.
srm
The SRM code has been made more robust against races between file deletions and copies.
systemtest
The ‘system-test’ script was updated to ensure anonymous dcap tests succeed.
Changelog 2.16.25..2.16.26
- 5cf4555
- [maven-release-plugin] prepare release 2.16.26
- e0a3229
- cleaner: show delete notification targets
- cc211e9
- systemtest: allow anonymous dcap activity
- e6e70d2
- [maven-release-plugin] prepare for next development iteration
- 9c2d941
- srm: fix recovery procedure in internal copy if source is deleted
Release 2.16.25
cells
Fixed a race condition and logging bug affecting cells relying on zookeeper. This could affect location manager, space manager and pool manager.
gplazma-ldap
Wrong quoting of two gplazma property strings could occasionally cause IllegalArgumentExceptions to be thrown when using LDAP-based logins. This problem has now been fixed.
locationmanager
Fixed a problem in Location Manager in which it would fail to shut down and could leave two overlapping connectors running.
Changelog 2.16.24..2.16.25
- e6a8458
- [maven-release-plugin] prepare release 2.16.25
- c7d9aaa
- lm: Fix lost thread interrupt that prevents shutdown
- ca1fde2
- cells: Schedule watcher notifications on cell event thread
- 4ab14f3
- gplazma-ldap: fix default properties for root and home
- df3b016
- [maven-release-plugin] prepare for next development iteration
Release 2.16.24
frontend
Included a new version of dCacheView with minor improvements.
Changelog 2.16.23..2.16.24
- f22295c
- [maven-release-plugin] prepare release 2.16.24
- b2571fc
- dcache: update dcache-view version to 1.0.4
- 411cafb
- resilience: fix fairness algorithm bugs in file operation map
- 281d16b
- [maven-release-plugin] prepare for next development iteration
Release 2.16.23
dcache
The current release updated dcache-view version.
poolmanager
Ability to restrict staging to specific protocol was missing in stage protection specification. Additionally it was not possible to specify DN, FQAN, storage group and protocol that are not allowed to stage.
This is now fixed and stage protection configuration can be used as a blacklist.
Changelog 2.16.22..2.16.23
- 4dd2ef9
- [maven-release-plugin] prepare release 2.16.23
- 4e6cd73
- PoolManager : stage protection add protocol to the list of discriminators
- c65cd28
- dcache: correct the output directory
- 65c8c58
- dcache: update dcache-view version
- 3c1c44e
- [maven-release-plugin] prepare for next development iteration
Release 2.16.22
chimera-enstore
Several improvements added to db chimera so that sql statements works as expected.
nfs
Remote I/O error was generated on the client side accompanied by stack trace on server side when user fumbled .(get) command. The is fixed now and client side sees no such file or directory error and no stack trace in log file.
transfermanager
The output of info
, queue
, ls internal
and ls external
admin
commands in RemoteTransferManager has been improved.
Changelog 2.16.21..2.16.22
- 9fcf032
- [maven-release-plugin] prepare release 2.16.22
- f192439
- transfermanager: tidy up output from ‘info’ ‘queue’ and the two ‘ls’ commands
- af9fd50
- chimera-enstore: specify fields order on insert
- 3f9e8ad
- NFS: throw FileNotFoundHimeraFsException in case of fumbled “.(get)” command
- 4b2ecc7
- [maven-release-plugin] prepare for next development iteration
Release 2.16.21
cleaner
Users reported that they wanted to see space freed up by cleaner processes to be reported as free as soon as possible. This patch sends notifications about freed up space more often, resulting in quicker status updates.
pool
A problem was fixed that could cause the csm check to fail on pools containing broken files.
When deleting files, the cleaner submits batches of files to delete to a pool, but this batch is then processed sequentially. In order to improve deletion performance, such deletion tasks are now run concurrently. The increase in concurrency can reduce the overhead of syncing the meta data to disk after every deletion, thus increasing the deletion rate.
Changelog 2.16.20..2.16.21
- 01ef131
- [maven-release-plugin] prepare release 2.16.21
- e463b0a
- pool: Delete files concurrently
- 2ae269f
- cleaner: Send notification more often
- 57d29bc
- pool: Fix csm check command in the pressence of broken files
- b7168e7
- [maven-release-plugin] prepare for next development iteration
Release 2.16.20
cells
The current release improves reply handling in batch processing.
ftpclient
The current release improves compatibility between dCache FTP client and Globus GridFTP server.
Changelog 2.16.19..2.16.20
- 4467d93
- [maven-release-plugin] prepare release 2.16.20
- 059450a
- cells+dcache: fix Reply handling in batch processing
- fc86472
- ftpclient: fix multiline ADAT reponses
- 9a22094
- ftpclient: encrypt SITE CLIENTINFO command
- 8a062ae
- [maven-release-plugin] prepare for next development iteration
Release 2.16.19
dcap
Connections of non-DCAP clients to a dCache no longer result in stack-traces in the logs.
pool
A lock contention scenario in the migration module was removed, increasing performance and throughput.
poolmanager
PoolManager was updated to properly handle the dcache.authz.staging.pep
and dcache.authz.staging
parameters. This allows to enable stage protection properly.
resilience
When building the central pool information data structure for resilience (PoolInfoMap), it was (erroneously) assumed that pool group names would never conflict with storage unit names, since the latter would always contain an ‘@’.
Obviously, there is no restriction on the naming of pool groups, so it is possible (though perhaps unlikely) that the name of a group could be identical to that of a storage unit. In this case, when the PoolInfoMap is updated, the storage group will overwrite the values for the pool group. This will eventually lead to the failure of all files belonging to that pool group to be replicated.
This potential problem was mitigated by this patch.
Resilience Manager was updated to support globbed expressions for PoolSelectionUnits.
A Python script, share/migration/migrate_from_repman_to_resilience.py
, is provided for migrating from replica manager to the new resilience service.
srmclient
SrmShell has been modified to actively probe for connection information that the user has not specified. This reduces unnecessary reconnection attempts.
If an SRM request originating from a BeStMan system has to be rejected, the client now properly reports the situation.
Changelog 2.16.18..2.16.19
- a4d5824
- [maven-release-plugin] prepare release 2.16.19
- 53e953a
- resilience: python script for making database changes necessary for migration from old replica manager setup
- 1c9704b
- pool: Reduce lock contention in migration module
- 5e8e9c7
- srmclient: fix compatibility with Bestman
- d2e7b95
- srmclient: probe for SRM endpoint parameters that the user doesn’t specify
- 6752620
- Resilience: separate out the data structures for handling storage units from those for pool groups
- 98ec759
- Resilience: Support matching the universal and class default storage unit expressions
- d430a02
- dcap: don’t create stack-trace if tunnel fails due to bad client
- de42e02
- PoolManager : stage protection, fix error in stage.fragment
- 5bd8b17
- [maven-release-plugin] prepare for next development iteration
Release 2.16.18
Changes affecting multiple services
Fixed a bug in admin and in the pool manager rebalancer component that caused these to not detect certain error replies from other components.
Internal notification processing between cleaners and Pin Manager, Replica Manager and Space Manager was improved and runs quicker now.
admin
Fixed a problem in which connecting the admin shell to a named queue caused messages to be sent to one of several consumers rather than the one cell than the admin prompt would indicate as the connected cell.
billing
If a dCache instance was shut down while the billing service was in the middle of a refresh, an exception was logged and shutdown was delayed. A change in exception handling fixes this rare scenario, ensuring a quick shutdown and no unnecessary log entries.
cells
In some cases, problems with two connectors running to the same core domain have been observed. This release improves message handling in connectors and improves debugging for problem cases.
Fixed a null pointer exception in location manager caused by loss of connection to zookeeper.
common
The representation parameter lists in help texts was improved.
Text output has been improved so that no trailing whitespace characters cause layout problems any more.
dcap
This release makes it possible for admins to ban outdated, problematic versions of dcap clients. Some old client versions contain a bug that causes the client to make unsatisfiable requests to a pool with no way for dCache to reject the request: the client will simply retry.
The client version limits are exposed using the new configuration property dcap.limits.client-version
.
The default is to allow all dcap client versions, unchanged from the previous behaviour.
This release fixes a regression through which Kerberos dcap would not work for host principals containing a ‘-’ character. GSI dcap was not affected.
doors
Fixed a bug in the lb set tags command in doors that prevented setting an empty list of tags.
pool
A bug that caused non-critical stack traces to be logged on the pool after stage or deletion failure from nearline storage has been fixed.
srm
SRM logging has been improved: When SRM request tokens appeared in the access log, a regression, fixed by this release, caused them to be listed without a backend ID prefix even if they carried one.
srmclient
The shell’s built-in help support now includes some additional information about available commands.
The SRM client now allows glob patterns for the “check permission”, “get permission”, “lrm”, “lrmdir”, “rm” and “rmdir” commands.
Until now, SrmFS could not connect to StoRM instances. This release updates the cd
command so that a connection to StoRM is now possible.
Changelog 2.16.17..2.16.18
- f3056d5
- [maven-release-plugin] prepare release 2.16.18
- 38b9a40
- dcap: expose dcap client version limit
- 66daa5d
- dcap: fix Kerberos dcap if principal contains a ‘-’
- 74d2303
- dcap: fix regression in handling old version
- c06470c
- cells: Fix NPE in location manager
- de159f1
- cells: Add safe guards to connector create and process update events
- fe480b3
- srmclient: fix ‘cd’ compatibility with StoRM
- 02ef254
- pool: Suppress two stack traces in nearline storage handling
- 76ec624
- srm: Log correct request token in access log
- 685502e
- srmclient: add initial support for glob expansion
- 4a53beb
- billing: fix stacktrace and slow shutdown if in refresh
- 7c95752
- cleaner: Send notifications concurrently
- de80f81
- doors: Allow setting an empty list of login broker tags
- ce367b9
- common: update ColumnWriter to remove any trailing whitespace
- 15e3528
- srmclient: add additional help text to srmfs
- 5b1cf01
- dcache: Fix detection of message errors in poolmanager and admin
- c040a50
- admin: Send requests to a fully qualified address once connected to a cell
- 8d1d657
- common: do not markup ellipses as a user-value
- c430295
- [maven-release-plugin] prepare for next development iteration
Release 2.16.17
doors
Fixed a bug in which information on stage pool and number of attempts were lost when retrying pool selection requests.
frontend
This release fixes a regression that prevented anonymous users from accessing dCache.
srmclient
A bug in srmclient was fixed that occasionally reported checksum errors when trying to upload small files.
Changelog 2.16.16..2.16.17
- 12fe4b2
- [maven-release-plugin] prepare release 2.16.17
- 6f113d1
- frontend: reinstate support for truly anonymous access
- a3a68b7
- doors: Ensure that pool selection context survives between retries
- fe0e9c2
- srmclient: fix checksum mismatch when uploading small files
- a5bd65d
- [maven-release-plugin] prepare for next development iteration
Release 2.16.16
Changes affecting multiple services
Fix potential inconsistencies with RoutingManager
The srmmanager and statistic services included hard-coded cell name for the billing service. This release updates these services to honour the configured name for billing.
billing
Fixed a problem in the output of the dcache billing
command using
JSON or YAML when the billing format includes a custom date format.
dcap
Add support for the dcap client supplying additional version information.
frontend
Upgrade dCache-view to v1.0.2, which brings several improvements.
gplazma
Fix explain login
and test login
commands so they are able to test
logging in with username and password.
Add examples to explain login
command help. Our thanks to Onno
Zweers for this change.
httpd
Fix regression in the transfers.txt
output format.
Update the transfers.html page so it no longer includes <unknown>
for default/unknown protocols. With this version of dCache, these are
represented by a ?
character. The webadmin page is updated to give
consistent output.
scripts
Fix the billing indexer to ignore the format
string, if present.
Changelog 2.16.15..2.16.16
- 22aedeb
- [maven-release-plugin] prepare release 2.16.16
- 59b7c4d
- srmclient: fix srmfs to shutdown cleanly if user issues Ctrl-C
- 4b6ebf9
- srmclient: clean shutdown srmfs after transfer
- 0e5984a
- eliminated ETWS on line 569 of TransferObserverV1.
- 87ededf
- Active Transfers: substitute ? for <unknown> on html pages
- 799a240
- srmclient: avoid stack-trace if lcd with incorrect path
- 5afc6b3
- cells: Guarantee event order for cell even listeners
- 11d4d35
- srmclient: give meaningful error message if credential is missing
- 4f86863
- common: add support for UserNamePrincipal as user:<name>
- 56e514f
- srmclient: provide better error message if credential has expired
- c791ca7
- Added ‘explain login’ examples to help text in Gplazma2LoginStrategy.java
- b4629be
- srm,statistics: Parameterize billing service name
- 16d3edc
- billing: Strip format string from attribute name
- 5223e82
- transferObserverV1: replace Args with Joiner to construct transfers.txt lines
- f3427e7
- billing: Make billing indexer work with custom format strings
- 4fbf078
- dcache: update dcache-view version to 1.0.2
- 08316b6
- [maven-release-plugin] prepare for next development iteration
- d9843a7
- dcap: add support for clients presenting more version metadata
Release 2.16.15
commons
If dCache’s internal ShellApplication framework detects a critical behaviour that might indicate a bug in the application, error messages now include an explicit request to send a mail to the developers and more relevant information for assessing and reproducing the situation.
gplazma2
This release fixes a small bug in GPlazma which inappropriately tried to handle non-DN subjects in x509 certificates. These will usually fail in gPlazma anyway, but the reported error was confusing.
srm
Handling of DNS names without trailing dots in certificates has been made more robust and universal.
srmclient
SRM client gained limited UNIX-like commands for navigating file systems.
From this release on, it offers the lpwd
, lcd
, lls
, lrm
, lrmdir
,
lmkdir
and lmv
commands, which work similar to the corresponding
UNIX commands that do not use the ‘l’-prefix.
The get
and put
commands have been updated and do now resolve
local paths relative to the local current working directory.
A new prompt
command allows users to modify their prompt.
This means that users are now able to modify their local directory when preparing for upload/download to an SRM endpoint. The local directories for uploads and downloads may also be modified.
The handling of empty directories in SRM’s ls
has been improved.
srmfs checks whether a user can change into a given SURL, but did previously not handle cases gracefully where the server declined such a request. This behaviour has been improved.
Error messages for SRM client are now more informative.
Changelog 2.16.14..2.16.15
- 69dc991
- [maven-release-plugin] prepare release 2.16.15
- 5822e1c
- srmclient: update checkPermissions to be more robust
- 7f3b41a
- srmclient: print friendly message on SRMException
- aa609fd
- srmclient: update ls to be more robust
- 71d02ee
- commons: log bugs with stack-trace and instructions
- c463889
- gplazma2-xacml: remove erroneous creation of placeholder extensions
- f104955
- srmclient: implement useful subset of local filesystem commands
- b6141f0
- [maven-release-plugin] prepare for next development iteration
- 13056cb
- srm: remove trailing dot from reverse lookup result
Release 2.16.14
cells
The current release fixes a race condition through which routes to non-existing cells could be set up.
A race condition could lead to default routes pointing to tunnels that no longer exist. This has been fixed.
In rare cases, an interrupt needed to cleanly shut down the location manager connector would not arrive. This issue was corrected, ensuring more reliable behaviour on cell shutdown.
Fixed a problem with phantom tunnel connectors causing tunnel connections between satellite and core domains to be reestablished about every 10 seconds.
A regression in which cell startup failures were not propagated to the starting component was fixed. As with the related fixes, this provides more robust cell lifecycle changes.
Fixed a problem in which threads could inappropriately be created as daemon threads, causing problems in killing those threads when the cell shuts down.
Fixed a race that could lead to corrupted state in the routing manager.
srmclient
Previously, sending SIGKILL to srmclient did not properly terminate the process. This was mainly caused by bash not propagating SIGKILL to child processes. This has been worked around, and SIGKILL now works as expected against srmclient.
When used against a server that responds asynchronuously, the stat command would never terminate. This behaviour was changed by introducing a short delay between successive retries, making stat more robust.
Currently dCache distinguishes between SURLs that have a port number and
those that do not; therefore srmrm srm://srm.example.org/file-1
will not
abort an upload issued with srmcp file:///path/to/local-file
srm://srm.example.org/file-1
.
This has been changed, so that srmclient can now be used to upload files to a port-less SURL.
Error messages for srmclient were improved by adding more detailed information about stat
and ls
failures.
The -debug option of srmclient now provides much more useful output.
webdav
Currently, the WebDAV door returns a 401 status on permission problems even when the user is authenticated. This results in clients prompting users to reauthenticate whenever the client makes a request that is denied for authorisation reasons. Since this is not in line with the generally accepted meaning of the status codes (401 standing for “please try again, but with Authorization header” and 403 for “please don’t try again, admin intervention needed”), the behaviour of the door was changed.
Note: If any installation used the fact that dCache returned the wrong status-code as a “hackish” mechanism to clear the browser’s BASIC credential cache (e.g., to “log out”), this is no longer possible.
Changelog 2.16.13..2.16.14
- 61b5d7f
- [maven-release-plugin] prepare release 2.16.14
- d2199bf
- srmclient: fix async stat command
- 42e33ac
- srmclient: fix error message for ls
- e0708ce
- srmclient: refrain from adding default port to SURL in srmcp
- a4d9895
- srmclient: fix -debug mode
- f0f86b8b7
- srmclient: support SIGKILL
- a0e5ae1
- cells: Remove race in routing manager
- cc8364f
- cells: Fix lost interrupt exception
- 9c64caf
- cells: Ensure that newly created threads are non-daemon normal priority threads
- 27caf3c
- webdav: fix Unauthorized vs Forbidden response
- 5b86a5d
- cells: Propagate startup failures to caller
- 090e6f9
- cells: Fix race when adding routes
- 422d9e5
- cells: Remove delayed default route if tunnel shuts down
- 03019d0
- cells: Fix race in cell shutdown
- 72ff73c
- [maven-release-plugin] prepare for next development iteration
Release 2.16.13
commons
The \s admin command uses the toString method to serialise the requested so that the remote cell may correctly parse it. This did not always work: ‘=’ characters in arguments were escaped but not unescaped; arguments that start with a ‘-’ character were not escaped; empty words were lost. In the current release this is fixed and \s command works as expected.
dcache
The current release fixes a regression in which the exit code of check-config would always be zero even when errors were detected.
doors
The current release fixes a bug that cost pool manager to adjust cost estimates on the wrong mover queue when selecting pools.
srmclient
The current release adds support in srmfs for uploading and downloading files.
Changelog 2.16.12..2.16.13
- 4e572ab
- [maven-release-plugin] prepare release 2.16.13
- ed3b2a7
- srmclient: add support in srmfs for uploading and downloading files
- 3951b1b
- dcache: Generate proper exit code for check-config command
- fcba804
- commons: fix Args string parsing and toString method
- d81194b
- doors: Include IO queue in pool selection request
- fa2b249
- [maven-release-plugin] prepare for next development iteration
Release 2.16.12
alarms
Alarm email notifications are now sent only on the first occurrence of given alarm (i.e., for that alarm instance’s unique ID).
If an alarm has been closed and not deleted, but then occurs again, the counter for receiving that alarm is now reset to 1, in order to treat this as a new (set of) occurrences, and to guarantee a new notification will be sent.
cells
Several bugs related to cell shutdown were fixed in this release. Most noticeably, tunnel connections could appear to fail, which has been corrected.
webadmin
Due to an implementation detail in a library used for the webadmin pages, filtering tables was a bit unintuitive until now: A filter that was set in a certain table column would reappear on tables in other browser windows if they had similar columns.
This behaviour was corrected, and tables on different pages exposed simultaneously in different browser tabs are now filtered independently. However, the fix also has the side effect that now with page reloads and form submissions the filters are cleared. Any commands, however, will always be issued correctly.
Filtering rows in webadmin tables could occasionally lead to unintuitive behaviour with regard to selections: Filtering a table and hiding rows may be included in a “select all” or “deselect all” operation. This was fixed, and selection/Deselection of hidden rows is now prevented.
Issues with filter boxes disappearing or filters resetting have been solved by disabling AJAX auto refresh for the affected pages.
Changelog 2.16.11..2.16.12
- 63ccd34
- [maven-release-plugin] prepare release 2.16.12
- 762f772
- cells: Backport fixes for cell shutdown
- 775d5af
- dcache-webadmin: synchronize client-side filtering with server-side selection of rows on pages using picnet table filters
- 92e5f39
- alarms: reset count history on reopened alarm
- e3e9bbb
- dcache-webadmin: disable saving table filter settings to browser cookies
- 5fe3119
- dcache-webadmin: disable AJAX autorefresh on pages using picnet table filter library
- 331c196
- alarms: only send email on first alarm occurrence
- 8378505
- [maven-release-plugin] prepare for next development iteration
Release 2.16.11
chimera
Chimera occasionally suffered from (operationally irrelevant) IllegalStateExceptions. Those are now avoided.
doors
Doors could get stuck temporarily if a file was deleted during pool selection. This has been fixed, and in such cases, transfers are now aborted properly.
Changelog 2.16.10..2.16.11
- e9ee828
- [maven-release-plugin] prepare release 2.16.11
- 2b526ee
- doors: Abort transfer if file is deleted during pool selection
- 2598c17
- chimera: Fix IllegalStateException in inode cache
- 6bd2ff7
- [maven-release-plugin] prepare for next development iteration
Release 2.16.10
billing
Formatting strings for the billing system have been extended to allow the names of the cell and cell domain to be addressed directly. Those names are accessible using $cellName.cell$
and $cellName.domain$
, respectively. This allows greater flexibility when creating informative messages. No existing formatting strings are affected by this change.
When using the dcache billing
command with a non-default date format in the billing file, an unneccessary stack trace was printed. This has been corrected.
cells
The handling of failures during cell startup was improved.
Fixed a RejectedExecutionException that could occur during shutdown.
srm
The SRM should periodically (by default every 10 minutes) delete obsolete historic data (older than 10 days by default) from the database. For cases where there are problems with that process, error logging and robustness against temporary database problems have been improved.
webadmin
The Wicket library used by dCache internally issued warnings about upcoming naming changes. Those cluttered the log files, and are silenced from the current version on.
zookeeper
A race condition caused Zookeeper to report non-critical CancelledKeyException instances on shutdown. This has been fixed, streamlining the shutdown and reducing operationally irrelevant log entries.
Previously, when shutting down dCache, the zookeeper cell was sufficiently slow that it required the System cell to explicitly interrupt still running threads. The shutdown has been accelerated, resulting also in less operationally irrelevant stack traces and errors reported in the process.
Changelog 2.16.9..2.16.10
- 598fea8
- [maven-release-plugin] prepare release 2.16.10
- c7f5145
- srm: make out-of-date historic data deletion more robust
- 7220392
- cells: Do not call shutdown from message thread
- 392ade9
- zookeeper: avoid one race in shutdown
- 3d2d2d1
- zookeeper: work-around slow shutdown of SessionTrackerImpl
- 78679bc
- webadmin: silence warning about future change in wicket
- 43a7fae
- [maven-release-plugin] prepare for next development iteration
- 263e224
- cells: Fix RejectedExecutionException during shutdown
- cc3155a
- billing: Removing erroneous stack trace output
- 7a6b7e0
- billing: Wrap cellName attribute to allow access to the cell and domain components
Release 2.16.9
Changes affecting multiple services
The current release fixes compatibility with OpenSSH 7.2.
dcache-restful-api
The current release adds a restfull call to change the QoS of a file from disk+tape
to tape
.
ftp
The Apache Commons FtpClient can issue the LIST command with the
non-standard -a
option. Which was causing dCache to switch
output format from the long (ls -l
-like) to the short (ls
-like)
response. This is fixed now and dCache is more compatible with Apache Commons FtpClient.
Changelog 2.16.8..2.16.9
- e9d5186
- [maven-release-plugin] prepare release 2.16.9
- 3efac50
- dcache-restful-api/pinmanager: modify current QoS fom disk+tape to tape
- ae798a6
- ftp: improve compatibility with Apache Commons FtpClient
- 4bacfe0
- Upgrade to CANL 2.1 and BC 1.50
- 2cb69bd
- [maven-release-plugin] prepare for next development iteration
Release 2.16.8
billing
Deployments may use more than one billing service. In such a deployment one may want all instances to log the same billing messages. This may be achieved by treating ‘billing’ as a topic rather than a named queue.
The billing.cell.subscribe
property was added, allowing the billing service to subscribe to PubSub topics.
cells
If the create
command in CellShell fails because of unreadable setup files, it throws an IOException. This was incorrectly reported as a bug. Reporting has been corrected now.
In rare cases, active transfers would show up with an incorrect state in the active transfers page of the admin backend. This was fixed, so that ransfers which are staging from non-DCAP doors are correctly indicated (in yellow) on the active transfers page, instead of showing up as “No Mover found” (in red).
common
If there is an IOException when trying to read a setup file, the corresponding file name is now listed in the error message.
dcache-nearline-spi
Since AbstractBlockingNearlineStorage is a useful baseclass for third party nearline storage implementations, it was made available as part of the dcache-nearline-spi artifact.
poolmanager
An issue with PoolManager prevented it from delivering correct cost estimates. This was fixed, resulting in improved estimations of pool load.
srm
The error messages for aborted srmRM uploads were improved. Instead of just stating “Upload aborted because the file was deleted by another request”, the message now includes the session ID of the remove request. This ID can be used by admins to find the remove request in the SRM access log.
srmclient
Mac OS-X BSD has an implementation of readlink that doesn’t support the -f option. A recent upgrade to SRM Client used that option, so a workaround was necessary. This release fixes the issue, and SRM Client works again on MacOS.
transfermanager
Due to a regression, 3rd-party WebDAV transfers would show a NullPointerException in transfer manager, although they completed correctly. This problem was fixed, and the progress report information is correct again.
Changelog 2.16.7..2.16.8
- 47dfc4f
- [maven-release-plugin] prepare release 2.16.8
- 531ab45
- srm: Include session identifier in error message when srmRm aborts an upload
- 284a31e
- billing: Expose property to subscribe to topics
- 486f797
- dcache-nearline-spi: Move AbstractBlockingNearlineStorage to dcache-nearline-spi
- 5fbd3e3
- cells: IOException is not a bug in create command
- b58a557
- transfermanager: fix querying of 3rd-party transfer
- 0da9df7
- common: include filename in error message
- c391fdd
- srmclient: do not rely on -f option of readlink
- 52a709b
- poolmanager: Fix incorrect correction of pool cost
- 4fd3879
- cells: handle empty string pool value on staging in TransferObserver
- c5c9607
- [maven-release-plugin] prepare for next development iteration
Release 2.16.7
alarms
Log entries that were promoted to alarm status and that show up in the webadmin table can now contain more detailed information.
Changelog 2.16.6..2.16.7
- ea871c1
- [maven-release-plugin] prepare release 2.16.7
- 332cd22
- Fixed package on NDC import.
- 3348eb3
- alarms: add ndc info to alarm info
- b5fd159
- resilience: fix alarm log level
- eb95101
- [maven-release-plugin] prepare for next development iteration
Release 2.16.6
frontend
Users can query Qos for file objects. The current release provides rest-api call to change QoS for files.
The current release added several enhancements to dcache-view.
gplazma2-argus
Fixed a problem with the gPlazma argus plugin that caused it to fail with a ClassNotFoundException.
Changelog 2.16.5..2.16.6
- 256f750
- [maven-release-plugin] prepare release 2.16.6
- fa9e8c2
- dcache-restful-api: RestfulAPI for QoS(CDMI) CHANGE current QoS for the specified file
- a6a02a2
- packages/dcache-view: depolyment of dcache-view through nexus
- 9169cbf
- dcache-nearline-plugin-archetype: Include correct service loader definition
- 00cf3a9
- frontend/dcache-restful-api: remove www-authenticate header
- 6c01567
- gplazma2-argus: Update to Argus client 2.2.0 to fix dependency on VOMS library
- f084045
- [maven-release-plugin] prepare for next development iteration
Release 2.16.5
alarms
A change to the alarms system improves handling of alarms with unset types.
frontend
Users can query existing Quality of Services, for example, storage requirements such as flexible allocation of disk or tape storage space. Possibility to query QoS for the specified file object.
httpd
An improvement to error checking avoids an unnecessary NullPointerException being logged.
nfs
A race condition in the NFS door that could result in the creation of multiple inconsistent copies of a file being uploaded has been fixed.
pool
A regression was fixed that caused the jtm go
command to occasionally not work.
When a transfer’s status is queried before the transfer is initiated, which can occasionally happen for queued requests, Exceptions were logged. This behaviour has now been corrected, providing more robust operation.
Permanent migration jobs were persisted in the pool setup file. The current release fixes a problem causing permanent migration jobs to be duplicated if the pool setup file was reloaded.
Several race conditions in the pool’s migration module are fixed now.
A regression prevented queues to be set to not handle any jobs at all. The fix allows to pass a limit of 0 to mover set max active
.
A regression was fixed that caused pools to “leak” movers if those were cancelled while still being queued.
resilience
Until now, a check was in place that made it impossible to link resilient storage units to non-resilient pool groups. This constraint has now been removed in order to accommodate use cases such as writing to a “staging” pool and then transferring from there to a resilient pool.
A potential race condition in Resilience Manager has been fixed.
File operations run concurrently, but are serialized with respect to pnfsid. For any logical file, only one copy or remove is thus allowed to run at a given time. In situations with several concurrent scans, this update improves the handling of metadata on the affected files.
Removing pools from a resilient group is now handled in a more robust manner.
File operations run concurrently, but are serialized with respect to pnfsid. For any logical file, only one copy or remove is thus allowed to run at a given time. In situations with several concurrent scans, this update improves the handling of metadata on the affected files.
A minor bug in target selection for a corner case has been resolved.
Resilience Manager now implements improved checks when determining whether or not a pool needs scanning after a state change.
Pools which were once resilient, but for which resilience has been deactivated, no longer stick around in the operation map and thus disappear from the “pool ls” listing.
After the initial commit of Resilience, some classes were renamed to use “File” rather than “Pnfs” as a more proper designation. With this release, all such occurences have been renamed.
spacemanager
Fixes a compatibility problem with NFS in which space manager would fail with a duplicate key error.
srm
A hint to describe the necessity to include escaping has been added.
The current release adds small fixes to SRM shell.
The current release adds small fixes to SRM shell.
The current release fixes issues in which the use of SRM third party copy operations could cause the SRM cell to become unresponsive, possibly even run out of memory.
Changelog 2.16.4..2.16.5
- e81cf7b
- [maven-release-plugin] prepare release 2.16.5
- 2cdc7b9
- pom: use project.version instead of derived dcache.version
- 0926362
- pool: Cancel permanent migration jobs when reload pool setup file
- 0dd0545
- archetypes: Adjust version numbers to match parent artifact
- 92982a2
- dcache-restful-api: fix data type for cdmi_geographic_placement
- b3e362c
- resilience: eliminate unnecessary update calls and fix synchronization of file op registration
- ff437ee
- dcache-restful-api: RestfulAPI for QoS(CDMI) get current QoS for the specified file
- 02bfe0f
- dcache-restful-api: RestfulAPI for QoS(CDMI)
- 89781d2
- srm: add hint to escape IDs
- 283456b
- archetype: Add archetype for creating nearline storage plugins
- a70e0b7
- nfs: Fix race condition in transfer startup
- 7328a70
- common-cli: Fix compatibility with Java 7
- 7c802fe
- SrmShell.java: changed ‘rw–’ to ‘rw-’ in case TPermissionMode._RW
- ed7e347
- SrmShell.java: changed ‘rx-’ to ‘r-x’
- 6c9f639
- info: fix broken unit-test
- 1c01d12
- srm: Resolve message thead blocking issues with SRM third party copy
- da1240c
- restful-api: make exception and error handling resful
- 085afd7
- spacemanager: Work around for doors resubmitting PoolAcceptFileMessage
- 8a6210b
- pool: Fix several race conditions in migration module
- 6193fe1
- pool: Fix regression in mover set max active command
- 8b71c93
- pool: Fix mover leak
- 5a44dd2
- pool: Fix synchronization regression in jtm
- 70fa317
- pool: avoid NPE when querying status of a 3rd-party HTTP transfer
- 1af8be6
- resilience: finish the renaming of ‘pnfs(id) operation’ to ‘file operation’
- 4649224
- [maven-release-plugin] prepare for next development iteration
- 2750d1e
- resilience: repair (subtle) bug in target selection
- 6dbbdcb
- resilience: fix error in file operation updating
- 5f4239b
- resilience: fix the way removal of a pool from a resilient group is handled
- eb8f5c3
- resilience: fix several related bugs in transition checking when scheduling scans
- f987f5e
- resilience: remove restriction on storage unit linkage
- bedc68c
- resilience: remove pool from operation table when it has been removed from a resilient group
- 6db604c
- resilience: allow file consumer to propagate Exception
- dcb4de1
- httpd: change condition from numeric inequality to non-null check on TransferInfo value
- 8a497a4
- alarms: fix NPE in type setter
- 9db791d
- chimera: update unit-test to log ChimeraFsExceptions
Release 2.16.4
Changes affecting multiple services
Fix security vulnerability. Affects following services: dcap, ftp, srm, transfermanagers, webdav and xrootd.
Release 2.16.3
admin
The admin door now generates SSH keys to ensure compatibility with OpenSSH 7. Additionally,
a new property admin.paths.host-keys
was introduced in the admin.properties file,
allowing to specify the location of keys.
build
A code coverage
profile was added to the build system to check the coverage of our unit tests.
The coverage analysis uses JaCoCo as the technical foundation.
cells
This release improves local message delivery.
A bug was fixed that caused delivery of messages to cells to fail if the cell name was different from the cell service name.
frontend
The login behaviour of the WebDAV frontend was changed. Now, if a user presents invalid credentials, the login attempt will fail and result in a 401 status code. Previously, that case was treated as if the user had not presented any credentials at all.
Normally, users get prompted for authentication when interacting with the WebDAV interface. This is not desirable for AJAX requests coming from the web frontend. With this release, Javascript can submit AJAX requests with username ‘anonymous’ to achieve desired anonymous interaction without the browser prompting the user for credentials if it turns out authentication is required.
pool
A minor documentation error for the rep ls
command was fixed.
resilience
The behaviour of resilience manager was improved to allow easier operation in cases
where a pool needs to be intentionally taken offline. Previously, a bug prevented
resilience from fully taking into account pools which were deliberately ‘excluded’
by the administrator. The redundant pool suppress resilience
command on the pools has been removed; pool exclude $POOLNAME
is the replacement.
Excluded pools are now correctly handled by resilience manager, and any marking of
pools as excluded will survive a restart of the resilience manager.
script
When a pool’s metadata conversion operations would fail, an error caused a script to report successful conversions. This error has been fixed now.
webdav
There are WebDAV clients that do not send a User-Agent
header along with
their requests. dCache’s WebDAV code has been updated to avoid
NullPointerExceptions occuring in those cases.
zookeeper
There was a race condition when creating log directories on a system using a local Zookeeper service. In rare cases, this led to messages stating (System) [] URL [file:/usr/share/dcache/services/zookeeper.batch]: line 42: Command failed ((3) Unable to create data directory /var/lib/dcache/zookeeper/version–2; caused by: Unable to create data directory /var/lib/dcache/zookeeper/version–2) and dCache not starting successfully. This race condition has now been fixed, and the log file creation is thus reliable now.
Changelog 2.16.2..2.16.3
- e36c8a0
- [maven-release-plugin] prepare release 2.16.3
- 2a3a3ba
- webdav: avoid NPE if client fails to send a User-Agent header
- 3b3bf36
- cells: Allow local delivery of messages through queue routes
- 3cce13c
- zookeeper: avoid race in creating log directories
- 77e5f86
- build: add code-coverage reports
- 2f4cd02
- cells: Fix local delivery to queues
- 04a194f
- pool: Report file in transient state as locked when setting sticky flag
- bdcef9d
- admin: Fix compatibility with OpenSSH 7
- aa30ef9
- pool: Fix documentation error for -storage option of rep ls command
- 0193cf3
- script: Do not claim success if meta data conversion failed
- 0f749a8
- frontend: add support for user ‘anonymous’
- 6bc8178
- resilience: fix incomplete behavior for pools intentionally marked “excluded”
- da3f0a3
- frontend: always fail request if wrong credentials are presented
- b197331
- [maven-release-plugin] prepare for next development iteration
Release 2.16.2
admin
First observed on Ubuntu Xenial, dCache fails to install on modern Linux distributions due to the short key length of the SSH 1 keys generated in the post install script. This patch removes those keys and their generation code. dCache has been supporting only modern key formats for quite some time now, so this change should not have any impact on users.
frontend
Removed unnecessary login prompts from dCacheView.
Previously, the REST interface did not provide a reliable way to check whether credentials supplied by a user also led to that user being authenticated. With this change, clients can discover information such as whether or not a username+password combination is valid and which uid and gid(s) the user has.
In order to improve operation of dCacheView, corrected error codes are introduced for situations where users try to query resources they are not allowed to.
pool
This change provides more detailed internal error reporting on P2P transfer failures, supporting the new resilience manager in its decisions.
This patch corrects the pools’ reporting behaviour when trying to access a broken replica. In that case, pools would previously report that the replica is locked which usually indicates that retrying will eventually allow the request to succeed. This change supports the operation of the new resilience manager.
When creating movers, some error conditions are expected to occur and dCache is designed to transparently recover from these. Consequently, this patch lowers the log level for the related error messages to reflect that their causes are harmless.
Fixed a staging problem that would lead to failures in nearline COPY operations.
Improved efficiency for pools under high resilience load.
The queue activate
command was accidentally renamed queue setIncluded
. This patch corrects the command name.
resilience
When a p2p copy fails because it is trying to access a source file before the pool repository has completed processing it, the client receives a “Source pool failure (File is incomplete)” error. Previously, the resilience manager would incorrectly consider this situation as an error requiring a retry from a different data source. This patch corrects that mistake.
This change improves the efficiency of resilience manager by avoiding repeated attempts to read from a location in case it is the only one available.
Improved exception handling in the resilience manager.
srm
Some race conditions during SRM startup were fixed. Those race conditions could potentially have lead to failures to expire jobs and to wrong job counts in the SRM schedulers.
webdav
Until now, trying to access a file for which the client was not authorized would generate a reply with a status code 200 OK, but an empty body, rather than an error page. This patch corrects that behaviour and also improves exception handling for that case.
Changelog 2.16.1..2.16.2
- 7553257
- [maven-release-plugin] prepare release 2.16.2
- 870abc0
- restful: add ability to discover information about current user
- aba7f76
- resilience: fix classification of file incomplete and other errors
- cf682b0
- pool: Don’t use transient error for a broken file
- d2a63bd
- admin: Drop old ssh 1 keys
- 1245be2
- pool: Add error codes to p2p failures
- 0ec3315
- pool: Make final state update after upload atomic
- f4e6b26
- pool: Lower log level of certain failures to create mover
- 4687641
- pool: fix staging for CopyNearlineStorage
- e571099
- REST-api: fix permission denied.
- 1044919
- frontend: remove prompt login onloading dCacheView
- 9674b36
- resilience: add checked location selection exception
- 12168e6
- resilience: fail when source retry is requested with only one possible source
- 7a5cd64
- pools: restore correct command names
- f125cd1
- webdav: Fix error reporting when client is unauthorized
- e2d6f8b
- srm: Fix job expiration during service startup
- f9c4cf2
- [maven-release-plugin] prepare for next development iteration
Release 2.16.1
billing
Fixes a regression that made using the billing system with an HSQLDB backend impossible. Since that backend can only be used for testing purposes, the impact of this regression was low.
The data used to create the 24 hour billing overviews is aggregated in hourly intervals before creating the plots. However, if there is very high activity on the system during an entire 24 hour period, there have occasionally been timeouts when querying the database for this aggregate data. This patch makes the data aggregation more robust against such situations, resulting in lower latency for histogram generation and no more timeouts.
gplazma
Previously, attempts to authenticate users against an htpasswd entry that was malformed resulted in a stack trace. This patch modifies the error handling so that only a detailed error message (“Bad entry in file: hash does not start ‘\(1\)’ or ’\(apr1\)”) is logged.
Changelog 2.16.0..2.16.1
- de72dba
- [maven-release-plugin] prepare release 2.16.1
- 0b85b5d
- (2.16) billing: use in-memory buffer for hourly aggregate data
- 43b3e55
- billing : fix issue wih HSQLDB backend
- edc52da
- [maven-release-plugin] prepare for next development iteration
- b788188
- gplazma: don’t generate a stack-trace if htaccess is malformed
Release 2.16.0
Introduced Apache ZooKeeper as an external coordination service
Apache ZooKeeper is a naming, coordination, and synchronization services. Starting with the release of dCache 2.16, Apache ZooKeeper is a required dependency to run dCache. It is used by several components in dCache – e.g. for locating dCache domains, locating services, and leader election. Future versions are most likely increasing reliance on Apache ZooKeeper.
To make migration of existing deployments easy, we support using an
embedded ZooKeeper, i.e. the ZooKeeper server runs inside a regular
dCache service in a dCache domain. It is configured through
dCache’s own configuration system. For easy upgrade, add the following
as the first domain in the layout file of the host with
dCacheDomain
:
[zookeeperDomain]
[zookeeperDomain/zookeeper]
By placing this domain on the host running dCacheDomain
, other
domains in dCache will be able to locate the ZooKeeper instance without
configuration changes. Eventually you should configure
dcache.zookeeper.connection
rather than dcache.broker.host
to
allow dCache domains to locate ZooKeeper.
If using the embedded ZooKeeper service, we recommend running
zookeeper
in its own dCache domain to avoid runtime dependencies
between the ZooKeeper server and the client inside dCache connecting
to it. In particular the low level cells system itself relies on
ZooKeeper, presenting problems during startup and shutdown of the
domain hosting zookeeper
. Such problems are minimized by running the
service in its own domain, but error messages in the log files during
startup and shutdown are inevitable.
For improved stability, we recommend installing a stand-alone ZooKeeper independently from dCache. Many Linux distributions have packaged ZooKeeper already.
ZooKeeper can be deployment in a redundant setup, typically on three or five servers, allowing respectively one or two instances to fail without loosing the service.
We have placed further information on using ZooKeeper with dCache at https://github.com/dCache/dcache/wiki/ZooKeeper.
Add user information to billing database
If using the billing database backend, the billinginfo
and
doorinfo
tables are extended with additional columns to represent
information about the user performing the transfer. This includes the
owner string, the primary FQAN, the UID and primary GID.
Upon downgrading to an earlier release, the schema must be rolled back
using the dcache database rollbackToDate
command. Doing so will
erase the data stored in the new columns.
Storage class was added when listing movers
The output of the mover ls
, st ls
and rh ls
commands have been
extended to show the storage class of the file being transferred,
flushed, or restored.
Obsoleted dcache.authn.gsi.crl.cache.lifetime
dCache no longer uses JGlobus and thus this property is no longer used.
New resilience
service
The resilience
service replaces the replica
service. The replica
service is still included in dCache 2.16, but we encourage sites that
rely on it to migrate to the resilience
service. The replica
service will eventually be removed from dCache.
In contrast to the replica
service, resilience
offers many
benefits, such as not relying on its own database and thus not
suffering from state synchronization issues, more flexible
configuration in terms of which files are to be replicated, not
relying on dedicated pools, more robust operation, works with any
retention policy, detecting and recovering broken replicas, allowing
temporary suppression of replication on particular pools, integration
with the alarm
service, and more.
Separate documentation for the resilience
service will be posted.
Overhaul of cell routing topology
dCache cell communication used to have a complex configuration system allowing arbitrary topologies of dCache domains to be formed. A hidden service called the location manager was embedded in all dCache domains and coordinated how connections between domains were established. Inside dCacheDomain the location manager deployed a simple UDP service that other domains used to discover how they should interconnect. To our knowledge all sites used the default star topology in which the dCacheDomain was the central messaging hub for all other domains.
In dCache 2.16 this has been completely reimplemented. The UDP based configuration discovery system previously used by location manager has been replaced by Apache ZooKeeper. The UDP based service is still supported in a legacy mode to allow rolling upgrades from earlier versions. dCache 2.17 will not support the legacy mode.
The location manager service no longer supports arbitrary communication topologies to be formed. Domains are configured individually to be either core domains or satellite domains, with the former being messaging hubs. The default is for dCacheDomain to be the only core domain. This maintains the previous default behavior.
The major new feature is the ability to have multiple core domains, i.e. multiple message hubs, with all other domains connecting to all core domains. This enables a redundant communication topology in which a core domain can fail and the remaining services can maintain connectivity through the remaining core domains.
We have published documentation about the dCache message passing system at https://github.com/dCache/dcache/wiki/Message-passing. This also includes information on redundant topologies.
The System
cell embedded into every dCache domain provides the new
traceroute
command which allows the route between two cells to be
discovered.
dCache Service Names
Traditionally central services in dCache have had well-known names like
PnfsManager
and PoolManager
. Other services – and the operator
through the admin
service – could communicate with those services
using the short well-known name rather than the longer fully qualified
cell address.
Starting with dCache 2.16 such services have a configurable service name instead. The service name typically coincides with the cell name, but they are no longer linked.
E.g. the pnfsmanager
service defaults to a cell name PnfsManager
(defined by the pnfsmanager.cell.name
property), which when
instantiated in a domain called namespaceDomain
may be addressed as
PnfsManager@namespaceDomain
. The pnfsmanager
service now also has
the new pnfsmanager.cell.service
property, which happens to default
to PnfsManager
too, allowing the cell to be addressed with the
shorter PnfsManager
name, just like before.
The difference is that these are no longer coupled. The cell name and the service name may be changed independently. This change applies to all central services in dCache, i.e. most services except pools and doors.
The cell.export
properties are all marked deprecated. Affected
services have the new cell.service
property.
More information about service names may be found at https://github.com/dCache/dcache/wiki/Message-passing.
Replicable services
Under the hood, the changes related to the new service names described above is much bigger than what may be apparent from the simple description. Several services may now be replicated. In such a setup, each instance of the service shares the same service name. This was the motivation for separating cell names from service names.
In dCache 2.16 not all services are replicable. Those that are define
a cell.replicable
property set to true
,
e.g. pnfsmanager.cell.replicable
. The dcache services
command
shows whether services are replicable or not.
This feature is considered experimental. Documentation will be published at https://github.com/dCache/dcache/wiki/Replicable-services.
Split of SRM into a frontend and a backend
The SRM service has been split into a frontend service called srm
and a backend service called srmmanager
. The frontend listens to the
SRM TCP port and is the endpoint clients communicate with. The backend
does all the request processing, including scheduling and persistence
in the SRM database.
This change allows multiple frontends to be deployed in a load
balanced setup while using a single srmmanager
backend. Furthermore
the srmmanager
backend may be replicated for improved fault
tolerance, scalability and rolling upgrades, but this is currently
considered experimental.
Upon upgrading, the simplest course of action is to add an
srmmanager
service in the same domain as the existing srm
service. It is recommended to place srmmanager
in from of the srm
service to ensure the backend starts before the frontend.
Note that many configuration properties previously defined for the
srm
service now need to be defined for srmmanager
instead. Such
properties have been renamed accordingly. Use dcache check-config
to
validate your configuration prior to starting dCache.
Also note that if the export point of the namespace is changed using
srmmanager.root
(formerly srm.root
), then a similar change must be
applied to srm.loginbroker.root
. Both default to the new
dcache.root
property, so in most deployments it may be easier to
change the export point for doors by setting that property instead.
Upon upgrading, it is important that the scheduler ID of the
srmmanager
matches the original cell name of the srm
service. This
is because entries in the database are tagged with this ID and using a
different value means that old SRM requests are not cleaned upon
restart. The default scheduler ID matches the old default cell name of
the srm
service and thus if deploying srm
and srmmanager
on the
same host and if the original service did no use a custom cell name,
no additional configuration changes are needed on upgrade.
Also note that the default values for the
srm.cell.limits.message.threads.max
and
srm.cell.limits.message.queue.max
properties have been updated. The
default total number of queued messages remains the same, but (by
default) only a single thread is operating on those messages.
More documentation will be published at https://github.com/dCache/dcache/wiki/Replicable-services.
Chimera auto-discovers the correct database driver
Chimera used to have the chimera.db.dialect
property to select the
database driver. The correct driver is now selected automatically and
chimera.db.dialect
and similar service specific properties are now
deprecated.
New frontend
service
The new frontend
service offers a RESTful webapi and a modern user
facing web interface. The service is currently fairly limited, but we
expect it to be aggressively extended over the next feature
releases. It currently offers a RESTful API to navigate the name
space, and a browser interface to navigate this name space. The
regular webdav
service is used as a backend for file transfers.
More documentation on this service will be published soon.
New property to define name space export point
Many doors now respect the new dcache.root
property to define the
directory in the name space that is exposed to users. If defaults to
the chimera root directory. Note that this also applies to FTP
doors. Thus by default, FTP doors no longer use the per account root
directory defined in gPlazma. To restore the old behavior, set
ftp.root
to the empty value.
gPlazma LDAP plugin now supports authenticating LDAP servers
The gplazma
LDAP plugin can now authenticate with the LDAP server
using the new gplazma.ldap.auth
, gplazma.ldap.binddn
, and
gplazma.ldap.bindpw
configuration properties.
Add user information to active transfer pages of httpd
The various active transfer pages now show the UID, primary GID and primary FQAN of the user doing a transfer.
JSON output for active transfers page
The classic httpd
web pages provides JSON output for ongoing
transfers. This complements the classic txt output. The information can
be accessed with something like curl -s
http://localhost:2288/context/transfers.json
.
Pin manager database backend was rewritten
The pinmanager
database backend was rewritten to no longer use the
DataNucleus ORM. Minor schema changes are applied during upgrade. Upon
downgrade these schema changes must be rolled back using the dcache
database rollbackToDate
command.
The use of Infinity
as a pool size is deprecated
Pools could be configured with Infinity
as a size to indicate that
the size should be derived automatically from available disk
space. This value is now deprecated and the pool.size
should be left
empty instead for automatically sized pools.
Improved poolmanager
configuration reload
Several race conditions have been fixed in the reload
command of the
poolmanager
service. The command now also preserves the already
cached pool state, thus reducing the side effects of reloading the
poolmanager
configuration.
Improved publishing of pool state
The poolmanager
service provides information about pools to several
other services. This has been refactored to use publish-subscribe
messaging rather than have each service pool poolmanager
for
updates. This change also allows poolmanager
to publish updates on
pool status changes immediately, i.e. pools going offline or coming
back online are immediately propagated to e.g. spacemanager
and
resilience
.
/var/lib/dcache
is owned by the dcache
user
When using the Debian package, /var/lib/dcache
was owned by user
dcache
all along. This is now also the case for the RPM package.
SRM 1 support has been dropped
The dCache srm
service no longer supports version 1 of the SRM protocol.
The internal cells of the transfermanagers
service have been merged
The RemoteTransferManager
and CopyManager
cells have been
merged. Several properties related to the CopyManager
cell have been
marked obsolete. Some of the runtime configuration commands of these
cells have been dropped.
Added caching of gPlazma mappings in webdav
service
The WebDAV service now caches gPlazma mappings. See
webdav.service.gplazma.cache.size
and
webdav.service.gplazma.cache.timeout
for the new configuration
properties.
Improved pool queue plots in webadmin
service
The pool queue plots in the webadmin
service have been restructured
and are now easier to use.
Limited OpenID Connect support
An OpenID Connect plugin (oidc
) for gPlazma was added as well as
limited support in the webdav
service to accept OpenID Connect
bearer tokens. Separate documentation on how to make use of this will
be published later.
Generic gPlazma mapping plugin
A new generic mapping plugin (multimap
) was added to gPlazma. It
allows mappings from any principal to multiple other
principals. Currently only dn, oidc, username, uid, gid, kerberos,
email principals are supported. An example configuration file looks
like
"dn:/C=DE/O=Hamburg/OU=desy.de/CN=Kermit The Frog" username:kermit
oidc:googleoidcsubject gid:1000,true uid:1000
"dn:/C=DE/O=Hamburg/OU=desy.de/CN=Kermit The Frog" uid:1000
"dn:/C=DE/O=Hamburg/OU=desy.de/CN=Kermit The Frog" uid:1000 gid:500,true gid:100
Cell address of the info
service is no longer hard-coded in httpd
The new httpd.service.info
property allows the cell address of the
info
service to be configured in the httpd
service. Sites with
custom httpd.conf
definitions will have to update the configuration
such that the info alias looks something like
set alias info class org.dcache.services.info.InfoHttpEngine -- -cell=${httpd.service.info}
NFS is better
Better spec compliance, faster.
Changelog from 2.15.0 to 2.16.0
- 5c5c643
- [maven-release-plugin] prepare release 2.16.0
- af2072d
- pool: Fix race in pool initialization
- 9488367
- Revert “srmmanager: Enable multiple instances to share a database”
- a7aecbf
- pool: Close stores after use in meta data utilities
- 1cd8d59
- pool: Fix regressions in meta data utilities
- 9771c6f
- Disable OCSP by default
- 24524ae
- gplazma: Add PrefixRestriction and use it to validate root and upload paths
- cf55408
- srm: Fix listing of completed list requests
- 5b3a4ef
- billing: re-implement population of aggregate tables
- e4dc028
- billing: Fix NoClassDefFoundError when invoking dcache billing command
- b42d51e
- httpd: Fail gracefully in case of missing options
- 70c3fe7
- httpd: Update cell info view for named queues and make srmmanager monitored
- cb677e0
- cells: Suppress RejectedExecutionException on shutdown
- 303397a
- cells: Fix NPE during shutdown
- d68c993
- zookeeper: Document ports used by embedded zookeeper
- 360c2ab
- ftp: Fix regression in resolving path
- 7449f96
- frontend: Expose dCacheView configuration in dCache configuration
- 789d5f4
- zookeeper: Add automatic purging of old snapshots
- 59f6e8e
- dCacheView: new end-user web interface for dCache
- f92249c
- api: Rename service to frontend
- 3846957
- api: Move rest API to separate service
- 0a0f4cb
- billing: fix issue with database change rollback
- 0033793
- srmmanager: Make scheduler ID configurable and compatible with older versions
- 0c86fb7
- httpd: restore original fields and format to the TransferObserverV1 ascii output
- 8deddbb
- [maven-release-plugin] prepare branch 2.16
- c39133d
- httpd: fix three small null-related bugs in accessing active transfer fields
- 4236e4a
- chimera: fix writing into file level
- 43fac20
- Work around races in Curator PathChildrenCache recipe
- c7020ce
- gplazma2-ldap: add support for simple authentication
- 1987838
- gplazma: improve log messages for oidc auth plugin and multi-map plugin
- c3c741b
- webdav: ensure WWW-Authenticate header is present when appropriate
- b08065e
- webdav: add support for ‘client’ template in StringTemplate file
- fc174cf
- srmmanager: Enable multiple instances to share a database
- 2b1ab17
- poolmanager: Don’t block message thread in rebalancer
- bda62f9
- script: Fail when pattern doesn’t match any domains
- 890ca05
- dcache: fix heap dump to simple file names
- f5756bc
- script: Make dump heap –force work for non-root processes
- b247cd8
- gplazma: Added a new map plugin with support for
- b13e271
- httpd: add VO-level information to Active Transfer representations
- 818da05
- common: add parsing and formatting utilities for time duration
- c2f407e
- gplazma: Add an authentication plugin to gplazma for validating OpenID Bearer Token
- 5846cae
- webdav: Support for accepting OpenID Connect Bearer Tokens in HTTP requests
- 6fecc55
- webdav: Separate the authentication from checking of root/home path from AuthenticationHandler
- b36c76f
- pnfsmanager: Revert “Use a single shared request queue for threads”
- 4f60561
- script: Add missing she-bang
- cfd6bfc
- pnfsmanager: change the timing of PnfsModifyCacheLocationMessage relays to follow reply to pools
- f4dcb5b
- pnfsmanager: Move flush notification out of chimera transaction
- b8caf01
- pom: Update various third party dependencies
- bcd71cc
- srm: Fix regression in get request tokens and get request summary processing
- dcafd19
- spacemanager: Make space manager replicable
- 205cb6d
- pinmanager: Make pinmanager replicable
- 36ee5db
- pinmanager: Drop DataNucleus
- 29eee00
- pnfsmanager: Fix precondition failure when querying the root path with PG backend
- 8a8687c
- pool: Fix p2p cancellation
- de98dbe
- srm: Do not expose TURL before request is ready
- 0ca62f6
- pool: Don’t log illegal state exception on migration job cancellation as a bug
- b5c7213
- pool: Reduce lock contention on mover creation
- 358dc91
- pool: Fix race condition in request scheduler
- 79192a2
- pool: Avoid reverse DNS lookup in HTTP mover
- 5317659
- resilience: fix handling of broken file where only one copy exists
- a7730ed
- billing : add primary key on date columns to daily tables
- 0ae362d
- srmclient: fix execution on SL–7 / CENTOS–7 / RHEL–7
- 5201a7e
- nfs: mover: use direct-io buffer for reading from the disk
- bb4ad95
- resilience: add filter support for matching null pool values on file cancel and list commands
- 05e8687
- resilience: fix two admin command types
- 84480cf
- billing: additional fixes to insert triggers
- 9683cac
- pom: update nfs4j to 0.12.2
- 3e3e03f
- webdav: remove accidently inserted comment from webdav.xml
- b4c7e0b
- resilience: restore liquibase
- 255a77b
- zookeeper: Fix regression in processing the listen property
- 0409589
- pool: add storage class to mover ls and rh/st ls output
- 9be2532
- webdav: fixed broken HTTP Basic Authentication
- 451be4b
- srmmanager: Allow horizontal scaling of the srm backend
- 27ceae7
- doors: Add properties to allow loginbroker address, port and root path to be configured
- fa47496
- srm: Split into frontend and backend services
- 373a3b9
- pool: Suppress logging of delivery failure of DoorTransferFinished
- ddcac02
- pinmanager: Drop compatibility with 1.9.11
- 01f040e
- script: Add input validation to size in dcache pool create command
- cb4d1b8
- cells: Shutdown ZooKeeper client on shutdown
- 846608e
- doors: Introduce common root property
- 672408d
- topo: Declare topo service as replicable
- e3f6b1d
- gplazma: Declare gplazma service as replicable
- 597a2c8
- pnfsmanager: Declare pnfsmanager service as replicable
- 2481943
- info: Declare info service as replicable
- 77d28c1
- script: Add replicable column to dcache service output
- 1ccea45
- pools: add CorruptFileTopic to excluded destinations on no route to cell
- 743c3d3
- system-test: update disposable-CA generated credentials
- c6b6765
- chimera: remove leftovers from db dialect removal
- ea56b53
- billing: modify insert to use single queue and thread pool executor service
- 487095d
- pnfsmanager: fix atime update regression
- 371037a
- chimera: auto-discover db dialect
- 494a2a4
- dcache-restful-api: fix code errors
- 8923530
- srm: Drop SRM 1 support
- 0d8cba5
- Introduce consume properties for services
- 05b5595
- cells: Implement support for named queues in cells
- dbf3bf3
- transfermanager: Merge CopyManager and RemoteTranferManager
- 619dd31
- poolmanager: fix double printout of poolmanager configuration
- 00aecbd
- resilience: add limits node to appropriate properties
- cc6f17c
- cells: Fix logging formatting string
- 6cf6857
- cells: Restore compatbility with legacy domains
- caf526a
- cells: Controlled removal of downstream default routes in case of core domain shutdown
- 5a76da7
- cells: Strip out source routing information when reverting a cell path
- ceb45d3
- cells: Delay installation of second default route
- bafd493
- cells: Add traceroute command to CellShell
- ba24710
- cells: Drop old routing manager and extend the new to handle all routes
- e22a6c0
- cells: Clean up tunnel shutdown
- c252c02
- cells: Drop support for arbitrary cell topologies
- 4f8f2f0
- cells: Add routing manager for routes involving core domains
- fb10403
- cells: Install default route to core domains
- eed7f63
- cells: Do not forward messages from core domains
- d89f01b
- cells: Allow domains to be marked as core domains
- d2cfa9b
- cells: Extend routing manager to work with multiple upstream domains
- c80b048
- cells: Allow multiple default routes
- 641aca8
- Motivation:
- 27e1c75
- dcache-restful-apiUpdated: Basic/draft version of Rest-Api functionality (UpdateRestfulAPI)
- a30e873
- webadmin: remove stale pool queue database files and plots
- f9e35c8
- webadmin: change pool queue plots to drill down from pool group data
- e3c84d9
- resilience: eliminate abstract commands and reintegrate annotation text
- fe03e75
- resilience: add command to find inaccessible files
- 722537f
- Add the dCache bird to the github README.md
- ce8e369
- Fix unit test failures after recent commit
- ecf33d6
- webdav: fixed webdav.xml
- f715deb
- cells: Avoid bouncing message on no-route errors in System cell
- 77eeeeb
- webdav: Changed the login strategy from Remote Login Strategy to CachingLoginStrategy
- fe46598
- webadmin: add destination to pool manager cell stub
- 6d8a4be
- srm: Suppress message delivery failures for credential service announcements
- d555a0e
- cells: Fix tunnel shutdown order
- 3595a50
- cells: Do not subscribe to topics in per-session door instances
- 8eab1dc
- cells: Allow local topic delivery
- 725899b
- cells: Let type of map reflect content
- a0f7366
- resilience: remove counts command
- d4ca39c
- webadmin: change PoolMonitorCollector to use SerializedPoolMonitor messages
- c98b0b2
- resilience: eliminate the inaccessible file api and handler
- d0001ae
- nfs: fix NPE when remove sent to billing
- bdc989f
- nearline storage: allow masking of HSM return codes from billing and logging
- 0399fe5
- nfs: fail quickly if we know that file is offline or lost
- e292832
- resilience: rename ‘pnfs’ objects, fields and methods to ‘file’
- 6fbe461
- resilience: rename ‘pnfs’ command to ‘file’
- 14295d4
- resilience: move source out of dcache and into dcache-resilience
- aedd966
- srm: Fix regression in SRM startup
- fbcece5
- Resilience: eliminate dead null-check
- efa242b
- Resilience: eliminate stray autocommits
- 552fc76
- chimera : fix trigger that populates data in t_locationinfo and t_inodes on insert or update of t_level_4
- 9692cb8
- chimera : fix trigger that populates data in t_locationinfo and t_inodes on insert or update of t_level_4
- 047e1d1
- common: fix ChecksumType.toString()
- 72a9d68
- http: avoid dns reverse lookup on HttpProtocolInfo#toString()
- 64ae699
- rpm: /var/lib/dcache must be owned by user dcache
- 8901251
- chimera: do not update ctime on atime only attribute update
- f48a423
- resilience: allow system-test to support old replica manager and new resilience service
- 28b010b
- Remove JGlobus properties
- f6c4973
- dcache-vehicles: Drop pre–2.12 serialization compatibility
- 817da9b
- srm: Delay announcing credential service after cell start
- 5bca6f5
- resilience: remove the nores option on pool enable/disable
- 109f9ca
- resilience: eliminate unnecessary executor service classes
- 100d1e9
- psu: remove super.toString() from pgroup toString
- 6b6e93a
- resilience: change handling of Corrupt/Broken files to use StateChangeListener
- 4fc0dee
- pool: Improve scalability of nearline storage subsystem
- 6c34485
- pool: Fix regression breaking hopping mananger
- 5ed88b3
- fix regression from 859431218e3d7cdddcf906144c1b8928bb625fed
- 555192b
- resilience: remove unnecessary selector interface, and use optimized Iterables.get
- 59fcf4a
- resilience: refactor ExceptionMessage
- 268d1a1
- resilience: remove AtomicInteger from NonReindexableList
- 367e4b8
- cells: Fix route removal in routing manager
- a0225c4
- cells: Preserve port number in lmd listen command
- bacf3c2
- resilience: eliminate CellStubFactory class and uses
- 5364603
- resilience: remove subscribe parameter from pnfsmanager batch script
- 2d6c612
- rpm: explicit requre ’which` package
- 070ccad
- doors: Preserve name when publishing the address of doors
- c8e63dc
- cells: Port location manager to ZooKeeper
- 5c270ae
- Add ZooKeeper support
- 81f3605
- Support asynchronous cell startup
- f680fed
- resilience: fix unit test broken by previous commit
- 01c5b4c
- resilience: prevent scan of uninitialized pool
- fdd6c30
- resilience: do not return null if tag map is null
- 69cc5b9
- Resilience, Version 2, March 2016
- 655671a
- revert 46bac35711e4dfef63b461876f1f4d57aa1e27c7 instead of adding changelog that adds columns, modify existing changelog that creates HSQL billing database content
- 46bac35
- added billing changelog also to HSQLDB plugin directory RB : https://rb.dcache.org/r/9142/ Acked-by : Gerd Behrmann behrmann@ndgf.org Target : trunk
- f927b94
- srm: add short request lifetime work-around
- fefeef7
- pool: Fix unit test failures
- 8c6bdeb
- cells: Set correct logging context in cell callbacks
- 6ad7cd3
- pool: Fix compilation failure
- 8d25340
- cells: Improve robustness of message time to live
- 320b00f
- cells: Fix erroneous reuse of message envelope in location manager registration
- 2b8493c
- cells: Drop support for persistent location manager maps
- 2e6bc05
- cells: Isolate location manager config interpreter
- 8f21dae
- cells: Fixed race conditions in location manager
- d3c4cf0
- cells: Document and rename onerror commands
- c4d4e4f
- pool: Refactor representation of pool size
- d1aaf74
- libs: update to nfs4j–0.12.1
- aa8256c
- cells: replace most CellInfoAware declarations with CellIdentityAware
- 16dd734
- cells: remove call arguments from CellEndpoint
- 0a7d8c5
- cells: refactor discovery of CellInfo from CellEndpoint
- dc5cfdd
- billing: add primay FQAN to doorinfo and billinginfo, also add owner, UID, GID to billinginfo
- e2d6113
- chimera: Fix regression in inheriting ACLs on directory creation
- cda7d91
- pools: fix bug in set breakeven command
- 5b0cf32
- cells: Limit commands exposed over UDP
- 330a982
- cells: Fix race conditions in counters of LocationManager
- 474afa3
- cells: Fix race conditions in LocationManager
- 057df19
- cells: Clean up LocationManager
- 57df18d
- authorisation: reintroduce ReadOnly class
- 14afb0d
- cells: Reformat LocationManager
- 2ab15eb
- cells: admin interface cmds -> version, check, ping and sleep command (method to class conversion)
- cb6ef5a
- Reimplement FsPath
- 8a5ab2c
- ftp: fix several problems with restrictions
- c14b150
- PnfsManager: fix restriction to prevent directory items
- f36bf64
- srm: add request lifetime to error message
- a3b9b2d
- Remove various redundant code constructs
- e28ccea
- Add missing serialVersionUID fields
- 91157dd
- srm-client: Fix type mismatch in third party copy client
- 9f9bcbe
- srm-client: Fix type error when aborting copy requests
- ead8df6
- pool: Fix type mismatch in collection to array conversion
- e33b4d6
- Use expression lambdas rather than statement lambdas where applicable
- 652e583
- Use lambda expressions where applicable
- bdb8c32
- Remove unused imports
- 56fc8af
- Make fields final
- e8b82bd
- Delete dead code
- 07d807c
- Fix static member access vis instance reference
- 20003b9
- Removed redundant local variables
- b4250b8
- PnfManager: implementing annotated command syntax for admin commands (PnfsManagerV3)
- 86002db
- poolmanager: Fixed typo
- 43cef58
- pnfsmanager: Fix regression in SRM billing entries
- d15e95e
- poolmanager: Allow fallback on write when pools are full
- 5f7ac60
- pool: Fix filtering of options in script HSM driver
- 369bcdf
- Replace StringBuilder with String
- 2c51832
- srm: Disable delegation on srmCopy to or from other SRMs
- 6faedb3
- srm: Return SRM_INVALID_PATH when target directory is a file
- 77b450d
- pnfsmanager: Fix race leading to transaction failures in Chimera
- d663e4e
- system-test: Restrict port range to avoid conflicts with services
- a2e697d
- Remove unnecessary semicolons
- 02100c8
- Fix deprecation and other warnings
- 95d0966
- Remove broadcast compatibility with pre–2.13 pools
- 11ade1b
- ftp: Fix race on short transfers
- 2fadc5b
- poolmanager: Publish pool monitor on update
- a1cce6f
- poolmanager: Add callbacks to pool selection unit
- 094e17f
- poolmanager: Preserve pool state on configuration reload
- f74fd78
- poolmanager: Fix race condition in pool heartbeat registration
- 8f47342
- poolmanager: Fix race when reloading configuration
- 0b6ee57
- poolmanager: Add topic to publish pool monitor
- 4ac82f1
- common: Username should not be empty string
- 91b4696
- nfs: notify billing on file removes
- 3e94b45
- Fix compatibility issues with ctlcluster
- 093579e
- poolmanager: Acquire read lock when serializing cost module and partition manager
- be1d495
- poolmanager: Acquire read lock when serializing pool selection unit
- 77d6b41
- poolmanager: Fix race condition in pool selection unit
- 6e94d36
- webdav: fix 404 error if attempting to delete a nonexistent file
- b9ec799
- poolmanager: Make pm set accept all options
- f2cdf27
- pool: Fix obsoletion of pp set pnfs timeout command
- f1886e6
- pool: Replace -si option with -storage in rep ls
- 7aa645e
- gplazma: Mark gplazma.cell.limits.threads as deprecated
- 9fd38d2
- spacemanager: Fix pool monitor fetch race during startup
- 1f28f55
- doors: Fix contention point and race in login broker publishing
- 806d11e
- [maven-release-plugin] prepare for next development iteration