dCache 9.2 Release Notes

Highlights

Performance improvements for the concurrent directory creation and removal

Acknolagements

Many Thanks to Lars Jansse and Sandro Grizzo for their contributions.

Incompatibilities

prior to version 9.2 dCache NFSv4.1 door will publish only nfs4_1_files layout. Now on the door publishes all available layout types
dropped reference count tracking for directory tags
dropped support of java options chimera_soft_update and chimera_lazy_wcc in favor of chimera.attr-consistency property
If upgrading from 8.2, be sure to read also the release notes for 9.0 and 9.1, important changes are described there. See the 9.2 migration guide for more complete information
With version 9.2.8+, empty and non-existent banfiles will be treated the same

Release 9.2.32

bulk

Restore the ability to use absolute paths when using bulk REST API.

cells

Cells will always try to re-establish dead tunnel connections.

qos

This fix will take care of the ‘Attribute is not defined: QOS_POLICY’ error in logs.

Changelog 9.2.31..9.2.32

3f4b9aca2e: [maven-release-plugin] prepare release 9.2.32
6794baa4c0: bulk: handle absolute/relative paths in uniform fashion
2010271998: qos: QOS fails with ’Attribute is not defined: QOS_POLICY’
c8eab52302: cells: always try to re-establish dead tunnel, unless stopped
a4c0df8587: [maven-release-plugin] prepare for next development iteration

Release 9.2.31

jvm

Removed default JVM option UseCompressedOops so dCache works with large heap size.

Changelog 9.2.30..9.2.31

5603028626: [maven-release-plugin] prepare release 9.2.31
b9e3025d27: jvm: drop UseCompressedOops JVM option
ba393f0db2: [maven-release-plugin] prepare for next development iteration

Release 9.2.30

cells

When Zookeeper updates core domain infos, dCache will first kill the existing cell tunnels and then later try to read and parse the new value. If the new value is an empty string (for whatever reason), parsing will fail, but a new connection will not be established. This now fixed.

Changelog 9.2.29..9.2.30

8aa2e85a7d: [maven-release-plugin] prepare release 9.2.30
9ceac4d97c: cells: ignore empty core domain uris propagated by zk
c7b8a50b22: [maven-release-plugin] prepare for next development iteration

Release 9.2.29

tape

Users reported 2 day pin lifetime on staged files (which is a default) despite specifying different values. This is now fixed.

Changelog 9.2.28..9.2.29

0d911615bc: [maven-release-plugin] prepare release 9.2.29
ab9a74538d: tape REST api: additional fix tohandling of prefixed paths
499959a572: [maven-release-plugin] prepare for next development iteration

Release 9.2.28

xroot

Return destination address (that is, the haproxy address) if xrootd.enable.proxy-protocol=true is set, instead of the actual door address.

Changelog 9.2.27..9.2.28

57579ad5de: [maven-release-plugin] prepare release 9.2.28
95cad94b15: xroot: handle haproxy and checksum command
89133016c8: [maven-release-plugin] prepare for next development iteration

Release 9.2.27

CI

Pipeline optimizations.

pool

Fix double decrement on active hsm requests. This addresses the issue of pools stopping flushing to tape with “Negative number of active requests”.

When a thread performing I/O gets interrupted, then an InterruptedIOException might be thrown. A DCAP mover will treat and propagate such an exception as a disk I/O error, thus disabling the pool. This fix reduces false positive disk I/O errors.

Changelog 9.2.26..9.2.27

c2f32fc913: [maven-release-plugin] prepare release 9.2.27
d8d5541241: pool: don’t treat InterruptedIOException as a disk IO error
8e376b7181: pool: fix double decrement of hsm requests
1215548076: ci: split container image registry and repository
0863dacffa: [maven-release-plugin] prepare for next development iteration

Release 9.2.26

bulk

The current release fixed broken command activities.

Changelog 9.2.25..9.2.26

9e9ea4fa6d: [maven-release-plugin] prepare release 9.2.26
6664395575: ci: run spotbugs only on master branch
b075c04d19: bulk: fix broken command activities
9c0b160e42: [maven-release-plugin] prepare for next development iteration

Release 9.2.25

bulk

When specifying an empty target, bulk proceeded to process the request instead of failing fast. This is now fixed.

gplazma

A previous commit, leading to the last bugfix releases being blacklisted, introduced a regression in the multimap plugin. Where the ‘op’ principal type is used, logins will fail with dCache logging a stacktrace like java.lang.RuntimeException: Failed to create principal: java.lang.NoSuchMethodException: org.dcache.auth.OAuthProviderPrincipal.<init>(java.lang.String). This is now fixed.

Changelog 9.2.24..9.2.25

8fecee871f: [maven-release-plugin] prepare release 9.2.25
4d14890f7d: gplazma: fix broken commit d74d9568167f4
729ce8f76a: gplazma: multimap fix op regression
5708994ab6: bulk: check targets for empty strings
6043a20af4: [maven-release-plugin] prepare for next development iteration

Release 9.2.24

doc

Better documentation clarifying OIDC provider ID and issue claim.

webdav

Bumps org.eclipse.jetty:jetty-servlets from 9.4.51.v20230217 to 9.4.52.v20230823. Thi has fixed java.lang.NullPointerException: null error on WebDAV Domain due to jetty 9.4.51 bug.

Changelog 9.2.23..9.2.24

bfaeae101e: [maven-release-plugin] prepare release 9.2.24
6394fbecd4: github: add action for atumatic github-release
6ab84e51ea: gplazma alise initial version of plugin
db6ea0f464: common: add issuer URI to OAuthProviderPrincipal
09e5a49937: build(deps): bump org.eclipse.jetty:jetty-servlets
eb858c9725: docs: clarify OIDC provider ID and issue claim
7e6098c30e: [maven-release-plugin] prepare for next development iteration

Release 9.2.23

CI

Improve our CI pipleline.

core

Pool migration : fixed behavior of migration copy with -replicas=n (where n>1) to behave as expected.

Libs

Improve and maintain our testing library.

Keeping libraries up to date.

TAPE-API

Release by relative path works.

Changelog 9.2.22..9.2.23

63803072d4: [maven-release-plugin] prepare release 9.2.23
a7aa1b3689: WLG TAPE REST API: fix handling of frontend.root in release API
a219e4457a: gitlab: mirror tags
3de9e9301c: libs: update mina-sshd to version 2.13.1
96ef3777ca: Fix issue with infinite replicas when replicas > 1
6b3ff705aa: github: change mirroring action
7bd322edc4: ci: use shorter k8s namespace names
abe0f45324: pom: add exta java option to run powermock test under java17
118a5cc34d: ci: sync CI pipeline with master
fb954fb8ef: ci: use desy nims repo for CentOS7
562668fc1e: ci: use almalinux9 for rpm install test
2991f5ef12: [maven-release-plugin] prepare for next development iteration

Release 9.2.22

chimera

Now it is possible to n read/write CTA and migrated Enstore files without having to set hsmInstance tag in the full directory tree.

frontend

The current release fixed the issue about failures to invoke WLCG tape API to stage files on files relative to {webdav,frontend}.root

It is now possible to call stage API successully on paths ralative to frontend.root.

Changelog 9.2.21..9.2.22

b7f34147e4: [maven-release-plugin] prepare release 9.2.22
743c52948e: pom: compile code in the same jvm as maven
8a428d3316: chimera: add CTA HSM StorageInfo extractor
69f2f109ca: frontend: handle frontend.root variable properly
5652f98acf: [maven-release-plugin] prepare for next development iteration

Release 9.2.21

chimera

Adapt existing uri_encode function so that the Enstore HSM script does not fail and the files go to tape successfully.

ci

CI pipeline improvements.

dependencies

Update build dependency to be compatible with modern tools.

Changelog 9.2.20..9.2.21

f48df60d4e: [maven-release-plugin] prepare release 9.2.21
9691e0096c: dependencies: update modernizer plugin to be compatible with maven 3.9
6e2c1186bd: chimera: fix uri_encode to handle special characters
f52a0ee2f4: ci: add property to control upload options
24759f2e3c: [maven-release-plugin] prepare for next development iteration

Release 9.2.20

bulk

Bulk truncates path to 256 characters and this seems to be causing problemsd.

This is now fixed.

pool

With CentOS7, the client sends the last request, such that offset+count == filesize. REHL9 sends the last request so that the count is multiple of 4096, which is legal.

The current release fixed miscalculation of offset on short read.

Changelog 9.2.19..9.2.20

f3b6d8e7c9: [maven-release-plugin] prepare release 9.2.20
8d7d4661d2: ci: disable srmcp test
a710037a84: ci: use python3 for robot test
4158c80ae5: bulk: do not truncate target paths
9b9714f267: ci: pinpoint postgres helm version
15ea025a3f: pool: fix miscalculation of offset on short read.
39bb2f532d: [maven-release-plugin] prepare for next development iteration

Release 9.2.19

chimera

Fixed a bug that made it possible to create a loop on directory move.

Changelog 9.2.18..9.2.19

368ce200f4: [maven-release-plugin] prepare release 9.2.19
465e00e82a: chimera: fix loop creation on directory move
9a46793677: ci: drop –ftp-create-dirs option (as we switch to https)
c0357f4f81: [maven-release-plugin] prepare for next development iteration

Release 9.2.18

webdav

When running multiple Remote Transfermanager, two transfer that were started simultaneously had the same ID which led to the second transfer becoming orphan after the first one completed. This is fixed now.

Changelog 9.2.17..9.2.18

9cb77bc505: [maven-release-plugin] prepare release 9.2.18
b0e6eaf759: ci: eplocitly specify kubernetes namespace
3f06a7a6ad: ci: fix typo
5cf1c757c9: webdav: use transfermanager+id to identify TPC transfer
73fbb8b77f: docs: update oidc chapter to explain trust anchors
25fe810ba9: [maven-release-plugin] prepare for next development iteration

Release 9.2.17

common

The out-of-box version of ProxyCSRGenerator#generate from CAnL uses SHA1 for proxy delegation, which is banned by modern OSes.

this now fixed and RHEL9 clients works with proxy delegation without enabling SHA1.

xroot

Switch to xrootd4j–4.6.0, ew major release with bug fixes and enhancements including the reload TLS certificate.

Changelog 9.2.16..9.2.17

42faeb1884: [maven-release-plugin] prepare release 9.2.17
973a06353d: xroot: switch to xrootd4j–4.6.0
3eeefe01c2: common-security: add custom version of ProxyCSRGenerator#generate
177d75f696: [maven-release-plugin] prepare for next development iteration

Release 9.2.16

qos

Admins now have the option to disable role based authorization for QoS transitions.

webdav

dCache now supports HTTP-TPC transfers where the dCache-local path includes HTTP reserved characters.

Changelog 9.2.15..9.2.16

c4035193dd: [maven-release-plugin] prepare release 9.2.16
4f8f2099ed: qos: add flag to enable/disable role based authorization for transitions
92c2350067: webdav: httptpc percent-decode local path
e08dc2a171: [maven-release-plugin] prepare for next development iteration

Release 9.2.15

bulk

Some of the format strings that manage the formatting of command results were sending to the user expect a different number of parameters than provided. This is now fixed and the returned bulk info strings return all expected values.

info

DGAs send messages to other dCache cells to discover their current status. These messages have a hard-coded one second timeout.

Some queries are data-intensive and could take longer than one second to build the answer, resulting in no information being provided.

this is now fixed and the info service will now wait longer for a cell to respond to a query for information.

nfs

Grizzly memory management uses heap memory size fraction even if direct memory is used (issue 7529). This issue has been fixed and dCache starts with 256m of direct memory.

Changelog 9.2.14..9.2.15

04a4d03830: [maven-release-plugin] prepare release 9.2.15
253e07ab7d: info: use DGA refresh rate as message timeout
58cc1f30f5: nfs: calculate desired memory fraction for correct memory allocation
f51fcde6db: bulk: fix divergent command params format strings
cfa39a1960: [maven-release-plugin] prepare for next development iteration

Release 9.2.14

cells

The error reporting on tunnel disconnect has been fixed.

srr

The current release fixed empty path annotation warning.

tape-api

Request supplied paths are now mapped based on frontend.root.

Changelog 9.2.13..9.2.14

7ff7f5ba6a: [maven-release-plugin] prepare release 9.2.14
f9ef94a8dd: tape-api: map request supplied paths based on frontend.root
2c8a17f3ad: srr: fix “empty path annotation” warning
fa7cd0ec2e: cells: fix error reporting on tunnel disconnect
bb4b067fb9: [maven-release-plugin] prepare for next development iteration

Release 9.2.13

core

This patch has been shown to fix the issue in which poolmanager will in some instances not load parts of its configuration.

packaging

Packaging infrastructure improvements.

Changelog 9.2.12..9.2.13

3d735b9766: [maven-release-plugin] prepare release 9.2.13
23469021c9: Revert “poolmanager: delete property for switchingon/off caching for psu”
861b5a104c: ci: use minimal almalinux–9 to upload packages
453fa408a8: [maven-release-plugin] prepare for next development iteration

Release 9.2.12

pool

Buffer initialization is now done at pool start instead of first NFS read request. This leads to more predictable buffer initialization.

Changelog 9.2.11..9.2.12

7ee8097efc: [maven-release-plugin] prepare release 9.2.12
fee2057919: ci: use dtzar helm-kubectl image
481408f770: dcache-core: fix psu logging
764779040b: pool: mover grizzly IO buffer initialization into NfsTransferService
82cd451b32: [maven-release-plugin] prepare for next development iteration

Release 9.2.11

webdav

Users have reported severe slow down when runnig listing using webdav.

This is currently fixed.

Changelog 9.2.10..9.2.11

46f47153a8: [maven-release-plugin] prepare release 9.2.11
da22a29c73: webdav: fix slow listing
2d73d8ba98: [maven-release-plugin] prepare for next development iteration

Release 9.2.10

webdav

A recent path introduced support for adding the Link HTTP response header, according to RFC 6249. Unfortunately, the code added the Link header for all requests, not just the intended GET and HEAD requests.

Additionally, due to peculiarities of how Milton generates PROPFIND results, the Link header is added multiple times: once for each subdirectory.

This results in the HTTP response headers taking up too much space and dCache failing the request, returning a 500 status code.

This regression has been now fixed where PROPFIND requests would fail if the directory contains too many subdirectories. The cut-off point depends on the length of the URL the PROPFIND request targets.

Changelog 9.2.9..9.2.10

f58389b35b: [maven-release-plugin] prepare release 9.2.10
f15e622cb4: webdav: fix link header
3125694915: [maven-release-plugin] prepare for next development iteration

Release 9.2.9

gplazma

A bug is fixed in gPlazma that is triggered when users attempt to authenticate with broken gPlazma configuration or a gPlazma plugin’s configuration is broken.

Changelog 9.2.8..9.2.9

ffef10d130: [maven-release-plugin] prepare release 9.2.9
c7993fe0f3: gplazma: fix NPE if gPlazma is rejecting all logins
42d497e172: [maven-release-plugin] prepare for next development iteration

Release 9.2.8

chimera

A recent commit as a side effect resulted in failure to upload a file to an xroot door with the overwrite (“-f”) option, returning okay but removing the original file without creating the new one. This patch reverts the offending commit: copy with an overwrite flag works again.

CI

Improve build pipeline.

gplazma

Previously, a missing banfile would result in every successive login attempt failing with an NPE. Now, even if configured, an empty or inexistent banfile will be ignored and logins should succeed. WARNING: This changes the banfile plugin behaviour! Empty and non-existent banfiles will be treated the same.

system-test

Increases the small default direct memory value for system-test: NFS on system-test works again.

webdav

Update webdav door to retry the request to the namespace if checksum is not present in files attributes.

xroot

A recent commit introduced handling of relative paths in xrootd with inadvertent side effect of dropping support xroot.root variable. This patch reverts that commit: pre–9.2 behavior is restored.

Changelog 9.2.7..9.2.8

63c0dfbd70: [maven-release-plugin] prepare release 9.2.8
7dcd2cc2fd: ci: dont’t pull build artefact for kubernetes-based jobs
1af0614d0c: Fix unit test for commit 6b47354
0deb226953: gplazma: configured banfile plugin should ignore non-existent ban file
2ac2180724: chimera: Revert “chimera: update FsSqlDriver#inodeOf to throw exception if file not found”
38d2cae1ff: system-test: increase direct memory
3840f87195: xrootd: fix xrootd.root regression
baa135bab9: webdav: wait for upload to complete
16d3bb94a3: [maven-release-plugin] prepare for next development iteration

Release 9.2.7

ci

Build and test system improvements.

cleaner-disk

No more occasional ConcurrentHashMap exceptions in cleaner-disk runs due to concurrent pool status changes.

core

Admin commands with hyphen-containing arguments are no longer trunkated prematurely, they are now properly shown and autocompleted.

qos

Prevent PropertySetterException in qos.

xroot

The change to use effectiveRoot did not take into account Subject = Nobody.

Regression eliminated, previous behavior with anonymous read restored.

Fixes previous fix for xroot descriptor: NPE risk mitigated.

Changelog 9.2.6..9.2.7

a91a3bbb1c: [maven-release-plugin] prepare release 9.2.7
8a60a76165: dcache-xroot: check that descriptor is not null before calling close – fix
84e5af54cd: dcache-xroot: check that descriptor is not null before calling close
711b4a38f8: cleaner-disk: prevent ConcurrentModificationException in cleaning run
84026b3779: dcache-xroot: fix effective root when subject is nobody
95590b4a5f: dcache-core: fix admin command completion
e8bc2c0274: qos: comment out property description
c2022ea407: ci: add exta helm ops to extend timeout and simplify retries
53afeee632: [maven-release-plugin] prepare for next development iteration

Release 9.2.6

dependencies

The MongoDB driver was updated. This might lead to different log messages.

frontend

A bug was fixed that led to an Exception when using qos without policy.

webdav

If a non-existing well-known resource is requested, it won’t log a stacktrace anymore.

Changelog 9.2.5..9.2.6

c0bc7fa627: [maven-release-plugin] prepare release 9.2.6
a83cc7fbfd: pom: Update mongodb-driver
1b10c15afb: dcache-frontend: check for defined arguments with namespace qos
1748b3b887: webdav: do not log stacktrace if non-existing well-known is requested
9ad93f688c: [maven-release-plugin] prepare for next development iteration

Release 9.2.5

bulk

The current releae fixed handling of uncaught exception.

dcache-view

dCache-view has been updated to 2.1.0 and it includes modifications to use new RolePrincipal instead of the roles plugin role, eliminating the need for additional logins for admin privileges.

qos

With ingest queues servicing large numbers of requests, as we have for performance reasons configured as defaults on QoS and Bulk, there was a potential for a race between the processing of the original modify request and a subsequent cancellation request, such that cancellation could not find the request as it was still in the executor queue.

This is fixed now and are is a trailing stream of requests still being processed after cancellation (from Bulk) completes.

Changelog 9.2.4..9.2.5

7969d6bbd7: [maven-release-plugin] prepare release 9.2.5
dee8cbc483: dcache-qos: cache modify requests until processed by executor
3a255bb92c: dcache-bulk,dcache-qos: repair mass cancellation issues
4bed61af7e: dcache-bulk: fix handling of uncaught exceptions
73b49ce730: pom.xml: update dcache-view to 2.1.0
ea69bea591: Fixed QoSPolicyTest duration strings
347c668ce9: dcache-frontend,common: check parsing of Duration in STAGE
470e0f65a3: [maven-release-plugin] prepare for next development iteration

Release 9.2.4

chimera

The rollback of 9.1 to 9.0 db schema was fixed and possibility to rollback namespace db to an earlier schema version is restored now.

webdav

The urls similar to this https://door.domain.foo:1234//pnfs/domain.foo/path/to/file were strip into /domain.foo/path/to/file.

The current release fixed the parsing of urls with two slashes in the path.

Changelog 9.2.3..9.2.4

cd710f3992: [maven-release-plugin] prepare release 9.2.4
cefea2f3cb: chimera: fix rollback of 9.1 to 9.0 db schema
95fcd5d230: webdav: fix infinite recursion in Requests.stripToPath
0e33db68c5: webdav: fix parsing of urls with two slashes in the path
428f7c77e5: ci: generate release-notes template
8fff819a39: Revert “ci: use rancher to create k8s namespace”
9e4b101bcf: [maven-release-plugin] prepare for next development iteration

Release 9.2.3

dcache-bulk

Performance and stability improved, but throughput continues when the submitted task activities are in a state of waiting for future completion.

door

Exception handling is improuved for Kafka.

oidc

Storgae scopes without path will be rejected now.

Changelog 9.2.2..9.2.3

b8134f8db3: [maven-release-plugin] prepare release 9.2.3
ec5783ca89: dcache-bulk: use rate limiter to throttle semaphore release
4e9864a336: oidc: fix remove invalid testcase
8059c50501: oidc: reject storage scopes without path
af96f5c6d6: door,pool: handle multiple possible KafkaExceptions
c9fe2f61f9: docker: use exec for start java process
674bb383df: [maven-release-plugin] prepare for next development iteration

Release 9.2.2

bulk

A bug was fixed that may lead to the archiver deleting ongoing requests.

qos

The default size of the task queues are set to “qos.limits.verifier.max-running-operations” now to increase the throughput. IMPORTANT: The default size of the QoS Engine’s modify thread pool was also updated to 500 as the current value, 32, is too little.

webdav

This patch fixes a bug that led to wrong paths on redirects.

Changelog 9.2.1..9.2.2

16ffa5d557: [maven-release-plugin] prepare release 9.2.2
3278975650: ci: use pynfs:0.5, enable LOCK24 test
09c872a80a: ci: use rancher to create k8s namespace
38678c9209: dcache-qos: correction to the threshold warning
1a8bb02376: dcache-qos: set default task thread pool sizes all to max concurrent running
b3a1bb7b6b: dcache-bulk: fix bug in archiver deletion query
ea7e26199b: dcache-bulk: fix thread executor injection
d466c360c0: [maven-release-plugin] prepare for next development iteration
08c0ac9c60: wevdav: fix redirect path

Release 9.2.1

bulk

Correct bulk cancellation semantics.

The problem with cancellation (hanging perpetually in the CANCELLING state) was solved.

Return directory listing to its own executor.

Same performance as before, but with a total memory footprint at a little more than half available physical memory instead of being very close to it.

Add commands to display the count instead of the actual entry (as with requests and targets), and also a command to clear/delete from the archive table.

documentation

Add commands to display the count instead of the actual entry (as with requests and targets), and also a command to clear/delete from the archive table.

Fixes link in documentation.

Add section in WebDAV door chapter on metalink.

Changelog 9.2.0..9.2.1

8d4bf82df5: [maven-release-plugin] prepare release 9.2.1
8cac6fa53e: dcache-bulk: fix cancellation issues
66b67eabe3: dcache-bulk: give directory listing a separate executor
151b4cb86a: doc: add QoS policy and role documentation to the dCache Book
0ffd722bd6: docs: UserGuide fix angle-braket in metalink
02924858c5: docs: add description of webdav’s metalink support
be306f450e: dcache-bulk: refine container executor model
3cd0912aca: dcache-bulk: cancel activity future on target cancel
201359918b: dcache-bulk: add count and clear to archive admin commands
95f53d8478: [maven-release-plugin] prepare for next development iteration

Release 9.2.0

Known issues

Cancel is currently broken in bulk service and recursive requests could freeze up.

Admin

Paging capability was added to the AnsiTerminalCommand (and DirectCommand), where results exceeding 10K lines prompt the user with [Y/N] for further results. When executing a shell command in non-terminal mode, all results are streamed back continuously. The Bulk commands have been converted to use this feature.

Bulk

There have been many small fixes and improvements to this service since 8.2.

The most significant changes or additions include:

The storage layer has been redesigned to conserve space and for efficiency/throughput. Please note that moving up to the new database schema may require some time.
The amount of time can be generally computed in terms of the number of entries in the request_target table; figure about 1 hour for every 10 million. If it is necessary to maintain all such entries, we recommend you do the upgrade offline, using the dcache database update command-line tool. If there is no need to keep completed requests in the database, we would advise truncation or at least deletion of the completed requests before the upgrade.
Periodic archiving of requests has been added; this is configurable via properties and admin commands. The archive table maintains an abbreviated summary of the requests, and can be purged via admin command as well.
The container job has been rewritten to bring it in line with SRM bringonline performance; multithreaded directory listing has also been implemented to improve recursive request time-to-completion.
Path resolution (for relative paths) has been integrated into bulk request processing.
Activity providers now can capture the environment so that defaults can be customized (this presently pertains only to pin and stage lifetime attributes).
Support has been added for the QoS update request to handle policy arguments.
Bulk has been made replicable (HA). Please read the requirements in the cookbook section on High Availability.
The delay clear option has been eliminated from bulk requests.
The prestore option has also been eliminated, as it is no longer necessary (since all initial targets are immediately batch stored synchronously) before the submission request returns.

A few minor points:

Default limits for request size and number of requests per user have been set on the basis of WLCG requirements.
The counts displayed by the info command are from startup and are cumulative; for actual (current) counts based on the data store, use the status counts command.

There are numerous properties which have been deprecated, particularly those dealing with limits. Please review the bulk.properties file for details. It should not be necessary under normal usage to adjust the limits for thread pools, database connections and semaphores from the defaults. Depending on the volume of activity and the site requirements, the archiver period and window may need to be shortened from the default. Note that the options for clearOnSuccess and clearOnFailure that can be included in an individual bulk request to indicate immediate deletion are not available for STAGE requests as this feature is not part of the WLCG specification, so the only way to remove such requests automatically is through the archiver.

Chimera

With dcache version 9.0 chimera has introduced chimera_soft_update and chimera_lazy_wcc Java properties to control the behavior of the parent directory attribute update policy. Now those properties are obsolete and replaced by a regular dcache configuration property chimera.attr-consistency, which takes the following values:

policy	behaviour
strong	a creation of a filesystem object will right away update parent directory’s mtime, ctime, nlink and generation attributes
weak	a creation of a filesystem object will eventually update (after 30 seconds) parent directory’s mtime, ctime, nlink and generation attributes. Multiple concurrent modifications to a directory are aggregated into a single attribute update.
soft	same as weak, however, reading of directory attributes will take into account pending attribute updates.

Read-write exported NFS doors SHOULD run with strong consistency or soft consistency to maintain POSIX compliance. Read-only NFS doors might run with weak consistency if non-up-to-date directory attributes can be tolerated, for example, when accessing existing data, or soft consistency, if up-to-date information is desired, typically when seeking newly arrived files through other doors.

Frontend

Aside from bug fixes, the following changes should be noted:

Support for .well-known/security.txt was added to both the frontend and WebDav ports.
More detailed description of request objects for bulk and stage.
Improved error messages in several places.
Support for relative paths and symlink prefix resolution for bulk and namespace resources.
The frontend.wellknown paths have been deprecated in favor of dcache.wellknown.
Authz checks have been removed in Quota GET methods.
Use of RolePrincipal (see under gPlazma) replaces reliance on LoginAttributes and the old admin role (special gid) as defined by the roles plugin.
Support for QoS Rule Engine policies. A new qos-policy resource allows one to add, remove, list and retrieve policy definitions. See the Swagger pages for details.
An -optional query parameter was added to the namespace resource to retrieve extra information about a file; the new QoS Policy file attributes (QOS_POLICY and QOS_STATE) are included with this option.
Support for pool migration has been introduced (migrations resource). See Swagger pages for details.

gplazma

A RolePrincipal has been added to support the use of role definitions in the multimap file. Currently the available roles are: admin, qos-user and qos-group. The second allows the user to transition files owned by the user’s uid; the third allows the user to transition files whose group is the user’s primary gid. The two qos roles can be combined. Admin grants full admin privileges. A multimap example:

dn:"/DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Al Rossi/CN=UID:arossi" username:arossi  uid:8773  gid:1530,true roles:admin

These roles do not depend on the presence of the roles plugin. Since the frontend has been changed to use the new RolePrincipal and not the old role definitions (where a special gid must be defined), one can easily drop that plugin from the gplamza.conf file.

There was also an important fix for a bug that was preventing upload (e.g., xroot POSC) when using tokens.

The gplazma-xacml plugin has been dropped.

NFS

Prior to version 9.2 dCache, to support RHEL6 based clients, if no species export options are specified, the NFSv4.1 door were publishing only nfs4_1_files layout. Now on the door publishes all available layout types. If for whatever reason RHEL6 clients are still used, the old behavior can be enforced by lt=nfsv4_1_files export option.

PNFS Manager

To remove unused directory tags chimera keeps the reference count (nlink) of tags. This approach creates a ‘hot’ record that serializes all updates to a given top-level tag. Starting 9.2 dCache doesn’t rely on ref count anymore and uses conditional DELETE, which should improve the concurrent directory creation/deletion rate.

Pool

The mover ls command is updated to display with the -u option the subject associated with the mover.

Added pool.mover.https.port.min and pool.mover.https.port.max to control TPC port number used by HTTPS mover.

Pools info command displays HTTP, HTTPS, NFS and XROOT movers listen ports and interface.

Billing information from DCAP and FTP movers now on include local socket end point

Poolmanager

Pnfsid and path options have been added to ac psu match.

Re-introduce wrandom partition type - a weighted-random pool selection partition, which works as a WASS partition, but ignores LRU metric and number of movers.

QoS

Aside from bug fixes, the following significant changes to QoS should be noted:

QoS was made to support migration using a new pool mode, “DRAINING”; please consult the book chapter for further details.
The DB namespace endpoint can now be configured to be separate from the main Chimera database (originally introduced/changed for Resilience). In this way, the scanner, whose namespace queries are read-only, could be pointed at a database replica.

This remains possible for QoS, even though the QoSEngine now also is responsible for updating Chimera with file policy state. These writes are all done via messaging (PnfsHandler) rather than by direct DB connection, so they go to the master Chimera instance.
Requests for QoS transitions are now authorized on the basis of role (see under gPlazma).
The first version of the QoS Rule Engine has been added. With this, one can define a QoS policy to apply to files either through a directory tag or via a requested transition; the engine tracks the necessary changes in state over time. A new database table has been added to the qos database. Remember that if you are deploying QoS for the first time, you need to create the database:

createdb -U <user> qos

Fuller explanation of the rule/policy engine is forthcoming in the Book chapter.
In conformity with the new rule engine changes, the scanner has been modified in terms of how it runs scans. There are now two types of system scans, the (NEARLINE) QoS scans (to ensure that files with a policy requiring them to be flushed have indeed been written to tape) and a system-wide ONLINE scan (there are two versions of this and an option to choose which one). Details in the Book; also refer to the relevant admin commands.

Note that the singleton QoS service (where all four components are plugged into each other directly) is no longer available; the four services can, however, still be run together or in separate domains, as with any dCache cell.

Resilience

Resilience is still available in 9.2, but should be considered as superseded by the QoS services. We encourage you to switch to the latter as soon as is feasible. Remember not to run Resilience and QoS simultaneously.

A recipe has been added to the cookbook for migration/draining of resilient pools. This continues to apply in QoS.

WebDAV

The WebDAV door now provides limited support for metalink format. Metalink is an XML format, described by RFC 5854, that describes how to download multiple files. The initial support is limited to describing the files within the targeted directory: there is no support for recursion. The metalink information is available via HTTP Content Negotiation (client sends a Accept: application/metalink4+xml header) or via a Link HTTP response header (see RFC 6249) when generating the normal HTML output.

XRootD

Aside from bug fixes and various documentation additions and clarifications, the following should be noted:

Proxying through the xroot door is now available.
Relative paths are supported in the xroot URL.
Resolution of symlinks in path prefixes and paths is supported.
The efficiency of the stat list (ls -l) has been greatly improved.

An Xroot User Guide page has been started.

Changelog from 9.1.0 to 9.2.0

95f53d8478: [maven-release-plugin] prepare for next development iteration
86ab6a2f5d: [maven-release-plugin] prepare release 9.2.0
8d0d926ab1: gplazma: oidc update explicit AuthZ parsing
11ceff7a92: gplazma: oidc increase cache duration for OP public key material
b640b5e92c: dcache-bulk: container rewrite to optimize threading
b2b76e9e15: dcache-bulk: add admin command and query to reset all requests with failed targets
74bab6447e: dcache-bulk: add convenience admin command for state counts
12a1fbf5f4: dcache-qos: fix scanner operation completion logic
28b0bebc2d: ci: by-pass docker.io for bitnami charts
ab16db9102: dcache-bulk: implement HA
1378259897: dcache-cli: convert IllegalArgumentException to CommandException on call()
2507eea428: dcache-bulk: do not PIN or STAGE files with AL ONLINE
a29351aae1: dcache-bulk: only set request status to QUEUED when permissions and targets are all inserted
8479f44e1e: Create test.txt
d23083495f: [maven-release-plugin] prepare branch @{releaseLabel}
707cf4f1cb: pool: on failed upload set file size to zero for space reporting
afe112ed48: dcache-bulk: guard against erroneous argument names
bf9bb6c5e3: dcache-qos: remove entry from rule engine table if file not found or other namespace exception
34b411557f: dcache-frontend: include new QoS Policy attributes in -optional
805735ff23: book: remove v6.2 references in HA chapter
449651ff22: poolmanager: fix wrandom partition incompatibilities
fb04d1c990: Revert “poolmanager: remove wrandom partition type”
3a33ae774c: Add a correct check for the existance of the specified link, poolgroup or HSM when using Migration via REST.
0fea8a858f: docs: gplazma document oidc plugin
67508e933b: pnfsmanager: inroduce limit on number of concurrent listing of the same directory
c1e3728412: packages: fix system test populate sed expression
2a01b13aa3: dcache-webdav: revert improve efficiency of directory listing (14085/14088)
29f340d3c0: dcache-qos,system-test: override db property to hsqldb
8ee98957db: ci: collect billing records from kafka
a518359f78: doc: clean up the multiple protocol configuration explanation for xroot
1266bb7372: dcache-bulk: fix warning for premature stop of job container
17453da77a: dcache (qos): move required attributes to Transfer.readNamespaceAttributesAsync
a1140fceff: pool: include local endpoint for ftp transfers
effabfe737: pool: report dcap local endpoint
52602da836: ci: pin all jobs to dcache-dev runners (temporary)
a8de5478b5: dcache-frontend,dcache-bulk,dcache-qos: regularize observance of the admin role
49cf4062a9: dcache-bulk: handle InterruptedException in DirListTask
41f34d8b43: ci: trace node where gitlab agent is running
af134b683e: ci: use egi-trust repo
560dce66e4: ci: use almalinux9-minimal images for rpm signing
3567dde692: dcache-webdav: fix incorrect parameter value given to DcacheDirectoryResource constructor
253859a532: dcache-qos: drop subject from qos_operation table (qos engine 9)
8cb60b96cd: dcache-frontend: add support for qos policies (qos rule engine 8)
13580d972c: dcache-bulk: modify qos update activity to support policies (qos rule engine 7)
0ebfa0553b: dcache-qos: add policy support to scanner (qos rule engine 6)
710ee615e2: dcache-qos: throttle verifier requests to engine (qos rule engine 5)
77bd7a1fb9: dcache-qos: add support for policy handling in qos-engine (qos rule engine 4)
e2a1f371eb: dcache,pnfs: implement qos policy support (qos engine 3)
e3d67b8c30: chimera,dcache-chimera: add support for inode policy info (qos engine 2)
21af809d58: chimera: implement qos policy storage (qos engine 1)
1f79ce55ce: webdav: add metalink support
707000c026: dcache-webdav: improve efficiency of directory listing
7e691a4f53: dcache-frontend,dcache-webdav: use RolePrincipal instead of LoginAttributes roles
7609229430: common,gplazma-multimap,dcache-qos: qos role authorization, revised
fe522a5409: ci: enable srm tests
603ab40ccc: ci: run junit test during rpm build
ec1ff2b5f4: ci: add srmls tests
2a40be361c: ci: report gridtest test results
2fb9170283: chimera: update FsSqlDriver#inodeOf to throw exception if file not found
c425a62257: ci: add first robot test
09e8e32390: dcache-qos: rework verifier operation handling so most of it is in memory (as in resilience)
8c92f2c413: common: fix NPE in AbstractUidPrincipal#equals
2b83b2c135: ci: upload_rpm must match all other upload rules
410762eb55: common,gplazma-multimap,dcache-qos: authorize qos transition based on role
de47e03e3a: dcache-qos,dcache-bulk: allow qos update to call engine asynchronously
4af8e26ce0: skel,logback: make the .resilience and .qos log files singletons
87dc1a4ea5: build(deps): bump org.springframework.kafka:spring-kafka
6643935656: pool: remove hidden defaults for rh/rm/sh operations
0dde36e585: ci: use CI_REGISTRY_IMAGE variable to address container image
99bfff3c52: dcache-bulk: adjust semaphore permits to something more reasonable
b5e8dd5e3f: ci: add xroot based tests
9e88a1ae07: ci: better readablity and only logic based refactoring
841d53731c: dcache-bulk: reconfigure activity providers to capture environment
a32aa038a6: rpm: add required packages
84c2caa61d: docs: fix external link to nfsmapid and DNS TXT Records docs
4071315e36: docs: add dcache door generic workflow
21f441b4ee: Revert “ci: pass extra pgk and rpm to dcache container”
1398b38eef: dcache-bulk: cancel all stored targets on abort
ef26f40e9a: dcache-bulk: implement periodic archiving of old completed requests (part 2)
e36714d42a: CI: Inheritance for Kubernetes
d5c7c9d3c6: ci: fix image labels
ccb6c8d491: ci: add labels to produced containers
f7b8b4de3a: ci: pass extra pgk and rpm to dcache container
e5fdc5d741: ci: fix manual upload logic
8dcb21e0a2: ci: install required OS tools for dcache container
9cc8f8a376: ci: add workaround installation of shadow-utils
a62142e6f9: pool: Save hsm load provider to setup file
02e0ef6028: ci: use dependencies instread of command not found: needs
1de87b4131: remove extra listing of workspace
e8d7e9fc10: ci: use pynfs–0.4 container
b396a798ab: ci: collect logs before cleanup
e007ede840: dcache-bulk: implement periodic archiving of old completed requests (part 1)
a29d8f4f08: dcache-xroot: improve efficiency of stat list (ls -l)
83e5974301: dcache-qos: remove singleton service configuration
d09c80074a: dcache-bulk: remove in memory running state counts
c0021411ec: pom.xml: upgrade to xrootd4j 4.5.8
ab7c3b1a4e: ci: install shadow-utils before other packages
4f7f78160f: ci: add possibility to manually publish non tagged packages
27d348c711: ci: almalinux:9-minimal as based image with java17
a565601190: common: modify the way the flag on isRestricted works
aa0ce5b75e: ci: add publishing to github
e1a450dc80: ci: initialize worker node
f9622f2fb2: ci: deply test WN with shared cvmfs
801358fc31: pool: show the listen port of nfs mover with admin info command
2cbd89fdf9: pool: introduce property for https mover port range
dab3637221: pnfsmanager: make list scheduling behavior optional (selectable)
a848b94a1c: chimera: rename FsSqlDriver#removeTag into removeAllTags
77c2dd60ce: chimera: fix commit 8fe67b2046
8fe67b2046: chimera: introduce attribute to control attribute update consistency
f56ac72e4b: Fixed string name in deserializer
cdcb84ae3a: Revert “Revert ”dcache-common: add QoS Json policy objects“”
fa9ffb95cf: Revert “dcache-common: add QoS Json policy objects”
ea28525852: dcache-common: add QoS Json policy objects
ff857831df: scripts: update default heap and mempry sizes
b9dd2f410e: ci: fix vesion of kafka helm chart (v2)
13de8e7a1a: ci: fix vesion of kafka helm chart
7cbee56df8: allow action only main repo
d9ed86fcde: docs: more pnfs flow diagrams
5eb4e70418: docs: more pnfs flow diagrams
dbd85ee88c: docs: add pnfs flow diagram
cdc47c7287: github: add gitlab mirroring action
f34a0cb4fa: dcache-gplazma: add serialVersionUID to MultiTargetedRestriction
ff86d14551: chimera: drop nlink ref count for directory tags
2acfaf8d14: dcache-bulk: allow multi-threaded directory listing on expandDirectories
33a37d6774: fix bash-completion
4be34ff6c3: pool: handle double remove of IdleStateHandler
633d1333ed: nfs: expose dcache version in EXCHANGE_ID operation
572f76cf90: dcache-qos: fix bug in storing and retrieving ‘tried’ value for verify operation
9f6910ece9: dcache-bulk: deprecate “prestore” option and remove related container
5f5a5a96f8: libs: use nfs4j–0.25.x
3fb66514d9: pool: add option to display user subject with mover ls
c70248cc26: ci: fix negation of disabled tests
d767580166: ci: disable unsupported pynfs tests
587ab61da1: ci: envorce nfsv4.2 for pynfs tests
3c1cd449f1: ci: use latest pynfs test set
ae44287d91: ci: fix exclude/include for pynfs tests
d73359de02: ci: set pynfs test error code according to test results
25f6b1ea32: ci: fail build if pynfs tests are failing
17e8f603a1: Fix Migration Resources
6e9e80c1da: ci: remove obsolete kybernetes deployment script
2970ebaa37: ci: use helm based dcache deployment
ad253a69ef: dcache-xroot: modify mkdir to ignore dir exists on make parent option
9e1955734a: book: fix rpm download links
108f2f5a3a: Reformat code in HsmSet.java
1b7e07c08e: ci: add pynfs test into pipeline
88109a7d74: ci: remove unused pipeline job
0edcfb12d0: rpm: depend on java–11-headless
b406b78bdb: dcache-vehicles: fix NPE in resolve symlink message
421e1dc932: nfs: fix race condition of LAYOUTRETURN and LAYOUTGET
1e54a504f9: xrootd: throw FileNotFoundException if vomsdir doesn’t exist
a3ca140698: ci: deploy in kubernetes
a9ebbc044f: dcache-qos: fix adjuster task cloning to reset state
31346a795b: dcache-qos: improve exception reported on aborted verification
fd38a52850: ci: prepare k8s environment for deploy and test
1ae82cc59c: pool: remote http movers to provide local endpoint info
fbdc448493: ci: add gitlab-ci.yml
62e945c4e7: docs: add DOI lable/badge
7f6675d7db: gplazma-roles: add QoS role support
ae83a08319: dcache-http: add http request header for role assertion
8de990a241: dcache-frontend: add symlink resolution to /api/v1/id GET and /api/v1/namespace
024b1cf5c8: dcache-frontend: remove authz checks in Quota GET methods
98ad434afb: dcache-qos: remove trigger to rescan pools on tag change
0d682c68de: Revert “dcache-qos: remove trigger to rescan pools on tag change”
825c40d4a0: pom.xml: bump xrootd4j to next version (4.5.7, 4.4.8, 4.3.9, 4.2.13)
adda59a455: dcache-qos: remove trigger to rescan pools on tag change
ad2632019d: several: remove non-ASCII dash occurrences
9eae61bbac: dcache,pnfs: skip symlink resolution when checking restrictions on directory children during listing
19e5341e58: dcache-qos: repair faulty queue refresh algorithm in verifier
6dde34e643: dcache-qos(verifier,engine): fix incompatibility issues with Subject and attributes
34adbb0828: skel,dcache-frontend,dcache-webdav: use dcache.well-known for all doors
d85ec22dff: dcache-bulk: catch Exception from getSubject()
d8e1826672: book: update config-message-passing.md
5217a1da4a: build(deps): bump netty-handler from 4.1.92.Final to 4.1.94.Final
9d4c75a102: Docs: frontend.wellknown!wlcg-tape-rest-api.path also applies to WebDAV doors
93ed1709f4: [maven-release-plugin] prepare for next development iteration
80f485f13b: build(deps): bump guava from 31.1-jre to 32.0.0-jre

What’s new in dCache 9.2 Release notes

Highlights

Acknolagements

Incompatibilities

Release 9.2.32

bulk

cells

qos

Changelog 9.2.31..9.2.32

Release 9.2.31

jvm

Changelog 9.2.30..9.2.31

Release 9.2.30

cells

Changelog 9.2.29..9.2.30

Release 9.2.29

tape

Changelog 9.2.28..9.2.29

Release 9.2.28

xroot

Changelog 9.2.27..9.2.28

Release 9.2.27

CI

pool

Changelog 9.2.26..9.2.27

Release 9.2.26

bulk

Changelog 9.2.25..9.2.26

Release 9.2.25

bulk

gplazma

Changelog 9.2.24..9.2.25

Release 9.2.24

doc

webdav

Changelog 9.2.23..9.2.24

Release 9.2.23

CI

core

Libs

TAPE-API

Changelog 9.2.22..9.2.23

Release 9.2.22

chimera

frontend

Changelog 9.2.21..9.2.22

Release 9.2.21

chimera

ci

dependencies

Changelog 9.2.20..9.2.21

Release 9.2.20

bulk

pool

Changelog 9.2.19..9.2.20

Release 9.2.19

chimera

Changelog 9.2.18..9.2.19

Release 9.2.18

webdav

Changelog 9.2.17..9.2.18

Release 9.2.17

common

xroot

Changelog 9.2.16..9.2.17

Release 9.2.16

qos

webdav

Changelog 9.2.15..9.2.16

Release 9.2.15

bulk

info

nfs

Changelog 9.2.14..9.2.15

Release 9.2.14

cells

srr

tape-api

Changelog 9.2.13..9.2.14

Release 9.2.13

What’s new in dCache 9.2
Release notes