What's new in dCache 12.1
Release notes
Highlights
- Beta support for S3 interface
Incompatibilities
- Files' last access time is enabled by default.
- The
systemdservices are now started with even more restrictions.
Acknowledgments
We want to thank Jerome Pansanel and Geonmo Ryu for their contributions.
Release 12.1.0
Httpd
Since dCache version 11.0, we have changed the look and feel of the admin pages.
Not all admins like that change. Starting with version 12.1, we allow the creation
of custom styles with the option to default back to the old style.
The new config property httpd.look-and-feel is introduced with values classic and new.
Packaging
The systemd services are now started with even more restrictions:
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
RemoveIPC=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
SystemCallArchitectures=native
UMask=0027
KeyringMode=private
LockPersonality=yes
MemoryDenyWriteExecute=no
PnfsManager
Site admins often need to find files that haven’t been accessed for some time. Usually such information requires a merge of namespace and
billing information. However, dCache can maintain the last access time of files, which is disabled by default. With this version,
the atime gap is set to 604800 (7x24x3600) seconds, which will update files' atime if last access was more than a week.
S3
dCache now provides an S3 door (dcache-s3), letting S3 clients use dCache
as an S3-compatible object store. By default, the s3-door listens on port 9000.
File access granted based on Unix permissions. The objects with forward slash (‘/’)
in the names are mapped as files with subdirectories. On object removal, the created
subdirectories are left behind.
Integration with gPlazma:
File /etc/dcache/s3-keys.conf:
AKIASYSTEMTEST system-test-secret
File /etc/dcache/multi-mapfile
s3:AKIASYSTEMTEST username:tigran uid:1 gid:2,true
The new “The S3 Service” chapter in The Book covers setup and limitations.
SRM
The SRM support is completely removed from the dCache code base.
Changelog from 12.0.0 to 12.1.0
- d8b34cc8b1
- [maven-release-plugin] prepare release 12.1.0
- 3e515c9214
- [maven-release-plugin] prepare branch @{releaseLabel}
- 9bd02ecdcd
- s3: enforce S3 as protocol name
- 3f678c43ed
- [skip-ci] docs: describe frontend.srr.paths SRR path mappings
- 26c911bda9
- cells: suppress error when killing non-existing cell
- 3e6f0edd09
- [skip-ci] docs: document streaming verification of signed S3 bodies
- 4ca12a5aa3
- pool: remove file hopping / replication
- e6efa763c4
- s3proxy: verify signed request bodies while streaming them to the pool
- 2398041c00
- xrootd: reduce number of DNS requests
- 537de7590c
- docs: xrootd: clarify how the door selects pools for capacity reporting
- b4dddf5fd9
- [skip-ci] docs: document capacity reporting in the NFS and xrootd doors (#8277)
- 03a34ea8b2
- nfs, xrootd: report the capacity of the pools or a space reservation
- 7d00903f4b
- s3: fix handling of IPv6 addresses
- 3d33e59c84
- dcache-vehicles: remove orphan srm messages
- 15682a772c
- [skip-ci] docs: correct wildcard description for allowed tunnel CNs (#8274)
- 4945b39f20
- LocationManager: check for allowed CN before opening tunnel
- 32e8630e14
- pom: use xrootd4j-4.6.7
- 3dc032b423
- [skip-ci] docs: document SciTag propagation to remote party in HTTP-TPC
- 1a5f361e21
- pool: propagate SciTag to remote party in HTTP-TPC transfers
- b742bfaa70
- cells: fix broken junit text
- 3c3ff60890
- cells: announce correct domain role on outbound core tunnels
- 70b0684d02
- ci: allow duplicates for jackson-core and jackson-databind
- c485d1b111
- [skip-ci] docs: add S3 door chapter to the dCache Book
- d9a34ee72c
- [skip-ci] docs: add S3 door chapter to the dCache Book
- 09f87606d2
- s3: fix issues found with the ceph s3-tests
- a12453d91c
- s3proxy: answer ListObjectVersions for stores without versioning
- 30c1bec7dc
- s3: support multipart uploads
- e5256da1ea
- s3: support writes in the S3 door
- 9671df5ca0
- s3proxy: accept signatures over a Content-Type charset changed by Jetty 9
- 292513ee8a
- s3: verify the door with the AWS SDK and the system test
- 64e792cc8d
- s3: add S3 door
- 9f4a1f25a5
- s3: support RestoreObject for tape-only objects
- 76dd5863e6
- s3: add proxied read transfers
- 373c7c7805
- s3: add read-only dCache blob store
- 09a44d853d
- s3: add blob store locator with lazy gPlazma login
- 1c00bec742
- s3: add file based S3 key store
- 0ce4ad3bbb
- gplazma: add S3 access key principal
- 0ce4f82c97
- s3: vendor s3proxy module
- 2062c440d2
- xrootd: improve handling of prepare options
- e344de3d8a
- pool: expose dcache version in NFS mover
- 1a362ebe48
- pool: make nearline request lifecycle transitions atomic
- 38d8a956fd
- libs: update to slf4j-2.0.20 and logback-1.6.5
- e893c66e01
- [skip-ci] docs: document pool startup failure when no IP addresses are found
- 97b72f274a
- pool: fail pool startup if we can’t identify pool’s IPs
- 9684cedf14
- [skip-ci] docs: document multiple OIDC providers in dCache View
- d013535cc9
- dcache-view: support multiple OIDC providers end-to-end
- 982259da49
- [skip-ci] docs: document httpd.look-and-feel property
- 00810bb092
- https: add optional new look-and-feel
- d5e80fe89c
- [skip-ci] docs: document exclusive lock on files open for write over NFS
- 5bd6b33394
- nfs: deny any concurrent access to a file opened for write
- ecdd82e074
- nfs: remove transfer from the door list if pool selection failed
- 94833441ac
- [skip-ci] docs: clarify zone-scoped admin-shell command examples
- c6c7999c1e
- admin-shell: bulk send commands to zone
- b06732627c
- dcache-view: fix OIDC logout not ending the Keycloak session
- b03065e7a0
- fix: restore comment and kafka clean up
- 7fd50715c1
- transfer: fix missing mover info before sending billing notification
- 62e3713583
- build(deps): bump com.fasterxml.jackson.core:jackson-databind
- 3e4e2eb55b
- billing:upgrading kafka version two 3.9.2
- 8ed4742510
- kafka:deleting kafka cleannig up
- dc5244c193
- build(deps): bump com.fasterxml.jackson.core:jackson-databind
- 46a26ceed2
- add mutual TLS to domain tunnels
- e035243c80
- retry mover start if redirect arrives before moverId
- 1e186bfe18
- pom: bump xtootd4j version to 2.6.5
- 86e23539da
- ci: add robot emoji to AI-driven jobs
- a5592a6c71
- xrootd: implement “ping” call
- a436b23e15
- [skip-ci] ci: improve instruction to docubot on how to push to github
- d7646623a7
- jetty: drop per-client and global rate limiter
- ddf3cfeb0b
- transfermanagers: deleting kafka producer
- 02b012753f
- pool: deleting kafka producer
- b3d32d7950
- [skip-ci] docs: note removal of Kafka support from NFS door
- edfe9a5ed5
- nfs: deleting kafk producer
- 082a687d1a
- xrootd: deleting kafk producer
- ecee1b75cf
- webdav: deleting the kafka producer for webdav
- 7ceae99207
- [skip-ci] docs: note removal of Kafka producer from FTP door
- 961276bbe9
- ftp: deleting kafka producer
- 6555df8e99
- xroot: enforce SHA1 for certificate hash calculation
- 793723d5f8
- build(deps): bump org.bouncycastle:bcprov-jdk18on from 1.79 to 1.85
- 2d7470f275
- ci: add [fast_pass] and [only: <suite>] options for selective pipeline execution
- 17b2a90a52
- [skip-ci] docs: document VOMS root FQAN matching in grid-vorolemap
- 8dca32a9bd
- voms: fix FQAN parsing for root /vo group
- 67c068d33c
- ci: add OIDC authorization code flow test
- 0651d3ae85
- ci: fix the PR checks failing
- c1620a5659
- poolmanager: fix zone selection for IPv6 clients
- d687e15c34
- [skip-ci] docs: remove obsolete DCAP Kafka properties from Kafka documentation
- c3f770a106
- dcap:deleting kafka producer
- fe195ff85a
- ci: add AI-based job to auto update documentation
- 501e1ba74c
- dcahce-view: fixing issue 8165, dCache View UI after upgrade
- 932dc9d508
- docs: remove SRM from documentation
- e0c0ee426c
- xrootd: implement prepare function
- 5d13b535d6
- utils: fix buildGenericWantDigest for RFC9530 (issue8191)
- 2002409302
- src: remove remaining SRM code
- fbaec62d74
- gplazma2-voms: enforce openssl SHA1 mode for certificate hash calculation
- 34bb7b71ca
- srm: remove SRM releated code
- 6410753db1
- libs: update CaNL to 2.9.0
- 7d0495d9d6
- REST: fix 500 for GET on api/v1/cells
- 1988d67456
- remove unused @Autowired field
- 8baef5db60
- REST: add Last-Modified Header to all rest Resource responses to GET
- f8d9a84838
- dcache-view: adapt OIDC login UI to discovery-based config
- 2d6a740110
- libs: removed unused jdom library
- 9da3e4d2e8
- door: p2p to clients zone derived by ip address
- 35e66fe7d3
- pnfsmanager: set default atime gap to one week
- 82b83ae3c8
- pom: use nfs4j-0.28.0 with client reboot recovery fixes
- 9d78a559a6
- use OIDC Provider Discovery endpoint
- 7adec58914
- common-security: add workaround bouncycastle EC algorithm name mismatch
- 3f5eb52f19
- REST: return user info for mover
- 05a0b726f0
- frontend: let OIDC callback complete when anonymous-operations=NONE
- 9e3525c4b5
- pool: remove unused class members in dcap mover
- 7922a28729
- pool: fix treadsafety when displaying DCAP mover last access time
- 19b5372682
- libs: drop custom milton2 version
- 504bd0ace3
- libs: use xrootd4j-4.6.2
- e76a959cca
- dcache-nearline-plugin-archetype: use latest plugin release for testing
- 249d639346
- gplazma2-ldap: drop dependency on ldap4testing
- de34cf9eb2
- pom: explicitly specify maven central to enforce the search order
- 1a14b1c7c0
- libs: switch to nfs4j-0.28.4
- f2a18942eb
- libs: use Sleepycat from maven central
- 7a19a22e61
- add jwdp agent to dockerfile
- d243c7108e
- pool: fix pool shutdown message order
- 72a7c1e13a
- systemd: harden systemd-based deployments
- 55b369feb9
- webdav: reintroduce workaround for checksum race condition
- 2b9e9a4a8f
- build(deps): bump io.netty:netty-codec-haproxy
- 8774837587
- build(deps): bump com.fasterxml.jackson.core:jackson-core
- a283b80298
- build(deps): bump org.postgresql:postgresql from 42.7.11 to 42.7.12
- 77340d6671
- Book: add section about Last-Modified Header for REST monitoring resources
- c11937e162
- REST: add Last-Modified Header to GET response
- 5b085d2a51
- domain: generate certificate files for zookeeper TLS in tests
- 4ead0b29fe
- Transfer: remove TransferTest that is wrong
- bd48e47a28
- rep ls: fix crash when access latency or retention policy is undefined
- 42802f236a
- transfer: restore ability to define alternative/additional checksum
- 4f4384c39d
- pool: Address slow initialization of hot file migration jobs
- 7823bae321
- dcache-view: vendoring npm and bower
- 6288b6c75f
- nfs: restart pool selection process on timeout for reads
- 644a889442
- pom: enable java21 to build dCache
- 0db178dc91
- ci: use java25 for dCache runtime
- 903d0f2e2f
- chimera: uuidv7 for new pnfsid generation
- 3d55bb1e99
- smc: remove extra semicolons
- 466c692a29
- scripts: accept java25 as runtime environment
- 5c9d83c40f
- ci: fix upload_container job
- 13c63246c3
- chimera: update shell command to set a proper error code on exit
- 4077ddd9cd
- [maven-release-plugin] prepare for next development iteration
- 306c412a0c
- docs: update TheBook SciTags docs for PR 8044
- c995f7888e
- frontend: Add an option to set SRR path mappings