1.9.6 Release Notes
The main focus areas of the 1.9.6 release are the addition of a
WebDAV door, complete ACL support in the SRM, the ability to easily
move cells between domains, and experimental Terracotta support in the
SRM door.
Upgrade Instructions
Incompatibilities
Please consider the following changes when upgrading from a version
before 1.9.6-1:
- Persistent command history in the admin door now needs to enabled,
see Other changes.
Compatibility
It is safe to mix pools of releases 1.9.4, 1.9.5 and 1.9.6. Head
nodes and doors must be upgraded to 1.9.6 together and cannot be mixed
with head nodes or doors of releases before 1.9.6. Components of
different 1.9.6 releases can be mixed freely.
Compatibility Matrix
We distinguish between pool components and head nodes. Any
component which is not a pool is considered a head node, including
doors. The following table shows compatibility between different
releases.
| | 1.9.6-5 Head | 1.9.6-5 Pool |
Head |
1.9.1-1..7,9..11 | no | no |
1.9.1-8 | no | no |
1.9.2-1..5,8..11 | no | no |
1.9.2-6,7 | no | no |
1.9.3-1..4 | no | no |
1.9.4-1..5 | no | no |
1.9.5-1..15 | no | no |
1.9.6-1..5 | yes | yes |
Pool |
1.9.1-1..7,9..11 | no | no |
1.9.1-8 | no | no |
1.9.2-1..5,8..11 | no | no |
1.9.2-6,7 | no | no |
1.9.3-1..4 | no | no |
1.9.4-1..5 | no | no |
1.9.5-1..15 | yes[1] | yes[1] |
1.9.6-1..5 | yes | yes |
- dCache 1.9.6-1 head nodes were not compatible with dCache 1.9.5
pools.
Changes in 1.9.6-5
WebDAV
Propagate namespace permission errors: user will see an informative
message instead of "500 Internal error".
PnfsManager
Fix interaction between ACLs and POSIX-like permissions that resulted
in some valid requests being rejected for certain dCache
configurations (if PnfsManager is the Policy Enforcement Point).
Chimera cleaner
Fix message timeout handling: messages that timed-out were previously
ignored.
Space Manager
Fix reserve admin command so it works if no retention policy
is specified.
SRM
Fix handling of srmSetPermission operation to allow
interoperability with DPM.
Provide Space Manager with all VOMS groups when a user attempts to
reserve or release space. (NB: doesn't affect implicit space
reservation)
FTP client
Check whether the remote FTP server supports uploading checksums
before attempting to upload a file's checksum. This fixes an
interoperability issue with DPM.
dcap door
The dcap door now rejects certain malformed URLs created by incorrect
client usage of dcap client software.
Web-admin
Cells are initially hidden in the Cell Services page until they start
to respond. If a cell should go off-line then it will continue to be
displayed and marked as being off-line (as happens prior to 1.6.5-17).
Migrating namespace
Fix bug that resulted in the storage-info migration test to fail.
Info provider
Fix bug where some sites would publish only a subset of their SRM
reservations.
JMS
Fix a couple of minor timing issues that resulted in some harmless
errors during starting up and incorrect behaviour when resolving
well-known cells under rare conditions.
Detailed changelog 1.9.6-4 to 1.9.6-5
- [13663] Name space: Add file attribute mapping for SIMPLE_TYPE
- [13669] WebDAV: Propagate permission denied
- [13679] webadmin: hide cells that have not responded to any xgetcellinfo requests
- [13684] PnfsManager: Workaround for ACL regression in attribute mapping
- [13720] ChimeraCleaner: Fix initialization
- [13756] Fix handling of unspecified retention policy in ac_reserve function
- [13769] dcap: make door more robust
- [13776] info-provider: publish all reservations
- [13782] SRM Space Manager, use complete Authorization Record for space reservations
- [13791] migration: fix storage-info check
Changes in 1.9.6-4
DCAP
Fixed read permission check when
permissionPolicyEnforcementPoint is configured to
PnfsManager. We urge sites using earlier builds of 1.9.6 to
upgrade to at least 1.9.6-4.
Info Provider
Glue requires that published SA objects (which represent some
storage) has ChunkKey, which is used to describe the relationship
between the SA and its containing storage element (SE) object. These
attributes were missing for nearline and offline accounting SA
objects.
Pools
Fixed the meta2yaml script. The script didn't work for
shells other than bash.
FTP
Added support for running multiple FTP doors on the same host. The
mechanism for doing this is the same as for running multiple DCAP
doors on the same host.
SRM
Starting with 1.9.6-1, the mounted name space is no longer needed
by SRM. The init script however still insisted on mounting the name
space. Starting with 1.9.6-4, the init script no longer mounts the
name space on the SRM node.
Fixed an interoperability issue that affected srmCopy transfers
between dCache and DPM.
WebDAV
Fixed the batch script for the WebDAV door. It accidentally
contained leftovers from an SVN merge conflicts.
Detailed changelog 1.9.6-3 to 1.9.6-4
- [r13523] infoprovider: add missing GlueChunkKey attributes for nearline and offline accounting
- [r13527] meta2yaml: Fix POSIX shell compatibility and don't require dCacheSetup
- [r13537] add support for running kerberos ftp, weak ftp and dcap doors, support for running multiple doors of the same type
- [r13575] ftp, srmcp client: properly implement checksum feature negotiation, send or request checksum only if server support it
- [r13592] 1.9.6 SRM does not need to mount name space
- [r13598] DCAP: Fix subject and access mask handling
- [r13614] pool: Relax requirements in PoolCostInfo
- [r13619] fixing previous reject
- [r13620] 1.9.6 since name space mounting is not needed in chimera and only for pnfs manager install.sh shoudl be updated accordingly
- [r13634] poolcost: ignore failing unit test
- [r13640] WebDAV: Check that keystore password is not empty
Changes in 1.9.6-3
SRM
The DB schema detection code for the srmLs related tables has been
fixed. The bug caused those tables to be recreated on every restart.
A deadlock in processing of file download requests has been
fixed. The bug could when triggered cause all download request
processing to halt until the next restart.
Fixed a bug in the init script that caused the init script to fail
to stop or restart the SRM. The bug was introduced in 1.9.5-13.
PoolManager
An error propagation bug was fixed. The bug caused failure to
resolve client host names to be retried forever.
Fixed a bug in which Pool Manager in rare occasions could create an
orphaned file.
DCAP
Failure logging in the DCAP door was improved. A number of
non-critical stack traces have been removed.
Fixed an issue with stage authentication.
Pool
The migration module now ignores files that have been deleted in
the name space. Before such files would be retried until the file was
removed by the cleaner.
The fsync failure handling that was changed in 1.9.5-12 has been
changed again. In 1.9.5-12 failure to fsync caused a pool to be
disabled. We have discovered a number of common sources to fsync
failures that are not to be considered critical. Hence in 1.9.5-13 we
changed the behaviour such that failure to fsync causes a transfer to
fail, but the pool is not disabled.
A bug related to HSM script failures has been fixed. The bug would
cause the pool to essentially ignore an error code returned by the HSM
script when trying to stage a file. The stage would still fail if
checksum verification was turned on (which it is by default), but it
would cause the error message in the log files to be misleading.
Fixed several interoperability issues and bugs in the xrootd
mover. If xrootd is used we strongly recommend upgrading to
1.9.5-15 or 1.9.6-3.
WebDAV
File upload on multi-homed doors has been fixed. Without this fix
pools would possibly connect to the wrong IP. The configuration
parameters webdavAddress and webdavInternalAddress
have been added to solve this problem.
Error reporting to billing has been improved.
A race condition in handling of mover errors has been resolved.
The HTML rendering of a directory listing is now styled using
CSS. The styling can be customized. Please read the comments in
etc/dCacheSetup.template for details on how to customize the
styling.
Authentication through the gPlazma module has been
fixed. Authentication through the gPlazma cell worked fine, however
when using the module all attempts to connect to the WebDAV door would
result in an empty page.
Info
Fixed a potential starvation issue in the info service. This should
improve responsiveness of the info service on systems that push lots
of updates into the info service.
PnfsManager
Fixed a problem in which the Chimera cleaner would wait
indefinitely for a lost reply from a pool. When the problem occurred
the cleaner had to be restarted.
Added HSM support to Chimera cleaner. When enabled in
config/dCacheSetup then Chimera cleaner now sends remove
requests to HSM attached pools for files on tape. The pool will call
the HSM integration script with the remove command. This
mimics the behaviour of the HSM cleaner for PNFS.
Improved handling of flush notification. This improves the error
reporting when using HSM integration scripts using the URI
approach.
Chimera
Fixed a problem with setting file mode for files created through
the mounted NFS 3 file system. The file mode for such files was
previously hardcoded. This problem did not affect files created
through dCache.
Admin
Fixed compatibility with the GUI.
Space Manager
Fixed update space reservation command such that
lifetime="-1" sets reservation to be non-expiring.
Pin Manager
Added the bulk unpin command.
All
Error reporting in the dCache batch processor has been
improved. The line number and file name is now reported on fatal
errors.
A bug in the dCache init script (bin/dcache) related to
stopping domains has been fixed. The bug would cause domains that were
in the middle of an automatic restart to be ignored by the
stop command. Also the start and restart
commands would not recognize such domains as already running and would
start a second instance. Part of the fix is that the bin/dcache
status command now shows when a domain is restarting. One
consequence of the fix is that we now store two PID files; one for the
Java process and one for the shell script that handles the restart
logic. The scripts shipped with dCache have been updated to handle
this change, but if you have custom scripts that rely on the PID
files, then those need to be updated.
A script compatibility issue with Solaris has been fixed. The bug
caused the automatic restart of domains to fail on Solaris.
The init script now uses an environment variable to pass the
CLASSPATH to the Java Virtual Machine. This solves a problem where the
command line length exceeded the maximum length allowed. It also makes
the output of utilities like ps nicer to read as the
excessively long CLASSPATH is no longer included.
The log4j related commands previously exposed through the System
cell are now available in all cells. The commands still affect the
logging of the complete domain.
Detailed changelog 1.9.6-2 to 1.9.6-3
- [r13307] http://www.dcache.org/rt/Ticket/Display.html?id=5397
- [r13311] Removal of two not working example-files
- [r13316] core: pool selection: propagate UnknownHostException
- [r13322] dcap: refactor command processing loop
- [r13331] cells: Refactor CommandExitException handling
- [r13334] pool: Fix migration of deleted files
- [r13337] scripts: Fix 'dcache stop' behaviour in case of domain restart
- [r13340] WebDAV: Fix address resolution for PUT
- [r13342] WebDAV: Refine error message reporting to billing
- [r13357] Stylesheet for the webdav door
- [r13359] file icon for the webdav door
- [r13360] Directory icon for the webdav door
- [r13361] dCache logo for the webdav door
- [r13365] WebDAV: Fix error handling in filters
- [r13367] WebDAV: Fix compatibility with using GPlazma as a module
- [r13369] srm: Fix deadlock in GetRequest
- [r13371] script: Fix daemon script Solaris compatibility
- [r13374] scripts: Use environment variable for classpath
- [r13377] pool: Fix stage error handling
- [r13381] pool: Don't disable pool on sync
- [r13399] all: Expose log4j cell commands in all cells
- [r13403] init script: Fix SRM restart regression
- [r13407] Job scripts: Fix Solaris compatibility
- [r13411] info: switch state read/write lock to use the fair variant
- [r13418] chimera: respect file mode specified by client
- [r13422] pool: Enlarge thread pool size for xrootd accept tasks
- [r13427] Changed method call for checking stage permissions
- [r13428] Chimera cleaner: added reply timeout for the disk-pool when sending PoolRemoveFilesMessage
- [r13430] admin: Fix binary mode (aka GUI support)
- [r13434] poolmanager: do not clean files location in waiting state
- [r13435] pool: Fix xrootd inoperability issue for vector read
- [r13438] xrootd: Fix retry on bind failure
- [r13441] pool: Fix reuse of TCP port in xrootd mover
- [r13448] scripts: fix Solaris grep compatibility
- [r13459] fix update space reservation function such that lifetime="-1" sets
- [r13470] Adding css support for the webdav door
- [r13471] adding css support for the webdav door
- [r13472] Adding css support for the webdav door
- [r13473] adding css support for the webdav door
- [r13474] adding css support for the webdav door
- [r13475] adding css support for the webdav door
- [r13476] renaming file name and variables
- [r13477] renaming file name and variable for css webdav door
- [r13478] renaming file name and variables for the css support for the webdav
- [r13479] renaming file name and variables for the css support for the webdav door
- [r13480] renaming file
- [r13485] This patch adds HSM-Cleaner functionality to ChimeraCleaner
- [r13486] Adding HSM-ChimeraCleaner related configuration
- [r13493] core: include unsupported hsmType into the error message
- [r13498] core: make hsm locations handling more robust
- [r13500] PinManager: add bulk unpin, remove potential memory leak due to the incorrect usage of the replied requiered property of the messsage
- [r13505] unit-tests: fix storage-info used in PnfsManagerTest
Changes in 1.9.6-2
Important fixes
An XROOTD write corruption issue has been fixed in the pool. This
issue was introduced in version 1.9.5-1 and we recommend that all data
written through XROOTD to an affected version of the pool code is
verified for integrity. Notice that the dCache checksum cannot be used
to verify the integrity of the file, as the checksum is computed from
corrupted data. We urge all sites running a prior version of 1.9.5 and
using XROOTD for writing into dCache to immediately upgrade to at
least version 1.9.5-11 or 1.9.6-2.
Compatibility
The SrmSpaceManager and the transfer managers have been taken out
of the srm.cell file and moved into the
spacemanager.cell and transfermanagers.cell
files. Both of these new files are executed from
config/srm.batch and the cells are still started inside the
SRM domain, however care should be taken on upgrade if
config/srm.batch has been customized.
dCache 1.9.6-1 was unfortunately not compatible with pools from
earlier versions. This has now been resolved and dCache 1.9.6-2 head
nodes can work with dCache 1.9.5 pools. Pools from earlier versions
are not supported.
Head nodes and doors
The Kerberos FTP door has been broken since the release of dCache
1.9.5-1. The problem has been resolved in dCache 1.9.5-11 and
1.9.6-2.
The SRM list implementation has been reconstructured to no longer
require the mounted file system. Thus starting with dCache 1.9.6-2,
the only node requiring access to the mounted file system is
PnfsManager when using PNFS. With Chimera no dCache node requires
access to the mounted file system. The performance characteristics of
the new implementation are different. Single file listings and
non-verbose listings should perform as before, while verbose listing
on some setups appear faster and other setups a bit slower. If you
observe drastic changes in srmLs performance, please notify support at
dcache dot org.
A new version of PNFS was released in November 2009. The new
version supports registering all deleted files in a special table in
the PNFS database. dCache 1.9.5-12 has new options to configure the
PnfsManager to let dCache consult this table when it cannot find a
PNFS entry for a file. If found in this table, dCache pools will
delete such orphaned files. The configuration options are
pnfsDeleteRegistration,
pnfsDeleteRegistrationDbUser,
pnfsDeleteRegistrationDbPass. Please consult
etc/dCacheSetup.template for details.
Several possile sources of the spurious "Already have one record"
write failures have been fixed. Please inform us if you observe any
changes in behaviour after upgrading to dCache 1.9.6-2.
The command to stage files administratively through the PinManager
was broken since dCache 1.9.5-1. This problem has been fixed.
Two configuration parameters of thread manager are now
configurable. The configuration parameters are
threadManagerThreads and threadManagerTimeout and
can be redefined in config/dCacheSetup.
Several deadlocks in the SRM have been resolved.
dCache supports running multiple DCAP doors on the same
host. Support for this has however been broken since dCache 1.9.3-1
due to bugs in the init script. This has now been fixed.
The feature to override configuration parameters on a per cell
level by adding a key value per list in the batch files did
unfortunately not work in dCache 1.9.6-1. In dCache 1.9.6-2 this
should work as described in the 1.9.6 release notes.
Pools
An issue with recovery of broken SI files in the pool's control/
directory has been fixed. That issue prevented pools with such files
from starting. With dCache 1.9.5-12 the SI file is recovered from
PnfsManager.
Another problem affecting recovery of meta data on pools was that
the recovery logic did not respect the lfs=volatile setting. This has
been fixed to correctly mark files on such pools as cached.
Starting with dCache 1.9.3-1, dCache would call fsync(2)
after upload to synchronize the new file's in-core state with the
storage device. Failure during sync would be logged, but was not
considered a fatal error. After talking to several sites, we have
decided to consider failure to sync to be a fatal error. Starting with
dCache 1.9.5-12 the transfer will be considered failed and the pool
will be disabled.
The callout to the HSM script on dCache pools has been updated such
that all HSM locations of a file are passed to the script.
Detailed changelog 1.9.6-1 to 1.9.6-2
- [r13106] srm: extract space manager and transfer managers from srm domain
- [r13109] srm: fix info output
- [r13113] chimera: switch to new chimera
- [r13115] pool: Fix xrootd write corruption
- [r13120] FTP: Log JGlobus exception on authentication
- [r13134] Scripts: Fix Solaris sed compatibility
- [r13136] FTP: Restore Kerberos FTP Door functionality
- [r13137] GssFtp door, set subject appropriately in case of kpwd authentication
- [r13139] Implemenation of SrmLs that utilized ListDirecroryHandler
- [r13148] PnfsManager: add pnfs manager deletion registration interface configuration
- [r13151] make SrmUnmarkSpaceAsBeingUsedCompanion.unmarkSpace blocking
- [r13159] pool: Recover corrupted SI file
- [r13163] pool: Log pool size when complaining about file system size
- [r13167] core: changing the name of a method: getUpdateTime() instead of
- [r13169] PinManager: restore admin stage functionality
- [r13172] srm: improve error reporting by pgpass, do not hide errors by returing null pass, but rather propogate SQLException
- [r13181] http://rb.dcache.org/r/1345/
- [r13185] pool: Respect LFS mode during meta data recovery
- [r13189] pool: Log lock file name on failure to acquire lock
- [r13192] pool: Disable pool on sync failure
- [r13199] infoprovider: update BDII-bug workaround so all RFC-2253 must-markup characters are converted to a colon
- [r13215] PnfsManager: Fix compatibility with 1.9.5 cells
- [r13218] cells: Fix misleading error message in CellShell
- [r13220] deployment: Clean up a couple of batch file related issues
- [r13225] xylophone: tidy-up of DN calculation and link-breaking
- [r13237] hsm: pass all hsm locations to restore script
- [r13241] hsmcleaner: Fix shutdown sequence
- [r13244] httpd: Fix shutdown sequence
- [r13247] srm: Fix another deadlock
- [r13249] srm: Fix minor locking bug
- [r13252] srm: Remove deadlock
- [r13254] ThreadManager: Fix shutdown sequence
- [r13263] srm: add Tranafer Managers and Space Manager cells to utility domain to preserve configuration compatibility with 1.9.6-1, add Tranafer Managers and Space Manager cells to utility domain
- [r13266] chimera: sort locations according priorities
- [r13271] srm: fix startup-ping so failures are reported
- [r13275] chimera: ignore empty tags
- [r13277] srm: Fixed yet another deadlock
- [r13281] cells: Make cellClass configurable
- [r13282] srm: reverting recent changes to srm and utility domains to preserve compatibility, trasfer manager and space manager are started in srm domain as before
- [r13285] webdav: Moved defaults from Spring XML to batch file
- [r13287] core: add linkGroup name into error message
- [r13290] webdav: Move configuration defaults from Spring XML to batch file
- [r13292] config: Fix configuration parameter override in batch files
- [r13294] scripts: Fix handling of multiple dcap doors
- [r13300] cells: Log bugs in command processing
Changed in 1.9.6-1
WebDAV
From Wikipedia: "Web-based Distributed Authoring and Versioning,
or WebDAV, is a set of extensions to the Hypertext Transfer Protocol
(HTTP) that allows computer-users to edit and manage files
collaboratively on remote World Wide Web servers. RFC 4918 defines the
extensions." Wikipedia further notes: "The WebDAV protocol
allows interactivity, making the Web a readable and writable medium,
in line with Tim Berners-Lee's original vision. It allows users to
create, change and move documents on a remote server (typically a web
server or "web share"). This has obvious uses when authoring the
documents that a web server serves, but it can also be used for
storing files on the web, so that the files can be accessed from
anywhere.
WebDAV is supported by all modern operating systems out of the
box. This includes Windows XP, Windows Vista, Windows 7, Mac OS X, and
Gnome and KDE shells for Linux and Unixes.
dCache 1.9.6 introduces the WebDAV door. The WebDAV door supports
both unauthenticated HTTP and HTTPS and client certificate based
authentication over HTTPS. The current version does not support HTTP
basic authentication, but this will likely be added in dCache
1.9.7. The WebDAV door supersedes the old HTTP door and we therefore
no longer ship httpdoor.batch with dCache.
To enable the WebDAV door, specify webdav as a service in
etc/node_config. You may want to adjust the default
parameters for the WebDAV door in config/dCacheSetup. Consult
etc/dCacheSetup.template for available parameters and
documentation on those parameters.
To enable WebDAV over HTTPS, set webdavProtocol=https in
config/dCacheSetup. The host and CA certificates need to be
imported to be readable by the Java SSL libraries. To do this, run the
commands bin/dcache import hostcert and bin/dcache import
cacerts. Consult the man page of the dcache script and the
documentation in etc/dCacheSetup.template for available
options.
dCache domains and batch files
A dCache installationen consists of a number of dCache domains,
each containing a number of cells. Each cell represent a
service. dCache domains are defined through batch files in the
config/ directory. We have urged sites not to modify these
batch files as doing so would make upgrading more difficult. However
many sites wish to change which cells are running in which
domain. Until now, there was no way to move a cell from one domain to
another without modifying the batch files.
To solve this problem, we have refactored all batch files in dCache
1.9.6. Each batch file has been split into a number of
.fragment and .cell files. Those .fragment
and .cell files are not user editable. The batch files in
config/ now consist of exec instructions that include
the .fragment and .cell files. By rearranging those
exec commands in the batch files, one can easily move a cell
from one domain to another. To show an example, the definition of the
dCacheDomain looks like this:
onerror shutdown
exec -shell file:${ourHomeDir}/share/cells/logging.fragment
exec -shell file:${ourHomeDir}/share/cells/setup.fragment
exec -shell file:${ourHomeDir}/share/cells/tunnel.fragment
exec -shell file:${ourHomeDir}/share/cells/poolmanager.cell
exec -shell file:${ourHomeDir}/share/cells/dummy-prestager.cell
exec -shell file:${ourHomeDir}/share/cells/broadcast.cell
exec -shell file:${ourHomeDir}/share/cells/loginbroker.cell
All domains must execute the first three fragment files, but the
remaining execution of the cell files can be moved between
domains.
To make it easier to have several instances of the same cell, for
instance a door, we have restructured the parameter parsing such that
configuration parameters can be overriden on the exec line in
the batch file. To show an example, this is the default batch file for
a domain containing a GridFTP door:
onerror shutdown
exec -shell file:${ourHomeDir}/share/cells/logging.fragment
exec -shell file:${ourHomeDir}/share/cells/setup.fragment
exec -shell file:${ourHomeDir}/share/cells/tunnel.fragment
exec -shell file:${ourHomeDir}/share/cells/gridftp.cell GFTP-${thisHostname}
To start two GridFTP cells within the same domain, we could change
this to:
onerror shutdown
exec -shell file:${ourHomeDir}/share/cells/logging.fragment
exec -shell file:${ourHomeDir}/share/cells/setup.fragment
exec -shell file:${ourHomeDir}/share/cells/tunnel.fragment
exec -shell file:${ourHomeDir}/share/cells/gridftp.cell GFTP1-${thisHostname} \
gsiFtpPortNumber=2811
exec -shell file:${ourHomeDir}/share/cells/gridftp.cell GFTP2-${thisHostname} \
gsiFtpPortNumber=2812 gsiftpIoQueue=local
This would start one GridFTP cell called
GFTP1-<hostname> on port 2811 and another GridFTP cell
called GFTP2-<hostname> on port 2812. The first would use
the default mover queue on pools, the second would use the queue
called local. All parameters definable in
etc/dCacheSetup can be overriden in this way (this is
currently not true for PoolManager and pools, but that will be fixed
soon).
SRM on Terracotta
Terracotta is a distribute shared memory platform. With Terracotta
SRM can be scaled to run on multiple hosts. dCache 1.9.6 introduces
Terracotta support for SRM. This is currently considered an
experimental feature which we are testing in preproduction
environments. If you are interested in this feature, please contact
support at dcache.org for detailed instructions.
SRM and Access Control Lists
dCache 1.9.4 added ACL support to FTP and DCAP. dCache 1.9.5 added
partial ACL support to the SRM door. In the dCache 1.9.6 release ACL
support has been extended to all SRM operations, including srmCopy,
srmPrepareToGet and srmPrepareToPut. ACL support needs to be
configured in PnfsManager by setting aclEnabled to
true in config/dCacheSetup.
Related to ACL support is that all permission checks for SRM are
now delegated to PnfsManager.
Other changes
We have further cleaned up logging in SRM, SpaceManager and
gPlazma. The log files should be less noisy now.
NFS 4.1 compliance has been improved a lot in dCache 1.9.6.
Persistent command history in the admin shell now has to be
explicitly enabled by defining adminHistoryFile in
config/dCacheSetup.
Detailed changelog 1.9.5-1 to 1.9.6-1
- [r12364] Added openssl-devel to the build required packages on the spec file
- [r12367] Remove a couple of redundant checks
- [r12372] http://rb.dcache.org/r/780/, Refactor the discovery of AbstractStorageElement, which is now given by getStorage of srm, eliminate hard links to this object in requests
- [r12373] Add intended remote domain name to tunnel cell listening for incoming connection
- [r12374] servlet-api: Upgrade to version 2.5
- [r12385] Pure white space change
- [r12400] TWO environment variables in dcap lib
- [r12401] TWO environment variables in dcap lib
- [r12403] http://rb.dcache.org/r/781/, srm: add options for running srm under the management of the terracotta server
- [r12404] http://rb.dcache.org/r/794/, srm: Refactor the discovery of SRM configuration, which is now given by getConfiguration of srm, eliminate hard links to this object in requests
- [r12423] srm: Ensure that ACLs are checked for advisory delete
- [r12425] pool: Add checksum verification on update
- [r12427] PnfsManager: Add STORAGEINFO file attribute
- [r12428] pnfsManager: Add last access time til FileAttributes
- [r12429] http://rb.dcache.org/r/804,
- [r12430] Alter Info service to use AbstractCell instead of CellAdaptor
- [r12436] pnfsManager: Add support for setting MODE, OWNER and OWNER_GROUP
- [r12437] Remove hard-coded registration of the info cell as well-known
- [r12443] Remove trailing white space
- [r12444] namespace: refactor chimera based cleaner
- [r12445] srm: hold a write lock during cache cleanup
- [r12451] srm: Fix infinite job reload loop
- [r12452] Adjust stop and restart, and add condrestart operations to the dcache script
- [r12459] Fixing typos in install.sh which broke mounting correctly
- [r12460] srm: removed redundant job state update
- [r12462] pool: Implement accept timeout for xrootd
- [r12463] PnfsManager: Add CREATION_TIME to FileAttributes
- [r12466] SRM: Remove deadlock caused by implicitly calling job's toString() method in log statements of the SharedMemoryCache
- [r12469] pool: Fix recovery of BROKEN files with Berkeley DB backend
- [r12471] Removed trailing white space
- [r12473] namespace: fix regression in chimera cleaner
- [r12476] srm: fix regression introduced by r12451
- [r12477] PnfsManager: Make PnfsGetFileMetaDataMessage a subclass of PnfsGetFileAttributes
- [r12478] ChimeraCleaner: make use of c3po db connection pool
- [r12479] srm: Transition PinCompanion to CellStub and MessageCallback
- [r12481] test: Update test cases to match changes in r12468 and r12469
- [r12483] srm: Delete unused PutInSpaceCompanion class
- [r12484] srm: Java 5 and logging cleanup for PutCompanion
- [r12485] Added missing class ThreadManagerMessageCallback
- [r12486] Quick and dirty improvement of exception
- [r12489] Adding an rpm dependency to SRM server and SRM client RPM's on java > 1.5 so the client can be deployed on glite 3.2
- [r12494] get rid of warnings on any srmclient call
- [r12496] dcap: removed unused code
- [r12498] external: update postgres jdbc driver to latest version
- [r12503] get rid of annoying SRM_PATH warning http://rb.dcache.org/r/843/
- [r12508] Changing to the new node.config format for dCacheConfigure.sh
- [r12521] http://rb.dcache.org/r/805/, make setState of job synchronized, remove synchronization on job in places where this is done only to setState, improve error handling and reporting
- [r12522] http://rb.dcache.org/r/852/, remove unused imports
- [r12523] http://rb.dcache.org/r/849/, Remove unnecessary comments and some of the unused code
- [r12524] dcap: remote pending movers on client disconnect
- [r12525] Maintenance of the metric expunge: increase available logging and simplify flow very slightly
- [r12527] cells: Unify exec implementation and URI handling
- [r12528] pool: Fix sticky bit expiration
- [r12530] dcap: door: cleanups missed in patch r12524
- [r12532] nfs: door: fixed orphan movers
- [r12534] Add a couple more debug statements
- [r12536] cells: Unify interpretation of ${0} when doing exec
- [r12538] namespace: do not store access latency and retention policy on crate
- [r12540] core: remove obsolete PoolReserveSpaceMessage
- [r12542] Make the build for gsi and kerberos tunnels optional, controlled with
- [r12544] pool: Fix deadlock in MigrationModule
- [r12545] Fix exception when info message times out
- [r12547] Catch any Throwable thrown by a scheduled task in the ScheduledExecutorService in StateMaintainer
- [r12549] build: drop build dependencies
- [r12552] PoolManager: Always enable watchdog
- [r12554] PoolManager: Fix PoolUp handling to take lost message into account
- [r12557] pool: Handle UnresolvedAddressException in FTP mover
- [r12559] Downgrade an info SIP logging message since it isn't an error
- [r12564] http://rb.dcache.org/r/880/,
- [r12571] pool: Fix Xrootd connection timeout
- [r12575] xrootd: Initialize request ID in pool request message
- [r12578] Fix info service SIP NAS naming and PaintInfo equals
- [r12580] scripts: Don't mount name space on GridFTP doors
- [r12582] scripts: Fix generic.lib.sh compatibility with Solaris
- [r12584] pool: Fix file system size check in case Java doesn't know the size
- [r12589] add Obsoletes: dcache-dcap in new dcap packaging spec files
- [r12592] pinmanager: Fix unauthorized pinning
- [r12594] config: Refactors batch files to make it easy to move cells
- [r12595] cells: Protect cell against SerializationException
- [r12599] pinmanager: Adjust timeout settings
- [r12600] deploy: remove runtime dependencies on java
- [r12602] dcap: fix cell framgment
- [r12603] extend "update space reservation" admin command with ability to modify
- [r12605] xylophone: fix bugs in attr support for dynamic, repeated attributes
- [r12607] libdcap: steer client active/passive mode by environment variable
- [r12608] changing release number for next RC candidates
- [r12610] Setting code version to match package version
- [r12615] xylophone: fix bug preventing use of list-iterator item from evaluation within a predicate
- [r12616] http://rb.dcache.org/r/926/, srm: amend the terracotta config file with options needed to run all supported scheduled srm requests under the terracotta management
- [r12617] rpc: disable Grizzly's default idle connection timeout
- [r12620] Update info provider to publish space tokens and installed capacity information
- [r12621] Add file mistakenly omitted from previous commit
- [r12622] srm: Use new PermissionHandler in SRM
- [r12624] Remove old comment
- [r12625] srm: Transition GetFileInfoCompanion to CellStub
- [r12626] nfs: mover: keep track of space allocation
- [r12628] Provides dcap
- [r12632] Update of version numbers to avoid conflicts
- [r12635] http://rb.dcache.org/r/925/, srm : slightly refactor scheduler queue sync code, remove scheduler queue usage of job operations in syncronized code to eliminate potential deadlock
- [r12636] review in http://rb.dcache.org/r/948/
- [r12638] Stripped trailing white space
- [r12639] Striped trailing white space
- [r12640] srm: Transition SRM from PnfsGetFileMetaData/StorageInfo to PnfsGetFileAttributes
- [r12641] reverting old patch untill abiguity is removed
- [r12646] pool: Make P2P Acceptor more robust
- [r12649] cells: Clean up NDC references on thread termination
- [r12650] deploy: assume that servicelocator is localhost
- [r12653] PoolManager: Stop ping handler on error
- [r12654] namespace: always store file size in level2
- [r12655] srm: Add new listDirectory method to AbstractStorageElement
- [r12657] PnfsManager: Fix permission checking for list and lookup
- [r12660] SpaceManager: Renamed and fixed group and role options of update command
- [r12664] PnfsManager: Make permission update more robust
- [r12665] chimera: update inodes ctime on metadata change
- [r12668] namespace: use global path on id to name mapping
- [r12671] http://rb.dcache.org/r/959, change the printing of the message that sending checksum failed from error output to debug output
- [r12673] http://rb.dcache.org/r/946/, Remove unused GLUE Web Service Libraries, and all the legacy code that is related to the GLUE in srm clients and servers
- [r12674] http://rb.dcache.org/r/958/, srm: fix a bug: srm getJob returns null if request was created in different jvm, and placed in this jvm's SharedMemoryCache by Terracotta
- [r12675] http://rb.dcache.org/r/968/, update gPlazma logging to using log4j directly and to use loggers that have names equal to fully qualified class names
- [r12676] http://rb.dcache.org/r/978/, gPlazma: add authorization id to the CDC context before sending message to gPlazma to enable the auth id logging on gPlazma side
- [r12678] PnfsManager: Add workaround for negative directory entry cache problem
- [r12679] fix inode to path mapping
- [r12680] nfsv3: fix reply on server failure
- [r12683] UniversalSpringCell: Fix argument parsing in the presence of -${<number>}
- [r12689] core: pool selection unit interface change
- [r12690] http://rb.dcache.org/r/930/, http://rb.dcache.org/r/932/, http://rb.dcache.org/r/933/,
- [r12691] get rid of "Already have 1 record(s)" for non-SRM transfers
- [r12694] libs: Add Jetty
- [r12695] LoginBrokerHandler: Refactor how we handle addresses
- [r12697] info-provider: work-around lcg-infosites bug with GlueSA.ACBR
- [r12699] info-provider: add missing NearlineSize attributes
- [r12701] info-provider: add support for generating LDIF for a site-level BDII
- [r12703] info-provider: work-around lcg-infosites bug in using only deprecated attributes
- [r12705] info-provider: work-around bug in BDII due to its non-complience with RFC 2253
- [r12707] info-provider: work-around bug in how FTS obtains list of supported VOs
- [r12709] info-provider: fix publishing of SRM so (single) aliased FQDNs are supported
- [r12711] libs: Update JLine to 0.9.94
- [r12713] info-provider: improve the work-around for BDII bug
- [r12715] info-provider: suppress VOInfo objects without a VO
- [r12717] xylophone: fix lookup so they use the default value when XPath result is empty
- [r12718] Stripped trailing white space
- [r12719] core: pass storageinfo to PoolMgrQueryPoolsMsg
- [r12720] srm: Call Storage.exists rather than getFileMetaData at end of transfer
- [r12722] review in http://rb.dcache.org/r/1006
- [r12724] acl: Avoid catching RuntimeException
- [r12726] http://rb.dcache.org/r/999/, gPlazma: prevent unessesary mapping services callouts by synchronizing the gPlazma code which uses/updates caches of authorizations
- [r12731] cells: Avoid InterruptedException in CellNucleus.addToEventQueue
- [r12736] Added Jettycell - a cell with an embedded jetty server
- [r12737] http://rb.dcache.org/r/1015/, srm: remove necessary commented out lines
- [r12738] Introduce a new flag "-noTXT" for the billing cell to have more control
- [r12739] core: Let CellStub propagate NoRouteToCellException
- [r12741] doors: Fix message reordering issue in ListDirectoryHandler
- [r12743] infoprovider: add default GlueSE.ImplementationVersion
- [r12745] infoprovider: support suppress elements inside attr elements
- [r12747] infoprovider: suppress publishing empty VO GlueSA.ACBR attributes
- [r12749] http://rb.dcache.org/r/1024/, restore ls functionality, make admin srm ls look at memory cache only, do not look at databases, admin ls does not affect states of the requests, improved performance
- [r12750] srm: Get rid of dead code
- [r12752] xrootd: Fix parse error for open request
- [r12754] cells: Fix typo in EventLogger
- [r12755] nfsv4: fix session id generation
- [r12758] http://rb.dcache.org/r/1012/, srm: extract scheduler notification into a new method, move call for trying to ready state method one level up, make these methods clustered
- [r12759] deployment: no tomcat stacktrace on install
- [r12762] info: fix summary of SRM reservations so changes in ownership propagate
- [r12764] Treat any exceptions thrown by getStorageInfo in the same fashion as verification failures
- [r12767] namespace: add file owner uid and gid into storageinfo
- [r12768] infoprovider: fix typos found by Stephen Burke
- [r12770] get rid of error messages if SrmSpaceManager is disabled
- [r12771] PnfsManager: Don't check lookup permissions on root directory
- [r12773] PnfsManager: Call correct constructor in PnfsGetFileMetaDataMessage
- [r12774] webdav: Add first version of a WebDAV door
- [r12777] This patch fixes PinManager to avoid the problem of using SRM door with stage protection
- [r12778] http://rb.dcache.org/r/1052/, srm: remove absolete code for space management direcly in pools
- [r12779] http://rb.dcache.org/r/1051/, srm: PinCompanion, remove unecessary comments
- [r12781] core: remove PoolIoFileMessage(String pnfsId ..
- [r12783] infoprovider: update comments to include SAM bug Savannah references
- [r12784] xylophone: add support for additional external XML files; support for xpath-based addition
- [r12785] core: cleanup of PoolIoFileMessage.java
- [r12786] core: fix regression introduced by r12781
- [r12787] dcap: use UUID as a challenge
- [r12790] jglobus: Add CRL caching
- [r12793] infoprovider: update README to document that additional stale files must be removed
- [r12795] chimera migration: refactor comparator, extract comparison logic
- [r12796] config: Remove unused buffer size parameters from dCacheSetup.template
- [r12798] Fix trunk compilation introduced by 12795
- [r12799] Remove old unit-tests that have been replaced by new tests
- [r12800] http://rb.dcache.org/r/1050/, srm: PutCompanion is refactored to be an implementation of the MessageCallback interface and is using CellStub for communication with PnfsManager, no changes in logic
- [r12801] http://rb.dcache.org/r/1053/, srm: re-factor the space manager companions to using MessageCallback interface and CellStub, use log4j for logging
- [r12804] Fix to include path for libdcap-tunnel-gsi-1.9.3-6.x86_64.rpm compilation against EPEL
- [r12809] bumping the version number for Xrc
- [r12810] This is just confusing and gets in the way casuing the error'error: line 249: second %prep'
- [r12819] xylophone: use indirect reference to external XML file locations
- [r12821] infoprovider: add support for nearline accounting
- [r12823] add CellVersion support to AbstractCell
- [r12829] moved Ranges/Range from diskCacheV111.util one package into
- [r12830] moved Ranges/Range from diskCacheV111.util one package into
- [r12832] xylophone: add support for unique constraints for objects and attributes
- [r12834] infoprovider: remove some duplicate VOInfo objects and SA.ACBR attributes
- [r12837] http://rb.dcache.org/r/1087/, srm: allow SRM_PARTIAL_SUCCESS as a request status
- [r12842] gPlazma: Update client API to use CellStub
- [r12845] core: Fix race condition in FileMetaData.toString
- [r12846] webdav: Add SSL authentication
- [r12847] WebDAV: Updated Spring definitions
- [r12848] dcap: do not send kill to finished mover
- [r12852] webdav: Add proxied write support
- [r12853] http://rb.dcache.org/r/1061/, srm: remove explicit checking of the read permissions in the srm get and bring online requests
- [r12854] http://rb.dcache.org/r/1072/, srm: add support for ACL permission checking for srmPrepareToPut operation, move to log4j in srmPutFileRequest
- [r12857] jglobus: Upstream version of CRL cache patch
- [r12859] review: http://rb.dcache.org/r/1089/
- [r12862] pool: Use FireAndForgetTask for our thread pools
- [r12868] http://rb.dcache.org/r/1068 : fix getFileMetaData to throw proper
- [r12869] http://rb.dcache.org/r/1088 specify minimum attributes necessary for
- [r12870] http://rb.dcache.org/r/1103/ changed location of Range and Ranges
- [r12871] http://rb.dcache.org/r/1104/, srm: fix a bug that could lead to ArrayIndexOutOfBounds exception in multi file copy request
- [r12876] http://rb.dcache.org/r/1090/, srm: Remote Gridftp Transfer Manager ACL aware
- [r12877] PoolManager: Fix the magic feature
- [r12880] pool: Don't log a kill of a missing mover as an error
- [r12883] Fixed Unittest org.dcache.tests.poolmanager.LinkGroupInfoTest
- [r12885] PoolManager: Ensure that magic feature doesn't make cost negative
- [r12894] http://rb.dcache.org/r/1108/, srm: Transfer Manager uses log4j for loggin
- [r12895] PnfsManager: Refactor getFileMetaData
- [r12896] Pool: Teach HTTP mover about port ranges
- [r12902] general: fix compiler warnings about UTF-8 literals
- [r12903] Forgot to remove PnfsFileMetaDataSource in r12895
- [r12905] migration comparator: fix output if FileMetaData comparison fails
- [r12908] Changed Unittest Buildfile to include only Classes ending with "Test.java" or "Tests.java"
- [r12910] Corrected Unittest hsmRestoreTest
- [r12914] settign executable flag on infoprovidercms.rb
- [r12915] Summary: Set /opt/d-cache/libexec/infoprovidercms.rb as an executable flag
- [r12916] http://rb.dcache.org/r/1127/, srm: add new srmHosts configuration parameter, use collection of hosts when testing if surl is local
- [r12917] nfsv41: handle clients with bad stateid
- [r12919] nfs: make use of PnfsHandler
- [r12921] deploy: use rpcinfo to discover portmap service
- [r12923] Stripped trailing white space
- [r12925] fixed RepositorySubsystemTest and StorageInfoTest
- [r12926] srm: Fix get protocol check
- [r12929] srm: Let SRM know about the WebDAV root path
- [r12930] webdav: Publish correct protocol in LoginBroker
- [r12931] WebDAV: Include host name in cells domain name
- [r12932] Permissions check is added when deleting a file
- [r12934] PnfsManager: Add SIMPLE_TYPE and optimize listing
- [r12935] srm: Only request cheap attributes for non-verbose listing
- [r12936] http://rb.dcache.org/r/1145/, srm: move server code to using log4j, cleanup code, no logic changes whatsoever [1 of 4]
- [r12937] converted DOS style new lines to unix style
- [r12938] converted DOS style new lines to unix style
- [r12939] PnfsManager: Fix getFileAttributes after r12934
- [r12941] Fix auto-discover of PNFS mount-points
- [r12942] srmls: bug fix - request pnfsid only in verbose mode, request
- [r12943] core: use Set<T> interface in constructor of UserAuthRecord
- [r12945] WebDAV: Add request logging
- [r12946] WebDAV: Don't expose full path in directory listing
- [r12952] update info-provider docs (README-GLUE) with fault-finding information
- [r12953] namespace: call correct constructor in PnfsSetFileAttributes
- [r12958] Pool: Fix ConcurrentModificationException on upload
- [r12959] core: remove obsolete classes
- [r12960] Reverted part of r12958
- [r12965] -ttest poolMonitorTest
- [r12968] pnfs handler: remove methods with string representation of pnfsids
- [r12972] infoprovider: add some require GLUE attributes that were missing
- [r12975] infoprovider: publish namespace implementation within GlueSEImplementationVersion
- [r12976] Added command line options to process information system xml
- [r12979] Make dCacheConfigure.sh work with new information system configuration management ruby script
- [r12981] srm: move server code to using log4j, cleanup code, no logic changes whatsoever
- [r12982] Specify paths to JDBC files that contains the DDL statements used by toplink
- [r12984] srm: when start with terracotta ping client does not need terracotta related options
- [r12985] srmclient: do not pass logger to SrmClientV1 and GridftpClient
- [r12988] gridftp: Implement ALLO command
- [r12990] cells: Fix and clean up admin shell
- [r12991] scripts: Add commands to import certificates
- [r12992] webdav: Add CDC session to WebDAV requests
- [r12994] migration comparator: various improvements
- [r12995] BillingDB: avoid NullPointerException-message in httpdDomain.log
- [r12997] srm: copy accepts srm urls without srm port specified and assumes it is 8443
- [r12999] PoolManager: Extend magic feature to take file size into account
- [r13004] comparator: add script to run comparator
- [r13005] http: use log4j instead of System.out
- [r13007] gridftp: Replace GetFileMetaData and GetFileStorageInfo with GetFileAttributes
- [r13008] WebDAV: Refactor Transfer class; add billing and transfer monitoring
- [r13009] cells: Forgot to delete JLineReader in r12990
- [r13010] WebDAV: Generate 403 Forbidden when user lack permission to access resource
- [r13012] srm: PutCompanion: do not try to unregister dir creators if dir creation was not attempted
- [r13013] ChimeraCleaner: avoid UnsupportedOperationException
- [r13015] Pool: Restore Volatile Mode support
- [r13017] Remove tabs
- [r13018] Fixed Unittest threadManagerTest
- [r13020] http: remove replace System.out with log4j
- [r13022] dcap: do not issue prestage on non file objects
- [r13026] infoprovider: add support for default store unit (*@*)
- [r13028] Changing one parameter name according to the current settings in cleaner-chimera.cell
- [r13029] Stripped trailing white space
- [r13030] Stripped trailing white space
- [r13031] Stripped trailing white space
- [r13032] Fixed Unittest org.dcache.tests.auth.UserAuthRecordTest
- [r13034] Added a comment to AbstractCell in parse Options
- [r13035] external: Downgrade to JGlobus 1.4
- [r13039] PinManager: remove unknown sticky bits in response to MovePin request
- [r13040] dcap: decrease debug level to info for all error messages from the door
- [r13041] HSM Chimera Cleaner patch
- [r13042] Unittest pinmanagerTest temporary solution
- [r13043] Some improvements to the code of administration commands for Chimera disk-Cleaner
- [r13048] srm: protect the overwriteMode of PutRequest with locks, and include TOverwriteMode in the terracotta config
- [r13049] srm: Move log4j to Tomcat common/lib directory
- [r13050] Reverting r13049
- [r13051] srm: Move log4j to Tomcat common/lib directory
- [r13053] PnfsManager: Add access mask to PnfsMessage
- [r13054] PnfsManager, doors: Make read permission check explicit
- [r13055] srm: Fix read permission check
- [r13056] PnfsManager: Use per thread group queue for list operations
- [r13057] core: remove constructors from messages with pnfsid as a string
- [r13058] gPlazma, remove posibillity of the NullPointerException in the clean up due to exception in the constructor
- [r13062] Reverting r13035
- [r13063] PnfsManager: Don't modify attribute set returned by PermissionHandler
- [r13064] PnfsManager: Move to log4j and log exceptions correctly
- [r13065] configuration: Polish the text in dCacheSetup.template
- [r13067] junit: switch to hsqldb-1.9-rc6
- [r13068] infoprovider: remove old info-provider xylophone configuration file
- [r13069] junit: iswitch to 1.9-rc6 version of hsqldb
- [r13070] pnfs manager: pull files owner and group if storageinfo is requested
- [r13071] junit: migrate acl tests to new hsqldb
- [r13074] nfsv41: move to the recent version of NFS code
- [r13076] Reverting r13051
- [r13077] WebDAV: Update Milton from 1.4.2 to 1.4.3
- [r13078] WebDAV: Upgrade Jetty to version 20091125
- [r13079] Reverted r13077
- [r13080] WebDAV: Fix NPE in LoggingFilter
- [r13081] WebDAV: Recast exception on upload
- [r13082] pool: Fix initialization failure in migration module
- [r13086] external: Add fix for CRL race condition in JGlobus
- [r13088] 1.9.6 branch
- [r13094] WebDAV: Redirect GETs on directories with missing /
- [r13096] WebDAV: Request TYPE attribute in Transfer.readNameSpaceEntry
- [r13098] PnfsManager: Fix getFileAttribute on root