The purpose of the
pinmanager service in dCache is ensuring the presence of a file replica on disk.
It can be used by explicitly (un)pinning files via the
admin interface or the
bulk service, but is also used by the
resilience component to ensure having a certain number of replicas available. PinManager is also used for keeping a replica online after fetching (staging) it from a connected tertiary storage system (sometimes called HSM - hierarchical storage manager), if staging is allowed.
pin, also called
sticky-ness, is a concept describing a file replica on a pool that cannot be deleted for a certain duration. The pin effectively suppresses automatic garbage collection for the lifetime of the pin.
Pins may have a finite or infinite lifetime. Pins also have an owner, which may be a dCache service (such as
resilience) or client through a protocol such as
srm. Only the owner is allowed to remove unexpired pins. Several pins (for different users) can exist for the same
pnfsid, and a file is considered pinned as long as at least one unexpired pin exists.
When a pin is created, it will initially appear in state
PINNING, then transition to state
PINNED once the attempt is successful.
When a pin either has a finite lifetime that has expired or is directly requested to be removed, it is put into state
READY_TO_UNPIN. An ‘unpinning’ background task runs regularly (default every minute), which directly removes all pins in state
READY_TO_UNPIN that don’t require pool contact, then selects a certain number of pins (default 200) in state
READY_TO_UNPIN for which the corresponding pool needs to be contacted and attempts to remove them as well. During unpinning the pins are in state
On success, the pin is deleted from the pool in question as well as the database, on failure the pin is put into state
FAILED_TO_UNPIN. Another background process regularly (default every 2h) resets all pins in state
FAILED_TO_UNPIN back to state
READY_TO_UNPIN in order to make them eligible to be attempted again.
The PinManager service can be run in a shared domain. It may also be deployed in high availability ( HA) mode (coordinated via ZooKeeper) by having several PinManager cells in a dCache instance, which then need to share the same database and configuration.
Pins are managed in this central database as well as on the pools containing the replicas.
Pin expiration and pin unpinning are background tasks which are executed regularly. The property
pinmanager.expiration-period controls how often to execute these tasks. The default value is 60 seconds.
The number of pins that should at most be attempted to be removed and necessitate pool contact per unpinning task run can be configured with the property
pinmanager.max-unpins-per-run and default to 200. A value of -1 indicates that there is no limit, which might lead to large CPU and memory loads if there are many pending unpin operations.
Another background task takes care of resetting pins that previously failed to be removed. It can be configured via
pinmanager.reset-failed-unpins-period and defaults to 2h.