Questions and Answers about dCache


Q: Question
A: Answer
 
Q: Can dCache be used to make many separate discs appear as a single disc space?
A: The dCache provides a single logical filenamespace. You may add an arbitrary number of partitions ( ~ 500Gbyes /each ) located on a single host or (better) on different hosts to this space. Per default there is no relation between the file within the filesystem and the actual physical location within this set of partitions. We even allow replicas. The locical filenamespace is provided by NFS2. It allows to do all regular nfs v2 namespace operations (rm,mv,get attr, readdir...) except the IO (read and write) calls. To actually transfer files you may :
  • use the 'dccp' command.
  • change open,read,write,close to dc_xxx in your code and link against the libdcap.so.
  • use the PRELOAD feature which should work fine with most applications which are not linked statically.
You may use clinet application against the dCache nfs filesystem or against any other fs. The library knows when to switch.
 
Q: How can we get dCache? (I assume it is free?) Are there any formalities we need to go through (letters of authorisation etc) to get permission to use it?
A: No formalities from our side. Its free software but not open source. Code and expertise is at DESY and at FNAL. So, no need to worry that either of both may suddenly disappear.
 
Q: Why should I use dccp instead of native HSM copy application (osmcp, encp, tsmcp)?
A: On the read: with a bit of luck, you can access the file which already in the dCache and avoid the "slow" tape mount operation as well as waiting in the queue for tape drive.
On the write: you always can write without waiting for HSM, even when the HSM is off-line.
 
Q: Why dccp does not support a recursive copy?
A: To protect from human error. The dCache is virtually unlimited disk space and by misstake you could copy hundreds TB before realize it. For example:
    dccp  -r  /afs /your.realm/user/y/your_home/some.file  /pnfs/your/data/space
will copy whole AFS to your local HSM as there a space after /afs.
 
Q: Are the files safe in the dCache?
A: The files in the read pools as safe as the disk or the file system on the disks, but we have a originals in the HSM. The files in the write pools are more critical. The write pools are built on top of trusted hardware/software components and are as safe as HSM storage.
 
Q: What is the policy of flushing the write caches to the HSM.
A: There are two conditions, which trigger the flushing. The first is the amount of data stored for the same storage group , usually 10Gb and the second is a time period, usually 1 hour.
 
Q: Ok, I decided to use dccp, but others still use HSM native store/restore. How will I be affected?
A: There will be no effects on your side.
 
Q: Can I modify a file in the dCache?
A: No. The access rights of the dCache are Create/Read/Delete. The workaround: remove the old file and put the new file with the same name.
 
Q: I removed a file in pnfs(/acs) and added a new one with the same name. What will happened with the cached copies?
A: The dCache does not use the file names to address datasets. The new and the old file are different objects for the dCache and you will never get the cached copy of the old file.
 
Q: I just copied a file with dccp into pnfs(/acs). Why can't I copy it back with HSM native restore?
A: The file is still in the cache only and not visible for the HSM yet, but it's already accessible via dccp. The file will migrate to the HSM within one or two hours.
 
Q: Are there any file size limitations ?
A: The dCache itself has not a file size limit, but file within pnfs are limited to 2 Gb, as NFS2 dictates.
 
Q: Is there a way to have a file to be cached without reading it?
A: Yes. Look at the prestage mechanism in the manual of dccp or libdcap for details.
 
Q: Is there a way to check file existence in the cache?
A: Yes. Look at the check mechanism in the manual of dccp or libdcap for details. Be careful - check is an expensive operation for dCache.
 
Q: Is the dccp only way to access dCache?
A: No. There also library with POSIX like API (open/read/write/...). Check the libdcap manual for details.
 
Q: Can I use my existing application to access dCache?
A: In some cases, you can use the preload library libpdcap.so, but not all applications will work and there is no additional effort to maintain preload library.
 
Q: Is it possible to tune library parameters of the preload library?
A: Yes. There is a set of environment variables to do it.
    DCACHE_DEBUG=debug level
    DCACHE_DEBUG_FILE=output file name
    DCACHE_RA_BUFFER=buffer size in bytes
    DCACHE_RAHEAD=true/false - enable read ahead
    DCACHE_REPLY=interface name
    DCACHE_DOOR=door location
    DCACHE_IO_TUNNEL=path to tunnel pluggin
    DCACHE_WRBUFFER=true/false - enable defered write
    DCACHE_WA_BUFFER=buffer size in bytes
    DCACHE_USE_UNSAFE=true/false - disable write acknowledge
    DCACHE_CLIENT_ACTIVE=true/false - let client connect to the pool
Check the manual of dccp or libdcap for details.
 
Q: Is the library thread-safe?
A: Yes. We did our best to make the library thread-safe as mush as possible.
 
Q: Is all platform are supported?
A: No. Currently we are supporting only Solaris, Linux, IRIX and Win32. The library and dccp tested on Solaris 7,8,9, Linux with libc >= 2.1.x, IRIX 6.5 and MS Windows NT/XP.
 
Q: I have a question which is not mentioned in this FAQ. What shell I do?
A: Send your question to support@dcache.org.