release notes | UserGuide: 4.2, 5.0, 5.1, 5.2, 6.0, 6.1, 6.2, 7.0, 7.1, 7.2 (unreleased), 8.0, 8.1, 8.2, 9.0, 9.1, 9.2, 7.2 (unreleased) | Book: 4.2, 5.0, 5.1, 5.2, 6.0, 6.1, 6.2, 7.0, 7.1, 7.2 (unreleased), 8.0, 8.1, 8.2, 9.0, 9.1, 9.2, 7.2 (unreleased) | Wiki | Q&A black_bg
Edit me on GitHub

Macaroon Usage Examples

For the general introduction, see Macaroons

Hands-on examples on Macaroons with dCache

Macaroons can be generated and used on a HTTP level. In principle, most HTTP clients like curl or HTTP libraries should be usable to generate and use macaroons, when header information can be managed.

Good to Know

Environment Variables

Grid CA Certificates

Since the Grid infrasturcture uses separate certificate chains than the web, you have to explicitly point your tools to the Grid CA certificates. E.g., for curl use --capath ${CAPATH} where $CAPATH might point to

export CAPATH="/cvmfs/grid.cern.ch/etc/grid-security/certificates"

User Proxy/Certificate Envvars

A user needs to authenticate against a dCache instance to request an authorization macaroon. In the following, we will use a Grid user proxy for authz. Generate a valid user proxy for a mapped user, who is authorized to read/write paths on your instance.

For easier handling, we ensure that the user proxy is set in a ${X509_USER_PROXY} environment variable

export X509_USER_PROXY="/tmp/x509up_u##UID##"`

Necessary flags in curl to use the Grid user proxy for authentication are

curl --key ${X509_USER_PROXY} --cert ${X509_USER_PROXY} --cacert ${X509_USER_PROXY} ...

curl vs. HTTP redirects

curl does not follows HTTP redirects like 301 or 302. Use curl with flags --location/-L, else a curl HTTP request going to a door for a file might not follow the door’s redirect to the actual pool.

Macaroon Requests

Example Macaroon Request

Basic request to request a full-power Macaroon with a valid Grid proxy

> curl [--include,--fail] --location --key ${X509_USER_PROXY}  --cert ${X509_USER_PROXY}  --cacert ${X509_USER_PROXY}  --capath ${CAPTAH} -X POST -H 'Content-Type: application/macaroon-request' https://dcache-se-doma.desy.de:2880
HTTP/1.1 200 OK
Date: Mon, 29 Jul 2019 10:04:38 GMT
Server: dCache/5.2.1
Content-Type: application/json
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, DELETE, PUT, PROPFIND
Access-Control-Allow-Headers: Content-Type, Authorization
Content-Length: 1082
 
{
"macaroon": "MDAxY2xvY2F0...",
"uri": {
"targetWithMacaroon": "https://dcache-se-doma.desy.de:2880/?authz=MDAxY2xvY2F0aW9uIE9wd....",
"baseWithMacaroon": "https://dcache-se-doma.desy.de:2880/?authz=MDAxY2xvY2F0....",
"target": "https://dcache-se-doma.desy.de:2880/",
"base": "https://dcache-se-doma.desy.de:2880/"
}
}

For easier handling in the following, export the macaroon in an environment variable

export MACAROON="MDAxY2xvY2F0aW9uIE9wd..."

Macaoon limited in time and on the path

Generate a macaroon limited to list and download from a specific directory and is valid for 1 day and 12 hours. The caveats have top be put into the request’s header. For curl

> curl [--include,--fail] -L --key $X509_USER_PROXY  --cert ${X509_USER_PROXY}  --cacert ${X509_USER_PROXY}  --capath ${CAPATH}  -X POST  -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD,LIST"],"validity": "P1DT12H"}' https://dcache-se-doma.desy.de:2880/path/allowed/to/access

See the ISO 8061 documentaion on the syntax for durations .

At maximum, a macaroon can be valid for one week.

Listing & Reading

Listing a Directory

Listing a directory content via the PROFIND WebDAV extension to HTTP. Put the previously requested macaroon into the request’s header. Here, we pipe the resulting info page from dCache with the directory content to xmllint for better readability on the command line

> curl [--include,--fail] -L --capath ${CAPATH} -X PROPFIND -H "Depth: 1" -H "Authorization: Bearer ${MACAROON}" https://dcache-se-doma.desy.de:2880/path/allowed/to/access | xmllint --format -

Note, that we had to give two header fields separately, one for the listing depth and one containing the macaroon.

Downloading a File

Get a file with HTTP GET

 curl [--include,--fail] -L --capath ${CAPATH} -X GET -H "Authorization: Bearer ${MACAROON}" https://dcache-se-doma.desy.de:2880/path/allowed/to/access/file.foo --output /tmp/output.foo

Changing the root Directory

With the ‘root’ caveat, the base path can be limited to a subtree of the namespace. I.e., You can generate a macaroon allowing access to files living under /path/allowed/to/access/ - and limit the visible namespace to the macaroon bearer to the same subdirectory

Request a macaroon allowing for listing and downloading files under /path/allowed/to/access/ with a lifetime of 1.5 days and the root direcory constraint to the same subdirectory

> curl [--include,--fail] -L --key $X509_USER_PROXY  --cert ${X509_USER_PROXY}  --cacert ${X509_USER_PROXY} --capath ${CAPATH}  -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD,LIST","root:/path/allowed/to/access/"],"validity": "P1DT12H"}' https://dcache-se-doma.desy.de:2880/path/allowed/to/access/

For the bearer of the resulting macaroon, listing of the root directory content will then be put into the afromentioned path of the namespace

> curl [--include,--fail] -L--capath ${CAPATH} -X PROPFIND -H "Depth: 1" -H "Authorization: Bearer ${MACAROON}" https://dcache-se-doma.desy.de:2880/

Limiting the path

Similarly, one can generate a macaroon limiting the bearer’s path to a specific path, i.e., only the given file/path will be available to the macaroon bearer.

> curl [--include,--fail] -L  --key $X509_USER_PROXY  --cert ${X509_USER_PROXY}  --cacert ${X509_USER_PROXY} --capath ${CAPATH} -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD,LIST","path:/path/allowed/to/access/file.foo"],"validity": "P1DT12H"}' https://dcache-se-doma.desy.de:2880

or

> curl [--include,--fail] -L  --key $X509_USER_PROXY  --cert ${X509_USER_PROXY}  --cacert ${X509_USER_PROXY} --capath ${CAPATH} -X POST -H 'Content-Type: application/macaroon-request' -d '{"caveats": ["activity:DOWNLOAD,LIST"],"validity": "P1DT12H"}' https://dcache-se-doma.desy.de:2880/path/allowed/to/access/file.foo

More Caveats

IP Range

To limit for example the IP range to the DESY subnet, add a caveat to the header

curl ... -d '{"caveats": ["ip:2001:638:700::/48","ip:131.169.0.0/16"],"validity":...} ...

Limiting Writing/Upload

For a write-only dropbox macaroon, that would allow the bearer to write only onto https://dcache-se-doma.desy.de:2880/path/allowed/to/access/file.foo

curl ...  -d '{"caveats": ["activity:UPLOAD"],"validity":...}' https://dcache-se-doma.desy.de:2880/path/allowed/to/access/file.foo

(to allow the macaroon bearer to also list and read the path, do not forget the DOWNLOAD & LIST caveats)

File Upload/HTTP PUT

To write onto the path as the macaroon bearer

curl --location --capath ${CAPATH}  -T /path/to/file/to/be/uploaded https://dcache-se-dome.desy.de:2880/path/allowed/to/access/file.foo?authz=${MACAROON}

Optionally, one could use also the root caveat and constraint the root pathfor the bearer, so that one would not need to know the full path and probably just write to the remapped /

Directory Creation

Create a macaroon, that allows the bearer to create subdirectories

curl ... -d '{"caveats": ["activity:MANAGE,LIST"],"validity": ...}' https://dcache-se-dome.desy.de:2880/path/allowed/to/access/

and as bearer create a new sub-directory with the MKCOL method in WebDAV

curl --capath ${CAPATH} -X MKCOL https://dcache-se-doma.desy.de:2880/path/allowed/to/access/mynew.d -H "Authorization: Bearer ${MACAROON}"

Deletion

To allow a macaroon bearer the deletion of files etc., apply the DELETE caveat during creation

{"caveats": ["activity:DELETE"],"validity": ...}

The corresponding method/curl request is -X DELETE - use with care.


Macaroon I/O in ROOT

Since the HTTP libraries used by ROOT support all the I/O methods for streaming read/writes over HTTP, macaroons can in principle be used directly. For example, to read a ntuple root file from HTTP authorized with a macaroon and write a plot locally

export CAPATH=...
export INMACAROON="https://dcache-se-doma.desy.de:2880/https://dcache-se-doma.desy.de:2880/path/allowed/to/access/myfootuple.root?authz=MDA0MWxvY..."
export OUTMACAROON="/tmp/macaroonio_out.png"
#include "TH1.h"
#include "TList.h"
#include "TCanvas.h"
#include "TStyle.h"
#include "TSystem.h"
 
void macaroonio() {
  const char * INMACAROON = gSystem->Getenv("INMACAROON");
  const char * OUTMACAROON = gSystem->Getenv("OUTMACAROON");
  TFile *inFile = TFile::Open(INMACAROON);
  TH1D *h1 = new TH1D("h1","Hello Macaroon",200,5000,7000);
  TCanvas * c1 = new TCanvas("c1", "c1", 900, 600);
  c1->cd();
  TObject * inTree = inFile->Get("DecayTree");
  inTree->Draw("Bplus_MM>>h1");
  c1->SaveAs(OUTMACAROON);
}