SciTokens and X.509 cert authentication
Added authentication for both SciTokens and X.509 certificates for all current views. This MR shouldn't be merged into the master branch, I'm just using it so that the current code can be reviewed. A few changes will still need to be made once this project has progressed further, these include:
- Adding a list of public keys, either from cluster issuers, or from a SciTokens server.
- Determining the allowed scopes that can be used.
- Setting the token subject and working out how this can be checked. Possibly from the grid-mapfile.
I have been testing this on the datafind-test VM, both with SciTokens that I've generated myself, and with proxy certificates. For the grid proxy certs, I've tested the cluster issued proxies, impersonation proxies that are generated on my laptop and then used to 'gsissh' into a cluster, and then UWM issued proxies. The first two are able to access the server, while the UWM proxy fails to authenticate.
Closes #3 (closed).
Merge request reports
Activity
@duncanmmacleod, @warren-anderson, @scott-koranda, here's is my current cleaned up code that I've been using. Would you be able to look through gwdatafind_server/authentication.py and check that you're happy with how the authentication is being done please? Thanks
This looks great!
Are you able to modularise the
validatefunction in the new auth module? In the end something like this would be easier to maintain:def validate(**outer_kwargs): def real_decorator(some_function): @wraps(some_function) def wrapper(*args, **kwargs): # Check for SciToken, use if available if 'Authorization' in request.headers: return _validate_scitoken(request, *args, **kwargs) # If no SciToken, check for and use X509 certificate if 'SSL_CLIENT_S_DN' and 'SSL_CLIENT_I_DN' in request.headers: return _validate_x509(request, *args, **kwargs) return some_function(*args, **kwargs) ...etc...
added 1 commit
- 43abcca6 - Update authentication.py: Check if subject is in /etc/grid-security/grid-mapfile file.
- gwdatafind_server/authentication.py 0 → 100644
91 92 # If no SciToken, check for and use X509 certificate 93 elif 'SSL_CLIENT_S_DN' and 'SSL_CLIENT_I_DN' in request.headers: 94 95 # Get subject and issuer from header 96 subject_dn_header = request.headers.get('SSL_CLIENT_S_DN') 97 issuer_dn_header = request.headers.get('SSL_CLIENT_I_DN') 98 99 # Clean up impersonation proxies. See: 100 # https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/api/backends.py#L119 101 subject_pattern = re.compile(r'^(.*?)(/CN=\d+)*$') 102 subject = subject_pattern.match(subject_dn_header).group(1) 103 104 # Check if subject is contained within grid-mapfile 105 with open('/etc/grid-security/grid-mapfile') as gridmap: 106 if subject in gridmap.read(): where
_parse_gridmap_subjectis just [ref]:def _parse_gridmap_subject(line): parts = line.strip().split('"') if len(parts) in {2, 3}: return parts[1] if len(parts) == 1: return parts[0] raise RuntimeError("error parsing gridmap line: {!r}".format(line))
30 32 _DEFAULT_GSIFTP_HOST = socket.gethostbyaddr(socket.gethostname())[0] 31 33 _DEFAULT_GSIFTP_PORT = 15000 32 34 35 aud = "https://ligo.org/oauth" 36 37 x509_issuers = [ 38 "/DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1", 39 "/DC=org/DC=cilogon/C=US/O=LIGO/CN=", 40 ] 58 66 # -- routes ------------------------------------------------------------------- 59 67 60 68 @blueprint.route('/', methods=['GET', 'POST']) 69 @authentication.validate(audience=aud,scp="read:/protected",x509_issuer_list=x509_issuers) - Edited by Duncan Macleod
- gwdatafind_server/authentication.py 0 → 100644
90 return ("Incorrect scope", 403, headers) 91 92 # If no SciToken, check for and use X509 certificate 93 elif 'SSL_CLIENT_S_DN' and 'SSL_CLIENT_I_DN' in request.headers: 94 95 # Get subject and issuer from header 96 subject_dn_header = request.headers.get('SSL_CLIENT_S_DN') 97 issuer_dn_header = request.headers.get('SSL_CLIENT_I_DN') 98 99 # Clean up impersonation proxies. See: 100 # https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/api/backends.py#L119 101 subject_pattern = re.compile(r'^(.*?)(/CN=\d+)*$') 102 subject = subject_pattern.match(subject_dn_header).group(1) 103 104 # Check if subject is contained within grid-mapfile 105 with open('/etc/grid-security/grid-mapfile') as gridmap: the path of the grid mapfile should be taken from the datafind server configuration file, default to the path you have used. See https://git.ligo.org/computing/ligo-data-replicator/-/blob/9ff125b6cb7b5efb26b5f18a4d18bb5bb1b87044/datafind-server/DataFindServer/LDRDataFindServer.py#L84
mentioned in merge request !5 (merged)
@duncan.meacher, I've created a new 'epic' for gwdatafind scitokens support, see gwdatafind&1. This should tie together this work on the server, and my work on the client (no MR just yet, see gwdatafind/gwdatafind#6).
What's the status of this server upgrade?
On the LIGO-SciTokens call this week (29/11/21, agenda/minutes) it was decided to set the namespace for the gwdatafind scope to be "read:/frames", though this can be changed later if needed. I've now started working on gwdatafind server again to process SciTokens.
It is possible to generate your own SciTokens on LHO ( instructons provided by @james-clark). This can only be done from:
- xpcdev1.ligo-wa.caltech.edu (only accessible from LHO, so ldas-grid.ligo-wa.caltech.edu first)
- ldas-osg-dev.ligo-wa.caltech.edu
- ldas-osg2.ligo.caltech.edu (but condor is not fully configured here)
From any of those you can then generate a SciToken with:
[duncan.meacher@ldas-grid ~]$ ssh -A duncan.meacher@xpcdev1 ... ... [duncan.meacher@xpcdev1 ~]$ htgettoken -v -a vault.ligo.org -i ligo Credkey from /home/duncan.meacher/.config/htgettoken/credkey-ligo-default: duncan.meacher Initializing kerberos client for host@vault.ligo.org Kerberos init failed: GSSError: Unspecified GSS failure. Minor code may provide more information. SPNEGO cannot find mechanisms to negotiate. Attempting OIDC authentication with https://vault.ligo.org:8200 Complete the authentication via web browser at: https://cilogon.org/device/?user_code=<user_id> Couldn't open browser with 'xdg-open', please open URL manually Storing vault token in /tmp/vt_u<user_id> Saving credkey to /home/duncan.meacher/.config/htgettoken/credkey-ligo-default: duncan.meacher Saving refresh token to https://vault.ligo.org:8200 at path secret/oauth-ligo/creds/duncan.meacher:default Getting bearer token from https://vault.ligo.org:8200 at path secret/oauth-ligo/creds/duncan.meacher:default Storing bearer token in /run/user/<user_id>/bt_<user_id>I'll continue to use my own code to generate tokens and develop the server code until the client code is set up, but I've modified it to produce a similar token.
mentioned in merge request !6 (merged)
