...
 
Commits (83)
This diff is collapsed.
recursive-include debian *
recursive-include ligo/gracedb/test *
include ligo-gracedb.spec
include COPYING
# GraceDB client software
Client software for the <b>Gra</b>vitational-wave <b>C</b>andidate <b>E</b>vent <b>D</b>ata<b>b</b>ase, a web service that organizes candidate events from gravitational wave searches and provides an environment to record information about follow-ups.
Further documentation is available at https://wiki.ligo.org/DASWG/GraceDB and https://gracedb.ligo.org/documentation/.
Further documentation is available at https://wiki.ligo.org/Computing/DASWG/GraceDB and https://gracedb.ligo.org/documentation/.
## Workflow
* Bug reports and feature requests: submit at https://bugs.ligo.org/redmine/projects/gracedb.
......
ligo-gracedb (2.1.1-1) unstable; urgency=low
* Add new optional parameters and remove one optional parameter for VOEvent creation
* Minor bugfixes/changes to integration tests
-- Tanner Prestegard <tanner.prestegard@ligo.org> Wed, 05 Dec 2018 14:30:17 -0600
ligo-gracedb (2.1.0-1) unstable; urgency=low
* Improved error handling for failed attempts to get service information from API
* Consolidate all auth types under a single client class
* Add unit tests for handling of credentials
-- Tanner Prestegard <tanner.prestegard@ligo.org> Thu, 29 Nov 2018 10:29:36 -0600
ligo-gracedb (2.0.1-1) unstable; urgency=low
* Bugfix for truthiness checks for numeric arguments
* Remove permissions detail retrieval (no longer available on server)
-- Tanner Prestegard <tanner.prestegard@ligo.org> Tue, 02 Oct 2018 21:41:01 -0500
ligo-gracedb (2.0.0-1) unstable; urgency=low
* Package now has comprehensive Python 3.4+ compatibility
* Added functionality for managing superevent signoffs and permissions
* Removed temporary deprecation measure for including VOEvent file text in response
* Added an option for specifying a server API version
* Updated unit tests
-- Tanner Prestegard <tanner.prestegard@ligo.org> Wed, 19 Sep 2018 15:30:32 -0500
ligo-gracedb (2.0.0~1-1) unstable; urgency=low
* Added CI configuration
* Updated how package version is handled
* Removed bin/gracedb executable, now built by setup.py as an entry_point
* Added client version to request headers
* Bugfix for "faking" VOEvent file text in response
* Added "fake" VOEvent file text in response of createVOEvent
* Added superevent categories
-- Tanner Prestegard <tanner.prestegard@ligo.org> Wed, 25 Jul 2018 11:38:40 -0500
ligo-gracedb (2.0.0~0-1) unstable; urgency=low
* Identical to 1.29~1-1 other than packaging changes, but decided that we should bump the major version number
-- Tanner Prestegard <tanner.prestegard@ligo.org> Wed, 27 Jun 2018 14:22:52 -0500
ligo-gracedb (1.29~1-1) unstable; urgency=low
* More features for creating and managing superevents
-- Tanner Prestegard <tanner.prestegard@ligo.org> Thu, 21 Jun 2018 11:09:05 -0500
ligo-gracedb (1.29~0-1) unstable; urgency=low
* New features for creating and managing superevents
-- Tanner Prestegard <tanner.prestegard@ligo.org> Mon, 31 May 2018 10:30:00 -0500
ligo-gracedb (1.28-1) unstable; urgency=low
* Various fixes for Python 3
......
......@@ -9,7 +9,7 @@ X-Python3-Version: >=3.4
Package: python-ligo-gracedb
Architecture: all
Depends: ${misc:Depends}, ${python:Depends}, python-ligo-common, python-setuptools
Depends: ${misc:Depends}, ${python:Depends}, python-ligo-common, python-future, python-setuptools
Provides: ${python:Provides}
Description: Gravitational-wave Candidate Event Database - Python
The gravitational-wave candidate event database (GraCEDb) is a prototype
......@@ -21,7 +21,7 @@ Description: Gravitational-wave Candidate Event Database - Python
Package: python3-ligo-gracedb
Architecture: all
Depends: ${misc:Depends}, ${python3:Depends}, python3-ligo-common, python3-setuptools
Depends: ${misc:Depends}, ${python3:Depends}, python3-ligo-common, python3-future, python3-setuptools
Provides: ${python3:Provides}
Description: Gravitational-wave Candidate Event Database - Python 3
The gravitational-wave candidate event database (GraCEDb) is a prototype
......
%define name ligo-gracedb
%define version 1.28
%define unmangled_version 1.28
%define release 2
%define version 2.1.1
%define unmangled_version 2.1.1
%define release 1
Summary: Gravity Wave Candidate Event Database
Name: %{name}
Version: %{version}
Release: %{release}%{?dist}
Source0: %{name}-%{unmangled_version}.tar.gz
License: GPL
License: GPLv2+
Group: Development/Libraries
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
Prefix: %{_prefix}
......@@ -34,10 +34,10 @@ client tool is provided to submit a candidate event to the database.
%package -n python2-%{name}
Summary: %{summary}
Provides: %{name}
Obsoletes: %{name} < 1.27-2
Conflicts: %{name}
Obsoletes: %{name}
Requires: python-six
Requires: python2-ligo-common
Requires: python-future
%{?python_provide:%python_provide python2-%{name}}
......@@ -53,6 +53,7 @@ client tool is provided to submit a candidate event to the database.
Summary: %{summary}
Requires: python%{python3_pkgversion}-six
Requires: python%{python3_pkgversion}-ligo-common
Requires: python%{python3_pkgversion}-future
%{?python_provide:%python_provide python%{python3_pkgversion}-%{name}}
......@@ -81,10 +82,12 @@ client tool is provided to submit a candidate event to the database.
rm -rf $RPM_BUILD_ROOT
%files -n python2-%{name}
%license COPYING
%{_bindir}/gracedb
%{python2_sitelib}/*
%exclude %{python_sitelib}/ligo/gracedb/*pyo
%files -n python%{python3_pkgversion}-%{name}
%license COPYING
%{python3_sitelib}/*
%exclude %{python3_sitelib}/ligo/gracedb/test/*pyo
......@@ -17,4 +17,4 @@
# along with gracedb. If not, see <http://www.gnu.org/licenses/>.
from .version import __version__
__all__ = ["cli", "exceptions", "rest"]
__all__ = ["cli", "exceptions", "rest", "utils"]
This diff is collapsed.
......@@ -6,4 +6,4 @@ class HTTPError(Exception):
self.status = status
self.reason = reason
self.message = message
Exception.__init__(self, (status, '{} / {}'.format(reason, message)))
Exception.__init__(self, status, reason, message)
This diff is collapsed.
Testing gracedb-client
======================
Integration testing for gracedb-client
======================================
*Last updated 2 Aug 2017*
General information
......
{"BCI": 1.111, "frequency_posterior_mean": 721.23, "instruments": "H1,L1", "hrss_posterior_mean": 8.12e-23, "quality_posterior_mean": 15.2, "hrss_posterior_median": 2.19e-23, "frequency_posterior_median": 718.03, "timeslides_H1": "0.0", "Omicron_SNR_H1": 4.98, "FAR": 7.22e-06, "timeslides_L1": "0.0", "Omicron_SNR_Network": 6.91, "Omicron_SNR_L1": 4.99, "BSN": 7.19, "logBSN": 0.82, "trials_factor": 4, "likelihood": null, "gpstime": "1216336200.66", "quality_posterior_median": 15.1, "search_bin": "high_Q", "nevents": 1, "raw_FAR": 4.16e-06}
This diff is collapsed.
......@@ -34,13 +34,15 @@ For help:
Environment Variables:
TEST_SERVICE: defaults to https://gracedb-test.ligo.org/api/
TEST_DATA_DIR: defaults to $PATH_TO_GRACEDB_LIB/test/data/
TEST_DATA_DIR: defaults to $PATH_TO_GRACEDB_LIB/test/integration/data/
Files expected (in ./data/):
burst-cwb.txt
cbc-lm2.xml
cbc-lm.xml
cbc-mbta.gwf
olib-test.json
spiir-test.xml
upload2.data
upload.data
upload.data.gz
......@@ -59,7 +61,7 @@ import os
from datetime import datetime
import time
from ligo.gracedb.rest import GraceDb, GraceDbBasic
from ligo.gracedb.rest import GraceDb
from six.moves import range
......@@ -84,7 +86,7 @@ class TestGraceDb(unittest.TestCase):
# Set up client
cls._gracedb = GraceDb(TEST_SERVICE)
print("Using service {0}".format(TEST_SERVICE))
print("Using service {0}".format(cls._gracedb._versioned_service_url))
class TestMain(TestGraceDb):
......@@ -168,7 +170,6 @@ class TestMain(TestGraceDb):
Upload a large file.
Issue https://bugs.ligo.org/redmine/issues/951
"""
uploadFile = os.path.join(self.TEST_DATA_DIR, "big.data")
r = self._gracedb.writeLog(self._eventId, "FILE UPLOAD", uploadFile)
self.assertEqual(r.status, 201) # CREATED
......@@ -252,6 +253,28 @@ class TestMain(TestGraceDb):
original_far = 4.006953918826065e-7
self.assertTrue((mbta_event['far'] - original_far) < original_far*1e-14)
def test_create_olib(self):
"""Create an oLIB event"""
time.sleep(self.SLEEP_TIME)
eventFile = os.path.join(self.TEST_DATA_DIR, "olib-test.json")
event = self._gracedb.createEvent("Test", "oLIB", eventFile).json()
self.assertEqual(event['group'], "Test")
self.assertEqual(event['pipeline'], "oLIB")
self.assertEqual(event['far'], 7.22e-06)
self.assertEqual(event['extra_attributes']['LalInferenceBurst']['bci'],
1.111)
def test_create_spiir(self):
"""Create a spiir event"""
time.sleep(self.SLEEP_TIME)
eventFile = os.path.join(self.TEST_DATA_DIR, "spiir-test.xml")
event = self._gracedb.createEvent("Test", "spiir", eventFile).json()
self.assertEqual(event['group'], "Test")
self.assertEqual(event['pipeline'], "spiir")
self.assertEqual(event['far'], 3.27e-07)
self.assertEqual(event['extra_attributes']['CoincInspiral']['mass'],
3.98)
def test_create_hardwareinjection(self):
"""Create a HardwareInjection event"""
"""sim-inj.xml"""
......@@ -316,7 +339,7 @@ class TestMain(TestGraceDb):
def test_remove_label_event(self):
"""Remove label added by test_label_event"""
r = self._gracedb.removeLabel(self._eventId, self._labelName)
self.assertEqual(r.status, 200)
self.assertEqual(r.status, 204)
label_list = [l['name'] for l in
self._gracedb.labels(self._eventId).json()['labels']]
self.assertNotIn(self._labelName, label_list)
......@@ -349,7 +372,7 @@ class TestMain(TestGraceDb):
log.addHandler(handler)
try:
message = "Message is {0}".format(random.random())
log.warn(message)
log.warning(message)
finally:
log.removeHandler(handler)
finally:
......@@ -364,12 +387,13 @@ if __name__ == "__main__":
# Import other unit tests
from test_labels import TestLabels
from test_voevents import VOEventTestSuite
from test_superevents import TestSuperevents
# Set up arguments
parser = argparse.ArgumentParser(formatter_class=
argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("type", nargs='?', default='all', type=str,
choices=['all', 'main', 'labels', 'voevents'],
choices=['all', 'main', 'labels', 'superevents', 'voevents'],
help="Set of unit tests to run.")
parser.add_argument("-v", "--verbosity", default=2, type=int,
choices=[0,1,2], help="Verbosity level")
......@@ -380,6 +404,8 @@ if __name__ == "__main__":
suite_dict['main'] = unittest.TestLoader().loadTestsFromTestCase(TestMain)
suite_dict['labels'] = unittest.TestLoader().loadTestsFromTestCase(
TestLabels)
suite_dict['superevents'] = unittest.TestLoader().loadTestsFromTestCase(
TestSuperevents)
suite_dict['voevents'] = VOEventTestSuite()
# Run unit test suites
......
......@@ -104,13 +104,13 @@ ${GRACEDB} search $GRACEID --ligolw >$OUTFILE 2>&1
recordTest "search $GRACEID --ligolw" "$?" "$(cat $OUTFILE)"
rm $OUTFILE
# Make sure FAR of created LM event is correct.
# Make sure GPS time of created LM event is correct.
OUTFILE=$(mktemp /tmp/tmp.XXXXXXXXX)
${GRACEDB} search "--columns=gpstime" $GRACEID > $OUTFILE 2>&1
RETCODE=$?
if [ $RETCODE == 0 ]
then
if [ "$(grep -v '#' <$OUTFILE)" == 971609248.152 ]
if [ "$(grep -v '#' <$OUTFILE)" == 971609248.151741 ]
then
RETCODE=0
else
......@@ -128,13 +128,13 @@ recordTest "replace $GRACEID" "$RETCODE" "$(cat $OUTFILE)"
rm $OUTFILE
# Make sure FAR of replaced LM event is correct.
# Make sure GPS time of replaced LM event is correct.
OUTFILE=$(mktemp /tmp/tmp.XXXXXXXXX)
${GRACEDB} search "--columns=gpstime" $GRACEID > $OUTFILE 2>&1
RETCODE=$?
if [ $RETCODE == 0 ]
then
if [ "$(grep -v '#' <$OUTFILE)" == 971609249.152 ]
if [ "$(grep -v '#' <$OUTFILE)" == 971609249.151741 ]
then
RETCODE=0
else
......@@ -146,7 +146,7 @@ rm $OUTFILE
# Upload a file
OUTFILE=$(mktemp /tmp/tmp.XXXXXXXXX)
${GRACEDB} --tag-name=tag_test upload $GRACEID "$TEST_DATA_DIR/upload.data.gz" > $OUTFILE 2>&1
${GRACEDB} --tag-name=tag_test upload $GRACEID "$TEST_DATA_DIR/upload.data.gz" "test comment" > $OUTFILE 2>&1
recordTest "upload file $GRACEID" "$?" "$(cat $OUTFILE)"
rm $OUTFILE
......
......@@ -20,7 +20,7 @@ from __future__ import print_function
import unittest
import os
import time
from test import TestGraceDb
from ligo.gracedb.test.integration.test import TestGraceDb
class TestLabels(TestGraceDb):
"""A suite for carefully testing event creation with labels"""
......
This diff is collapsed.
......@@ -20,7 +20,7 @@ from __future__ import print_function
import unittest
import os
import time
from test import TestGraceDb
from ligo.gracedb.test.integration.test import TestGraceDb
from xml.etree import ElementTree as ET
class TestVOEvents(TestGraceDb):
......@@ -44,10 +44,15 @@ class TestVOEvents(TestGraceDb):
filecontents="Fake skymap image.", tagname="sky_loc")
# Helper functions --------------------------------------------------------
def get_citations_dict(self, voevent):
def get_citations_dict(self, graceid, voevent_filename):
"""Gets a dictionary of ivorns and citation types"""
voevent_xml = ET.fromstring(voevent['text'])
# Get voevent file
voevent_file_text = self._gracedb.files(graceid,
voevent_filename).read()
# Parse XML
voevent_xml = ET.fromstring(voevent_file_text)
citations_dict = {}
for citations in voevent_xml.iterfind('Citations'):
for e in citations.iterfind('EventIVORN'):
......@@ -55,9 +60,12 @@ class TestVOEvents(TestGraceDb):
citations_dict[ivorn] = e.attrib['cite']
return citations_dict
def get_ivorn(self, voevent):
def get_ivorn(self, graceid, voevent_filename):
"""Extracts ivorn"""
return ET.fromstring(voevent['text']).get('ivorn')
# Get voevent file
voevent_file_text = self._gracedb.files(graceid,
voevent_filename).read()
return ET.fromstring(voevent_file_text).get('ivorn')
# Tests -------------------------------------------------------------------
def test_create_preliminary_voevent(self):
......@@ -65,7 +73,6 @@ class TestVOEvents(TestGraceDb):
r = self._gracedb.createVOEvent(self._graceid, "Preliminary")
rdict = r.json()
self.assertTrue('voevent_type' in list(rdict.keys()))
preliminary_voevent_text = rdict['text']
def test_retrieve_preliminary_voevent(self):
"""Retrieve preliminary VOEvent"""
......@@ -78,7 +85,8 @@ class TestVOEvents(TestGraceDb):
"""Create an update VOEvent"""
r = self._gracedb.createVOEvent(self._graceid, "Update",
skymap_filename="fake_skymap.txt", skymap_type="FAKE",
skymap_image_filename="fake_skymap_image.txt")
skymap_image_filename="fake_skymap_image.txt",
ProbHasRemnant=0.2, BBH=0.1, Terrestrial=0.3)
rdict = r.json()
self.assertTrue('voevent_type' in list(rdict.keys()))
......@@ -91,8 +99,10 @@ class TestVOEvents(TestGraceDb):
# Make sure the ivorns are different
voevent_dict = {v['voevent_type']: v for v in voevent_list}
preliminary_ivorn = self.get_ivorn(voevent_dict['PR'])
update_ivorn = self.get_ivorn(voevent_dict['UP'])
preliminary_ivorn = self.get_ivorn(self._graceid,
voevent_dict['PR']['filename'])
update_ivorn = self.get_ivorn(self._graceid,
voevent_dict['UP']['filename'])
self.assertNotEqual(preliminary_ivorn, update_ivorn)
def test_citation_section(self):
......@@ -101,8 +111,10 @@ class TestVOEvents(TestGraceDb):
voevent_list = r.json()['voevents']
voevent_dict = {v['voevent_type']: v for v in voevent_list}
preliminary_ivorn = self.get_ivorn(voevent_dict['PR'])
update_citations = self.get_citations_dict(voevent_dict['UP'])
preliminary_ivorn = self.get_ivorn(self._graceid,
voevent_dict['PR']['filename'])
update_citations = self.get_citations_dict(self._graceid,
voevent_dict['UP']['filename'])
self.assertEqual(update_citations[preliminary_ivorn], 'supersedes')
def test_create_retraction_voevent(self):
......@@ -118,9 +130,12 @@ class TestVOEvents(TestGraceDb):
voevent_dict = {v['voevent_type']: v for v in voevent_list}
# Parse retraction voevent and check for correct citations
retraction_citations = self.get_citations_dict(voevent_dict['RE'])
preliminary_ivorn = self.get_ivorn(voevent_dict['PR'])
update_ivorn = self.get_ivorn(voevent_dict['UP'])
retraction_citations = self.get_citations_dict(self._graceid,
voevent_dict['RE']['filename'])
preliminary_ivorn = self.get_ivorn(self._graceid,
voevent_dict['PR']['filename'])
update_ivorn = self.get_ivorn(self._graceid,
voevent_dict['UP']['filename'])
self.assertTrue(retraction_citations[preliminary_ivorn] == 'retraction'
and retraction_citations[update_ivorn] == 'retraction')
......
# -*- coding: utf-8 -*-
# Copyright (C) Brian Moe, Branson Stephens (2015)
#
# This file is part of gracedb
#
# gracedb is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# It is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with gracedb. If not, see <http://www.gnu.org/licenses/>.
from __future__ import print_function
import os
import six
import unittest
try:
from unittest import mock
except ImportError:
import mock
from ligo.gracedb.rest import GraceDb
@mock.patch.object(GraceDb, 'set_up_connector')
class TestClientSetup(unittest.TestCase):
"""Set of unit tests for client constructor"""
def test_provide_x509_cert_and_key(self, mock_set_up_connector):
"""Test providing X509 cert and key to constructor"""
# Setup
cert_file = '/tmp/cert_file'
key_file = '/tmp/key_file'
# Initialize client
g = GraceDb(cred=(cert_file, key_file))
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], cert_file)
self.assertEqual(g.credentials['key_file'], key_file)
self.assertEqual(g.auth_type, 'x509')
def test_provide_x509_proxy(self, mock_set_up_connector):
"""Test providing X509 combined proxy to constructor"""
# Setup
combined_proxy_file = '/tmp/proxy_file'
# Initialize client
g = GraceDb(cred=combined_proxy_file)
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], combined_proxy_file)
self.assertEqual(g.credentials['key_file'], combined_proxy_file)
self.assertEqual(g.auth_type, 'x509')
def test_provide_username_and_password(self, mock_set_up_connector):
"""Test providing username and password to constructor"""
# Setup
username = 'user'
password = 'pw'
# Initialize client
g = GraceDb(username=username, password=password)
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['username'], username)
self.assertEqual(g.credentials['password'], password)
self.assertEqual(g.auth_type, 'basic')
def test_provide_username_only(self, mock_set_up_connector):
"""Test providing only username to constructor"""
# Setup
username = 'user'
# Initialize client
with six.assertRaisesRegex(self, RuntimeError,
'Must provide both username AND password for basic auth.'):
g = GraceDb(username=username)
def test_provide_password_only(self, mock_set_up_connector):
"""Test providing only password to constructor"""
# Setup
password = 'pw'
# Initialize client
with six.assertRaisesRegex(self, RuntimeError,
'Must provide both username AND password for basic auth.'):
g = GraceDb(password=password)
def test_provide_all_creds(self, mock_set_up_connector):
"""
Test providing X509 credentials and username/password to constructor
"""
# Setup
cert_file = '/tmp/cert_file'
key_file = '/tmp/key_file'
username = 'user'
password = 'pw'
# Initialize client
g = GraceDb(cred=(cert_file, key_file), username=username,
password=password)
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], cert_file)
self.assertEqual(g.credentials['key_file'], key_file)
self.assertEqual(g.auth_type, 'x509')
@mock.patch.object(GraceDb, '_find_x509_credentials')
def test_x509_cred_lookup(self, mock_find_x509, mock_set_up_connector):
"""Test lookup of X509 credentials"""
# Setup
cert_file = '/tmp/cert_file'
key_file = '/tmp/key_file'
mock_find_x509.return_value = (cert_file, key_file)
# Initialize client
g = GraceDb()
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], cert_file)
self.assertEqual(g.credentials['key_file'], key_file)
self.assertEqual(g.auth_type, 'x509')
def test_x509_cred_lookup_cert_key_envvars(self, mock_set_up_connector):
"""Test lookup of X509 credentials from cert and key env variables"""
# Setup
cert_file = '/tmp/cert_file'
key_file = '/tmp/key_file'
# Initialize client
with mock.patch.dict(os.environ, {'X509_USER_CERT': cert_file,
'X509_USER_KEY': key_file}):
# Initialize client
g = GraceDb()
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], cert_file)
self.assertEqual(g.credentials['key_file'], key_file)
self.assertEqual(g.auth_type, 'x509')
def test_x509_cred_lookup_proxy_envvar(self, mock_set_up_connector):
"""Test lookup of X509 credentials from proxy env variable"""
# Setup
proxy_file = '/tmp/proxy_file'
with mock.patch.dict(os.environ, {'X509_USER_PROXY': proxy_file}):
# Initialize client
g = GraceDb()
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['cert_file'], proxy_file)
self.assertEqual(g.credentials['key_file'], proxy_file)
self.assertEqual(g.auth_type, 'x509')
@mock.patch.object(GraceDb, '_find_x509_credentials')
@mock.patch('ligo.gracedb.rest.safe_netrc')
def test_basic_auth_cred_lookup(self, mock_netrc, mock_find_x509,
mock_set_up_connector):
"""Test lookup of basic auth credentials from .netrc file"""
# Force lookup to not find any X509 credentials
mock_find_x509.return_value = None
# Set up credentials and mock_netrc return
fake_creds = {
'machine': 'fake.com',
'login': 'fake_user',
'password': 'fake_password',
}
mock_netrc().authenticators.return_value = (fake_creds['login'],
None, fake_creds['password'])
# Initialize client
g = GraceDb('https://{0}/api/'.format(fake_creds['machine']))
# Check credentials
self.assertEqual(len(g.credentials), 2)
self.assertEqual(g.credentials['username'], fake_creds['login'])
self.assertEqual(g.credentials['password'], fake_creds['password'])
self.assertEqual(g.auth_type, 'basic')
@mock.patch.object(GraceDb, '_find_x509_credentials')
@mock.patch('ligo.gracedb.rest.safe_netrc')
def test_no_creds(self, mock_netrc, mock_find_x509, mock_set_up_connector):
"""Test providing no credentials and having none to look up"""
# Force lookup to not find any X509 credentials
mock_find_x509.return_value = None
# Force lookup to not find credentials in a .netrc file
mock_netrc().authenticators.return_value = None
# Initialize client
g = GraceDb()
# Check credentials
self.assertEqual(len(g.credentials), 0)
self.assertIsNone(g.auth_type)
@mock.patch.object(GraceDb, '_find_x509_credentials')
@mock.patch('ligo.gracedb.rest.safe_netrc')
def test_no_creds_fail_noauth(self, mock_netrc, mock_find_x509,
mock_set_up_connector):
"""
Test providing no credentials and having none to look up, but
requiring auth
"""
# Force lookup to not find any X509 credentials
mock_find_x509.return_value = None
# Force lookup to not find credentials in a .netrc file
mock_netrc().authenticators.return_value = None
# Initialize client
with six.assertRaisesRegex(self, RuntimeError,
'No authentication credentials found'):
g = GraceDb(fail_if_noauth=True)
def test_force_noauth(self, mock_set_up_connector):
"""Test forcing no authentication, even with X509 certs available"""
# Setup
cert_file = '/tmp/cert_file'
key_file = '/tmp/key_file'
# Initialize client
with mock.patch.dict(os.environ, {'X509_USER_CERT': cert_file,
'X509_USER_KEY': key_file}):
# Initialize client
g = GraceDb(force_noauth=True)
# Check credentials
self.assertEqual(len(g.credentials), 0)
self.assertIsNone(g.auth_type)
import os
import re
import shlex
import six
import stat
from datetime import datetime
from functools import wraps
from netrc import netrc, NetrcParseError
from subprocess import Popen, PIPE
if os.name == 'posix':
import pwd
EVENT_PREFIXES = ['G', 'E', 'H', 'M', 'T']
event_prefix_regex = re.compile(r'^({prefixes})\d+'.format(
prefixes="|".join(EVENT_PREFIXES)))
# Decorator for class methods so that they work for events or superevents
def event_or_superevent(func):
@wraps(func)
def inner(self, object_id, *args, **kwargs):
is_superevent = True
if event_prefix_regex.match(object_id):
is_superevent = False
return func(self, object_id, is_superevent=is_superevent, *args,
**kwargs)
return inner
# Function for checking arguments which can be strings or lists
# If a string, converts to list for ease of processing
def handle_str_or_list_arg(arg, arg_name):
if arg:
if isinstance(arg, six.string_types):
arg = [arg]
elif isinstance(arg, list):
pass
else:
raise TypeError("{0} arg is {1}, should be str or list"
.format(arg_name, type(arg)))
return arg
# XXX It would be nice to get rid of this if we can.
# It seems that you can pass python lists via the requests package.
# That would make putting our lists into comma-separated strings
# unnecessary.
def cleanListInput(list_arg):
stringified_list = list_arg
if isinstance(list_arg, float) or isinstance(list_arg, int):
stringified_value = str(list_arg)
return stringified_value
if not isinstance(list_arg, six.string_types):
stringified_list = ','.join(map(str, list_arg))
return stringified_list
# The following are used to check whether a user has tried to use
# an expired certificate.
# Parse a datetime object out of the openssl output.
# Note that this returns a naive datetime object.
def get_dt_from_openssl_output(s):
dt = None
err = ''
# Openssl spits out a string like "notAfter=Feb 6 15:17:54 2016 GMT"
# So we first have to split off the bit after the equal sign.
try:
date_string = s.split('=')[1].strip()
except Exception as e:
err = 'Openssl output not understood: {0}'.format(e)
return dt, err
# Next, attempt to parse the date with strptime.
try:
dt = datetime.strptime(date_string, "%b %d %H:%M:%S %Y %Z")
except Exception as e:
err = 'Could not parse time string from openssl: {0}'.format(e)
return dt, err
return dt, err
# Given a path to a cert file, check whether it is expired.
def is_expired(cert_file):
cmd = 'openssl x509 -enddate -noout -in %s' % cert_file
p = Popen(shlex.split(cmd), stdout=PIPE, stderr=PIPE)
out, err = p.communicate()
expired = None
if p.returncode == 0:
dt, err = get_dt_from_openssl_output(out)
if dt:
# Note that our naive datetime must be compared with a UTC
# datetime that has been rendered naive.
expired = dt <= datetime.utcnow().replace(tzinfo=None)
return expired, err
class safe_netrc(netrc):
"""The netrc.netrc class from the Python standard library applies access
safety checks (requiring that the netrc file is readable only by the
current user, and not by group members or other users) only if using the
netrc file in the default location (~/.netrc). This subclass applies the
same access safety checks regardless of the path to the netrc file."""
def _parse(self, file, fp, *args, **kwargs):
# Copied and adapted from netrc.py from Python 2.7
if os.name == 'posix':
prop = os.fstat(fp.fileno())
if prop.st_uid != os.getuid():
try:
fowner = pwd.getpwuid(prop.st_uid)[0]
except KeyError:
fowner = 'uid %s' % prop.st_uid
try:
user = pwd.getpwuid(os.getuid())[0]
except KeyError:
user = 'uid %s' % os.getuid()
raise NetrcParseError(
("~/.netrc file owner (%s) does not match"
" current user (%s)") % (fowner, user),
file)
if (prop.st_mode & (stat.S_IRWXG | stat.S_IRWXO)):
raise NetrcParseError(
"~/.netrc access too permissive: access"
" permissions must restrict access to only"
" the owner", file)
return netrc._parse(self, file, fp, *args, **kwargs)
__version__ = '1.28'
__version__ = '2.1.1'
......@@ -4,3 +4,6 @@ release = 1
[bdist_wheel]
universal=1
[metadata]
license_file = COPYING
......@@ -16,26 +16,47 @@
# You should have received a copy of the GNU General Public License
# along with gracedb. If not, see <http://www.gnu.org/licenses/>.
import os
import re
import sys
from setuptools import setup, find_packages
from ligo.gracedb import __version__
def parse_version(path):
"""Extract the `__version__` string from the given file
"""
with open(path, 'r') as fp:
version_file = fp.read()
version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]",
version_file, re.M)
if version_match:
return version_match.group(1)
raise RuntimeError("Unable to find version string.")
# Required packages for tests
tests_require = []
if sys.version_info.major < 3:
tests_require.append('mock')
setup(
name = "ligo-gracedb",
version = __version__,
version = parse_version(os.path.join('ligo', 'gracedb', 'version.py')),
maintainer = "Tanner Prestegard, Alexander Pace",
maintainer_email = "tanner.prestegard@ligo.org, alexander.pace@ligo.org",
description = "Gravitational Wave Candidate Event Database",
long_description = "The gravitational wave candidate event database (GraceDB) is a system to organize candidate events from gravitational wave searches and to provide an environment to record information about follow-ups. A simple client tool is provided to submit a candidate event to the database.",
url = "https://wiki.ligo.org/DASWG/GraceDB",
license = 'GPL',
license = 'GPLv2+',
namespace_packages = ['ligo'],
#provides = ['ligo.gracedb'],
packages = find_packages(),
install_requires = ['six'],
install_requires = ['future', 'six'],
tests_require = tests_require,
package_data = { 'ligo.gracedb.test' : ['data/*', 'test.sh', 'README'] },
package_data = {'ligo.gracedb.test': ['integration/data/*', 'integration/test.sh', 'integration/README']},
entry_points={
'console_scripts': [
'gracedb=ligo.gracedb.cli:main',
......