Database exceptions
One of the recent condor test jobs for offline data resulted in:
Details: An unknown Database Exception has ocurred.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 100, in set_status
client.set_status(scope=scope, name=name, open=open_status)
File "/usr/local/lib/python3.6/site-packages/rucio/client/didclient.py", line 533, in set_status
raise exc_cls(exc_msg)
rucio.common.exception.DatabaseException: Database exception.
Details: An unknown Database Exception has ocurred.
2023-02-21 19:16:08,397 41030 ERROR Error setting DID LIGO.frames.O3:L-L1_HOFT_C00-12482 open_status=True
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 100, in set_status
client.set_status(scope=scope, name=name, open=open_status)
File "/usr/local/lib/python3.6/site-packages/rucio/client/didclient.py", line 533, in set_status
raise exc_cls(exc_msg)
rucio.common.exception.DatabaseException: Database exception.
Details: An unknown Database Exception has ocurred.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 553, in register_wrapper
did_set.add_replicas(prepend_str)
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 484, in add_replicas
self.add_dataset(dataset) # checks if it exists
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 380, in add_dataset
set_status(scope=self.scope, name=dataset, open_status=True, _client=self.client)
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 106, in set_status
f"={open_status}") from set_status_exc
RuntimeError: Error setting DID LIGO.frames.O3:L-L1_HOFT_C00-12482 open_status=True
The online clients also regularly show e.g.:
2023-02-22 15:45:10,613 1 INFO Finding registered, attached files matching pattern '*H1_HOFT_C00*'
2023-02-22 15:45:10,613 1 DEBUG Listing all datasets matching pattern '*H1_HOFT_C00*'
2023-02-22 15:45:10,617 1 DEBUG Starting new HTTP connection (1): rucio-server:80
2023-02-22 15:45:10,710 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/dids/search?type=collection&filters=%5B%7B%27name%27%3A+%27%2AH1_HOFT_C00%2A%27%2C+%27type%27%3A+%27dataset%27%7D%5D&long=False&recursive=False HTTP/1.1" 200 None
2023-02-22 15:45:11,117 1 DEBUG Found 74 datasets matching pattern '*H1_HOFT_C00*'
2023-02-22 15:45:11,117 1 DEBUG Compiling list of attached files from each dataset
2023-02-22 15:45:11,180 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/H-H1_HOFT_C00-13591/dids HTTP/1.1" 200 None
2023-02-22 15:45:11,260 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/H-H1_HOFT_C00-13532/dids HTTP/1.1" 200 None
2023-02-22 15:45:11,516 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/H-H1_HOFT_C00-13542/dids HTTP/1.1" 200 None
2023-02-22 15:45:12,913 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/H-H1_HOFT_C00-13519/dids HTTP/1.1" 200 None
2023-02-22 15:45:28,088 1 DEBUG http://rucio-server:80 "GET /dids/LIGO.frames.postO3/H-H1_HOFT_C00-13520/dids HTTP/1.1" 500 105
2023-02-22 15:45:28,090 1 DEBUG Encoding detection: ascii is most likely the one.
2023-02-22 15:45:28,090 1 DEBUG Encoding detection: ascii is most likely the one.
2023-02-22 15:45:28,090 1 DEBUG Encoding detection: ascii is most likely the one.
Traceback (most recent call last):
File "/usr/local/bin/gravcat", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/gravcat/__main__.py", line 260, in main
register(rse_opts, f'{scope}:{container}', disk_cache_params, did_metadata, exec_opts)
File "/usr/local/lib/python3.6/site-packages/gravcat/registration.py", line 613, in register
extension=diskcache_params['extension'])
File "/usr/local/lib/python3.6/site-packages/gravcat/utils.py", line 180, in files_to_register
attached_files.extend(list(client.list_content(scope=scope, name=dataset)))
File "/usr/local/lib/python3.6/site-packages/rucio/client/didclient.py", line 361, in list_content
raise exc_cls(exc_msg)
rucio.common.exception.DatabaseException: Database exception.
Details: An unknown Database Exception has ocurred.
Unclear if these are connected or what the cause is.