conveyor-submitter needs FTS cert: create an FTS-cronjob in k8s
rucio-conveyor-submitter
just failed with the following message. It needs the FTS certificate. One solution would be to put this in ceph but what does do other deployments do?
2019-09-24 20:34:31,670 6 DEBUG Sources before sorting: [(u'LIGO-CIT', u'gsiftp://ldas-grid.ligo.caltech.edu:2811/mnt/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf', '22f35a17b9a54d3dbce16b3473bc2fed', 0, 1)]
2019-09-24 20:34:31,670 6 DEBUG Sources after sorting: [(u'LIGO-CIT', u'gsiftp://ldas-grid.ligo.caltech.edu:2811/mnt/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf', '22f35a17b9a54d3dbce16b3473bc2fed', 0, 1)]
2019-09-24 20:34:31,670 6 DEBUG Transfer for request(9c57a1b364224153b325120f7cd6343b): {'schemes': [u'gsiftp', 'srm'], 'dest_spacetoken': None, 'bring_online': None, 'sources': [(u'LIGO-CIT', u'gsiftp://ldas-grid.ligo.caltech.edu:2811/mnt/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf', '22f35a17b9a54d3dbce16b3473bc2fed', 0)], 'external_host': u'https://fts3-pilot.cern.ch:8446', 'overwrite': True, 'selection_strategy': 'auto', 'file_metadata': {'dst_type': 'DISK', 'adler32': u'62eb0bdc', 'src_rse': u'LIGO-CIT', 'activity': u'User Subscriptions', 'request_type': 'transfer', 'dest_rse_id': '592f70ef82374b93abfb32a8c7c60512', 'src_type': 'DISK', 'dst_rse': u'LIGO-WA', 'md5': u'289f48db7d064bc3d5487e488c08dcbc', 'name': u'H-H1_HOFT_C02-1164558336-4096.gwf', 'src_rse_id': '22f35a17b9a54d3dbce16b3473bc2fed', 'filesize': 500544618L, 'request_id': '9c57a1b364224153b325120f7cd6343b', 'scope': O2, 'verify_checksum': 'both'}, 'copy_pin_lifetime': None, 'request_id': '9c57a1b364224153b325120f7cd6343b', 'dest_urls': [u'gsiftp://ldas-grid.ligo-wa.caltech.edu:2811/archive/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf'], 'src_spacetoken': None, 'rule_id': '0b47759a25524f3c9513750b26e5508f'}
2019-09-24 20:34:31,670 6 INFO Thread [2/2] : Got 1 transfers for User Subscriptions in 2.12567901611 seconds
2019-09-24 20:34:31,670 6 INFO Thread [2/2] : Starting to group transfers for User Subscriptions
Configuration not found.
Configuration not found.
2019-09-24 20:34:32,040 6 INFO Thread [2/2] : Starting to submit transfers for User Subscriptions
2019-09-24 20:34:32,040 6 INFO Thread [2/2] : PREPARING REQUEST 9c57a1b364224153b325120f7cd6343b DID O2:H-H1_HOFT_C02-1164558336-4096.gwf TO SUBMITTING STATE PREVIOUS None FROM [(u'LIGO-CIT', u'gsiftp://ldas-grid.ligo.caltech.edu:2811/mnt/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf', '22f35a17b9a54d3dbce16b3473bc2fed', 0)] TO [u'gsiftp://ldas-grid.ligo-wa.caltech.edu:2811/archive/rucio/O2/hoft_C02/H1/H-H1_HOFT_C02-11645/H-H1_HOFT_C02-1164558336-4096.gwf'] USING https://fts3-pilot.cern.ch:8446
2019-09-24 20:34:32,040 6 DEBUG Thread [2/2] : Start to prepare transfer
2019-09-24 20:34:32,649 6 DEBUG Thread [2/2] : Finished to prepare transfer
2019-09-24 20:34:32,650 6 INFO Thread [2/2] : About to submit job to https://fts3-pilot.cern.ch:8446 with timeout None
2019-09-24 20:34:32,658 6 WARNING Could not submit transfer to https://fts3-pilot.cern.ch:8446 - Could not find the TLS certificate file, invalid path: /opt/rucio/web/x509up
2019-09-24 20:34:32,658 6 WARNING Failed to submit transfer to https://fts3-pilot.cern.ch:8446, error: None
2019-09-24 20:34:32,659 6 ERROR Thread [2/2] : Failed to submit a job with error exceptions must be old-style classes or derived from BaseException, not unicode: Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/rucio/daemons/conveyor/common.py", line 125, in submit_transfer
user_transfer_job=user_transfer_job)
File "/usr/lib/python2.7/site-packages/rucio/core/transfer.py", line 91, in submit_bulk_transfers
transfer_id = FTS3Transfertool(external_host=external_host).submit(files=job_files, job_params=job_params, timeout=timeout)
File "/usr/lib/python2.7/site-packages/rucio/transfertool/fts3.py", line 221, in submit
raise('No transfer id returned by %s' % self.external_host)
TypeError: exceptions must be old-style classes or derived from BaseException, not unicode