gstlal_ll_inspiral_event_plotter: fix stale event clean up and change elifs to ifs

Various bug fixes to get this program running smoothly:

Corrupted coinc files

We have seen that this program periodically fails (on a cadence of once every few hours) with this error:

Traceback (most recent call last):
  File "/usr/bin/gstlal_ll_inspiral_event_plotter", line 552, in <module>
    event_plotter.start()
  File "/usr/lib64/python3.6/site-packages/gstlal/events.py", line 122, in start
    self.process()
  File "/usr/lib64/python3.6/site-packages/gstlal/events.py", line 111, in process
    self.handle()
  File "/usr/bin/gstlal_ll_inspiral_event_plotter", line 236, in handle
    self.upload_ranking_plots(event)
  File "/usr/bin/gstlal_ll_inspiral_event_plotter", line 328, in upload_ranking_plots
    sngl_inspirals = dict((row.ifo, row) for row in lsctables.SnglInspiralTable.get_table(event['coinc']))
  File "/usr/lib64/python3.6/site-packages/ligo/lw/table.py", line 606, in get_table
    raise ValueError("document must contain exactly one %s table" % cls.TableName(name))
ValueError: document must contain exactly one sngl_inspiral table

Carefully debugging this, I found that the coinc files were getting corrupted during the stale event clean up in handle(). Then downstream we get this error when trying to use the coinc file to make additional plots since all the content had been removed from the coinc files. I reverted the last commit made to this part of the code (which was fixing an error due to removing events from the self.events dict while iterating over it) and replaced it with a simpler solution. I also cleaned up some of the logic around this part of the code in general.

Incorrect elif logic

I also found that the elifs that control generating the various plots for each event in the handle() method are not correct. Each of the functions:

• self.upload_ranking_data
• self.upload_ranking_plots
• self.upload_psd_plots
• self.upload_snr_plots

are independent and there's no reason why they shouldn't all get executed during one iteration of the events loop. Changing them to ifs means that plots are generated and uploaded as soon as possible.

gstlal-inspiral/bin/gstlal_ll_inspiral_event_plotter

@@ -222,33 +222,35 @@ class EventPlotter(events.EventProcessor):
 				if not event['uploaded']['ranking_data'] and event['ranking_data_path']:
 					self.upload_ranking_data(event)
 					event['uploaded']['ranking_data'] = True
-				elif not event['uploaded']['ranking_plots'] and event['ranking_data_path']:
+				if not event['uploaded']['ranking_plots'] and event['ranking_data_path']:
 					self.upload_ranking_plots(event)
 					event['uploaded']['ranking_plots'] = True
-				elif not event['uploaded']['psd_plots'] and event['psd']:
+				if not event['uploaded']['psd_plots'] and event['psd']:
 					self.upload_psd_plots(event) 
 					event['uploaded']['psd_plots'] = True
-				elif not event['uploaded']['snr_plots']:
+				if not event['uploaded']['snr_plots']:
 					self.upload_snr_plots(event)
 					event['uploaded']['snr_plots'] = True
 
-		# clean out old events
+		# clean out events once all plots are uploaded
+		# and clean out old events
 		current_time = utils.gps_now()
-		events_to_pop = []
-		for event_key, event in self.events.items():
-			uploaded = event['uploaded']
-			if uploaded['ranking_data'] and uploaded['ranking_plots'] and uploaded['psd_plots'] and uploaded['snr_plots']:
-				event['coinc'].unlink()
-				event['psd'].unlink()
-				events_to_pop.append(event_key)
-			if current_time - event['time'] >= self.max_event_time:
-				logging.info('removing stale event from {} and bin {}'.format(event['time'], event['bin']))
+		for event_key, event in copy.copy(self.events).items():
+			if self.all_plots_uploaded(event['uploaded']) or current_time - event['time'] >= self.max_event_time:
+				logging.info('removing event from {} and bin {}'.format(event['time'], event['bin']))
 				if event['coinc'] is not None:
 					logging.info('Did not receive path of ranking data file associated with event from {} and bin {}'.format(event['time'], event['bin']))
 					event['coinc'].unlink()
 					event['psd'].unlink()
-				events_to_pop.append(event_key)
-		map(lambda x: self.events.pop(x), events_to_pop)
+				self.events.pop(event_key)
+
+	@staticmethod
+	def all_plots_uploaded(uploaded):
+		"""
+		return True if all plots for 
+		the event have been uploaded
+		"""
+		return uploaded['ranking_data'] and uploaded['ranking_plots'] and uploaded['psd_plots'] and uploaded['snr_plots']
 
 	def upload_file(self, message, filename, tag, contents, graceid):
 		"""