Add high-level Stream API to build GStreamer pipelines
This MR adds a high level Stream API to simplify the creation of building GStreamer pipelines within GstLAL. The two main drivers are to:
- Avoid excessive boilerplate
- Avoid exposing technical details of GStreamer's machinery to create pipelines
For this, a new module gstlal.stream has been created with a Stream class which:
- Manages the main event loop, pipeline and handlers in one central location
- Allows
pipeparts-style elements to be registered as Stream methods, allowing chaining - Folds in common boilerplate typically used when creating a pipeline:
- Gst imports and init
- Start mainloop, pipeline instances
- Handles state management, seeking
- Adds a wrapper around appsink, AppSync allowing registering callbacks when new buffers arrive via
stream.bufsink
This also converts all existing pipelines in gstlal, gstlal-ugly and gstlal-inspiral.
Example (gstlal_ll_dq):
tracker = NoiseTracker(options.out_path, instrument, agg_sink)
stream = Stream.from_datasource(gw_data_source_info, instrument, verbose=options.verbose)
stream.add_callback(MessageType.ELEMENT, "spectrum", tracker.on_spectrum_message)
stream.resample(quality=9) \
.capsfilter("audio/x-raw, rate=%d" % options.sample_rate) \
.queue(max_size_buffers=8) \
.whiten(fft_length=options.psd_fft_length, expand_gaps=True) \
.queue() \
.reblock() \
.bufsink(tracker.on_buffer)
stream.start()
Other changes as part of this:
-
multichannel_datasource.pyhad its main function,mkwhitened_multirate_src, split off into two parts,mkconditionandmkmultibandsince multiple locations only used the previous function with a single rate defeating the purpose of the multibanding feature (e.g.gstlal_fake_frames). The module was moved topipeparts/condition.pybut that was a personal choice I am happy to change if needed. -
mkhtgateindatasource.pywas also moved topipeparts.conditionas well. -
itacacelement was also added inpipepartsas part of this as this didn't exist previously and was needed to allow creation of this element as part of the stream API. -
-
gstlal_fake_frames: If the output directory does not exist, create it instead of silently failing.
-
Edited by Olivia Godwin