Add high-level Stream API to build GStreamer pipelines
This MR adds a high level Stream API to simplify the creation of building GStreamer pipelines within GstLAL. The two main drivers are to:
- Avoid excessive boilerplate
- Avoid exposing technical details of GStreamer's machinery to create pipelines
For this, a new module gstlal.stream
has been created with a Stream
class which:
- Manages the main event loop, pipeline and handlers in one central location
- Allows
pipeparts
-style elements to be registered as Stream methods, allowing chaining - Folds in common boilerplate typically used when creating a pipeline:
- Gst imports and init
- Start mainloop, pipeline instances
- Handles state management, seeking
- Adds a wrapper around appsink, AppSync allowing registering callbacks when new buffers arrive via
stream.bufsink
This also converts all existing pipelines in gstlal
, gstlal-ugly
and gstlal-inspiral
.
Example (gstlal_ll_dq
):
tracker = NoiseTracker(options.out_path, instrument, agg_sink)
stream = Stream.from_datasource(gw_data_source_info, instrument, verbose=options.verbose)
stream.add_callback(MessageType.ELEMENT, "spectrum", tracker.on_spectrum_message)
stream.resample(quality=9) \
.capsfilter("audio/x-raw, rate=%d" % options.sample_rate) \
.queue(max_size_buffers=8) \
.whiten(fft_length=options.psd_fft_length, expand_gaps=True) \
.queue() \
.reblock() \
.bufsink(tracker.on_buffer)
stream.start()
Other changes as part of this:
-
multichannel_datasource.py
had its main function,mkwhitened_multirate_src
, split off into two parts,mkcondition
andmkmultiband
since multiple locations only used the previous function with a single rate defeating the purpose of the multibanding feature (e.g.gstlal_fake_frames
). The module was moved topipeparts/condition.py
but that was a personal choice I am happy to change if needed. -
mkhtgate
indatasource.py
was also moved topipeparts.condition
as well. -
itacac
element was also added inpipeparts
as part of this as this didn't exist previously and was needed to allow creation of this element as part of the stream API. -
-
gstlal_fake_frames
: If the output directory does not exist, create it instead of silently failing.
-
Edited by Patrick Godwin