Skip to content

Add high-level Stream API to build GStreamer pipelines

Patrick Godwin requested to merge stream_api into master

This MR adds a high level Stream API to simplify the creation of building GStreamer pipelines within GstLAL. The two main drivers are to:

  • Avoid excessive boilerplate
  • Avoid exposing technical details of GStreamer's machinery to create pipelines

For this, a new module gstlal.stream has been created with a Stream class which:

  • Manages the main event loop, pipeline and handlers in one central location
  • Allows pipeparts-style elements to be registered as Stream methods, allowing chaining
  • Folds in common boilerplate typically used when creating a pipeline:
    • Gst imports and init
    • Start mainloop, pipeline instances
    • Handles state management, seeking
  • Adds a wrapper around appsink, AppSync allowing registering callbacks when new buffers arrive via stream.bufsink

This also converts all existing pipelines in gstlal, gstlal-ugly and gstlal-inspiral.

Example (gstlal_ll_dq):

tracker = NoiseTracker(options.out_path, instrument, agg_sink)

stream = Stream.from_datasource(gw_data_source_info, instrument, verbose=options.verbose)
stream.add_callback(MessageType.ELEMENT, "spectrum", tracker.on_spectrum_message)

stream.resample(quality=9) \
    .capsfilter("audio/x-raw, rate=%d" % options.sample_rate) \
    .queue(max_size_buffers=8) \
    .whiten(fft_length=options.psd_fft_length, expand_gaps=True) \
    .queue() \
    .reblock() \
    .bufsink(tracker.on_buffer)

stream.start()

Other changes as part of this:

  • multichannel_datasource.py had its main function, mkwhitened_multirate_src, split off into two parts, mkcondition and mkmultiband since multiple locations only used the previous function with a single rate defeating the purpose of the multibanding feature (e.g. gstlal_fake_frames). The module was moved to pipeparts/condition.py but that was a personal choice I am happy to change if needed.
  • mkhtgate in datasource.py was also moved to pipeparts.condition as well.
  • itacac element was also added in pipeparts as part of this as this didn't exist previously and was needed to allow creation of this element as part of the stream API.
    • gstlal_fake_frames: If the output directory does not exist, create it instead of silently failing.
Edited by Patrick Godwin

Merge request reports

Loading