Skip to content

SWIG: add option to redirect standard output/error through scripting language API

Karl Wette requested to merge ANU-CGA/lalsuite:swig-redirect-stdouterr into master

Description

In some environments (e.g. Jupyter notebooks) standard output/error within the scripting language are redirected in a non-trivial way (e.g. in Jupyter notebooks, standard output/error from Python functions is redirected to output cells in the notebook). These redirections may not capture standard output/error from external C libraries such as LAL, since they happen at the scripting-language level (e.g. replacing sys.stdout) rather than at the C/kernel level. This can result in e.g. error messages from LAL being printed to a terminal (if any) instead of the environment in which the scripting language is being used (i.e. Jupyter notebook output cells).

This MR adds a feature to the SWIG wrappers to redirect standard output/error from LAL functions through the scripting language API in such a way that they will then be captured by any scripting-language level redirects. Standard output/error from a LAL function call is redirected at the C/kernel level (using dup() and dup2()) to temporary files (created with tmpfile(), which are automatically deleted on closure). After the function call returns, the captured standard output/error is output again through the scripting language API (e.g. in Python, using the C API functions PySys_WriteStdout() and PySys_WriteStderr()).

The feature is turned on/off globally for all SWIG wrappers using the swig_redirect_standard_output_error() function in the LAL library. To share the feature state between all LAL libraries, I added a file SWIGSharedVars.c to LALSupport, which defines the internal variable which stores the feature state. This variable is then accessible from all SWIG wrappers which ultimately all link against liblalsupport.so. This is a much easier way of sharing a state variable between wrappers than storing a value at the scripting-language level, which would then need to be accessed in a scripting-language-dependent way (e.g. in Python, checking that lal has been imported, getting a reference to the module, accessing an attributes which gives the feature state, etc.)

This feature is not enabled by default, since it's not needed for terminal-level applications using SWIG, and probably has some small performance hit (i.e. opening/closing temporary files on each function call). It's most useful for interactive environments, e.g. Jupyter notebooks.

Usage

e.g. Python:

import lal
lal.swig_redirect_standard_output_error()
...

API Changes and Justification

Backwards Compatible Changes

  • This change does not modify any class/function/struct/type definitions in a public C header file or any Python class/function definitions
  • This change adds new classes/functions/structs/types to a public C header file or Python module

Backwards Incompatible Changes

  • This change modifies an existing class/function/struct/type definition in a public C header file or Python module
  • This change removes an existing class/function/struct/type from a public C header file or Python module

Review Status

I have tested the functions swiglal_redirect_stdouterr() and swiglal_restore_stdouterr() to make sure I've got the system calls right (e.g. I ran them through valgrind to check for memory leaks, file descriptors left open, etc.) I've tested the feature in Jupyter notebooks and confirmed that it captures standard output/error from LAL function calls and displays them as output cells. I've not tested the feature in the Octave GUI (which I don't have access to) but confirmed that from a terminal that the feature will redirect standard output/error through Octave's output pager (instead of being printed directly to the terminal).

Merge request reports