Skip to content

fix ipython3 environment bug

Alexander Pace requested to merge gracedb-2.22.0-1 into master

I noticed when attempting to do some manual queries on gracedb-test earlier today that since the python3.9/debian11 upgrade, launching the ipython-based Django shell fails with:

# python3 manage.py shell
Traceback (most recent call last):
  File "/app/gracedb_project/manage.py", line 44, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/commands/shell.py", line 100, in handle
    return getattr(self, shell)(options)
  File "/usr/local/lib/python3.9/dist-packages/django/core/management/commands/shell.py", line 36, in ipython
    start_ipython(argv=[])
  File "/usr/lib/python3.9/site-packages/IPython/__init__.py", line 129, in start_ipython
    return launch_new_instance(argv=argv, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/traitlets/config/application.py", line 1042, in launch_instance
    app.initialize(argv)
  File "/usr/local/lib/python3.9/dist-packages/traitlets/config/application.py", line 113, in inner
    return method(app, *args, **kwargs)
  File "/usr/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 279, in initialize
    self.init_shell()
  File "/usr/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 293, in init_shell
    self.shell = self.interactive_shell_class.instance(parent=self,
  File "/usr/local/lib/python3.9/dist-packages/traitlets/config/configurable.py", line 551, in instance
    inst = cls(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 854, in __init__
    super(TerminalInteractiveShell, self).__init__(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 577, in __init__
    self.init_virtualenv()
  File "/usr/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 866, in init_virtualenv
    if p_venv.parts[1] == "cygdrive":
IndexError: tuple index out of range

Which appears to be the same as this bug but hasn't been addressed in years by ipython.

Turns out it's related to how we set the VIRTUAL_ENV variable inside the containers. I haven't done the bisection method to determine which python3/ipython3 version broke it, but the traceback points to this line from ipython where there is an assumed length of the Path(VIRTUAL_ENV).parts tuple:

In [1]: from pathlib import Path

In [2]: test = 'dummy'

In [3]: a = Path(test)

In [4]: a
Out[4]: PosixPath('dummy')

In [5]: a.parts
Out[5]: ('dummy',)

In [6]: test = '/dummy/'

In [7]: a = Path(test)

In [8]: a.parts
Out[8]: ('/', 'dummy')

This merge request changes the name of this variable to look more like an actual path so that the Django console can launch correctly.

Note that this doesn't affect the runtime functionality of GraceDB (which has been running for weeks without issue on gracedb-test and gracedb-playground), it only manifested itself when i was trying to do some manual work.

Merge request reports