python3 compatibility fixes
This merge request should get us python 3 compatibility, or at least something close to it. The unit tests all pass and imports work correctly as well as some basic runs through the pipeline. There's probably something I missed since I didn't test all combinations of Reporters, etc.
Changes:
-
dict.has_key(key)
tokey in dict
-
Error, 'message'
toError('message')
-
configparser.py
: changeConfigParser
import tofrom six.moves import configparser
to support both easily. - added a function in
utils.py
calledunpack()
. This unpacks an argument tuple and calls the target function 'func', and is used as a decorator for any function that multiprocessing uses that needs to be passed in multiple arguments. The trick in python 2 was to wrap that all up in a tuple like:func((arg1, arg2))
but this doesn't work in python 3. I have the same function inligo-scald
for multiprocessing routines to get around this issue.- Example:
@utils.unpack def _multiprocessing_fetch_triggers(data, args, kwargs):
def _multiprocessing_fetch_triggers((data, args, kwargs)):
- Example:
-
PickleReporter
: Theopen()
context managers have been modified to handlepickle.dump()
andpickle.load()
properly, by changing the file modes from 'r' and 'w' to 'rb' and 'wb', respectively. This is required in python 3 but is done implicitly in python 2. -
__setstate__
and__getstate__
for some classes for use in pickling have been modified. The same problem in multiprocessing not able to define tuples spanning multiple args is also affected here. Instead of setting the state as a tuple, we use a dict. I forget which thread I found this from, but it was a pattern I found on stackoverflow. - Some extra changes in the test suite that weren't issues before but with a newer numpy and pytest, were giving errors.