OS X 32-bit wheels fail import in 1.15.0

[dhermes]

With the intel / fat python binaries provided by the python.org installers, trying to import in 32-bit mode fails with:

$ arch -32 /Library/Frameworks/Python.framework/Versions/3.6/bin/python -c "import numpy as np; print(np)"
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/__init__.py", line 16, in <module>
    from . import multiarray
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/gfortran/lib/i386/libgfortran.3.dylib
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/.dylibs/libopenblasp-r0.3.0.dev.dylib
  Reason: image not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
    from . import add_newdocs
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/__init__.py", line 26, in <module>
    raise ImportError(msg)
ImportError:
Importing the multiarray numpy extension module failed.  Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try `git clean -xdf` (removes all
files not under version control).  Otherwise reinstall numpy.

Original error was: dlopen(/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/gfortran/lib/i386/libgfortran.3.dylib
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/numpy/.dylibs/libopenblasp-r0.3.0.dev.dylib
  Reason: image not found

Similar errors occur in 2.7 and 3.5.

[charris]

I don't think we provide 32 bit wheels for OS X. There was a recent attempt to build fat binaries, but it failed. @matthew-brett Comment?

[charris]

Does this work for earlier NumPy wheels?

[charris]

Does Python.org use Intel?

[rgommers]

$ arch -32

IIRC this has never worked properly, even when we were distributing fat binaries as DMG installers.

[dhermes]

Does this work for earlier NumPy wheels?

$ /Library/Frameworks/Python.framework/Versions/3.6/bin/python -m virtualenv venv
...
$ venv/bin/python -m pip install 'numpy < 1.15'
...
$ venv/bin/python -m pip freeze
numpy==1.14.5
$ arch -32 venv/bin/python -c "import numpy as np; print(np)"
<module 'numpy' from '.../venv/lib/python3.6/site-packages/numpy/__init__.py'>
$ arch -64 venv/bin/python -c "import numpy as np; print(np)"
<module 'numpy' from '.../venv/lib/python3.6/site-packages/numpy/__init__.py'>

and I can say empirically that I have been running unit tests for bezier in 32-bit OS X for many releases (1.13 and 1.14 series at least) and there have been no numpy issues.

Does Python.org use Intel?

This is just the nice name for "fat" binaries on OS X. For the latest official 3.6:

$ /Library/Frameworks/Python.framework/Versions/3.6/bin/python -V
Python 3.6.6

the binary supports both architectures:

$ file /Library/Frameworks/Python.framework/Versions/3.6/bin/python
/Library/Frameworks/Python.framework/Versions/3.6/bin/python: Mach-O universal binary with 2 architectures: [i386:Mach-O executable i386] [x86_64:Mach-O 64-bit executable x86_64]
/Library/Frameworks/Python.framework/Versions/3.6/bin/python (for architecture i386):   Mach-O executable i386
/Library/Frameworks/Python.framework/Versions/3.6/bin/python (for architecture x86_64): Mach-O 64-bit executable x86_64

[matthew-brett]

Hmm - this should work. In fact, I thought we were testing 32-bit numpy, see: https://github.com/MacPython/numpy-wheels/blob/master/config.sh#L38

[dhermes]

The /usr/local/gfortran/lib/i386/libgfortran.3.dylib in particular sticks out to me since a recent-ish update to the gcc provided by Homebrew upgraded this to libgfortran.4.dylib:

$ find /usr/local/Cellar | grep -i 'i386.*libgfortran\..*\.dylib$'
/usr/local/Cellar/gcc/8.1.0/lib/gcc/8/i386/libgfortran.5.dylib
/usr/local/Cellar/gcc/7.3.0_1/lib/gcc/7/i386/libgfortran.4.dylib
/usr/local/Cellar/gcc/7.2.0/lib/gcc/7/i386/libgfortran.4.dylib

Though it may just be a red herring. It seems the < 1.15 series did not ship their own dylibs

$ venv1/bin/pip freeze
numpy==1.14.5
$ find venv1 | grep -i dylib

but 1.15 does

$ venv2/bin/pip freeze
numpy==1.15.0
$ find venv2 | grep -i dylib
venv2/lib/python3.6/site-packages/numpy/.dylibs
venv2/lib/python3.6/site-packages/numpy/.dylibs/libgfortran.3.dylib
venv2/lib/python3.6/site-packages/numpy/.dylibs/libquadmath.0.dylib
venv2/lib/python3.6/site-packages/numpy/.dylibs/libopenblasp-r0.3.0.dev.dylib
venv2/lib/python3.6/site-packages/numpy/.dylibs/libgcc_s.1.dylib

This (i.e. shipping portable dylibs) mirrors the current approach that SciPy has.

[matthew-brett]

I suspect what's happening is the delocate utility is only finding, and including, the libraries needed for the 64-bit build.

[dhermes]

Good call @matthew-brett. The gcc and openblasp-r libraries support both:

$ lipo -info venv2/lib/python3.6/site-packages/numpy/.dylibs/libopenblasp-r0.3.0.dev.dylib
Architectures in the fat file: venv2/lib/python3.6/site-packages/numpy/.dylibs/libopenblasp-r0.3.0.dev.dylib are: i386 x86_64
$ lipo -info venv2/lib/python3.6/site-packages/numpy/.dylibs/libgcc_s.1.dylib
Architectures in the fat file: venv2/lib/python3.6/site-packages/numpy/.dylibs/libgcc_s.1.dylib are: x86_64 i386

but the gfortran and quadmath libraries do not:

$ lipo -info venv2/lib/python3.6/site-packages/numpy/.dylibs/libgfortran.3.dylib
Non-fat file: venv2/lib/python3.6/site-packages/numpy/.dylibs/libgfortran.3.dylib is architecture: x86_64
$ lipo -info venv2/lib/python3.6/site-packages/numpy/.dylibs/libquadmath.0.dylib
Non-fat file: venv2/lib/python3.6/site-packages/numpy/.dylibs/libquadmath.0.dylib is architecture: x86_64

The issue is that the gcc distribution has a separate i386 and x86_64 version of libgfortran on OS X (or at least the one shipped by Homebrew):

$ find /usr/local/Cellar | grep -i 'libgfortran\.5\.dylib$' | xargs lipo -info
Non-fat file: /usr/local/Cellar/gcc/8.1.0/lib/gcc/8/i386/libgfortran.5.dylib is architecture: i386
Non-fat file: /usr/local/Cellar/gcc/8.1.0/lib/gcc/8/libgfortran.5.dylib is architecture: x86_64

@matthew-brett may remember I actually raised this issue (matthew-brett/delocate#23) with delocate. When I build OS X wheels for bezier I cobble together a "frankenstein" libgfortran that is 32-bit and 64-bit.

[matthew-brett]

@dhermes - do you also need scipy to work on 32-bit?

I could try and address this in delocate, but 32-bit is dying fast, and I'd rather just patch the build process to put in the 32-bit library, after the initial delocate run.

[dhermes]

No I don't have any need for 32-bit, was just trying to support users that might. I don't have a dog in the fight and am happy to stop building 32-bit wheels if they aren't deemed useful. (The existence of a 32-bit manylinux image was in part guiding me.)