Python package installation via pip error

Hello,
I’m trying to install few python packages via pip3, but at most of them I’m geting Errors that I don’t understand. For example:

root@turris:~# pip3 install pyshark
Collecting pyshark
  Downloading pyshark-0.4.2.9-py3-none-any.whl (31 kB)
Collecting py
  Downloading py-1.8.1-py2.py3-none-any.whl (83 kB)
     |████████████████████████████████| 83 kB 1.6 MB/s
Collecting lxml
  Downloading lxml-4.5.0.tar.gz (4.5 MB)
     |████████████████████████████████| 4.5 MB 3.6 MB/s
Installing collected packages: py, lxml, pyshark
    Running setup.py install for lxml ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"'                                                                     ; __file__='"'"'/tmp/pip-install-eln_tufx/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.c                                                                     lose();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-0uhmqqdg/install-record.txt --single-version-externally-managed --compile --insta                                                                     ll-headers /usr/include/python3.6/lxml Check the logs for full command output.

I have python3.6 installed and pip 20.0.2.
EDIT: I already updated setuptools and wheel. Now i get this:

root@turris:~/skripty# python3.6 -m pip install numpy
Collecting numpy
  Using cached numpy-1.18.2.zip (5.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3.6 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpf_8xl1dr Check the logs for full command output.

Does anyone know what is wrong? Thanks.

Hi,

I have a workaround for you. If you can not install some packages via pip[3]. You might want to check if those packages are not packaged for OpenWrt. I checked that python[3]-lxml is there.

You can install it with these following commands:

opkg update
opkg install python3-lxml
pip3 install pyshark

And in my case, I have been able to install it.

root@omnia:~# opkg install python3-lxml
Installing python3-lxml (4.4.1-3.7-1.17) to root...
Downloading https://repo.turris.cz/hbk/omnia/packages/packages/python3-lxml_4.4.1-3.7-1_arm_cortex-a9_vfpv3.ipk
Installing libxslt (1.1.33-4.0) to root...
Downloading https://repo.turris.cz/hbk/omnia/packages/packages/libxslt_1.1.33-4_arm_cortex-a9_vfpv3.ipk
Installing libexslt (1.1.33-4.0) to root...
Downloading https://repo.turris.cz/hbk/omnia/packages/packages/libexslt_1.1.33-4_arm_cortex-a9_vfpv3.ipk
Configuring libxslt.
Configuring libexslt.
Configuring python3-lxml.
root@omnia:~# pip3 install pyshark
Collecting pyshark
  Using cached https://files.pythonhosted.org/packages/b9/b0/ef87c71f7937ea8124944b2081210f9df10e47d2faa57d7c30d3e12af064/pyshark-0.4.2.9-py3-none-any.whl
Collecting py (from pyshark)
  Using cached https://files.pythonhosted.org/packages/99/8d/21e1767c009211a62a8e3067280bfce76e89c9f876180308515942304d2d/py-1.8.1-py2.py3-none-any.whl
Requirement already satisfied: lxml in /usr/lib/python3.7/site-packages (from pyshark) (4.4.1)
Installing collected packages: py, pyshark
Successfully installed py-1.8.1 pyshark-0.4.2.9
1 Like

Thank you! That worked for Pyshark, but I’m still getting another errors when installing e.g. numpy.

root@turris:~# pip3 install numpy
Collecting numpy
  Using cached numpy-1.18.2.zip (5.4 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp6bzl_666 Check the logs for full command output.

Or another package and different error:

root@turris:~# pip3 install matplotlib
Collecting matplotlib
  Downloading matplotlib-3.2.1.tar.gz (40.3 MB)
     |████████████████████████████████| 40.3 MB 11.3 MB/s
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

It will provide verbose insight as to why the installation fails, e.g.

File “numpy/core/setup.py”, line 661, in get_mathlib_info
raise RuntimeError(“Broken toolchain: cannot link a simple C program”)
RuntimeError: Broken toolchain: cannot link a simple C program
Preparing wheel metadata: finished with status ‘error’


From what I could gather numpy checks a dependency on gcc, just the libgcc does not appear to suffice. However, currently the installation of gcc does not seem to work [1]


[1] https://gitlab.labs.nic.cz/turris/turris-build/issues/136

1 Like

Here is a “small” part of pip error log.

...
2020-04-12T18:42:29,390     Running from numpy source directory.
2020-04-12T18:42:29,390     setup.py:461: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and                 expanding templates
2020-04-12T18:42:29,391       run_build = parse_setuppy_commands()
2020-04-12T18:42:29,391     /tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/system_info.py:1896: UserWarning:
2020-04-12T18:42:29,392         Optimized (vendor) Blas libraries are not found.
2020-04-12T18:42:29,392         Falls back to netlib Blas library which has worse performance.
2020-04-12T18:42:29,392         A better performance should be easily gained by switching
2020-04-12T18:42:29,393         Blas library.
2020-04-12T18:42:29,393       if self._calc_info(blas):
2020-04-12T18:42:29,393     /tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/system_info.py:1896: UserWarning:
2020-04-12T18:42:29,394         Blas (http://www.netlib.org/blas/) libraries not found.
2020-04-12T18:42:29,394         Directories to search for the libraries can be specified in the
2020-04-12T18:42:29,395         numpy/distutils/site.cfg file (section [blas]) or by setting
2020-04-12T18:42:29,395         the BLAS environment variable.
2020-04-12T18:42:29,395       if self._calc_info(blas):
2020-04-12T18:42:29,396     /tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/system_info.py:1896: UserWarning:
2020-04-12T18:42:29,396         Blas (http://www.netlib.org/blas/) sources not found.
2020-04-12T18:42:29,396         Directories to search for the sources can be specified in the
2020-04-12T18:42:29,397         numpy/distutils/site.cfg file (section [blas_src]) or by setting
2020-04-12T18:42:29,397         the BLAS_SRC environment variable.
2020-04-12T18:42:29,398       if self._calc_info(blas):
2020-04-12T18:42:29,398     /tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/system_info.py:1730: UserWarning:
2020-04-12T18:42:29,398         Lapack (http://www.netlib.org/lapack/) libraries not found.
2020-04-12T18:42:29,399         Directories to search for the libraries can be specified in the
2020-04-12T18:42:29,399         numpy/distutils/site.cfg file (section [lapack]) or by setting
2020-04-12T18:42:29,400         the LAPACK environment variable.
2020-04-12T18:42:29,400       return getattr(self, '_calc_info_{}'.format(name))()
2020-04-12T18:42:29,400     /tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/system_info.py:1730: UserWarning:
2020-04-12T18:42:29,401         Lapack (http://www.netlib.org/lapack/) sources not found.
2020-04-12T18:42:29,401         Directories to search for the sources can be specified in the
2020-04-12T18:42:29,401         numpy/distutils/site.cfg file (section [lapack_src]) or by setting
2020-04-12T18:42:29,402         the LAPACK_SRC environment variable.
2020-04-12T18:42:29,402       return getattr(self, '_calc_info_{}'.format(name))()
2020-04-12T18:42:29,402     /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'define_macros'
2020-04-12T18:42:29,404     Traceback (most recent call last):
2020-04-12T18:42:29,404       File "/usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
2020-04-12T18:42:29,405         main()
2020-04-12T18:42:29,405       File "/usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
2020-04-12T18:42:29,406         json_out['return_val'] = hook(**hook_input['kwargs'])
2020-04-12T18:42:29,406       File "/usr/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py", line 110, in prepare_metadat                a_for_build_wheel
2020-04-12T18:42:29,407         return hook(metadata_directory, config_settings)
2020-04-12T18:42:29,408       File "/tmp/pip-build-env-82ka3drp/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 1                58, in prepare_metadata_for_build_wheel
2020-04-12T18:42:29,408         self.run_setup()
2020-04-12T18:42:29,409       File "/tmp/pip-build-env-82ka3drp/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 2                50, in run_setup
2020-04-12T18:42:29,409         self).run_setup(setup_script=setup_script)
2020-04-12T18:42:29,410       File "/tmp/pip-build-env-82ka3drp/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 1                43, in run_setup
2020-04-12T18:42:29,410         exec(compile(code, __file__, 'exec'), locals())
2020-04-12T18:42:29,411       File "setup.py", line 488, in <module>
2020-04-12T18:42:29,411         setup_package()
2020-04-12T18:42:29,412       File "setup.py", line 480, in setup_package
2020-04-12T18:42:29,412         setup(**metadata)
2020-04-12T18:42:29,413       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/core.py", line 171, in setup
2020-04-12T18:42:29,413         return old_setup(**new_attr)
2020-04-12T18:42:29,414       File "/tmp/pip-build-env-82ka3drp/overlay/lib/python3.6/site-packages/setuptools/__init__.py", line 144                , in setup
2020-04-12T18:42:29,414         return distutils.core.setup(**attrs)
2020-04-12T18:42:29,415       File "/core.py", line 148, in setup
2020-04-12T18:42:29,415       File "/dist.py", line 955, in run_commands
2020-04-12T18:42:29,416       File "/dist.py", line 974, in run_command
2020-04-12T18:42:29,416       File "/tmp/pip-build-env-82ka3drp/overlay/lib/python3.6/site-packages/setuptools/command/dist_info.py",                 line 31, in run
2020-04-12T18:42:29,417         egg_info.run()
2020-04-12T18:42:29,417       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/command/egg_info.py", line 26, in run
2020-04-12T18:42:29,418         self.run_command("build_src")
2020-04-12T18:42:29,418       File "/cmd.py", line 313, in run_command
2020-04-12T18:42:29,419       File "/dist.py", line 974, in run_command
2020-04-12T18:42:29,419       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/command/build_src.py", line 146, in run
2020-04-12T18:42:29,420         self.build_sources()
2020-04-12T18:42:29,421       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/command/build_src.py", line 157, in build_sources
2020-04-12T18:42:29,421         self.build_library_sources(*libname_info)
2020-04-12T18:42:29,422       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/command/build_src.py", line 290, in build_library                _sources
2020-04-12T18:42:29,422         sources = self.generate_sources(sources, (lib_name, build_info))
2020-04-12T18:42:29,423       File "/tmp/pip-install-c0hcmv_o/numpy/numpy/distutils/command/build_src.py", line 380, in generate_sour                ces
2020-04-12T18:42:29,423         source = func(extension, build_dir)
2020-04-12T18:42:29,424       File "numpy/core/setup.py", line 661, in get_mathlib_info
2020-04-12T18:42:29,424         raise RuntimeError("Broken toolchain: cannot link a simple C program")
2020-04-12T18:42:29,424     RuntimeError: Broken toolchain: cannot link a simple C program
2020-04-12T18:42:29,579 Cleaning up...
2020-04-12T18:42:29,579   Removing source in /tmp/pip-install-c0hcmv_o/numpy
2020-04-12T18:42:29,723 Removed numpy from https://files.pythonhosted.org/packages/84/1e/ff467ac56bfeaea51d4a2e72d315c1fe440b20192fea                7e460f0f248acac8/numpy-1.18.2.zip#sha256=e7894793e6e8540dbeac77c87b489e331947813511108ae097f1715c018b8f3d from build tracker '/tmp/pi                p-req-tracker-6uffhcrv'
2020-04-12T18:42:29,724 Removed build tracker: '/tmp/pip-req-tracker-6uffhcrv'
2020-04-12T18:42:29,725 ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.6/site-packages/pip/_vendor/                pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpgm6t56qq Check the logs for full command output.
2020-04-12T18:42:29,726 Exception information:
2020-04-12T18:42:29,726 Traceback (most recent call last):
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
2020-04-12T18:42:29,726     status = self.run(options, args)
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 331, in run
2020-04-12T18:42:29,726     resolver.resolve(requirement_set)
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
2020-04-12T18:42:29,726     discovered_reqs.extend(self._resolve_one(requirement_set, req))
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
2020-04-12T18:42:29,726     abstract_dist = self._get_abstract_dist_for(req_to_install)
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_fo                r
2020-04-12T18:42:29,726     abstract_dist = self.preparer.prepare_linked_requirement(req)
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 516, in prepare_linked_re                quirement
2020-04-12T18:42:29,726     req, self.req_tracker, self.finder, self.build_isolation,
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 95, in _get_prepared_dist                ribution
2020-04-12T18:42:29,726     abstract_dist.prepare_distribution_metadata(finder, build_isolation)
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/distributions/sdist.py", line 40, in prepare_distribut                ion_metadata
2020-04-12T18:42:29,726     self.req.prepare_metadata()
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 564, in prepare_metadata
2020-04-12T18:42:29,726     self.metadata_directory = self._generate_metadata()
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 551, in _generate_metadata
2020-04-12T18:42:29,726     backend=self.pep517_backend,
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/operations/build/metadata.py", line 37, in generate_me                tadata
2020-04-12T18:42:29,726     metadata_dir
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_vendor/pep517/wrappers.py", line 170, in prepare_metadata_for_b                uild_wheel
2020-04-12T18:42:29,726     '_allow_fallback': _allow_fallback,
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_vendor/pep517/wrappers.py", line 248, in _call_hook
2020-04-12T18:42:29,726     extra_environ=extra_environ
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/utils/subprocess.py", line 275, in runner
2020-04-12T18:42:29,726     spinner=spinner,
2020-04-12T18:42:29,726   File "/usr/lib/python3.6/site-packages/pip/_internal/utils/subprocess.py", line 242, in call_subprocess
2020-04-12T18:42:29,726     raise InstallationError(exc_msg)
2020-04-12T18:42:29,726 pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /usr/bin/python3 /usr/lib                /python3.6/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpgm6t56qq Check the logs for full                 command output.

Interesting part is:

2020-04-12T18:42:29,424       File "numpy/core/setup.py", line 661, in get_mathlib_info
2020-04-12T18:42:29,424         raise RuntimeError("Broken toolchain: cannot link a simple C program")
2020-04-12T18:42:29,424     RuntimeError: Broken toolchain: cannot link a simple C program

So as you wrote, numpy needs gcc, which does not work on Turris right know. I suppose that same problem may be with other python packages… Maybe a stupid question of beginner, but is it possbile to uninstall binutils, then install gcc, install packages i need a then install binutils back?

package causality, e.g. dependency on numpy with its dependency on gcc or packages that depend on library linking from the OS.


It is entirely possible but installing binutils on its own will produce the same error. The issue lies with OpenWrt’s messy implementation of busybox plus package management; not for the first time such collision is observed.
OpenWrt is not very diverse about the target devices, foremost aiming to squeeze everything suitable for devices with (much) less capable hardware than what is provided from NIC.CZ and thus often leaves more powerful hardware in the lurch.
TOS is trying to work around that but facing challenges. IMHO busybox is completely misplaced on the NIC.CZ hardware, it would do well with full packages providing functionality squeezed into busybox otherwise.

With the aforementioned binutils installation issue been fixed it also requires python3-dev being installed for installing numpy via pip. But then/now hitting:

Failed to build numpy
ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
    status = self.run(options, args)
  File "/usr/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
    return func(self, options, args)
  File "/usr/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 378, in run
    ", ".join(r.name for r in pep517_build_failures)))
pip._internal.exceptions.InstallationError: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
Removed build tracker: '/tmp/pip-req-tracker-vc1ia4uq'

that is with setuptools v46.4.0 and wheel v0.34.2.