I'm having trouble installing the following packages in a new python 3.9.7 virtual environment on Arch Linux.
My requirements.txt file:
joblib
python-dotenv
azure-cli==2.29.2
azureml-core
Steps to reproduce:
python3 -m venv venv
source venv/bin/activate
pip install -U pip
pip install -r requirements.txt
Here's the relevant part of the output of the last command, it errors when trying to install ruamel.yaml:
Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.
Installing collected packages: pycparser, urllib3, idna, chardet, cffi, certifi, six, requests, PyJWT, oauthlib, cryptography, requests-oauthlib, python-dateutil, isodate, azure-nspkg, wrapt, tabulate, pyyaml, PySocks, pyopenssl, PyNaCl, pygments, portalocker, msrest, msal, MarkupSafe, jmespath, colorama, bcrypt, azure-mgmt-nspkg, azure-core, argcomplete, applicationinsights, adal, websocket-client, vsts, pyparsing, pyasn1, psutil, pkginfo, pathlib2, paramiko, msrestazure, msal-extensions, knack, jinja2, jeepney, invoke, humanfriendly, deprecated, backports.weakref, azure-mgmt-datalake-nspkg, azure-mgmt-core, azure-common, azure-cli-telemetry, xmltodict, sshtunnel, semver, SecretStorage, scp, ruamel.yaml, pytz, PyGithub, pathspec, packaging, ndg-httpsclient, jsonpickle, jsondiff, javaproperties, fabric, docker, distro, contextlib2, backports.tempfile, azure-synapse-spark, azure-synapse-managedprivateendpoints, azure-synapse-artifacts, azure-synapse-accesscontrol, azure-storage-common, azure-multiapi-storage, azure-mgmt-web, azure-mgmt-trafficmanager, azure-mgmt-synapse, azure-mgmt-storage, azure-mgmt-sqlvirtualmachine, azure-mgmt-sql, azure-mgmt-signalr, azure-mgmt-servicefabricmanagedclusters, azure-mgmt-servicefabric, azure-mgmt-servicebus, azure-mgmt-security, azure-mgmt-search, azure-mgmt-resource, azure-mgmt-reservations, azure-mgmt-relay, azure-mgmt-redis, azure-mgmt-redhatopenshift, azure-mgmt-recoveryservicesbackup, azure-mgmt-recoveryservices, azure-mgmt-rdbms, azure-mgmt-privatedns, azure-mgmt-policyinsights, azure-mgmt-network, azure-mgmt-netapp, azure-mgmt-msi, azure-mgmt-monitor, azure-mgmt-media, azure-mgmt-marketplaceordering, azure-mgmt-maps, azure-mgmt-managementgroups, azure-mgmt-managedservices, azure-mgmt-loganalytics, azure-mgmt-kusto, azure-mgmt-keyvault, azure-mgmt-iothubprovisioningservices, azure-mgmt-iothub, azure-mgmt-iotcentral, azure-mgmt-imagebuilder, azure-mgmt-hdinsight, azure-mgmt-extendedlocation, azure-mgmt-eventhub, azure-mgmt-eventgrid, azure-mgmt-dns, azure-mgmt-devtestlabs, azure-mgmt-deploymentmanager, azure-mgmt-datamigration, azure-mgmt-datalake-store, azure-mgmt-datalake-analytics, azure-mgmt-databoxedge, azure-mgmt-cosmosdb, azure-mgmt-containerservice, azure-mgmt-containerregistry, azure-mgmt-containerinstance, azure-mgmt-consumption, azure-mgmt-compute, azure-mgmt-cognitiveservices, azure-mgmt-cdn, azure-mgmt-botservice, azure-mgmt-billing, azure-mgmt-batchai, azure-mgmt-batch, azure-mgmt-authorization, azure-mgmt-applicationinsights, azure-mgmt-appconfiguration, azure-mgmt-apimanagement, azure-mgmt-advisor, azure-loganalytics, azure-keyvault-keys, azure-keyvault-administration, azure-keyvault, azure-identity, azure-graphrbac, azure-functions-devops-build, azure-datalake-store, azure-cosmos, azure-cli-core, azure-batch, azure-appconfiguration, antlr4-python3-runtime, python-dotenv, joblib, azureml-core, azure-cli
Running setup.py install for ruamel.yaml: started
Running setup.py install for ruamel.yaml: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8y_97owt/install-record.txt --single-version-externally-managed --compile --install-headers /home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml
cwd: /tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/
Complete output (649 lines):
sys.argv ['/tmp/pip-install-ouy4o2zq/ruamel-yaml_763ae0866f7e4425a0965a00130752b0/setup.py', 'install', '--record', '/tmp/pip-record-8y_97owt/install-record.txt', '--single-version-externally-managed', '--compile', '--install-headers', '/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include/site/python3.9/ruamel.yaml']
test compiling test_ruamel_yaml
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.9
creating build/lib.linux-x86_64-3.9/ruamel
copying .ruamel/__init__.py -> build/lib.linux-x86_64-3.9/ruamel
creating build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./main.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./util.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./tokens.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarfloat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scanner.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./comments.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./loader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./serializer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./resolver.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./nodes.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./dumper.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./configobjwalker.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./timestamp.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./error.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./reader.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./cyaml.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./parser.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./__init__.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./representer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./events.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarbool.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./composer.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarint.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./emitter.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./anchor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./compat.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./constructor.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./scalarstring.py -> build/lib.linux-x86_64-3.9/ruamel/yaml
copying ./LICENSE -> build/lib.linux-x86_64-3.9/ruamel/yaml
running build_ext
building '_ruamel_yaml' extension
creating build/temp.linux-x86_64-3.9
creating build/temp.linux-x86_64-3.9/ext
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fno-semantic-interposition -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fPIC -I/home/zjeffer/Documents/school/howest-mct/semester5/mlops/labo/project/mlops-project/venv/include -I/usr/include/python3.9 -c ext/_ruamel_yaml.c -o build/temp.linux-x86_64-3.9/ext/_ruamel_yaml.o
In file included from ext/_ruamel_yaml.c:523:
ext/_ruamel_yaml.h:10: warning: "PyString_CheckExact" redefined
<a whole bunch of compiling errors & warnings>
The full log file can be found here
Note the line Using legacy 'setup.py install' for ruamel.yaml, since package 'wheel' is not installed.. If I first install the wheel package, the same error appears when installing the packages.
Installing ruamel.yaml manually works fine, but the azure packages still can't be installed.
EDIT: Installing every package one by one installs everything without errors. Why?
Changing step 3 to pip install -U pip wheel ruamel.yaml doesn't change anything.
The ruamel.yaml documentation states that it should be installed using:
pip install ruamel.yaml
so using the legacy [python] setup.py install should not be used.
Your full log indicates that it is tried to install two versions of ruamel.yaml:
Collecting ruamel.yaml<=0.17.16,>=0.17.10
Using cached ruamel.yaml-0.17.16-py3-none-any.whl (109 kB)
and
Collecting ruamel.yaml<=0.15.89,>=0.15.35
Using cached ruamel.yaml-0.15.89.tar.gz (306 kB)
Apart from a potential problem with two versions being installed, the second one will not succeed as the 0.15 ruamel.yaml series (the last one uploaded to PyPI in July 2019) doesn't support Python 3.9 first released in Oct 2020.
So when you install each package step by step, you get the latest version or ruamel.yaml. Why that step by step installation doesn't at with some package tries to install ruamel.yaml==0.15.89 as a dependency is unclear.
When I install the four packages listed in your requirements.txt, then ruamel.yaml doesn't get installed but pipdeptree indicates that azureml-core==1.36.0.post2 has possible conflicting dependencies.
When I use your requirements.txt I get the same error.
When I use your requirements.txt in a Python 3.7 venv (so that the old ruamel.yaml wheel can be installed) installation works. From that you can see that the package azureml-core==1.14.0 is installed instead of the azureml-core==1.36.0.post2
I have a dataflow code in python 3.6 that works to copy data from pubsub topic into GCS bucket but when I create a template version of it with DataflowRunner I have this error:
Pip install failed for package: -r
Output from execution of subprocess: b'Collecting apache-beam==2.27.0 (from -r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\apache-beam-2.27.0.zip\r\nCollecting avro-python3!=1.9.2,<1.10.0,>=1.8.1 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\avro-python3-1.9.2.1.tar.gz\r\nCollecting crcmod<2.0,>=1.7 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\crcmod-1.7.tar.gz\r\nCollecting dill<0.3.2,>=0.3.1.1 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\dill-0.3.1.1.tar.gz\r\nCollecting fastavro<2,>=0.21.4 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\fastavro-1.2.3.tar.gz\r\nCollecting future<1.0.0,>=0.18.2 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\future-0.18.2.tar.gz\r\nCollecting grpcio<2,>=1.29.0 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\grpcio-1.34.0.tar.gz\r\nCollecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\hdfs-2.5.8.tar.gz\r\nCollecting httplib2<0.18.0,>=0.8 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\httplib2-0.17.4.tar.gz\r\nCollecting mock<3.0.0,>=1.0.1 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\mock-2.0.0.tar.gz\r\nCollecting numpy<2,>=1.14.3 (from apache-beam==2.27.0->-r ./requirements.txt (line 1))\r\n File was already downloaded c:\\users\\kaghole\\appdata\\local\\temp\\dataflow-requirements-cache\\numpy-1.19.5.zip\r\n Installing build dependencies: started\r\n Installing build dependencies: still running...\r\n Installing build dependencies: finished with status \'error\'\r\n Complete output from command C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\Scripts\\python.exe C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\lib\\site-packages\\pip-19.0.3-py3.6.egg\\pip install --ignore-installed --no-user --prefix C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-build-env-w2zl8_gl\\overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- setuptools<49.2.0 wheel<=0.35.1 Cython>=0.29.21,<3.0:\r\n Collecting setuptools<49.2.0\r\n Using cached https://files.pythonhosted.org/packages/d0/4a/22ee76842d8ffc123d4fc48d24a623c1d206b99968fe3960039f1efc2cbc/setuptools-49.1.3.zip\r\n Collecting wheel<=0.35.1\r\n Using cached https://files.pythonhosted.org/packages/83/72/611c121b6bd15479cb62f1a425b2e3372e121b324228df28e64cc28b01c2/wheel-0.35.1.tar.gz\r\n Collecting Cython<3.0,>=0.29.21\r\n Using cached https://files.pythonhosted.org/packages/6c/9f/f501ba9d178aeb1f5bf7da1ad5619b207c90ac235d9859961c11829d0160/Cython-0.29.21.tar.gz\r\n Installing collected packages: setuptools, wheel, Cython\r\n Running setup.py install for setuptools: started\r\n Running setup.py install for setuptools: finished with status \'done\'\r\n Running setup.py install for wheel: started\r\n Running setup.py install for wheel: finished with status \'done\'\r\n Running setup.py install for Cython: started\r\n Running setup.py install for Cython: finished with status \'error\'\r\n Complete output from command C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\Scripts\\python.exe -u -c "import setuptools, tokenize;__file__=\'C:\\\\Users\\\\kaghole\\\\AppData\\\\Local\\\\Temp\\\\pip-install-nn3jr_0n\\\\Cython\\\\setup.py\';f=getattr(tokenize, \'open\', open)(__file__);code=f.read().replace(\'\\r\\n\', \'\\n\');f.close();exec(compile(code, __file__, \'exec\'))" install --record C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-record-y1h5732j\\install-record.txt --single-version-externally-managed --prefix C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-build-env-w2zl8_gl\\overlay --compile --install-headers C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\include\\site\\python3.6\\Cython:\r\n Unable to find pgen, not compiling formal grammar.\r\n running install\r\n running build\r\n running build_py\r\n creating build\r\n creating build\\lib.win-amd64-3.6\r\n copying cython.py -> build\\lib.win-amd64-3.6\r\n creating build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\CodeWriter.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\Coverage.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\Debugging.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\Shadow.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\StringIOTree.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\TestUtils.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\Utils.py -> build\\lib.win-amd64-3.6\\Cython\r\n copying Cython\\__init__.py -> build\\lib.win-amd64-3.6\\Cython\r\n creating build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\BuildExecutable.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\Cythonize.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\Dependencies.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\Distutils.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\Inline.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\IpythonMagic.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n copying Cython\\Build\\__init__.py -> build\\lib.win-amd64-3.6\\Cython\\Build\r\n creating build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\AnalysedTreeTransforms.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\Annotate.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\AutoDocTransforms.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\Buffer.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\Builtin.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\CmdLine.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying Cython\\Compiler\\Code.py -> build\\lib.win-amd64-3.6\\Cython\\Compiler\r\n copying build_ext\r\n building \'Cython.Plex.Scanners\' extension\r\n error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/\r\n \r\n ----------------------------------------\r\n Command "C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\Scripts\\python.exe -u -c "import setuptools, tokenize;__file__=\'C:\\\\Users\\\\kaghole\\\\AppData\\\\Local\\\\Temp\\\\pip-install-nn3jr_0n\\\\Cython\\\\setup.py\';f=getattr(tokenize, \'open\', open)(__file__);code=f.read().replace(\'\\r\\n\', \'\\n\');f.close();exec(compile(code, __file__, \'exec\'))" install --record C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-record-y1h5732j\\install-record.txt --single-version-externally-managed --prefix C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-build-env-w2zl8_gl\\overlay --compile --install-headers C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\include\\site\\python3.6\\Cython" failed with error code 1 in C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-install-nn3jr_0n\\Cython\\\r\n \r\n ----------------------------------------\r\nCommand "C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\Scripts\\python.exe C:\\Users\\kaghole\\retention_for_retention\\retentionenv\\lib\\site-packages\\pip-19.0.3-py3.6.egg\\pip install --ignore-installed --no-user --prefix C:\\Users\\kaghole\\AppData\\Local\\Temp\\pip-build-env-w2zl8_gl\\overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- setuptools<49.2.0 wheel<=0.35.1 Cython>=0.29.21,<3.0" failed with error code 1 in None\r\n'
I am using below deployment command:
python -m df-pubsubRead-gcsWrite-Op --requirements_file requirements.txt --runner DataflowRunner --project ing-dev --staging_location gs://my_bucket/staging --temp_location gs://my_bucket/temp --template_location gs://my_bucket/templates/test/df-pubsubRead-gcsWrite-Op
The requirements.txt file:
apache-beam[gcp]==2.27.0
I tried:
Use setup.py per this Dataflow fails when I add requirements.txt [Python] but the setup_file argument is discarded:
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['setup.py', 'True'] WARNING:apache_beam.options.pipeline_options:Discarding unparseable args: ['setup.py', 'True']
Not including the requirements file, which successfully create the template but the flow fails because apache-beam is not installed. In other words, specifying dependencies is a must for me. Unless there are other ways to install dependencies on Dataflow.
When I try to import the sawtooth SDK into Pycharm it failes with the following error
Collecting sawtooth-sdk==1.0.1
Using cached sawtooth-sdk-1.0.1.tar.gz
Requirement already satisfied: colorlog in /home/user/PycharmProjects/test/venv/lib/python3.6/site-packages (from sawtooth-sdk==1.0.1)
Collecting sawtooth-signing (from sawtooth-sdk==1.0.1)
Using cached sawtooth-signing-1.0.1.tar.gz
Collecting protobuf (from sawtooth-sdk==1.0.1)
Using cached protobuf-3.5.2.post1-cp36-cp36m-manylinux1_x86_64.whl
Collecting pyzmq (from sawtooth-sdk==1.0.1)
Using cached pyzmq-17.0.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting toml (from sawtooth-sdk==1.0.1)
Using cached toml-0.9.4.tar.gz
Collecting PyYAML (from sawtooth-sdk==1.0.1)
Using cached PyYAML-3.12.tar.gz
Collecting secp256k1 (from sawtooth-signing->sawtooth-sdk==1.0.1)
Using cached secp256k1-0.13.2.tar.gz
Collecting six>=1.9 (from protobuf->sawtooth-sdk==1.0.1)
Using cached six-1.11.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /home/user/PycharmProjects/test/venv/lib/python3.6/site-packages (from protobuf->sawtooth-sdk==1.0.1)
Requirement already satisfied: cffi>=1.3.0 in /home/user/PycharmProjects/test/venv/lib/python3.6/site-packages (from secp256k1->sawtooth-signing->sawtooth-sdk==1.0.1)
Requirement already satisfied: pycparser in /home/user/PycharmProjects/test/venv/lib/python3.6/site-packages (from cffi>=1.3.0->secp256k1->sawtooth-signing->sawtooth-sdk==1.0.1)
Installing collected packages: secp256k1, sawtooth-signing, six, protobuf, pyzmq, toml, PyYAML, sawtooth-sdk
Running setup.py install for secp256k1: started
Running setup.py install for secp256k1: finished with status 'error'
Complete output from command /home/user/PycharmProjects/test/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pycharm-packaging/secp256k1/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-g3n5renb-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/PycharmProjects/test/venv/include/site/python3.6/secp256k1:
0.29.1
Installed libsecp256k1 is unusable falling back to bundled version.
Using bundled libsecp256k1
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/secp256k1
copying secp256k1/__init__.py -> build/lib.linux-x86_64-3.6/secp256k1
copying secp256k1/__main__.py -> build/lib.linux-x86_64-3.6/secp256k1
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.6/_libsecp256k1.c'
creating build/temp.linux-x86_64-3.6
building '_libsecp256k1' extension
creating build/temp.linux-x86_64-3.6/build
creating build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/user/PycharmProjects/test/venv/include -I/usr/include/python3.6m -c build/temp.linux-x86_64-3.6/_libsecp256k1.c -o build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6/_libsecp256k1.o
build/temp.linux-x86_64-3.6/_libsecp256k1.c:22:24: fatal error: pyconfig.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/home/user/PycharmProjects/test/venv/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pycharm-packaging/secp256k1/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-g3n5renb-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/PycharmProjects/test/venv/include/site/python3.6/secp256k1" failed with error code 1 in /tmp/pycharm-packaging/secp256k1/
When I run the pip command pip install sawtooth-sdk==1.0.1 it succeeds but it installs the files to python 2.7 version and I need to install it for python 3.6 which it is the interpreter I have use for this project. Or is there anyway to import them directly into pycharm from the folder of the sdk the sawtooth project provides?? Any help??
Use pip3 which is available for Ubuntu-16.04
sudo apt-get install python3-pip
sudo pip3 install sawtooth-sdk==1.0.1
Another solution for cases where you cannot change Python version: install secp256k1 library in system. For example, to install it on python3.7 docker image:
RUN apt update && apt install -y libsecp256k1-dev libsecp256k1-0
I changed the python interpreter to version 3.5 and it installed correctly
I'm using Windows, but trying to use cygwin on it (this probably doesn't have anything to do with it, but just to have some context). I have this problem installing awscli. Does anyone know how to fix it?
$ pip install awscli
Collecting awscli
Using cached awscli-1.11.117-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): s3transfer<0.2.0,>=0.1.9 in c:\program files\python 3.5\lib\site-packages (from awscli)
Requirement already satisfied (use --upgrade to upgrade): docutils>=0.10 in c:\program files\python 3.5\lib\site-packages (from awscli)
Requirement already satisfied (use --upgrade to upgrade): colorama<=0.3.7,>=0.2.5 in c:\program files\python 3.5\lib\site-packages (from awscli)
Requirement already satisfied (use --upgrade to upgrade): rsa<=3.5.0,>=3.1.2 in c:\program files\python 3.5\lib\site-packages (from awscli)
Collecting PyYAML<=3.12,>=3.10 (from awscli)
Using cached PyYAML-3.12.tar.gz
Requirement already satisfied (use --upgrade to upgrade): botocore==1.5.80 in c:\program files\python 3.5\lib\site-packages (from awscli)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.3 in c:\program files\python 3.5\lib\site-packages (from rsa<=3.5.0,>=3.1.2->awscli)
Requirement already satisfied (use --upgrade to upgrade): jmespath<1.0.0,>=0.7.1 in c:\program files\python 3.5\lib\site-packages (from botocore==1.5.80->awscli)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil<3.0.0,>=2.1 in c:\program files\python 3.5\lib\site-packages (from botocore==1.5.80->awscli)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5 in c:\program files\python 3.5\lib\site-packages (from python-dateutil<3.0.0,>=2.1->botocore==1.5.80->awscli)
Installing collected packages: PyYAML, awscli
Running setup.py install for PyYAML
Complete output from command "c:\program files\python 3.5\python.exe" -c "import setuptools, tokenize;file='C:\cygwin64\tmp\pip-build-97mqnqfq\PyYAML\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record C:\cygwin64\tmp\pip-rmh4ny9_-record\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.5
creating build\lib.win-amd64-3.5\yaml
copying lib3\yaml\composer.py -> build\lib.win-amd64-3.5\yaml
...
copying lib3\yaml\tokens.py -> build\lib.win-amd64-3.5\yaml
copying lib3\yaml__init__.py -> build\lib.win-amd64-3.5\yaml
running build_ext
creating build\temp.win-amd64-3.5
creating build\temp.win-amd64-3.5\Release
checking if libyaml is compilable
error: [WinError 2] The system cannot find the file specified
----------------------------------------
Command ""c:\program files\python 3.5\python.exe" -c "import setuptools, tokenize;file='C:\cygwin64\tmp\pip-build-97mqnqfq\PyYAML\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record C:\cygwin64\tmp\pip-rmh4ny9_-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\cygwin64\tmp\pip-build-97mqnqfq\PyYAML
The error output seems to be complaining about installing PyYAML
I would suggest you to follow this guide to install AWSCli on cygwin.
I am on a Mac OS El Capitan, running a Windows 10 64-bit VM inside Parallels. I have Cygwin installed and Anaconda3. I would like to install two packages (pyrsistent and rpy2) using pip. Both of them throw an error "error: [WinError 2] The system cannot find the file specified" without specifying the file it can't find.
Here's the output:
$ pip install pyrsistent
Collecting pyrsistent
Using cached pyrsistent-0.11.9.tar.gz
Requirement already satisfied (use --upgrade to upgrade): six in c:\anaconda3\lib\site-packages (from pyrsistent)
Building wheels for collected packages: pyrsistent
Running setup.py bdist_wheel for pyrsistent
Complete output from command C:\Anaconda3\python.exe -c "import setuptools;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d C:\cygwin64\tmp\tmpn25raothpip-wheel-:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.5
copying _pyrsistent_version.py -> build\lib.win-amd64-3.5
creating build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_checked_types.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_field_common.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_helpers.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_immutable.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pbag.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pclass.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pdeque.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_plist.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pmap.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_precord.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pset.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_pvector.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\_transformations.py -> build\lib.win-amd64-3.5\pyrsistent
copying pyrsistent\__init__.py -> build\lib.win-amd64-3.5\pyrsistent
running build_ext
building 'pvectorc' extension
error: [WinError 2] The system cannot find the file specified
----------------------------------------
Failed building wheel for pyrsistent
Failed to build pyrsistent
Installing collected packages: pyrsistent
Running setup.py install for pyrsistent
Complete output from command C:\Anaconda3\python.exe -c "import setuptools, tokenize;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\cygwin64\tmp\pip-_wbdiief-record\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
running build_ext
building 'pvectorc' extension
error: [WinError 2] The system cannot find the file specified
----------------------------------------
Command "C:\Anaconda3\python.exe -c "import setuptools, tokenize;__file__='C:\\cygwin64\\tmp\\pip-build-sqcinj9m\\pyrsistent\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\cygwin64\tmp\pip-_wbdiief-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\cygwin64\tmp\pip-build-sqcinj9m\pyrsistent
Both python3 and pip are in the path:
$ which pip
/cygdrive/c/Anaconda3/Scripts/pip
$ which python
/cygdrive/c/Anaconda3/python
The error is identical when trying to install rpy2 so it's not something particular to the package I'm trying to install. Does anyone have ideas of the problem or ideas for troubleshooting? I've tried debugging into the install.py but the code throwing the error is in C. I've tried doing the install in verbose mode, but it still doesn't say what file it cannot find in that case.
(and if you're wondering why I haven't used conda, it's because it doesn't install pyrsistent, and for the rpy2 install, it insists on linking it to its own install of R, and even when specifying my own install, something doesn't link correctly. I've already been down that road and would like to stick to pip).
Here is a workaround for anyone who is stuck installing python packages on Windows. There are several sites that have Windows binaries for python packages. I was able to find a binary for rpy2 here but there are several other sites too, if you just google something like "python Windows binaries". Then I did:
pip install rpy2-2.7.4-cp35-none-win_amd64.whl
pyrsistent was not in that website or any of several other binaries websites I found. But someone else on my dev team sent me the /pyrsistent directory inside his Anaconda3/Lib/site-packages directory and after copying it to the same directory on my machine, I was able to import pyrsistent as well.
It's not pretty, but it worked.