How can one fully replace distutils, which is deprecated in 3.10? - python-3.x

According to PEP 632, distutils will be formally marked as deprecated, and in Python 3.12, it will be removed. My product is soon going to support Python 3.10 and I don't want to put up with deprecation warnings, so I would like to remove references to distutils now. The problem is that I can't find good, comprehensive documentation that systematically lets me know that A in distutils can be replaced by B in modules C, D, and E. The Migration Advice in the PEP is surprisingly sketchy, and I haven't found standard documentation for distutils, or for whatever modules (such as setuptools?) that are required to replace distutils, that would let me fill in the gaps. Nor am I sure how to look at the content of the installed standard distribution (that is, the physical directories and files) in order to answer these questions for myself.
The "Migration Advice" section says:
For these modules or types, setuptools is the best substitute:
distutils.ccompiler
distutils.cmd.Command
distutils.command
distutils.config
distutils.core.Distribution
distutils.errors
...
For these modules or functions, use the standard library module shown:
...
distutils.util.get_platform — use the platform module
Presumably, that means that setuptools has either a drop-in replacement or something close to it for these modules or types (though I'm not sure how to verify that). So, for instance, perhaps setuptools.command.build_py can replace distutils.command.build_py. Is that correct? In any case, what about these?
distutils.core.setup
distutils.core.Extension
Furthermore, what am I supposed to make of the fact that setuptools does not appear under the modules or index list in the standard documentation? It is part of the standard distribution, right? I do see it under Lib/site-packages.
UPDATE 1: If setuptools is not currently part of the standard distribution, is it expected to become one, say, in Python 3.11 or 3.12? Are customers expected to install it (via pip?) before they can run a setup.py script that imports setuptools? Or is the thought that people shouldn't be running setup.py anymore at all?
Knowing how to replace distutils.core.setup and distutils.core.Extension is probably enough for my current needs, but answers to the other questions I've asked would be quite helpful.
UPDATE 2:
setuptools is indeed part of the python.org standard distribution, as can be determined by importing it from a freshly installed Python interpreter from python.org. The thing that was confusing me was that it is documented on a separate site, with a separate style, from python.org. However, as SuperStormer pointed out in the comments, some Linux distributions, such as Debian, don't install it by default when you install Python.
UPDATE 3:
This command:
Python-3.9.1/python -m ensurepip --default-pip
installs both pip and setuptools on Debian 10 on a Python installation freshly downloaded and built from python.org. Before the command is executed, pip is absent and setuptools cannot be imported.

Related

Use shared system libraries in Conda

I am using Conda on a shared compute cluster where numerical and io libraries have been tune for the system.
How can I tell Conda to use these and only worry about the libraries and packages which are not already there on path?
For example:
There is a openmpi library installed and the package which I would like to install and mange with Conda has it also as a dependency.
How can I tell Conda to just worry about what is not there?
One trick is to use a shell package - an empty package whose only purpose is to satisfy constraints for the solver. This is something that Conda Forge does with mpich, as mentioned in this section of the documentation. Namely, for every version, they include an external build variant, that one could install like
conda install mpich=3.4.2=external_*
signaling that it will be supplied by the host. One can consult the recipe's meta.yaml for a concrete example.
I don't think this is great (seems like a lot of work), but I also don't know of a better alternative.

setup.py not finding pre-installed dependencies

I am trying to install a package I have written by running
python setup.py install
whilst in the package directory. My folder structure is as follows:
package
-module
--__init__.py
--parent_class.py
--child_class.py
-setup.py
Here '-' denotes the folder level with package being the root directory.
When running the installation works as expected until:
Installed c:\users\me\appdata\local\continuum\anaconda3\lib\site-
packages\package-0.1-py3.7.egg
Processing dependencies for package==0.1
Searching for json
Reading https://pypi.org/simple/json/
Couldn't find index page for 'json' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.org/simple/
No local packages or working download links found for json
error: Could not find suitable distribution for Requirement.parse('json')
I don't believe this is an issue with the json package installation as:
I know I have it installed (I can run 'import json' in my env)
and because if I remove it.
If i remove 'json' from install_requires=['pandas', 'numpy', 'datetime', 'os', 'copy', 'json'] it will simply return the same error, this time with the 'copy' and then the 'os' package.
Thanks in advance
I believe json is a built-in module in Python 2 and 3 (e.g., like open). Thus, it's usually included in all Python distributions (that is, anyone who has Python probably also has the json module).
Thus, you should be able to omit json from your requirements (while still importing it in your code). Since json is a built-in module, you can assume that anyone who has Python installed will also have json.
I found this question that is also about installing the json module. The second answer notes that minimal Python installs (e.g., python-minimal in Ubuntu) may not include json, however. You should note that the linked question concerns Python 2 and not 3.
Therefore, if a user lacks json on their system, they should be able to install it through their system's package manager (in Linux), and not pip. In Windows, I would assume that all Python installations will include built-in modules (though I don't know for certain). Regardless, if a user doesn't have json installed, then I think it's up to the user to install it.
The same applies to os, copy, and datetime. Only pandas and numpy need to be included in your requirements.
You can find a list of built-in modules (for Python 3) here. Again, the modules listed here are ones that are not hosted on pypi (thus not available through pip install ..., conda install ..., etc.), but ones that are usually included in all Python distributions.
As an aside: this is my first answer here. Please let me know if I can improve it.

PyQt5 stubs files

I've just started a PyQt5 project that is currently running in a virtualenv.
PyQt5 was installed using a classic pip install pyqt5.
I want to type check my application using mypy.
But when I run it, I get an error telling me that there are no stubs file for PyQt5.
myapp/__main__.py:3: error: No library stub file for module 'PyQt5.QtWidgets
I've checked the site-package of my virtualenv, and indeed, there aren't any .pyi file in it.
Checking the documentation, I see that if compiled, stub files can be generated (and could at least exists beginning with PyQt5.6, I'm using 5.10).
Is there a way to obtain those file without the need to manually compile the library ?
I had the same problem with PyQt5. So I decided to put up PyQt5-stubs which contains the stub files for the main PyQt5 modules.
You can install it with pip:
$ pip install pyqt5-stubs
Another solution for you would be to change to Qt for Python (PySide2). They provide proper type annotations.
Not currently.
PEP 561, which specifies how packages should indicate they supply type information, was recently accepted. (Full disclosure, I am the author).
PyQt5 will need to become compliant with the PEP, and then as long as you are using mypy >=0.600, everything should work as expected.

SublimeCodeIntel isn't looking in python 3 paths

I installed SublimeCodeIntel package in sublime text, and I use it to code in python 3. However, the problem is that it uses python2 paths to do the import, so if a library is installed in python2 and not in python3, i won't find it when i use import or from X import y.
I did confirm that when I used the command SublimeCodeIntel: Dump Import Directories , so I saw both the files python and python3 in the directory ./codeintel/import_dir_stats and here is there content :
python
dedeeb56f744e507026fef17243da41f /home/bilal/.local/lib/python2.7/site-packages
6a1d0cac3d9e6148e2208b63a33a1e6f /home/bilal/.local/lib/python2.7/site-packages/impacket
16a4fccbb3beadfdfd72691ef8f7298c /home/bilal/.local/lib/python2.7/site-packages/mechanize
211d2b55059f6b634799fdae534decd9 /usr/lib/python2.7/dist-packages
be5448890caffe81686310f127d6efae /usr/lib/python2.7/dist-packages/_markerlib
cec69a0830a725e10ac4e364d44add8f /usr/lib/python2.7/dist-packages/appindicator
python3
dedeeb56f744e507026fef17243da41f /home/bilal/.local/lib/python2.7/site-packages
6a1d0cac3d9e6148e2208b63a33a1e6f /home/bilal/.local/lib/python2.7/site-packages/impacket
16a4fccbb3beadfdfd72691ef8f7298c /home/bilal/.local/lib/python2.7/site-packages/mechanize
211d2b55059f6b634799fdae534decd9 /usr/lib/python2.7/dist-packages
be5448890caffe81686310f127d6efae /usr/lib/python2.7/dist-packages/_markerlib
cec69a0830a725e10ac4e364d44add8f /usr/lib/python2.7/dist-packages/appindicator
I didn't put all the lines (because there is a lot), but the content of the two files is identical.
I don't understand from where is coming this problem, why SublimeCodeIntel is looking in python2 directories for the import ??!!
PS : I am using Ubuntu 15.10 (with Linux version > 4), and sublime text 3 build 3103.
Please help, I really consumed a lot of time and energy looking for this.
I would strongly recommend using the Anaconda plugin (no relation whatsoever to the Anaconda Python distribution) instead of SublimeCodeIntel. I struggled with SCI for a while on various machines, and could never get very good code completion out of it until I ran across Anaconda one day. The next day I removed SCI and have been completely satisfied ever since. It's super-easy to configure (just give it the path to your python executable and it reads sys.path and all the rest), and pretty much just works. It has linting with several different linters built-in (you can disable them if you want), has popups available for function signatures and other hints, works with virtualenvs right out of the box, and more.
(I didn't write it and I have no connection to the author(s) - it's just a great plugin!)

Using setuptools, can I make my Python package depend-on/install a non-Python utility/executable?

I'm trying to write a Python utility that is a thin wrapper around an existing command line program (wmctrl - https://sites.google.com/site/tstyblo/wmctrl/).
I'd like to make my program available on PyPI to share my work later on. But as wmctrl has the core functionality, my code depends heavily on it being installed.
Is there a way to configure setuptools to depend on a non-Python/non-setuptools dependency like wmctrl? I'd like setuptools to fail the install if the binary isn't there. (Ideally, I'd like setuptools to install it, but that seems less likely...).
Python Packaging/setuptools seems to be really geared towards working with other Python-packaged dependencies (PyPI packages, setuptools-based VCS repos, etc.). I haven't been able to find anything online about configuring it to depend on other third-party executables.
Thanks to anyone who can offer some guidance/help here.

Resources