I am using poetry for dependency and packaging of my project.
One of my python package maintained in gitlab package registry . And the pip installation command goes straight forward
pip install <my_package> --index-url https://__token__:<your_personal_token>#gitlab.com/api/v4/projects/<project_id>/packages/pypi/simple
But I am unable to add this to my pyproject.toml dependency from gitlab package registry
This command works for me
[tool.poetry.dependencies]
<my_package> = {git = "https://<token_name>:<>#gitlab.com/lululemon/global-tech-services/retail-engineering-department/dsp/dsp-logging.git"}
But its not getting installed from package registry rather directly from the codebase .
Related
It's extremely useful for the development workflow to be able to build and install the latest version of a package into a local environment. You can then interactively validate and debug by importing this latest version into a Python shell or a Jupyter notebook. The problem is I've recently adopted Poetry and cannot figure how to do this now. So...
How do I install the latest version of my package from the current working directory into my local environment using Poetry?
Moving on from setuptools
Back in the day, I used to always use setuptools and it worked great. I'd put a setup.py file in the root of my repository, create a virtual environment (let's say using conda) for the project, and do...
pip install -e .
From here, I could fire up a python shell, or even configure a jupyter kernel to use this virtual environment, and I'd always have the latest version of my package to interact with.
Now setuptools has its limitations, and we've since moved on to Poetry to more tightly control dependencies and handle more sophisticated build needs and such.
The problem with poetry
If you look up what's the pip install -e . equivalent in poetry you will find this issue. Looks like the creator of poetry thinks installing directly from source like this is a hack and has no interest in supporting it. (BTW: I've tried poetry build and then pulling out the setup.py file like he suggested and it does not work)
Linking directly to source is not necessary, I'm willing to run an install command to get the latest version of the package. And when I do this with poetry, it appears to work.
cd root/of/my/project
poetry install
Installing dependencies from lock file
No dependencies to install or update
Installing the current project: my-project (0.4.8) <-- this is the latest version according to the source code in the working directory
The problem is that if I open a Python shell and try and import my package for instance, it is linked to the last version of my package that installed from a remote artifact repository (via pip install my-package) – not what's in my working directory.
python
...
>>> import my_package
>>> my_package.__version__
'0.4.7`
Now, even though I'm using poetry I'm using a conda environment to specify the Python version for my project and then installing, and using, poetry from inside that.
source activate my-package
(my-package) ... $ poetry update
I also know that poetry (not very transparently) can create and manage its own virtual environment on your behalf. I thought maybe the reason this is not working is because I need to be inside this environment (whereas I was only inside my conda environment, while poetry is installing the 0.4.8 version of my package within the virtual environment it manages).
I tried both shell and run to test this out. I get the same result.
poetry shell
Virtual environment already activated: /Users/U6020643/.conda/envs/my-package
Python 3.8.5
...
>>> import my_package
>>> my_package.__version__
'0.4.7`
What gives?
The way I fixed this: I stopped using conda to manage the Python version for projects that involve poetry and instead use pyenv.
Below is how I made that work. This was very helpful!
1. Removing conda as default environment manager.
This involved removing conda base activate from ~/.bash_profile.
Now open a new shell and verify that there's no conda environment prefix, e.g. (base) ... $.
2. Installing pyenv using Homebrew
Had been awhile, and an OS upgrade or two, since I interacted with Homebrew. Needed to do some housekeeping.
brew cleanup # This made it so that brew update didn't take forever
brew update
brew upgrade
brew cleanup
Then...
brew install pyenv
3. Install Python version(s)
Let's say you need/want Python 3.
pyenv install 3.8.5
4. Set the local Python version for your project
cd your/project/root/
pyenv local 3.8.5
5. Install poetry
See here.
6. Use it
Now install and use shell. Check the version -> hey it works!
cd your/project/root
poetry install
poetry shell
my_package --version # Package has a CLI.
i am trying to install this a flask project on another server but later realized it was failing at ifxpy during the
pip install -r requirements
The github link for the project is https://github.com/OpenInformix/IfxPy. Is there a way i can install it from the github repository.
There is a need to install python packages on machine without internet connection
I used pip download to download the packages and their dependencies
I copied all the dependencies to the offline machine
I run pip from the local python packages repository using
pip install *
package with dependencies are trying to access the internet to download their dependencies even that they are locate in the same directory
I would like to avoid the requirement.txt file and would like it to install all the packages from the local directory with their dependencies.
Is there any way to do so?
It's possible to download the wheels directly for each package, and once you have them on the machine you can run pip install name-of-wheel.whl and it will install them without routing to pypi.
You can use on the online machine:
pip download -r requirements.txt
to download package without installing them.
Then, on the offline machine:
pip install --no-index --find-links /path/to/download/dir/ -r requirements.txt
Source: Python Packages Offline Installation
There is an open-source python package that I want to work on (toga-android). To test the code I write, I have to be able to build my own project that has said open-source package as a dependency. My project has to be built with setuptools, so I need setuptools to fulfill the dependency using my version of the package, and not get the package from PyPI.
The problem is that setuptools always gets the package from PyPI.
Whenever I build with setuptools I see:
Collecting toga-android==0.3.0.dev8
Downloading https://files.pythonhosted.org/packages/92/fe/348a39e2e0bbcac2d3ed511dd2b62943b488e7dcb8097c437416caf1c179/toga_android-0.3.0.dev8-py3-none-any.whl
or
Collecting toga-android==0.3.0.dev8
Using cached https://files.pythonhosted.org/packages/92/fe/348a39e2e0bbcac2d3ed511dd2b62943b488e7dcb8097c437416caf1c179/toga_android-0.3.0.dev8-py3-none-any.whl
Clearly it is getting the package from PyPI or using a cached version from it.
I have installed my version using pip install -e . , and that has no effect. I have also tried including the package's source in my project's directory with setup.py. Setuptools apparently includes this code because syntax errors there make the build fail, but it doesn't recognize that it can satisfy the dependency. It still gets the package from PyPI and any modules imported from the package are the PyPI versions.
How can I use of custom version of a package that is also in PyPI as a setuptools dependency?
Steps to reproduce:
pip install briefcase (using or not using virtualenv does not matter)
git clone https://github.com/pybee/toga.git
cd ~/toga/src/core; sudo pip install -e .
cd ~/toga/src/android/; sudo pip install -e .
cd ~/toga/examples/tutorial0
python setup.py android
The output will show that an older version of toga-android is downloaded even though it was already installed with pip.
Whats the successful way to import packages from github repo?
Here is what I tried
Tried Installing a python package- corepkg, which is available in git repository - git.example.com/corepkg.git
In another Project- Proj2, to import logic from above corepkg package, Kept an entry in requirements.txt and ran the following pip command.
pip install -r requirements.txt
Here are my Entries in requirement.txt for Proj2
...
PyYAML==3.12
requests==2.18.4
urllib3==1.22
git+https://git.example.com/corepkg.git#develop
But it did not create any src folder or .dist-info folder in virtual environment site-packages?
It just created corepkg-1-py3.6.egg_info file but not files required to import.
Whats the step I am missing here to import it successfully from git?
Try
git+https://git.example.com/corepkg.git#develop#egg=corepkg
See https://pip.readthedocs.io/en/stable/reference/pip_install/#vcs-support
The problem also could be in the very corepkg.git repo so it's hard to say something without looking at the repo.