I have been trying to install sent2vec on Amazon EC2. However, I think there's something wrong in what I am doing.
Could you please give me some guidance.
Thanks
I found it out myself.
Download the zipped version from github.
wget https://github.com/epfml/sent2vec/archive/master.zip
unzip master.zip
make
sudo python3.7 -m pip install . ( I am using python3.7, hence mentioned. you can try sudo pip install . also. )
(if you fail at step 4, try this as well sudo apt-get install python3.7-dev)
That's it!
These are the steps I followed in my local environment.
After running the following commands
conda activate <the environment name here>
git clone https://github.com/facebookresearch/fastText.git
cd fastText
pip install .
cd ..
git clone https://github.com/epfml/sent2vec.git
cd sent2vec
pip install .
it worked fine
Related
Building a docker image, I've installed Negbio in my Dockerfile using:
RUN git clone https://github.com/ncbi-nlp/NegBio.git xyz && \
python xyz/setup.py install
When I try to run my Django application at localhost:1227 I get:
No module named 'negbio' ModuleNotFoundError exception
When I run pip list I can see negbio. What am I missing?
As per your comment, It wouldn't install with pip and hence not installing via pip.
Firstly, to make sure the https://github.com/ncbi-nlp/NegBio is properly installed via python setup.py install, you need to install it's dependencies via pip install -r requirements first. So either ways, you are doomed to have pip inside Docker.
For example, this is the sample Dockerfile that would install the negbio package properly:
FROM python:3.6-slim
RUN mkdir -p /apps
WORKDIR /apps
# Steps for installing the package via Docker:
RUN apt-get update && apt-get -y upgrade && apt-get install -y git gcc build-essential
RUN git clone https://github.com/ncbi-nlp/NegBio.git
WORKDIR /apps/NegBio
RUN pip install -r requirements.txt
RUN python setup.py install
ENV PATH=~/.local/bin:$PATH
EXPOSE 8000
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
So wouldn't harm if you actually install it via requirements.txt
I would do it like this:
requirements.txt --> have all your requirements added here
negbio==0.9.4
And make sure it's installed on the fly inside the docker using RUN pip install -r requirements.txt
I ultimately resolved my issue by going to an all Anaconda environment. Thank you for everyone's input.
So i am aiming to install Ludwig for experimentation but i didn't found any solution on the internet to this issue enter image description here
screenshot of the error message
i am using windows subsytem for linux (debian)
Your python version is probably unsupported by tensorflow 1.15.3. I ran into the same issue trying to install with python 3.8.
https://github.com/tensorflow/tensorflow/issues/34302
When I compiled Ludwig from GitHub source, there were a lot of dependencies to patch. I experienced the same error message and I gave up solving it.
I retried with clean installation from the very beginning, I managed to install Ludwig successfully on Google Cloud - Debian 9 VM.
Now I redo the steps on Oracle Cloud - Ubuntu 20.04 VM.
Steps:
Ensure the following dependencies are ready, which I consolidate from various sources.
$ sudo apt update
$ sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev curl libbz2-dev lzma liblzma-dev python3-tk
Clean install Python 3.6. I choose to use older Python 3.6 to execute Ludwig, because Ludwig is using the old Tensorflow 1.15.3. I reserve Python 3.7 and 3.8 environment for other newer Python projects, for example Tensorflow 2.
# Simply use a temporary working folder.
$ cd /tmp
# Download the newest Python3.6 source.
$ curl -O https://www.python.org/ftp/python/3.6.12/Python-3.6.12.tgz
$ tar -xzvf Python-3.6.12.tgz
$ cd Python-3.6.12
# --prefix=/usr/local ensures the newly installed Python3.6 does not mess up with the default Python executables in the OS. This is specially warned in Google Cloud.
$ ./configure --prefix=/usr/local --enable-optimizations
$ sudo make altinstall
$ python3.6 --version
Python 3.6.12
# Upgrade pip and virtualenv
$ sudo python3.6 -m pip install --upgrade pip
$ sudo python3.6 -m pip install --upgrade virtualenv
Prepare the virtual environment for Ludwig.
Reference https://ludwig-ai.github.io/ludwig-docs/getting_started/#installation.
# Go back to home.
$ cd
# Create a Working directory.
$ mkdir Works
$ cd Works
# Initialize a virtual environment with Python3.6
$ virtualenv -p python3.6 ludwig
$ source ludwig/bin/activate
# Install Ludwig
$ pip install ludwig
You can see every dependencies are taken care of and Ludwig is ready to use.
$ pip list
Package Version
-------------------- -------
... ...
ludwig 0.2.2.8
...
tensorflow 1.15.3
...
# Execute Ludwig
$ ludwig
I am trying to set up a mysql connection that will work with SqlAlchemy in Python 3.6.5 . I have the following in my Dockerfile:
RUN pip3 install -r /event_git/requirements.txt
I also have, in requirements.txt:
mysql-connector-python==8.0.15
However, I am not able to connect to the DB. Is there anything else that I need to do to set this up?
Update:
I got 8.0.5 working but not 8.0.15 . Apparently, a protobuff dependency was added; does anyone know how to handle that?
docker file is:
RUN apt-get -y update && apt-get install -y python3 python3-pip fontconfig wget nodejs nodejs-legacy npm
RUN pip3 install --upgrade pip
# Copy contents of this directory (i.e. full source) to image
COPY . /my_project
# Install Python dependencies
RUN pip3 install -r /event_git/requirements.txt
# Set event_git folder as working directory
WORKDIR /my_project
ENV LANG C.UTF-8
I am running it via
docker build -t event_git .;docker run -t -i event_git /bin/bash
and then executing a script; the db is on my local machine. This is working on mysql-connector-python==8.0.5 but not 8.0.15, so the setup is ok; I think I just need to fulfill the protobuff dependency that was added (see https://github.com/pypa/warehouse/issues/5537 for mention of protobuff dependency).
The mysql-connector-python has the Python Protobuf as an installation requirement, this means that protobuf will be installed along mysql-connector-python.
If this doesn't work, try to add protobuf==3.6.1 in your requirements.txt.
Figured out the issue. The key is that import mysql.connector needs to be at the top of the file where the create_engine is. Still not sure of the exact reason, but at the very least that seems to define _CONNECTION_POOLS = {}. If anyone knows why, please do give your thoughts.
How can I have python3.6 in tensorflow docker images.
All the images I tried (latest, nighty) are using python3.5 and I don't want to modify all my scripts.
The Tensorflow images are based on Ubuntu 16.04, as you can see from the Dockerfile. This release ships with Python 3.5 as standard.
So you'll have to re-build the image, and the Dockerfile will need editing, even though you need to do the actual build with the parameterized_docker_build.sh script.
This answer on ask Ubuntu covers how to get Python 3.6 on Ubuntu 16.04
The simplest way would probably be just to change the From line in the Dockerfile to FROM ubuntu:16.10, and python to python3.6 in the initial apt-get install line
Of course, this may break some other Ubuntu version-specific thing, so an alternative would be to keep Ubuntu 16.04 and install one of the alternative ppa's also listed in the linked answer:
RUN add-apt-repository ppa:deadsnakes/ppa &&
apt-get update &&
apt-get install -y python3.6
Note that you'll need this after the initial apt-get install, because that installs software-properties-common, which you need to add the ppa.
Note also, as in the comments to the linked answer, that you will need to symlink to Python 3.6.
Finally, note that I haven't tried any of this. The may be gotchas, and you may need to make another change to ensure that the correct version of Python is used by the running container.
You can use stable images which are supplied by third parties, like ufoym/deepo.
One that fits TensorFlow, python3.6 and cuda10 can be found here or you can pull it directly using the command docker pull ufoym/deepo:py36-cu100
I use their images all the time, never had problems
With this anwer, I just wanted to specify how I solved this problem (the previous answer of SiHa helped me a lot but I had to add a few steps so that it worked completly).
Context:
I'm using a package (segmentation model for unet++) that requires tensorflow==1.4.0 and keras==2.2.2.
I tried to use the docker image for tensorflow 1.4.0, however, the default version of python of this image is 3.5 which is not compatible with my package.
I managed to install python3.6 on the docker images thanks to the following files:
My Dockerfile contains the following lines:
Dockerfile:
FROM tensorflow/tensorflow:1.4.0-gpu-py3
RUN mkdir /AI_PLATFORM
WORKDIR /AI_PLATFORM
COPY ./install.sh ./install.sh
COPY ./requirements.txt ./requirements.txt
COPY ./computer_vision ./computer_vision
COPY ./config.ini ./config.ini
RUN bash install.sh
Install.sh:
#!/urs/bin/env bash
pip install --upgrade pip
apt-get update
apt-get install -y python3-pip
add-apt-repository ppa:deadsnakes/ppa &&
apt-get update &&
apt-get install python3.6 --assume-yes
apt-get install libpython3.6
python3.6 -m pip install --upgrade pip
python3.6 -m pip install -r requirements.txt
Three things are important:
use python3.6 -m pip instead of pip, else the packages are installed on python 3.5 default version of Ubuntu 16.04
use docker run python3.6 <command> to run your containers with python==3.6
in the requirements.txt file, I had to specify the following things:
h5py==2.10.0
tensorflow-gpu==1.4.1
keras==2.2.2
keras-applications==1.0.4
keras-preprocessing==1.0.2
I hope that this answer will be useful
Maybe the image I created will help you. It is based on the cuda-10.0-devel image and has tensorflow 2.0a-gpu installed.
You can use it as base image for your own implementation. The image itself doesn't do anything. I put the image on dockerhub https://cloud.docker.com/repository/docker/patientzero/tensorflow2.0a-gpu-py3.6
The github repo is located here: https://github.com/patientzero/tensorflow2.0-python3.6-Docker
Pulling it won't do much, but for completeness:
$ docker pull patientzero/tensorflow2.0-gpu-py3.6
edit: changed to general tensorflow 2.0x image.
Also as mentioned here, the official image for the beta 2.0 release now comes with python 3.6 support
I recently noticed that I am running an old version of AWS CLI that is lacking some functionality I need:
$aws --version
aws-cli/1.2.9 Python/3.4.3 Linux/3.13.0-85-generic
How can I upgrade to the latest version of the AWS CLI (1.10.24)?
Edit:
Running the following command fails to update AWS CLI:
$ pip install --upgrade awscli
Requirement already up-to-date: awscli in /usr/local/lib/python2.7/dist-packages
Cleaning up...
Checking the version:
$ aws --version
aws-cli/1.2.9 Python/3.4.3 Linux/3.13.0-85-generic
From http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-with-pip
To upgrade an existing AWS CLI installation, use the --upgrade option:
pip install --upgrade awscli
On Linux and MacOS X, here are the three commands that correspond to each step:
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
This does not work:
pip install --upgrade awscli
This worked fine on Ubuntu 14.04( no need to reboot also .. You would have to first install pip3 ):
pip3 install --upgrade awscli
For Ubuntu 16.04 I used parts of the other answers and comments and just reloaded bash instead of rebooting.
I installed the aws-cli using apt so I removed that first:
sudo apt-get remove awscli
Then I could pip install (I chose to use sudo to install globally with pip2):
sudo pip install -U awscli
Since I was doing this on a server I didn't want to reboot it, but reloading bash did the trick:
source ~/.bashrc
At this point I could use the new version of aws cli
aws --version
Update: Upgrade instance using AWS CLI v1 to AWS CLI v2:
This question and answer was initially created when there was only an AWS CLI v1. There is now a AWS CLI v2. The installation instructions for the AWS CLI v2 can be found here.
The new AWS CLI v2 has different installation instructions based on whether your EC2 instance is using Linux x86 (64-bit) or Linux ARM architecture.
To upgrade to AWS CLI v2, on an EC2 instance using Linux ARM, I had to issue the following commands:
rm -rf /bin/aws
curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install -i /usr/local/aws -b /bin
Subsequently test your AWS CLI version by executing: aws --version
For the Linux x86 (64-bit) architecture I'm hoping the commands are the same except for replacing the curl command with the following: (as per the installation instructions)
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o
"awscliv2.zip"
The AMI I used was the most recent one currently available and it was still using the AWS CLI v1. In the future if AWS starts packaging AWS CLI v2 with their AMIs this answer might require an update.
Original answer: Upgrade instance using AWS CLI v1 to use the most recent version of AWS CLI v1:
If you are having trouble installing the AWS CLI using pip you can use the "Bundled Installer" as documented here.
The steps discussed there are as follows:
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Check your AWS CLI version subsequently as a sanity-check that everything executed correctly:
$ aws --version
If the AWS CLI didn't update to the latest version as expected maybe the AWS CLI binaries are located somewhere else as the previously-given commands assume.
Determine where AWS CLI is being executed from:
$ which aws
In my case, AWS CLI was being executed from /bin/aws, so I had to install the
"Bundled Installer" using that location as follows:
$ sudo ./awscli-bundle/install -i /user/local/aws -b /bin/aws
Try
sudo pip install --upgrade awscli, and open a new shell.
This worked well for me (no need to reboot).
Simple use
sudo pip install awscli --force-reinstall --upgrade
This will upgrade all the required modules.
On Mac you can use homebrew:
to install: brew install awscli
to upgrade: brew upgrade awscli
Make sure you don't have multiple installations: where aws
pip install awscli --upgrade --user
The --upgrade option tells pip to upgrade any requirements that are already installed. The --user option tells pip to install the program to a subdirectory of your user directory to avoid modifying libraries used by your operating system.
We can follow the below commands to install AWS CLI on UBUNTU:
sudo apt install curl
curl “https://s3.amazonaws.com/aws-cli/awscli-bundle.zip” -o
“awscli-bundle.zip”
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/ aws -b /usr/local/bin/aws
rm -rf awscli-bundle.zip awscli-bundle
To test: aws — version
For More Info :
https://gurudathbn.wordpress.com/2018/03/31/installing-aws-cli-on-ubuntu/
When using sudo pip install --upgrade awscli I got the following error:
ERROR: Could not install packages due to an EnvironmentError: [Errno 30] Read-only file system: '/lib'
By using sudo with -H option, I could fix the problem.
sudo -H pip install --upgrade awscli
Currently, using pip will get you the old version of awscli, 1.18.103.
The latest version of aws-cli, 2.0.33 is on the v2 branch. You can download the installer for Linux, Windows and macOS from here.
I was trying to install awscli on one of my ec2 instances where I tried both
sudo pip install --upgrade awscli,
sudo pip3 install --upgrade awscli
which didn't worked, as I was getting errors like
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-2nh71cs2/cryptography/
And rebooting servers were not an option.
Luckily, simple
sudo apt update
sudo apt install awscli worked.
I do it by removing & installing the awscli like described in this video
basically:
pip uninstall awscliv2
pip install awscliv2
pip install awscliv2==your-version
pip install awscliv
(you can keep v1 along with v2 if you want)
pip install --upgrade ...
works as well. sure.
I do not install it globally (like some ppl seems still do), btw. Because sometimes I need different v for different cases. so I keep it in separate python virtual environment.
For windows you can try this command
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
Install or update the AWS CLI on macOS
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg ./AWSCLIV2.pkg -target /
Done!
You can verify the same with below cmd
aws --version
Try AWS Cloud Shell, Quick and easy
AWS CloudShell is a browser-based shell that makes it easy to securely manage, explore, and interact with your AWS resources. CloudShell is pre-authenticated with your console credentials.
Benefits
No extra credentials to manage
Always up to date
No cost
More Details here https://aws.amazon.com/cloudshell/
Customizable
To install globally, get on the sudo access
sudo su & then upgrade aws cli by
pip3 install --upgrade awscli