I have followed a number of steps but could not get airflow up and running in my conda virtual environment.
below are the steps I have followed.
Issue below command in my conda environment.
pip install apache-airflow==1.10.10 --constraint https://raw.githubusercontent.com/apache/airflow/1.10.10/requirements/requirements-python3.7.txt.
There were issues regarding VC++ not present.This issue has been resolved post installation of VC++ and added it in my environment variable. I have even added AIRFLOW_HOME in my environment variable.
Unfortunately below command is not working.
airflow initdb
I have not been able to start the webserver as well.
airflow webserver -p 8080
my system is not aware of airflow.When I am issuing 'airflow initdb' commmand,I am getting below error.
(biometric) C:\Users\royan\Anaconda3\envs\biometric\lib\site-packages\airflow\bin>airflow initdb
'airflow' is not recognized as an internal or external command,
operable program or batch file.
Please suggest the steps I need to perform to resolve this error.
I believe the easiest way to install Airflow in an Anaconda environment is to use the conda-forge repository:
conda create -n airflow
conda activate airflow
conda config --env --add channels conda-forge
conda config --env --set channel_priority strict
conda install airflow
Related
I have been trying to install a Rust kernel for a Jupyter server inside a dedicated conda environment but I get errors.
Slighty adjusting steps from:
https://depth-first.com/articles/2020/09/21/interactive-rust-in-a-repl-and-jupyter-notebook-with-evcxr/
conda create -n rusttest
conda activate rusttest
conda install -c conda-forge rust jupyterlab
conda install -c anaconda cmake -y
cargo install evcxr_jupyter
Add $HOME/.cargo/bin to my PATH variable (export PATH)
evcxr_jupyter --install (here I already see the kernel is installed outside the env)
jupyter lab
The kernel is visible on the dashboard however when I try to start a notebook with it the connection fails and I get:
Error: Failed to find sysroot for Cargo.toml file /tmp/.tmpbZ0Pkw/Cargo.toml. Is rust-src installed?
I have tried manually:
jupyter kernelspec install {MY_PATH_DURING_PKG_INSTALLATION}/Jupyter/kernels/rust --sys-prefix
And I get:
[InstallKernelSpec] Installed kernelspec rust in {MY_PATH}/miniconda3/envs/rusttest/share/jupyter/kernels/rust
Which seems OK (inside the correct conda env) but the error persists.
Is there any way to add a working kernel just to this one jupyter server inside that env?
(I want Rust to be gone when I start jupyter lab from another conda env)
EDIT
After digging into this I think the first thing is to get rust-src installed inside that conda environment... I don't know how...
Solved:
I needed to download https://static.rust-lang.org/dist/2022-05-19/rust-src-1.61.0.tar.gz manually and extract it under the env dir such that XXX/miniconda3/envs/rusttest/lib/rustlib/src/rust exists.
I have python3.7 installed on my windows 10 laptop
But i need python3.6 for a specific project
Can i install it in virtualenv which will override python3.7 in that environment?
I don't know whether this may be an appropriate solution for you. But this is what I generally follow. Just install Anaconda in your system and create an environment according to your needs. For your case create an environment for Python 3.6.
conda create --name py36 python=3.6
//This lines will create an environment named py36
You then install libraries according to your needs in that environment. You work in that environment without interfering with the libraries of the other environment. To use anaconda kindly follow Anaconda cheatsheet. You will get everything that you need.
Has anyone found a way to successfully set up an environment for a jupyter notebook without having the full version of Anaconda installed on your system? I am using Miniconda and have been unable to use a conda environment with the jupyter notebook platform. Note that with Miniconda source activate myenv will not work.
I also tried using ipykernel which did successfully get an environment to appear in my notebook (via the command: python -m ipykernel install --user --name myenv). However, this environment is still made as a copy of my base environment. Commands similar to conda create -n test_env python=3.7 don't seem to be compatible with ipykernel unless I am missing something.
You don't need to run ipython from the environment you are adding.
Since you've got an environment with jupyter, lets call that Jenv.
go to your miniconda install location
Scripts/activate Jenv
conda create -n SomeOther_vEnv python=3.6.8 whateverPackagesYouWant
python -m ipykernel install --sys-prefix --name SomeOther_vEnv --display-name "foo"
Now jupyter when started from your Jenv should see "foo". You can't start jupyter from foo, but you can use it in a jupyter server started from Jenv. (unless whateverPackagesYouWant included jupyter of course)
After running conda init from the terminal I am only able to run from the terminal anaconda-navigator from the base environment of anaconda. I want to make it so that I can run anaconda-navigator from any environment including no environment.
I started down this because conda activate myenv would not work. It recommended running conda init; so I did. Previously I could run anaconda-navigator from any anaconda environment and it would pop up but now if I try from anything but base it doesn't work and I get this error message "anaconda-navigator: command not found". However if I enter conda activate into the terminal to bring on the base environment the command works.
I just got done with a clean install of anaconda3 using the recommended steps https://docs.anaconda.com/anaconda/install/linux/
If I delete what conda init added to my .bashrc and add in
export PATH=/home/yourUserName/anaconda3/bin:$PATH
then anaconda-navigator works again from any environement but conda activate myenv stops working.
FYI: I ran conda config --set auto_activate_base False after installing anaconda3
I expect to be able to run anaconda-navigator from any anaconda environment and to be able to use conda activate myenv. Right now I can do one or the other by modifying my .bashrc
I have python2.7 and 3.7 both installed on my Mac machine. I have set python3 default in .bash_profile file using
alias python=python3
alias pip=pip3
when i hit the below command
pip install apache-airflow
it says
Requirement already satisfied: apache-airflow in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (1.10.3)
But when i hit the command "airflow version", it says command not found.
So how to install apache airflow for python3
You might need setup adjust the path environment variable accordingly.
My way to solve such environment case is to run applications in docker containers as micro service.