databricks cli dbfs command is is throwing error - databricks

I am trying to execute below command in git bash as well as in cmd. However its not working. databricks cli is already installed using pip install databricks-cli installed
dbfs rm -r dbfs:/reco/user/feature/branch
Error: TypeError: an integer is required (got type bytes)

It's a known issue with Python 3.8, workaround is to use Python 3.7 or to uninstall pyspark 2.4.x that it's incompatible with Python 3.8 anyway...

uninstalling pyspark 3 worked for me. I am still using python 3.8.5 so looks like issue was after installing pyspark 3

Related

AWS CLI not working because of unsupported Python version

I'm trying to download the AWS CLI tools onto my mac. The error message is pretty clear Unsupported Python version detected: Python 2.7 To continue using this installer you must use Python 3.6 or later. The issue I'm having is that aliasing python to python3 isn't working. For some reason, after aliasing, the installer still references Python 2.7.
After aliasing through the cli didn't work for installing AWS CLI, I added alias python=python3 to my .zshrc file. Running python --version returns Python 3.9.6. Running the AWS installer still references the older version of python.
I'm hesitant to completely override the older version, because I've read from multiple sources that the default python on OS X should not be touched.
Can someone explain how I can reference the newer version of python when installing the AWS CLI tools?
while installation, choosing python3 worked well for me. Choose the "To install the AWS CLI version using the bundled installer" from here: https://docs.aws.amazon.com/cli/v1/userguide/install-macos.html
sudo /usr/local/bin/python3 awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Replace /usr/local/bin/python3 with the path to your python binary with version >= 3.6.
My problem was that I was trying to download an old version of the awscli. After downloading the newest version I ran into some issues with the credentials file. Upon updating the credentials file and adding a config file in the .aws directory, everything worked as expected.

awscli stopped working after upgrading python 3.6 to 3.7

Python has been upgraded from 3.6 to 3.7 on my mac and my awscli stopped working , I get this error every time I try use awscli commands :
-bash: /Users/myuser/Library/Python/3.6/bin/aws: /usr/local/opt/python/bin/python3.6: bad interpreter: No such file or directory
I have tried different things but none of them solved my issue. any ideas ?
You are trying to refer python3.6 aws cli and which is internally referring to /usr/local/opt/python/bin/python3.6 which doesn't exist.
solution is upgrade aws and also refer aws from python3.7

Does TensorFlow 1.9 support Python 3.7

I'm hesitating whether to downgrade to Python 3.6 or install a new version of TensorFlow.
Does TensorFlow 1.9 support Python 3.7?
I was able to install Tensorflow 1.12.0 with Python 3.7 on MacOS, with the following command.
sudo python3 -m pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.12.0-py3-none-any.whl
Not yet. It seems there are some variables named "async", which has become a keyword in 3.7. Shouldn't be too difficult to fix, but still a problem.
Source: https://github.com/tensorflow/tensorflow/issues/20444
Probably not yet.
First of all, you will probably get a SyntaxError: invalid syntax because some parameters and variables at the pywrap_tensorflow_internal.py module have the name async which is a reserved keyword in Python 3.7.0 (see also this post).
However, you can solve this by simply changing the name of all these (Ctrl + R and Replace All) from async to for example async1.
The problem is that then you will probably get a ImportError: No module named '_pywrap_tensorflow_internal' which perhaps may be fixed for any previous version before Python 3.7.0 as this post on StackOverflow suggests but personally I could not fix it on my laptop with Python 3.7.0 .
Consequently, I simply downgraded to Python 3.6.6 and now Tensorflow is working fine.
There is a whl built here. However in my run it was cpu version.
https://www.lfd.uci.edu/~gohlke/pythonlibs/
as of Aug 26tH, there is an update on the tensorflow github site
https://github.com/tensorflow/tensorflow/pull/21202
Tensorflow 1.13.1 now supports Python 3.7.
According to the comments on github
see
https://github.com/tensorflow/tensorflow/issues/20517#issuecomment-457185528
by member av8ramit
"We are aiming to try and have Windows and Ubuntu python binaries by rc2 or the official."
the estimated release date for 2019-02-03
however on
https://github.com/winpython/winpython/issues/687
one can read from a winpython contributor:
"Tensorflow-2 is 2019, Tensorflow-1 on Python-3.7 is still not in sight"
Finally Worked for me!
On Python 3.6.4 (even 3.6.5 failed for me)
Install Python3.6.4 from here:
https://www.python.org/downloads/mac-osx/
If you're facing issues with Python pkgs that you already had on you older version just revert it using this post:
python location on mac osx
I found this to work after searching for a while.
As Tensorflow only supports Python 3.6 as of now, you can install a different version of python alongside your standard one. Here are the steps I followed:
Download the Python3.6 tgz file from the official website (eg. Python-3.6.6.tgz)
Unpack it with tar -xvzf Python-3.6.6.tgz
cd Python-3.6.6
run ./configure
run make altinstall to install it (install vs altinstall explanation here Difference in details between "make install" and "make altinstall")
You'll normally find your new python install under /usr/local/bin. Now you can create a new virtualenv specifying the python version with:
virtualenv --python=python3.6 env3.6
Get into the virtualenv running the command source env3.6/source/bin/activate.
Install tensorflow with the classic pip install tensorflow
Profit
First of all,
*conda create -n yourenvname python=3.6
*conda activate yourenvname
*pip install tensorflow
then it will work like rock.

Error while installing psycopg2 in Windows

I tried to install psycopg2 as:
pip install psycopg2
It was giving error for pg_config. I checked some stackoverflow notes and installed PostgreSQL. Added the path of pg_config in $PATH.
Now it is giving the below error:
LINK : fatal error LNK1181: cannot open input file 'libpq.lib'
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\VC\\Tools\\MSVC\\14.14.26428\\bin\\HostX86\\x64\\link.exe' failed with exit status 1181
Any help on this?
Have you tried reinstall without pip?
sudo apt-get install python3-psycopg2
EDIT: Since you're on windows:
You can try this if you're using python 2.7:
pip uninstall psycopg2
pip install git+https://github.com/nwcell/psycopg2-windows.git#win64-py27#egg=psycopg2
If you're using python 3.4:
pip uninstall psycopg2
pip install git+https://github.com/nwcell/psycopg2-windows.git#win64-py34#egg=psycopg2
More info here.
If you prefer, you can download the installer (.exe) and install it.
You can find it here.
I found the issue.... I was using Python 3.7 version. Looks like the latest stable version of psycopg2 (2.7.4) is not yet supported on Python 3.7. It is supported till 3.6
I downgraded my python to 3.6.5 in Anaconda and also to 2.7.15 through PyCharm. I was able to install psycopg2 in both and use them.
Go to https://pypi.org/project/psycopg2
in release history and select the version you want to download.
There will be a table with files and the versions of python that can be compatible with the selected version.
Now download the wheel file that matches your system configuration(mac/linux/win-32/win-64)
I did this and was able to download the package.
For windows10(64-bit), download psycopg2-2.7.7-cp36-cp36m-win_amd64.whl (998.5 kB), in case you have selected 2.7.7
For Windows users, to install the psycopg2 without any errors first install Python 3.7.4 version and then install psycopg2 using command pip install psycopg2 as it worked complete fine with me. I think there is some issue in Python 3.8.0 version onwards because while installing psycopg it is showing an error in setup wheels.
In case you are here in 2020, this is a problem with version 3.8 of python. I had the issue recently. Try using virtualenv so that you can use lower versions of python in your workspace until psycopg2 is supported fully on Python version 3.8

cqlsh not able to locate cassandra running on localhost [duplicate]

I've tried all the measures from this post and Cassandra doc.
I've tried running all the versions of Cassandra including the latest release 3.7 from tarball and Debian package, but I keep getting errors when I execute cqlsh.
Error:
Connection error: ('Unable to connect to any servers', {'127.0.0.1': TypeError('ref() does not take keyword arguments',)})
I had no problem running Cassandra before I upgraded my Linux Mint from 17.3 to 18.
I believe I installed all the necessary packages such as java 8 and python 2.7.12.
I think the problem exists in cassandra.yaml file since the default setting isn't working, but I'm not sure how to configure properly to get it running.
Any suggestions appreciated.
You are running into CASSANDRA-11850, where cqlsh breaks with Python 2.7.11+. This ticket has been marked as "Resolved" and a patch has been applied to Cassandra 3.9 which has not been released yet.
I believe I installed all the necessary packages such as java 8 and python 2.7.12.
In the interim (until 3.9 is released) you can roll back to Python 2.7.10, and cqlsh should work (not trivial). Otherwise, DataStax DevCenter should work with Cassandra 3.7.
Edit 20161020
Cassandra 3.9 was released a few weeks ago, and can now be downloaded.
refer https://issues.apache.org/jira/browse/CASSANDRA-11850
After setting environment variable
CQLSH_NO_BUNDLED=TRUE
it resolved.
I use windows 7, python 2.7.12, cassandra 3.7
Need to add following command
sudo apt install python-pip
pip install cassandra-driver
export CQLSH_NO_BUNDLED=true
This works for Ubuntu 16.04 in Amazon EC2:
sudo apt-get --no-install-recommends install python-cassandra python3-cassandra
CQLSH_NO_BUNDLED=TRUE cqlsh "$(ec2metadata --local-ipv4)"
Even if after rolling back to python 2.7.10, the issue persists. It means the python 2.7.10 is not set as the default python version.
Go to /usr/bin directory and check the different python versions available, say python2.7 corresponds to version 2.7.10 ( you can check it by running command python2.7 in your terminal and the python version will be mentioned in the first line of the Interpreter,try the same with all other python versions available in the folder to find the one which corresponds to version 2.7.10).
Now, use the following commands to make correct python version (python2.7 in my case) as default choice
update-alternatives --install /usr/bin/python python /usr/bin/python2.7
It may be possible as you have not installed cassandra-driver.
As I also faced the same problem and I solved using following such steps.
Try installing python pip then install cassandra-driver.
1.sudo apt install python-pip
2.pip install cassandra-driver

Resources