I am trying to do a machine learning tutorial, but for some reason, I get an error when importing quandl from the script, yet I can import it fine from the shell?
file is called regression1.py:
#!/usr/bin/python3
import pandas as pd
import quandl
df = quandl.get('WIKI/GOOGL')
df = df[['Adj. Open', 'Adj. High', 'Adj. Low', 'Adj. Close', 'Adj. Volume',]]
df['HL_PCT'] = (df['Adj. High'] - df['Adj. Close']) / df['Adj. Close'] * 100.0
df['PCT_change'] = (df['Adj. Close'] - df['Adj. Open']) / df['Adj. Open'] * 100.0
df = df[['Adj. Close', 'HL_PCT', 'PCT_change', 'Adj. Volume']]
print(df.head)
and here is the error that I get:
Traceback (most recent call last):
File "./regression1.py", line 4, in <module>
import quandl
ImportError: No module named 'quandl'
What's up here, any ideas?
I've found the answer to this issue, and I refer to this question:
Python Script: Runs in Shell but fails in real life because cant import module
I am running virtualenv, but in my script I used the shebang:
#!/usr/bin/python3
If I change this to:
#/home/me/Programming/virtual_environments/mlearn_env/python
i.e. my virtual environment for this particular exercise, it works fine. So be aware of what you are running and from where. I have to admit, it has not happened before, I have been able to use virtualenv with the initial shebang for all other programs run, but it seems some particular modules are a little stubborn and are not written with a different path in mind.
Please correct me if I am incorrect in my thinking.
Related
While trying to write a NumPy array of floats to a NetCDF4 dataset, I am getting a RuntimeError: NetCDF: HDF error. I believe that somewhere TensorFlow2.0 is messing up with NetCDF4, but I do need to import both in the same class/function. It is not clear why the sequence of importing libraries is affecting I/O of a NetCDF4 file.
Here's a sample script:
##The sequence of import which doesn't work
import numpy as np
import tensorflow as tf ### <<< if imported here, saving .nc doesn't work
import netCDF4 as nc
#import tensorflow as tf ### <<< if imported here, saving .nc works properly
print("I am TensorFlow ", tf.__version__, " but I have no job here")
print("I would let NetCDF4 ", nc.__version__," do it's job, for now")
Nx = 160 # just another number
outputfile = "outputfile.nc" # just another filename
ArrayField = np.ones((Nx,Nx,1)) # sample array to write
print("Writing field data of shape", ArrayField.shape)
ncfile = nc.Dataset("outputfile.nc",'w',format='NETCDF4_CLASSIC')
ncfile.createDimension('X',ArrayField.shape[0]) #line is probably okay
newx = ncfile.createVariable('X','d',('X')) #line is probably okay
newx[:] = np.linspace(0.00,1.00,ArrayField.shape[0]) #line is probably okay
velx = ncfile.createVariable('Component_X','d',('X','X')) #line is probably okay
velx[:] = ArrayField[:,:,0].T #line is probably okay
print("Something written to: ", outputfile)
ncfile.close() ###### <<<<<<< Gives error here
print("Data successfully written to: ", outputfile)
Output/Error:
I am TensorFlow 2.0.0 but I have no job here
I would let NetCDF4 1.5.3 do it's job, for now
Writing field data of shape (160, 160, 1)
Something written to: outputfile.nc
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-1-1cfdf4070b97> in <module>
22
23 print("Something written to: ", outputfile)
---> 24 ncfile.close() ###### <<<<<<< Gives error here
25 print("Data successfully written to: ", outputfile)
netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.close()
netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset._close()
netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()
RuntimeError: NetCDF: HDF error
Expected Output:
I am TensorFlow 2.0.0 but I have no job here
I would let NetCDF4 1.5.3 do it's job, for now
Writing field data of shape (160, 160, 1)
Something written to: outputfile.nc
Data successfully written to: outputfile.nc
Though I can import TF2.0 after importing NetCDF4 for this particular sample to work, it doesn't really answer the question about getting a RuntimeError: NetCDF: HDF error and fixing the issue for its use in complex cases. Also, I would like to return more debugging information.
Tested with tensorflow==2.1.0 and 2.2.0, gives the same error.
Disk size of created garbage .nc file is ~204K which should be about ~209K for this array sample.
Issue persists on a different machine or a clean environment by installing only tensorflow==2.0.0, numpy==1.17.3, netCDF4==1.5.3.
Just in case, here's a pip freeze list of my clean environment: https://gist.github.com/aakash30jan/9ae0cf3dde8a63d28df5275873cb0f10
This is the program for the stock prediction to be simply printed...
from alpha_vantage.timeseries import TimeSeries
# Your key here
key = 'yourkeyhere'
ts = TimeSeries(key)
aapl, meta = ts.get_daily(symbol='AAPL')
print(aapl['2020-22-5'])
I get this error...
Traceback (most recent call last):
File "C:/Users/PycharmProjects/AlphaVantageTest/AlphaVantageTest.py", line 7, in <module>
print(aapl['2020-22-5'])
KeyError: '2020-22-5'
Since that didn't work, I tried getting a little more technical with it...
from alpha_vantage.timeseries import TimeSeries
from alpha_vantage.techindicators import TechIndicators
from matplotlib.pyplot import figure
import matplotlib.pyplot as plt
# Your key here
key = 'W01B6S3ALTS82VRF'
# Chose your output format, or default to JSON (python dict)
ts = TimeSeries(key, output_format='pandas')
ti = TechIndicators(key)
# Get the data, returns a tuple
# aapl_data is a pandas dataframe, aapl_meta_data is a dict
aapl_data, aapl_meta_data = ts.get_daily(symbol='AAPL')
# aapl_sma is a dict, aapl_meta_sma also a dict
aapl_sma, aapl_meta_sma = ti.get_sma(symbol='AAPL')
# Visualization
figure(num=None, figsize=(15, 6), dpi=80, facecolor='w', edgecolor='k')
aapl_data['4. close'].plot()
plt.tight_layout()
plt.grid()
plt.show()
I get these errors...
Traceback (most recent call last):
File "C:/Users/PycharmProjects/AlphaVantageTest/AlphaVantageTest.py", line 9, in <module>
ts = TimeSeries(key, output_format='pandas')
File "C:\Users\PycharmProjects\AlphaVantageTest\venv\lib\site-packages\alpha_vantage\alphavantage.py", line 66, in __init__
raise ValueError("The pandas library was not found, therefore can "
ValueError: The pandas library was not found, therefore can not be used as an output format, please install manually
How can I improve my program so that I don't receive these errors? None of these programs are bad syntax wise. Thank you to anyone that can help.
You need to install pandas. If you're just using pip, you can run pip install pandas if you are using conda to manage your envs you can use conda install pandas.
Glad it worked. According to this meta overflow post: What if I answer a question in a comment?
I am posting my comment as an answer so you can mark the question as answered.
I've some code like:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
data = pd.read_csv('StudentsPerformance.csv')
#print(data.isnull().sum()) // checking if there are some missing values or not
#print(data.dtypes)checking datatypes of the dataset
# ANALYSİS VALUES OF THE COLUMN'S
"""print(data['gender'].value_counts())
print(data['parental level of education'].value_counts())
print(data['race/ethnicity'].value_counts())
print(data['lunch'].value_counts())
print(data['test preparation course'].value_counts())"""
# Adding column total and average to the dataset
data['total'] = data['math score'] + data['reading score'] + data['writing score']
data['average'] = data ['total'] / 3
sns.distplot(data['average'])
I would like to see distplot of average for visualization but I run the program that gives me an error like
Traceback (most recent call last): File
"C:/Users/usersample/PycharmProjects/untitled1/sample.py", line 22, in
sns.distplot(data['average']) AttributeError: module 'seaborn' has no attribute 'distplot'
I've tried to reinstall and install seaborn and upgrade the seaborn to 0.9.0 but it doesn't work.
head of my data female,"group B","bachelor's
degree","standard","none","72","72","74" female,"group C","some
college","standard","completed","69","90","88" female,"group
B","master's degree","standard","none","90","95","93" male,"group
A","associate's degree","free/reduced","none","47","57","44"
this might be due to removal of paths in environment variables section. Try considering to add your IDE scripts and python folder. I am using pycharm IDE, and did the same and its working fine.
I'm following the tutorial for decision tree on scikit documentation.
I have pydotplus 2.0.2 but it is telling me that it does not have write method - error below. I've been struggling for a while with it now, any ideas, please? Many thanks!
from sklearn import tree
from sklearn.datasets import load_iris
iris = load_iris()
clf = tree.DecisionTreeClassifier()
clf = clf.fit(iris.data, iris.target)
from IPython.display import Image
dot_data = tree.export_graphviz(clf, out_file=None)
import pydotplus
graph = pydotplus.graphviz.graph_from_dot_data(dot_data)
Image(graph.create_png())
and my error is
/Users/air/anaconda/bin/python /Users/air/PycharmProjects/kiwi/hemr.py
Traceback (most recent call last):
File "/Users/air/PycharmProjects/kiwi/hemr.py", line 10, in <module>
dot_data = tree.export_graphviz(clf, out_file=None)
File "/Users/air/anaconda/lib/python2.7/site-packages/sklearn/tree/export.py", line 375, in export_graphviz
out_file.write('digraph Tree {\n')
AttributeError: 'NoneType' object has no attribute 'write'
Process finished with exit code 1
----- UPDATE -----
Using the fix with out_file, it throws another error:
Traceback (most recent call last):
File "/Users/air/PycharmProjects/kiwi/hemr.py", line 13, in <module>
graph = pydotplus.graphviz.graph_from_dot_data(dot_data)
File "/Users/air/anaconda/lib/python2.7/site-packages/pydotplus/graphviz.py", line 302, in graph_from_dot_data
return parser.parse_dot_data(data)
File "/Users/air/anaconda/lib/python2.7/site-packages/pydotplus/parser.py", line 548, in parse_dot_data
if data.startswith(codecs.BOM_UTF8):
AttributeError: 'NoneType' object has no attribute 'startswith'
---- UPDATE 2 -----
Also, se my own answer below which solves another problem
The problem is that you are setting the parameter out_file to None.
If you look at the documentation, if you set it at None it returns the string file directly and does not create a file. And of course a string does not have a write method.
Therefore, do as follows :
dot_data = tree.export_graphviz(clf)
graph = pydotplus.graphviz.graph_from_dot_data(dot_data)
Method graph_from_dot_data() didn't work for me even after specifying proper path for out_file.
Instead try using graph_from_dot_file method:
graph = pydotplus.graphviz.graph_from_dot_file("iris.dot")
I met the same error this morning. I use python 3.x and here is how I solve the problem.
from sklearn import tree
from sklearn.datasets import load_iris
from IPython.display import Image
import io
iris = load_iris()
clf = tree.DecisionTreeClassifier()
clf = clf.fit(iris.data, iris.target)
# Let's give dot_data some space so it will not feel nervous any more
dot_data = io.StringIO()
tree.export_graphviz(clf, out_file=dot_data)
import pydotplus
graph = pydotplus.graphviz.graph_from_dot_data(dot_data.getvalue())
# make sure you have graphviz installed and set in path
Image(graph.create_png())
if you use python 2.x, I believe you need to change "import io" as:
import StringIO
and,
dot_data = StringIO.StringIO()
Hope it helps.
Also another problem was the backend settings to my Graphviz!! It is solved nicely here. you just need to lookup that settings file and change backend, or in the code mpl.use("TkAgg") as suggested there in the comments. After I only got error that pydotplot couldn't find my Graphviz executable, hence I reinstalled Graphviz via homebrew: brew install graphviz which solved the issue and I can make plots now!!
What really helped me solve the problem was:-
I executed the code from the same user through which graphviz was installed. So executing from any other user would give your error
i would suggest avoid graphviz & use the following alternate approach
from sklearn.tree import plot_tree
plt.figure(figsize=(60,30))
plot_tree(clf, filled=True);
from __future__ import division
import math
from sympy import *
d=symbol('d')
x=solve(d**2 - 224*d + 400)
print(x)
Hi,I'm new to python.I just tried to solve a polynomial expression using symPy,but got the following error.
Traceback (most recent call last):
File "C:/Windows/System32/test.py", line 4, in <module>
d=symbol('d')
TypeError: 'module' object is not callable
Someone pls help me out with the correct function.Thank you
You're sure you are running python3, and your script is not named something that conflicts with any other modules? Do you have a file named sympy.py in your script's directory? (You shouldn't)
I never use import *, you never know what kind of namespace errors you'll run into. This code works 100% for me:
#!/usr/bin/env python3
import math
import sympy
d = sympy.Symbol('d')
x = sympy.solve(d**2 - 224*d + 400)
print(x) # Prints [-4*sqrt(759) + 112, 4*sqrt(759) + 112]
x = sympy.solve(d - 10)
print(x) # Prints 10
capitalize Symbol and try again.
try with d = Symbol('d'). Doing symbol('d') you are trying to use the symbol sub-module as a function which is invalid. Also as you are in python 3 you don't need that future import