I'm new to Theano and trying out some examples.
import numpy
import theano.tensor as T
from theano import function
import datetime
print datetime.datetime.now()
x = T.dscalar('x')
y = T.dscalar('y')
z = x + y
f = function([x, y], z)
print f(2, 3)
print numpy.allclose(f(16.3, 12.1), 28.4)
print datetime.datetime.now()
And it took 15 minutes to run this. I'm using a 2GB ram, and there aren't many processes running simultaneously.
Check the Theano Flags first.
If you didnt run with : THEANO_FLAGS=mode=FAST_RUN or ran with not the default flag or if you changed the .theanorc , it might take some time.
--
However, read here:
http://deeplearning.net/software/theano/tutorial/using_gpu.html
You can also see more about Theano Flags here:
http://deeplearning.net/software/theano/library/config.html
Since you must be running from an IDE, you will have to edit the .theanorc
As described on the Theano Link above:
"
It defaults to $HOME/.theanorc. On Windows, it defaults to $HOME/.theanorc:$HOME/.theanorc.txt to make Windows users’ life easier.
"
The exact flag is this :
config.mode
String value: 'Mode', 'ProfileMode' (deprecated), 'DebugMode', 'FAST_RUN', 'FAST_COMPILE'
In case this doesnt help, be sure to update Theano to bleeding edge and edit your question with the theanorc settings!
pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git
You can also use OpenMP to use extra Threads/cores :
THEANO_FLAGS=mode=FAST_RUN THEANO_FLAGS='openmp=True' OMP_NUM_THREADS=4 python x.py
Related
I am attempting to run some simple PyMC3 code in PyCharm on MacOS Big Sur but the execution fails and throws the following error:
Exception: ('Compilation failed (return status=1): ld: library not found for -lSystem. clang-10: error: linker command failed with exit code 1 (use -v to see invocation). ', '[Elemwise{add,no_inplace}(TensorConstant{1.0}, TensorConstant{1.0})]')
I am using PyCharm 2020.3.2. I am able to install pymc3 (version 3.8) and theano (1.0.4) into the environment. Code I am attempting to run follows:
import numpy as np
import scipy.stats as stats
import matplotlib.pyplot as plt
import pymc3 as pm
np.random.seed(123)
n_experiments = 4
theta_real = 0.35
data = stats.bernoulli.rvs(p=theta_real, size=n_experiments)
with pm.Model() as our_first_model:
theta = pm.Beta('theta', alpha=1, beta=1)
y = pm.Bernoulli('y', p=theta, observed=data)
start = pm.find_MAP()
step = pm.Metropolis()
trace = pm.sample(1000, step=step, start=start)
burnin = 100
chain = trace[burnin:]
pm.traceplot(chain, lines={'theta':theta_real});
I disabled Apple System Integrity Protection on some speculation that maybe Apple hiding /usr/include might be the source of the problem; it made no difference. No clue how else to fix this.
From the information in comments it seems like your anaconda is broken.
Update anaconda and try again.
If that doesn't work, try to install PyMC3 in the system Python not the Anaconda one.
This will use the system clang and not the Anaconda clang.
I wanted to use pandas-profiling to do some eda on a dataset but I'm getting an error : AttributeError: 'DataFrame' object has no attribute 'profile_report'
I have created a python script on spyder with the following code :
import pandas as pd
import pandas_profiling
data_abc = pd.read_csv('abc.csv')
profile = data_abc.profile_report(title='Pandas Profiling Report')
profile.to_file(output_file="abc_pandas_profiling.html")
AttributeError: 'DataFrame' object has no attribute 'profile_report'
The df.profile_report() entry point is available from v2.0.0. soln from here
Did you install pandas-profiling via pip or conda?
use : pip install -U pandas-profiling to solve this and restart your kernel
The issue is that the team hasn't updated the pip or conda installations yet (described here). If you installed using one of these, try this for the time being.
profile = pandas_profiling.ProfileReport(df)
print(profile)
This should work for those who want to use the latest version:
Run pip uninstall pandas_profiling from anaconda prompt (given you're using Spyder, I'd guess this would be your case) / or command prompt
Run pip install https://github.com/pandas-profiling/pandas-profiling/archive/master.zip
If you're using something like a Jupyter Notebook/Jupyter Lab, be sure to restart your kernel and re-import your packages.
I hope this helps.
For the those using google colabs, the profiling library is outdated, hence use the command below and restart the runtime
! pip install https://github.com/pandas-profiling/pandas-profiling/archive/master.zip
The only workaround I found was that the python script I made is getting executed from the command prompt and giving the correct output but the code is still giving an error in Spyder.
Some of the version of the pandas-profiling does not work for me and I installed 2.8.0 version and it work for me.
!pip install pandas-profiling==2.8.0
import numpy as np
import pandas as pd
import pandas_profiling as pp
df = pd.read_csv('/content/sample_data/california_housing_train.csv')
profile = df.profile_report(title = "Data Profiling Report")
profile.to_file("ProfileReportTest.html")
If none of the above worked, can you check by setting the encoding to unicode_escape in read_csv? It may be due to one of your columns
encoding = 'unicode_escape'
My solution
For me installation via pip was giving errors, therefore I installed it via conda from here.
Code Example
And here is the code example to use profile report:
import pandas as pd
from pandas_profiling import ProfileReport
data_abc = pd.read_csv('abc.csv')
profile = ProfileReport(data_abc, minimal=True)
profile.to_file("abc_pandas_profiling.html")
To read the html file I used the following code
df = pd.read_html("abc_pandas_profiling.html")
print(df[0])
Try in conda environment
!pip install --user pandas-profiling
import pandas_profiling
data.profile_report()
I used to use this module like this:
import community
if __name__ == '__main__':
G = nx.karate_club_graph()
pos = nx.spring_layout(G)
partition = community.best_partition(G)
I installed the correct module:
sudo pip3 install python-louvain
I get this error:
AttributeError: module 'community' has no attribute 'best_partition'
As far as I know it follows the documentation presented here.
It seems some others have run into this problem before, see:
https://bitbucket.org/taynaud/python-louvain/issues/23/module-has-no-attribute-best_partition
If you have another library called community installed that may be causing a problem. Here is one solution proposed in the thread I linked to:
from community import community_louvain
partition = community_louvain.best_partition(G)
I am a beginner in using Networkx as well but I used following syntax in Jupyter notebook and it worked fine for me.
!pip install python-louvain
from community import community_louvain
communities =community_louvain.best_partition(G)
Kind Regards,
You need the package named python-louvain from here.
!pip install python-louvain
import networkx as nx
import community
import numpy as np
np.random.seed(0)
W = np.random.rand(15,15)
np.fill_diagonal(W,0.0)
G = nx.from_numpy_array(W)
louvain_partition = community.best_partition(G, weight='weight')
modularity2 = community.modularity(louvain_partition, G, weight='weight')
print("The modularity Q based on networkx is {}".format(modularity2))
The modularity Q based on networkx is 0.0849022950503318
I am not sure why the following situation exists, but there appears to be another package called "community" that does not contain the function "community.best_partition". As stated above, you want the "python-louvain" package, which appears to include a "community" part?! In PyCharm 2020.3, under Preferences -> Project: Python Interpreter, I deleted the "community" package and added the "python-louvain" package. After that, "import community" still worked as did "community.best_partition".
you should install the package below. i use it and it works.
i install it in windows.
https://pypi.org/project/python-louvain/
write "pip install python-louvain" in cmd and after that write program like this:
import community
import networkx as nx
import matplotlib.pyplot as plt
G = nx.erdos_renyi_graph(30, 0.05)
partition = community.best_partition(G)
size = float(len(set(partition.values())))
pos = nx.spring_layout(G)
count = 0
for com in set(partition.values()) :
count = count + 1
list_nodes = [nodes for nodes in partition.keys()if partition[nodes] == com]
nx.draw_networkx_nodes(G, pos, list_nodes, node_size = 20,node_color = str(count / size))
nx.draw_networkx_edges(G, pos, alpha=0.5)
plt.show()
i use python 3.7
For what it's worth: I had to
pip uninstall community
then
pip install python-louvain
then
pip install networkx
in order to get my conda py37 environment to work correctly and to be able to call community.best_partition() without an attribute error.
I think, if you have networkx installed before python-louvain, it will claim the namespace for community and not allow you to run what you want.
I use this solution:
import community.community_louvain as community_louvain
part = community_louvain.best_partition(G)
Reference: https://programmerclick.com/article/70941941990/
But here is another alternative:
Figured out. Reinstalled, and reloaded, all works fine.
Ps. for who does not know how to reload a module in jupyter:
from importlib import reload
reload(community)
Reference: https://github.com/taynaud/python-louvain/issues/48
I am running Theano with Anaconda on Windows. Ihave pretty much followed the steps in the comments here. I can import theano with no problems:
import theano
from theano import tensor as T
x = T.vector('x')
W = T.matrix('W')
dot = T.dot(x, W)
This works fine. But when I do
f = theano.function(inputs=[x, W], outputs=dot)
I get warnings:
WARNING (theano.gof.compilelock): Overriding existing lock by dead process '4844' (I am process '3908')
WARNING:theano.gof.compilelock:Overriding existing lock by dead process '4844' (I am process '3908')
and a long error, which ends with:
ImportError: ('The following error happened while compiling the node', CGemv{inplace}(AllocEmpty{dtype='float64'}.0, TensorConstant{1.0}, W.T, x, TensorConstant{0.0}), '\n', 'DLL load failed: The specified module could not be found.', '[CGemv{inplace}(<TensorType(float64, vector)>, TensorConstant{1.0}, W.T, x, TensorConstant{0.0})]')
Any ideas on how to fix this?
Theano on Windows need Theano development version and not last Theano version:
http://deeplearning.net/software/theano/install.html#bleeding-edge-install-instructions
So just update Theano and it should work.
Having installed FiPy using pip with a brew installation of Python 2.7.3 on OSX 10.8.2, I ran the following sample test code:
from fipy import *
mesh = Grid3D(nx=50, ny=100, nz=10, dx=0.1, dy=0.01, dz=0.1)
x, y, z = mesh.cellCenters
xyzVar = CellVariable(mesh=mesh, name=r"x y z", value=x * y * z)
k = Variable(name="k", value=0.)
viewer = MayaviClient(vars=numerix.sin(k * xyzVar), limits={'ymin': 0.1, 'ymax': 0.9}, datamin=-0.9, datamax=2.0, title="MayaviClient test")
for kval in range(10):
k.setValue(kval)
viewer.plot()
viewer._promptForOpinion()
which generated the following error:
File "/usr/local/lib/python2.7/site-packages/pyface/qt/__init__.py", line 17, in prepare_pyqt4
sip.setapi('QString', 2)
ValueError: API 'QString' has already been set to version 1
I decided to see how far I could get by commenting out sip.setapi('QString', 2) and sip.setapi('QVariant', 2) in prepare_pyqt4. This simple hack got passed the versioning issue but presented a new problem.
File "/Library/Python/2.7/site-packages/fipy/viewers/mayaviViewer/mayaviDaemon.py", line 79, in <module>
from enthought.mayavi.plugins.app import Mayavi
ImportError: No module named enthought.mayavi.plugins.app
It seems now the FiPy Mayavi viewer is lacking a module.
I think my site-packages should all be in the same location too. Looks like pip installs to /Library/Python/2.7/site-packages while brew installs to /usr/local/lib/python2.7/site-packages, something I have to fix up (I think the problem is with pip).
The question I have is how can I get a clean FiPy installation working with Mayavi (without generating these errors) and fix up my site-packages?
I don't know for sure what's causing your problem. I run a Homebrew installation under Mac OS X 10.6.8 and Mayavi works for me (including your example script). I described my installation process at http://matforge.org/fipy/wiki/InstallFiPy/MacOSX/HomeBrew
The only issue I can guess is that you say your pip installs into /Library/Python/2.7/site-packages. I would guess that you are using the system python and not one installed by brew. The last time I did it, I had to brew install python and then easy_install pip, but https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python makes it sound like you get pip automatically with a brewed python, now.
Try doing
which python
which easy_install
which pip