Here is the code that I'm using to plot many plots and save them, but it is eating up all of the available RAM and causes the notebook to crash. I tried adding fig.clf(), del fig, gc.collect, and yet nothing seems to work.
I'm able to save only 38 figures around, then session gets crashed on Google Colab, since RAM gets full.
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
print(np.__version__) # 1.19.5
print(mpl.__version__) # 3.2.2, also tried with latest 3.4.1
x = np.arange(0, 280, 0.1)
y = np.sin(x)
for k in range(100):
fig, ax = plt.subplots(6, 2, sharex = True)
fig.set_size_inches(37.33, 21)
for i in range(2):
for j in range(6):
ax[j][i].plot(x, y)
fig.savefig(f'figure{k}.png', dpi = 300)
plt.close(fig)
This is related to the inline backend. The memory leak can be avoided by explicitly switching to the agg backend.
cross ref: matplotlib/issues/20067
maybe if you try to save each figure after it is generated, I mean try putting fig. savefig in the for loop.
Edit: after looking for the issue on google, I found that you might need to buy Colab pro.
Related
I'm trying to put the finishing touches to a small program but I'm stuck on the last 2 items and have gone nowhere for hours. The 2 problems are:
I get the following warning when I compile: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
I want the title of the plot to be center justified. And even though I have loc=center in the parameters for the title, what it is doing is center justifying over the last image (right most). Here is a link to the image: image
The warning mentioned in #1 above is on Line 15 of my code:
import keras
keras.__version__
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import ImageGrid
from keras.datasets import mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
def draw_with_mnist(theString):
for x in range(0, len(theString)):
c = int(theString[x],10)
for i in range(0, 100):
if (train_labels[i] == c):
digit = train_images[i]
plt.subplot(1, len(theString), x+1)
plt.imshow(digit)
plt.axis('off')
plt.grid(b=None)
plt.tight_layout(pad=0.00)
plt.title(theString+" -- My Name", loc='center')
plt.show()
draw_with_mnist("34085194")
Appreciate any help anyone can offer.
this is my first question here so let me know if I should make any improvements regarding e.g. formulation of the question, code and so on.
So I am creating several 3-D Scatter Plots in Python and want to safe them for later re usage and comparability. I am using Qt5 as Graphics Backend in Spyder, which perfectly displays my interactive (so I can rotate over the axes and flip the plot) 3-D Scatter plot using the origin Code.
Now I am able to successfully save the created plot and also load it into a new script, which opens the Plot in Qt5 as well. But somehow the interactivity is gone, meaning I am not able to rotate over the axes and flip the plot anymore.
I was unable to find any guidance to that issue or find any person with a similar problem. Do you guys have any idea? I'll put the relevant part of my sample Code below:
""" First script """
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import pandas as pd
import pickle
testdf = pd.DataFrame({"X" : x, "Y" : y, "Z" : z}) #x and y are the criteria, z the values, stored as lists
# Create 3d scatter plot
fig = plt.figure(figsize=(12, 12))
ax = fig.add_subplot(111, projection="3d")
ax.scatter(x, y, z, c=z, marker="o")
ax.set_xlabel("Initial Notional Cluster")
ax.set_ylabel("Laufzeit in Month Cluster")
ax.set_zlabel("Vol. Weighted Margin")
plt.show()
# Save the figure object as binary file
file = open(r"Init_Lfz_VolWeightedMargin.pkl", "wb")
pickle.dump(fig, file)
file.close()
""" Second script """
import matplotlib.pyplot as plt
figx = pickle.load(open(r"Init_Lfz_VolWeightedMargin.pkl", "rb"))
plt.show()
Any idea, why the interactivity is gone? According to the pickle library and other usercases, this shall not happen.
Many thanks.
So following a tutorial, I tried to create a graph using the following code:
time_values = [i for i in range(1,100)]
execution_time = [random.randint(0,100) for i in range(1,100)]
fig = plt.figure()
ax1 = plt.subplot()
threshold=[.8 for i in range(len(execution_time))]
ax1.plot(time_values, execution_time)
ax1.margins(x=-.49, y=0)
ax1.fill_between(time_values,execution_time, 1,where=(execution_time>1), color='r', alpha=.3)
This did not work as I got an error saying I could not compare a list and an int.
However, I then tried:
ax1.fill_between(time_values,execution_time, 1)
And that gave me a graph with all area in between the execution time and the y=1 line, filled in. Since I want the area above the y=1 line filled in, with the area below left un-shaded, I created a list called threshold, and populated it with 1 so that I could recreate the comparison. However,
ax1.fill_between(time_values,execution_time, 1,where=(execution_time>threshold)
and
ax1.fill_between(time_values,execution_time, 1)
create the exact same graph, even though the execution times values do go beyond 1.
I am confused for two reasons:
firstly, in the tutorial I was watching, the teacher was able to successfully compare a list and an integer within the fill_between function, why was I not able to do this?
Secondly, why is the where parameter not identifying the regions I want to fill? Ie, why is the graph shading in the areas between the y=1 and the value of the execution time?
The problem is mainly due the use of python lists instead of numpy arrays. Clearly you could use lists, but then you need to use them throughout the code.
import numpy as np
import matplotlib.pyplot as plt
time_values = list(range(1,100))
execution_time = [np.random.randint(0,100) for _ in range(len(time_values))]
threshold = 50
fig, ax = plt.subplots()
ax.plot(time_values, execution_time)
ax.fill_between(time_values, execution_time, threshold,
where= [e > threshold for e in execution_time],
color='r', alpha=.3)
ax.set_ylim(0,None)
plt.show()
Better is the use of numpy arrays throughout. It's not only faster, but also easier to code and understand.
import numpy as np
import matplotlib.pyplot as plt
time_values = np.arange(1,100)
execution_time = np.random.randint(0,100, size=len(time_values))
threshold = 50
fig, ax = plt.subplots()
ax.plot(time_values, execution_time)
ax.fill_between(time_values,execution_time, threshold,
where=(execution_time > threshold), color='r', alpha=.3)
ax.set_ylim(0,None)
plt.show()
Got very happy when I learned the potential of applying plotly in my use of Python. I use PyCharm, and found out that I could depict numbers, figures, stats, etc using the above option. Yet I am bit confused. First, executing code including 'import plotly.io as pio' and on the following line 'pio.renderers.default = "browser"' it takes ages for the data and graphics to load, but almost only a split second to open the browser. Second, is there an alternative to the "browser"-choice, e.g. a choice that allowed fig.show() directly in the PyCharm console? - for jupyter I think the alternative is "notebook", but that is not PyCharm. If alternatives exist to pio, i.e. that prompt rendering of code execution in the console, I'd be all ears and eyes. Thx, in advance, for any advice.
import plotly.figure_factory as ff
import plotly.graph_objects as go
import numpy as np
import plotly.io as pio
pio.renderers.default = "browser"
## Create first figure
import plotly.io as pio
pio.renderers.default = "browser"
x1,y1 = np.meshgrid(np.arange(0, 2, .2), np.arange(0, 2, .2))
u1 = np.cos(x1)*y1
v1 = np.sin(x1)*y1
fig1 = ff.create_quiver(x1, y1, u1, v1, name='Quiver')
fig1.show()
## Create second figure
import plotly.io as pio
pio.renderers.default = "browser"
x = np.linspace(-3, 3, 100)
y = np.linspace(-3, 3, 100)
Y, X = np.meshgrid(x, y)
u = -1 - X**2 + Y
v = 1 + X - Y**2
fig2 = ff.create_streamline(x, y, u, v, arrow_scale=.1, name='Steamline')
fig2.show()
Sometimes results are indeed rendered in a browser, mostly the browser stops loading and an utterly blank, white screen keeps staring at me. That is why I'd kind of fancied a rendering result as performed in matplotlib where graphics is shown in the console directly.
The official answer to your question is that you can use other renderers like “png” or “svg” as provided by Orca. If your figure is too complex, however, you may have trouble with any renderer, depending on your hardware setup.
More info here: https://plot.ly/python/renderers/
I am trying to display a sentinel-1 satellite image which is of size 26000(width) X 17000(height) pixels and using Python 3.5. I am able to load this image in numpy as an array and trying to display in matplotlib but unable to do as it gives Memory Error..The screen resolution is 1600(width) X 1200(height) pixels.I am using windows 7 with 8GB RAM. I agree that it might not be able to display due to memory constraints but is there any way I can display such huge image ?
Also I have seen many satellite image processing softwares such as SNAP(sentinel toolbox) which can display such huge images in the above specified screen resolution, but cannot find how they do it.
Kindly help.
Tried to create an image with the width and height as specified by you. My screen resolution is 1920 by 1080, FHD
import matplotlib
matplotlib.rcParams['figure.dpi'] = 120
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import cm
from numpy.random import randn
fig, ax = plt.subplots()
data = np.clip(randn(26000, 17000), -1, 1)
cax = ax.imshow(data, interpolation='nearest', cmap=cm.coolwarm)
cbar = fig.colorbar(cax, ticks=[-1, 0, 1])
cbar.ax.set_yticklabels(['< -1', '0', '> 1'])
plt.show()
The plot is generated but about 7GB of memory is eaten by python!