I have created an animated heatmap using holoviews with the 'bokeh' renderer. The code is below and runs in a Jupyter Notebook. Basically I compose a dictionary of heatmaps then I use 'hv.DynamicMap' and 'hv.streams' to create the animation ie. stream keys of dict and render the associated heatmap.
The code runs successfully as an animation but produces an output where a single frame looks like this:
I have two questions:
I want the slider to show the 'key' of the dictionary not just the
index of the list of keys as it does now. So in the image above 'a0' not '0'. How can I do that?
there are warnings being thrown about 'fixed_width'. I have set the
height and width of all the objects so why is there a warning?
import numpy as np
import holoviews as hv
from bokeh.io import show, curdoc, output_notebook
from bokeh.layouts import layout
from bokeh.models import Slider, Button
renderer = hv.renderer('bokeh').instance(mode='server')
output_notebook()
# generate a dict of heatmaps
heatmap_dict = {}
for i in range(10):
heatmap = hv.HeatMap((np.random.randint(0, 10, 100), np.random.choice(['A', 'B', 'C', 'D', 'E'], 100),
np.random.randn(100), np.random.randn(100)), vdims=['z', 'z2']).sort().aggregate(function=np.mean)
heatmap.opts(height=400, width=400)
heatmap_dict['a' + str(i)] = heatmap
heatmap_keys = list(heatmap_dict.keys())
# Create the holoviews app again
def mapping(phase):
key = heatmap_keys[phase]
return heatmap_dict[key]
stream = hv.streams.Stream.define('Phase', phase=0)()
dmap = hv.DynamicMap(mapping, streams=[stream])
# Define valid function for FunctionHandler
# when deploying as script, simply attach to curdoc
def modify_doc(doc):
# Create HoloViews plot and attach the document
hvplot = renderer.get_plot(dmap, doc)
# Create a slider and play buttons
def animate_update():
year = slider.value + 1
if year > end:
year = start
slider.value = year
def slider_update(attrname, old, new):
# Notify the HoloViews stream of the slider update
stream.event(phase=new)
start, end = 0, len(heatmap_keys) - 1
slider = Slider(start=start, end=end, value=start, step=1, title="Phase", height=30, width=180)
slider.on_change('value', slider_update)
callback_id = None
def animate():
global callback_id
if button.label == '► Play':
button.label = '❚❚ Pause'
callback_id = doc.add_periodic_callback(animate_update, 50)
else:
button.label = '► Play'
doc.remove_periodic_callback(callback_id)
button = Button(label='► Play', width=60, height=30)
button.on_click(animate)
# Combine the holoviews plot and widgets in a layout
plot = layout([
[hvplot.state],
[slider, button]], sizing_mode='fixed')
doc.add_root(plot)
return doc
# To display in the notebook
show(modify_doc, notebook_url='localhost:8888')
Related
For years, I've been struggling to get efficient live plotting in matplotlib, and to this day I remain unsatisfied.
I want a redraw_figure function that updates the figure "live" (as the code runs), and will display the latest plots if I stop at a breakpoint.
Here is some demo code:
import time
from matplotlib import pyplot as plt
import numpy as np
def live_update_demo():
plt.subplot(2, 1, 1)
h1 = plt.imshow(np.random.randn(30, 30))
redraw_figure()
plt.subplot(2, 1, 2)
h2, = plt.plot(np.random.randn(50))
redraw_figure()
t_start = time.time()
for i in xrange(1000):
h1.set_data(np.random.randn(30, 30))
redraw_figure()
h2.set_ydata(np.random.randn(50))
redraw_figure()
print 'Mean Frame Rate: %.3gFPS' % ((i+1) / (time.time() - t_start))
def redraw_figure():
plt.draw()
plt.pause(0.00001)
live_update_demo()
Plots should update live when the code is run, and we should see the latest data when stopping at any breakpoint after redraw_figure(). The question is how to best implement redraw_figure()
In the implementation above (plt.draw(); plt.pause(0.00001)), it works, but is very slow (~3.7FPS)
I can implement it as:
def redraw_figure():
plt.gcf().canvas.flush_events()
plt.show(block=False)
And it runs faster (~11FPS), but plots are not up-to date when you stop at breakpoints (eg if I put a breakpoint on the t_start = ... line, the second plot does not appear).
Strangely enough, what does actually work is calling the show twice:
def redraw_figure():
plt.gcf().canvas.flush_events()
plt.show(block=False)
plt.show(block=False)
Which gives ~11FPS and does keep plots up-to-data if your break on any line.
Now I've heard it said that the "block" keyword is deprecated. And calling the same function twice seems like a weird, probably-non-portable hack anyway.
So what can I put in this function that will plot at a reasonable frame rate, isn't a giant kludge, and preferably will work across backends and systems?
Some notes:
I'm on OSX, and using TkAgg backend, but solutions on any backend/system are welcome
Interactive mode "On" will not work, because it does not update live. It just updates when in the Python console when the interpreter waits for user input.
A blog suggested the implementation:
def redraw_figure():
fig = plt.gcf()
fig.canvas.draw()
fig.canvas.flush_events()
But at least on my system, that does not redraw the plots at all.
So, if anybody has an answer, you would directly make me and thousands of others very happy. Their happiness would probably trickle through to their friends and relatives, and their friends and relatives, and so on, so that you could potentially improve the lives of billions.
Conclusions
ImportanceOfBeingErnest shows how you can use blit for faster plotting, but it's not as simple as putting something different in the redraw_figure function (you need to keep track of what things to redraw).
First of all, the code that is posted in the question runs with 7 fps on my machine, with QT4Agg as backend.
Now, as has been suggested in many posts, like here or here, using blit might be an option. Although this article mentions that blit causes strong memory leakage, I could not observe that.
I have modified your code a bit and compared the frame rate with and without the use of blit. The code below gives
28 fps when run without blit
175 fps with blit
Code:
import time
from matplotlib import pyplot as plt
import numpy as np
def live_update_demo(blit = False):
x = np.linspace(0,50., num=100)
X,Y = np.meshgrid(x,x)
fig = plt.figure()
ax1 = fig.add_subplot(2, 1, 1)
ax2 = fig.add_subplot(2, 1, 2)
img = ax1.imshow(X, vmin=-1, vmax=1, interpolation="None", cmap="RdBu")
line, = ax2.plot([], lw=3)
text = ax2.text(0.8,0.5, "")
ax2.set_xlim(x.min(), x.max())
ax2.set_ylim([-1.1, 1.1])
fig.canvas.draw() # note that the first draw comes before setting data
if blit:
# cache the background
axbackground = fig.canvas.copy_from_bbox(ax1.bbox)
ax2background = fig.canvas.copy_from_bbox(ax2.bbox)
plt.show(block=False)
t_start = time.time()
k=0.
for i in np.arange(1000):
img.set_data(np.sin(X/3.+k)*np.cos(Y/3.+k))
line.set_data(x, np.sin(x/3.+k))
tx = 'Mean Frame Rate:\n {fps:.3f}FPS'.format(fps= ((i+1) / (time.time() - t_start)) )
text.set_text(tx)
#print tx
k+=0.11
if blit:
# restore background
fig.canvas.restore_region(axbackground)
fig.canvas.restore_region(ax2background)
# redraw just the points
ax1.draw_artist(img)
ax2.draw_artist(line)
ax2.draw_artist(text)
# fill in the axes rectangle
fig.canvas.blit(ax1.bbox)
fig.canvas.blit(ax2.bbox)
# in this post http://bastibe.de/2013-05-30-speeding-up-matplotlib.html
# it is mentionned that blit causes strong memory leakage.
# however, I did not observe that.
else:
# redraw everything
fig.canvas.draw()
fig.canvas.flush_events()
#alternatively you could use
#plt.pause(0.000000000001)
# however plt.pause calls canvas.draw(), as can be read here:
#http://bastibe.de/2013-05-30-speeding-up-matplotlib.html
live_update_demo(True) # 175 fps
#live_update_demo(False) # 28 fps
Update:
For faster plotting, one may consider using pyqtgraph.
As the pyqtgraph documentation puts it: "For plotting, pyqtgraph is not nearly as complete/mature as matplotlib, but runs much faster."
I ported the above example to pyqtgraph. And although it looks kind of ugly, it runs with 250 fps on my machine.
Summing that up,
matplotlib (without blitting): 28 fps
matplotlib (with blitting): 175 fps
pyqtgraph : 250 fps
pyqtgraph code:
import sys
import time
from pyqtgraph.Qt import QtCore, QtGui
import numpy as np
import pyqtgraph as pg
class App(QtGui.QMainWindow):
def __init__(self, parent=None):
super(App, self).__init__(parent)
#### Create Gui Elements ###########
self.mainbox = QtGui.QWidget()
self.setCentralWidget(self.mainbox)
self.mainbox.setLayout(QtGui.QVBoxLayout())
self.canvas = pg.GraphicsLayoutWidget()
self.mainbox.layout().addWidget(self.canvas)
self.label = QtGui.QLabel()
self.mainbox.layout().addWidget(self.label)
self.view = self.canvas.addViewBox()
self.view.setAspectLocked(True)
self.view.setRange(QtCore.QRectF(0,0, 100, 100))
# image plot
self.img = pg.ImageItem(border='w')
self.view.addItem(self.img)
self.canvas.nextRow()
# line plot
self.otherplot = self.canvas.addPlot()
self.h2 = self.otherplot.plot(pen='y')
#### Set Data #####################
self.x = np.linspace(0,50., num=100)
self.X,self.Y = np.meshgrid(self.x,self.x)
self.counter = 0
self.fps = 0.
self.lastupdate = time.time()
#### Start #####################
self._update()
def _update(self):
self.data = np.sin(self.X/3.+self.counter/9.)*np.cos(self.Y/3.+self.counter/9.)
self.ydata = np.sin(self.x/3.+ self.counter/9.)
self.img.setImage(self.data)
self.h2.setData(self.ydata)
now = time.time()
dt = (now-self.lastupdate)
if dt <= 0:
dt = 0.000000000001
fps2 = 1.0 / dt
self.lastupdate = now
self.fps = self.fps * 0.9 + fps2 * 0.1
tx = 'Mean Frame Rate: {fps:.3f} FPS'.format(fps=self.fps )
self.label.setText(tx)
QtCore.QTimer.singleShot(1, self._update)
self.counter += 1
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
thisapp = App()
thisapp.show()
sys.exit(app.exec_())
Here's one way to do live plotting: get the plot as an image array then draw the image to a multithreaded screen.
Example using a pyformulas screen (~30 FPS):
import pyformulas as pf
import matplotlib.pyplot as plt
import numpy as np
import time
fig = plt.figure()
screen = pf.screen(title='Plot')
start = time.time()
for i in range(10000):
t = time.time() - start
x = np.linspace(t-3, t, 100)
y = np.sin(2*np.pi*x) + np.sin(3*np.pi*x)
plt.xlim(t-3,t)
plt.ylim(-3,3)
plt.plot(x, y, c='black')
# If we haven't already shown or saved the plot, then we need to draw the figure first...
fig.canvas.draw()
image = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8, sep='')
image = image.reshape(fig.canvas.get_width_height()[::-1] + (3,))
screen.update(image)
#screen.close()
Disclaimer: I'm the maintainer of pyformulas
I made an animation for sorting algorithms and it it works great for animating one sorting algorithm, but when I try to animate multiple at the same time both windows come up but none of them are moving. I was wondering how I could go around to fix this.
When I run the code the first figure is stuck on the first frame and the second figure jumps to the last frame
import matplotlib.pyplot as plt
from matplotlib import animation
import random
# my class for getting data from sorting algorithms
from animationSorters import *
def sort_anim(samp_size=100, types=['bubblesort', 'quicksort']):
rndList = random.sample(range(1, samp_size+1), samp_size)
anim = []
for k in range(0, len(types)):
sort_type = types[k]
animation_speed = 1
def barlist(x):
if sort_type == 'bubblesort':
l = bubblesort_swaps(x)#returns bubble sort data
elif sort_type == 'quicksort':
l = quicksort_swaps(x)#returns quick sort data
final = splitSwaps(l, len(x))
return final
fin = barlist(rndList)
fig = plt.figure(k+1)
plt.rcParams['axes.facecolor'] = 'black'
n= len(fin)#Number of frames
x=range(1,len(rndList)+1)
barcollection = plt.bar(x,fin[0], color='w')
anim_title = sort_type.title() + '\nSize: ' + str(samp_size)
plt.title(anim_title)
def animate(i):
y=fin[i]
for i, b in enumerate(barcollection):
b.set_height(y[i])
anim.append(animation.FuncAnimation(fig,animate, repeat=False,
blit=False, frames=n, interval=animation_speed))
plt.show()
sort_anim()
As explained in the documentation for the animation module:
it is critical to keep a reference to the instance object. The
animation is advanced by a timer (typically from the host GUI
framework) which the Animation object holds the only reference to. If
you do not hold a reference to the Animation object, it (and hence the
timers), will be garbage collected which will stop the animation.
Therefore you need to return the references to your animations from your function, otherwise those objects are destroyed when exiting the function.
Consider the following simplification of your code:
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import numpy as np
def my_func(nfigs=2):
anims = []
for i in range(nfigs):
fig = plt.figure(num=i)
ax = fig.add_subplot(111)
col = ax.bar(x=range(10), height=np.zeros((10,)))
ax.set_ylim([0, 1])
def animate(k, bars):
new_data = np.random.random(size=(10,))
for j, b in enumerate(bars):
b.set_height(new_data[j])
return bars,
ani = animation.FuncAnimation(fig, animate, fargs=(col, ), frames=100)
anims.append(ani)
return anims
my_anims = my_func(3)
# calling simply my_func() here would not work, you need to keep the returned
# array in memory for the animations to stay alive
plt.show()
I am learning to plot live graphs using matplotlib, and so far have managed to read live data from serial port and plot it.
Following is my code:-
import serial
import numpy
import matplotlib.pyplot as plt
from drawnow import *
import binascii
import serial
import time
x = []
y = []
plt.ion()
cnt=0
z1baudrate = 9600
z1port = 'COM6'
z1serial = serial.Serial(port=z1port, baudrate=z1baudrate)
z1serial.timeout = 1
print (z1serial.is_open)
def makeFig():
plt.ylim(0,150)
plt.title('Live Data')
plt.grid(True)
plt.ylabel('Temperature')
plt.plot(x, y, 'ro-', label='F')
plt.legend(loc='upper left')
if z1serial.is_open:
while True:
size = z1serial.inWaiting()
if size:
data = z1serial.read(1)
data = (ord(data))
print (data)
if data:
cnt = cnt+1
x.append(cnt)
y.append(data)
drawnow(makeFig)
plt.pause(.000001)
cnt=cnt+1
if(cnt>50):
x.pop(0)
y.pop(0)
z1serial.flushInput()
z1serial.flushOutput()
else:
print ('no data')
time.sleep(1)
else:
print ('z1serial not open')
I want to two buttons on the live graph, Start and Stop
Pressing Stop button would stop the live plotting, leaving behind a blank graph.
Pressing Start button would start the live plotting again.
I referred to the link <
Matplotlib Python: How to add panel button> but couldn't use it properly to add to my code.
How can I add the buttons to the live plot?
Thanks in advance!
EDIT: Some users have mentioned that the question is unclear. My objective is to keep track of previous states.
I am trying to create a plot that can be modified using 3 widgets. However, every new widget change does not take into consideration previous widget selections (for example, if a selection is made using widget 1 and then widget 2 is modified, the modification of widget 2 considers the original graph and not the changes made with widget 1).
I am trying to avoid using Custom_JS as I have no experience with Javascript. Is there any way to combine the functions so that any change in the widgets takes into consideration previous widget interactions?
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib
from datetime import date
from ipywidgets import interact
from bokeh.io import push_notebook, show, output_notebook, curdoc
from bokeh.plotting import figure
from bokeh.layouts import column, layout, widgetbox, row
from bokeh.models import ColumnDataSource, HoverTool, CustomJS, Panel
from bokeh.layouts import widgetbox
from bokeh.models.widgets import RangeSlider, Slider, Select,
DateRangeSlider, Tabs
df = pd.read_csv("/Users/danielmontillanavas/Desktop/Tiller/00_Data/SF_may_correct_decimal.csv", sep =',',decimal=',')
df = df.drop(['Unnamed: 0'], axis=1)
df.rename(columns = {'Stamped phone from HS':'phone','Contact Email':'Email','Account Name':'Account_name',
'Opportunity ID':'ID', 'Close Date':'Close_date','Stamped-date of closed (DO NOT USE)':'Stamped_date',
'Quote Amount':'Quote_Amount', 'Lead Source':'Source','Desired activation date':'Activ_date'},
inplace=True)
df.Close_date = pd.to_datetime(df.Close_date, format='%Y-%m-%d')
cols_num = ['Quote_Amount','DISCOUNT']
df[cols_num] = df[cols_num].apply(pd.to_numeric)
df_closed = df[df['Stage']=='Closed']
df_closed.fillna("Unknown", inplace=True)
start_point = min(df['Quote_Amount'])
end_point = max(df['Quote_Amount'])
TOOLS = 'pan,wheel_zoom,box_zoom,reset,tap,save,box_select,lasso_select'
source = ColumnDataSource(df_closed)
hover = HoverTool(
tooltips=[
("Quote", "$x"),
("Discount", "$y")
]
)
p = figure(title='Quotes per Source - Closed deals',tools=[hover,TOOLS],
plot_height=800, plot_width=800)
p.circle('Quote_Amount','DISCOUNT',source=source, size = 8, color = 'CornflowerBlue', alpha = 0.6)
N = 20000
slider = Slider(start=start_point, end=end_point, step=10, value=N,
title='Select Quote Amount Cutoff')
dfList = df_closed.Source.unique().tolist()
All_view = ['All']
source_options = All_view + dfList
menu = Select(title = "Select Lead Source",options=source_options, value = 'All')
first_date = min(df['Close_date'])
last_date = max(df['Close_date'])
date_range_slider = DateRangeSlider(title="Select Date Range ", start=first_date, end=date.today(), value=(date(2017, 9, 7), date(2017, 10, 15)), step=1)
def slider_callback(attr, old, new):
N = new # this works also with slider.value but new is more explicit
new1 = ColumnDataSource(df_closed.loc[(df_closed.Quote_Amount < N)])
source.data = new1.data
slider.on_change('value',slider_callback)
def menu_callback(attr, old, new):
if menu.value == 'All': new2 = ColumnDataSource(df_closed)
else: new2 = ColumnDataSource(df_closed.loc[(df_closed.Source == menu.value)])
source.data = new2.data
menu.on_change('value',menu_callback)
def date_callback(attr, old, new):
start = date_range_slider.value_as_datetime[0].strftime("%Y-%m-%d")
end = date_range_slider.value_as_datetime[1].strftime("%Y-%m-%d")
df_closed_new = df_closed[df_closed['Close_date'] >= start]
df_closed_new = df_closed[df_closed['Close_date'] <= end]
new3 = ColumnDataSource(df_closed_new)
source.data = new3.data
date_range_slider.on_change('value',date_callback)
# Put controls in a single element
controls = widgetbox(menu, slider, date_range_slider)
# Create a row layout
layout = row(controls, p)
curdoc().add_root(layout)
You question is not very clear to me. Are you talking about the previous state of widgets? The callbacks have access to the current state of all the other widgets so if you want to maintain a history of their previous states you'll have to explicitly keep track of that.
However I immediately notice on thing that should not be done, so I am going to post an answer just to draw attention to it. Don't create new CDS objects just to use their .data attribute and throw them away:
new1 = ColumnDataSource(df_closed.loc[(df_closed.Quote_Amount < N)])
source.data = new1.data
There is alot of machinery under the covers that affords all the automatic synchronization other features of Bokeh. CDS in particular are extremely heavyweight, complicated objects, and doing this above is a known anti-pattern that can break things. Instead, if you just need a new suitable .data dict, then use from_df:
new_data = ColumnDataSource.from_df(df_closed.loc[(df_closed.Quote_Amount < N)])
source.data = new_data
I am trying to render a simple app on bokeh server. I took the script straight from the bokeh website.
# myapp.py
from random import random
from bokeh.layouts import column
from bokeh.models import Button
from bokeh.palettes import RdYlBu3
from bokeh.plotting import figure, curdoc
# create a plot and style its properties
p = figure(x_range=(0, 100), y_range=(0, 100), toolbar_location=None)
p.border_fill_color = 'black'
p.background_fill_color = 'black'
p.outline_line_color = None
p.grid.grid_line_color = None
# add a text renderer to our plot (no data yet)
r = p.text(x=[], y=[], text=[], text_color=[], text_font_size="20pt",
text_baseline="middle", text_align="center")
i = 0
ds = r.data_source
# create a callback that will add a number in a random location
def callback():
global i
# BEST PRACTICE --- update .data in one step with a new dict
new_data = dict()
new_data['x'] = ds.data['x'] + [random()*70 + 15]
new_data['y'] = ds.data['y'] + [random()*70 + 15]
new_data['text_color'] = ds.data['text_color'] + [RdYlBu3[i%3]]
new_data['text'] = ds.data['text'] + [str(i)]
ds.data = new_data
i = i + 1
# add a button widget and configure with the call back
button = Button(label="Press Me")
button.on_click(callback)
# put the button and plot in a layout and add to the document
curdoc().add_root(column(button, p))
I run this in my Win Python Command Prompt.exe using the below:
python -m bokeh serve --show main.py
This runs the app, but I am left with a blank webpage. See the output in the command prompt below:
Can anyone help with how to get this to show the doc?