I am new to plotly dash. I want to draw a table whose values (Rows) will
automatically be updated after certain interval of time but i do not know how
to use dash table experiments. The table is already saved as CSV file but i
am somehow unable make it live.
Please help!
Can some one guide me in the right direction what should i do
Your help will be highly appreciated. Following is the code.
import dash
import pandas as pd
from pandas import Series, DataFrame
from dash.dependencies import Input, Output, Event
import dash_core_components as dcc
import dash_html_components as html
import dash_table_experiments as dtable
app=dash.Dash()
def TP_Sort():
address = 'E:/Dats Science/POWER BI LAB DATA/PS CORE KPIS/Excel Sheets/Throughput.xlsx'
TP = pd.read_excel(address)
TP1=TP.head()
Current_Interval.to_csv('TP1.csv', index=False)
return app.layout = html.Div([
html.H1('Data Throughput Dashboard-NOC NPM Core'),
dcc.Interval(id='graph-update',interval=240000),
dtable.DataTable(id='my-table',
rows=[{}],
row_selectable=False,
filterable=True,
sortable=False,
editable=False)
])
#app.callback(
dash.dependencies.Output('my-table','row_update'),
events=[dash.dependencies.Event('graph-update', 'interval')])
def update_table(maxrows=4):
TP_Sort()
TP_Table1='C:/Users/muzamal.pervez/Desktop/Python Scripts/TP1.csv'
TP_Table2=pd.read_csv(TP_Table1)
return TP_Table2.to_dict('records')
if __name__ == '__main__':
app.run_server(debug=False)
I am trying the above approach. Please correct me where i am wrong as the output is error loading dependencies.
BR
Rana
Your callback is wrong.
It should be:
#app.callback(Output('my-table', 'rows'), [Input('graph-update', 'n_intervals')])
def update_table(n, maxrows=4):
# We're now in interval *n*
# Your code
return TP_Table2.to_dict('records')
Related
I am a GIS person fairly new to Plotly and exceptionally new to Dash. I'm trying to mostly copy an example solution from a post here:
drop down menu with dash / plotly
To build an interactive app to look at various choropleth maps based on choropleth_mapbox figures. The last solution from the above post, using Plotly and Dash by Rob Raymond, looks brilliant and close to what I am trying to do. But in my case, my figures built on several data 'columns' also require an individual update_layout call and a hovertemplate built for each data column; and I cannot figure out where to place those definitions within the solution posted above.
This is my code for a single data column's figure, which gives me the functionality I want in the layout and hover tool:
fig = px.choropleth_mapbox(
gdf_blks_results,
geojson = gdf_blks.geometry,
locations = gdf_blks_results.index,
color=classme.yb,
color_continuous_scale = "YlOrRd",
center={"lat": 18.2208, "lon": -66.49},
mapbox_style="open-street-map",
width=800,
height=500,
custom_data = [gdf_blks_results['GEOID'],
gdf_blks_results['overallBurden']]
)
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0},
coloraxis_colorbar=dict(
title="burden",
thicknessmode="pixels",
lenmode="pixels",
yanchor="top",y=1,
ticks="outside",
tickvals=[0,1,2,3,4],
ticktext=myclasses,
dtick=5
))
# hover template
hovertemp = '<i>Census ID :</i> %{customdata[0]}<br>'
hovertemp += '<i>burden : </i> %{customdata[1]:.5f}<br>'
fig.update_traces(hovertemplate=hovertemp)
fig.show()
My question is, how do I incorporate that into the list of figures for a set of columns of data with custom template and figure update info for each? I tried to add it to the figure definitions in the cited post example before the "for c, color in zip(...)" statement, but I cannot get the syntax right, and I am not sure why not.
I think you should create a Dropdown list with Options as gdf_blks_results columns the returns it with callback to update choropleth map. Please refer below code:
import pandas as pd
import numpy as np
import plotly.express as px
import dash
import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output
import dash_table
import dash_bootstrap_components as dbc
columns_list = list(gdf_blks_results.columns)
app = dash.Dash(__name__,external_stylesheets=[dbc.themes.LUX])
app.layout = html.Div([
dbc.Row([
dbc.Col([
html.H5('Columns',className='text-center'),
],width={'size':2,"offset":0,'order':1}),
dbc.Col([
dcc.Dropdown(id='columns',placeholder="Please select columns",
options=[{'label':x,'value':x} for x in columns_list],
value=[],
multi=False,
disabled=False,
clearable=True,
searchable=True),
],width={'size':10,"offset":0,'order':1})
], className='p-2 align-items-stretch'),
dbc.Row([
dbc.Col([
dcc.Graph(id="choropleth_maps",figure={},style={'height':500}), #Heatmap plot
],width={'size':12,'offset':0,'order':2}),
]),
])
#app.callback(Output('choropleth_maps', 'figure'),
[Input('columns', 'value')])
def update_graph(columns):
fig = px.choropleth_mapbox(
gdf_blks_results,
geojson = gdf_blks.geometry,
locations = gdf_blks_results.index,
color=columns,
color_continuous_scale = "YlOrRd",
center={"lat": 18.2208, "lon": -66.49},
mapbox_style="open-street-map",
width=800,
height=500,
custom_data = [gdf_blks_results['GEOID'],
gdf_blks_results['overallBurden']])
fig.update_layout(margin={"r":0,"t":0,"l":0,"b":0},
coloraxis_colorbar=dict(
title="burden",
thicknessmode="pixels",
lenmode="pixels",
yanchor="top",y=1,
ticks="outside",
tickvals=[0,1,2,3,4],
ticktext=myclasses,
dtick=5
))
# hover template
hovertemp = '<i>Census ID :</i> %{customdata[0]}<br>'
hovertemp += '<i>burden : </i> %{customdata[1]:.5f}<br>'
fig.update_traces(hovertemplate=hovertemp)
return fig
if __name__ == "__main__":
app.run_server(debug=False,port=1116)
I created a code (using Tkinter, Python3 and matplotlid) that could read data from different serial ports, save them to csv, then create graphs and finally preview data in GUI. The code was splited in two different scripts. The main script contained reading data, save data to csv an priview of data and the other script contained the graph creation.
Today I rewrote the code using the answer of #user2464430 here. The code is working, but I can't update the GUI. Opens once and then no refresh with new data.
The following code is a part of total code.
My code is:
from PIL import ImageTk, Image
import tkinter as Tk
import multiprocessing
from queue import Empty, Full
from time import strftime
import serial
import numpy as np
import matplotlib.pyplot as plt
from drawnow import *
from pylab import *
import pandas as pd
from datetime import timedelta
from datetime import datetime
import plotly.graph_objs as go
from plotly.subplots import make_subplots
import locale
import os
class GuiApp(object):
def __init__(self, image):
self.root = Tk.Tk()
self.root.resizable(width=False, height=False)
self.root.geometry("1600x800+0+0")
C = Canvas(self.root, bg="black", width=1600, height=800)
def BasicLabels():
....... # in this stage create multiple axis labels
Î¥AxisLabels()
BasicLabels()
def ValueLabels():
....... # Read and munipulate datas from CSV file and print in in labels
ValueLabels()
C.pack()
def GenerateData(q): #Read Serial Ports and store data to CSV file
file_exists = os.path.isfile("BigData.csv")
header = [["Daytime,T1"]]
if not file_exists:
with open("BigData.csv", "a+") as csvfile:
np.savetxt(csvfile, header, delimiter=",", fmt="%s", comments="")
while True:
try:
ser1 = serial.Serial(port="COM4", baudrate=9600)
read_ser1 = ser1.readline()
if read_ser1 == "":
read_ser1 = "Missing Value"
else:
read_ser1 = ser1.readline()
read_ser1 = str(read_ser1[0 : len(read_ser1)].decode("utf-8"))
# print("COM4:", read_ser1)
ser1.close()
except:
print("Failed 1")
read_ser1 = "9999,9999,9999,9999,9999"
daytime = strftime(" %d-%m-%Y %H:%M:%S")
rows = [
daytime
+ ","
+ read_ser1.strip()
]
with open("BigData.csv", "a+") as csvfile:
np.savetxt(csvfile, rows, delimiter=",", fmt="%s", comments="")
CreateGraphs()
def CreateGraphs():
#Code to generate graph. Called every time i have new line in CSV.
if __name__ == "__main__":
# Queue which will be used for storing Data
q = multiprocessing.Queue()
q.cancel_join_thread() # or else thread that puts data will not term
gui = GuiApp(q)
t1 = multiprocessing.Process(target=GenerateData, args=(q,))
t1.start()
gui.root.mainloop()
t1.join()
The graphs are generating after while True in GenerateData.
All datas for labels and graphs are coming from CSV file and not directly from serial port.
Is it possible to update GUI with latest datas from CSV and created graphs?
Thank for your time.
I'm working on a simulated osilloscope where the server PC collects data and ultimately will publish the streaming plot online. Below is a working script that can do the job. However, when I open multiple browsers, the streaming plots exhibit different data. (Although they are using the same data source). The example 'ohlc' seems to have the same problem. So, what is the right way to do this? I'm considering to write data to a file, but that will bring some issues like file i/o delay and disk storage limitation etc. Thank you for any help.
from bokeh.server.server import Server
from bokeh.models import ColumnDataSource, Label
from bokeh.plotting import figure
from bokeh.layouts import column
import numpy as np
import datetime as dt
from functools import partial
import time
# this will be replaced with the real data collector in the end
def f_emitter(p=0.1):
v = np.random.rand()
return (dt.datetime.now(), 0. if v>p else v)
def make_document(doc, functions, labels):
def update():
for index, func in enumerate(functions):
data = func()
sources[index].stream(new_data=dict(time=[data[0]], data=[data[1]]), rollover=1000)
annotations[index].text = f'{data[1]: .3f}'
sources = [ColumnDataSource(dict(time=[], data=[])) for _ in range(len(functions))]
figs = []
annotations = []
for i in range(len(functions)):
figs.append(figure(x_axis_type='datetime',
y_axis_label=labels[i], toolbar_location=None,
active_drag=None, active_scroll=None))
figs[i].line(x='time', y='data', source=sources[i])
annotations.append(Label(x=10, y=10, text='', text_font_size='40px', text_color='black',
x_units='screen', y_units='screen', background_fill_color='white'))
figs[i].add_layout(annotations[i])
# print(figs[i].plot_height)
doc.add_root(column([fig for fig in figs], sizing_mode='stretch_both'))
doc.add_periodic_callback(callback=update, period_milliseconds=100)
if __name__ == '__main__':
# list of functions and labels to feed into the scope
functions = [f_emitter]
labels = ['emitter']
server = Server({'/': partial(make_document, functions=functions, labels=labels)})
server.start()
server.io_loop.add_callback(server.show, "/")
try:
server.io_loop.start()
except KeyboardInterrupt:
print('keyboard interruption')
When you connect with a new client, by default Bokeh creates a new session. Each session has its own document, so the data source end up not being the same.
I am trying to make this web app to work but I am getting an error. these are the steps that web app is supposed to handle:
import a file
run the python script
export the results
when I run python script independently( without interfering with flask), it works fine( I use Jupyter notebook) on the other hand, when I run it with flask (from prompt) I get an error:
File "app.py", line 88, in <module>
for name, df in transformed_dict.items():
NameError: name 'transformed_dict' is not defined
Any idea of how can I make this web app to work?
This is my first time using flask and I will appreciate any suggestions or guidance.
python file & html file
from flask import Flask,render_template,request,send_file
from flask_sqlalchemy import SQLAlchemy
import os
import pandas as pd
from openpyxl import load_workbook
import sqlalchemy as db
def transform(df):
# Some data processing here
return df
app=Flask(__name__)
#app.route('/')
def index():
return render_template('firstpage.html')
#app.route('/upload',methods=['Get','POST'])
def upload():
file=request.files['inputfile']
xls=pd.ExcelFile(file)
name_dict = {}
snames = xls.sheet_names
for sn in snames:
name_dict[sn] = xls.parse(sn)
for key, value in name_dict.items():
transform(value)
transformed_dict={}
for key, value in name_dict.items():
transformed_dict[key]=transform(value)
#### wirte to excel example:
writer = pd.ExcelWriter("MyData.xlsx", engine='xlsxwriter')
for name, df in transformed_dict.items():
df.to_excel(writer, sheet_name=name)
writer.save()
if __name__=='__main__':
app.run(port=5000)
Your block:
#### wirte to excel example:
writer = pd.ExcelWriter("MyData.xlsx", engine='xlsxwriter')
for name, df in transformed_dict.items():
df.to_excel(writer, sheet_name=name)
writer.save()
should be part of your upload() function since that's where you define and fill transformed_dict. You just need to match the indentation there to the block above it.
The current error is coming up because it's trying to run that code as soon as you start your script, and transformed_dict doesn't exist at that point.
I'm trying to implement multiprocessing library for splitting up a dataframe into parts, process it on multiple cores of CPU and then concatenate the results back into a final dataframe in a python dash application. The code works fine when I try it outside of the dash application (when I run the code standalone without enclosing it in a dash application). But when I enclose the same code in a dash application, I get an error. I have shown the code below:
I have tried the multiprocessing code out of the dash framework and it works absolutely fine.
import dash
from dash.dependencies import Input, Output, State
import dash_core_components as dcc
import dash_html_components as html
import flask
import dash_table_experiments as dt
import dash_table
import dash.dependencies
import base64
import time
import os
import pandas as pd
from docx import *
from docx.text.paragraph import Paragraph
from docx.text.paragraph import Run
import xml.etree.ElementTree as ET
import multiprocessing as mp
from multiprocessing import Pool
from docx.document import Document as doctwo
from docx.oxml.table import CT_Tbl
from docx.oxml.text.paragraph import CT_P
from docx.table import _Cell, Table
from docx.text.paragraph import Paragraph
import io
import csv
import codecs
import numpy as np
app = dash.Dash(__name__)
application = app.server
app.config.supress_callback_exceptions = True
app.layout = html.Div(children=[
html.Div([
html.Div([
html.H4(children='Reader'),
html.Br(),
],style={'text-align':'center'}),
html.Br(),
html.Br(),
html.Div([
dcc.Upload(html.Button('Upload File'),id='upload-data',style = dict(display = 'inline-block')),
html.Br(),
]
),
html.Div(id='output-data-upload'),
])
])
#app.callback(Output('output-data-upload', 'children'),
[Input('upload-data', 'contents')],
[State('upload-data', 'filename')])
def update_output(contents, filename):
if contents is not None:
content_type, content_string = contents.split(',')
decoded = base64.b64decode(content_string)
document = Document(io.BytesIO(decoded))
combined_df = pd.read_csv('combined_df.csv')
def calc_tfidf(input1):
input1 = input1.reset_index(drop=True)
input1['samplecol'] = 'sample'
return input1
num_cores = mp.cpu_count() - 1 #number of cores on your machine
num_partitions = mp.cpu_count() - 1 #number of partitions to split dataframe
df_split = np.array_split(combined_df, num_partitions)
pool = Pool(num_cores)
df = pd.concat(pool.map(calc_tfidf, df_split))
pool.close()
pool.join()
return len(combined_df)
else:
return 'No File uploaded'
app.css.append_css({'external_url': 'https://codepen.io/plotly/pen/EQZeaW.css'})
if __name__ == '__main__':
app.run_server(debug=True)
The above dash application takes as input any file. Upon uploading the file in the front end, a local CSV file (any file, in my case it is combined_df.csv) is loaded into a dataframe. Now I want to split the dataframe into parts using multiprocessing, process it and combine it back. But the above code results in the following error:
AttributeError: Can't pickle local object 'update_output..calc_tfidf'
What's wrong with this piece of code?
Okay I've figured it out now!. The problem is that the function calc_tfidf was not defined as a global function. I changed the function to be a global function and it worked perfect.
Simple checks when left unsolved at times might lead to days of redundant efforts! :(