I'd like to produce a boxplot similar to that which is found here. Instead of by indexing by the month I'd like to index by the Question, that is, for each of the 12 Questions produced I'd like to have each question shown along with it's particular possible classes (similar to value_vars=["A", "B"] in the linked example). However, instead of just having the two possible classes, "A" and "B" for each of the Questions, I'd like to have each of the classes being "1" through "5" in which each class has the number of occurrences of each of the possible question values e.g. the number of 1's, 2's etc. for each question. To try to be a little more clear, I've included a picture of what I'm trying to achieve:
The issue that I'm having is that I don't know how to create the correct index scheme and how to melt the classes together.
import random
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
def SeverityvsFives(df):
df['question']= df.index.???
df_plot = df.melt(id_vars='question', value_vars=["1","2","3","4","5"])
sns.boxplot(x='question', y='value', hue='variable', data=df_plot)
plt.show()
n = 500
m = 12
columnnames = [str("Question_"+ str(j+1)) for j in range(m)]
a = [[random.randint(1, 5) for j in range(m)] for i in range(n)]
df = pd.DataFrame(data = a, columns = columnnames, index= ???)
SeverityvsFives(df)
Struggling to understand why this bokeh visual will not allow me to change plots and see the predicted data. The plot and select (dropdown-looking) menu appears, but I'm not able to change the plot for items in the menu.
Running Bokeh 1.2.0 via Anaconda. The code has been run both inside & outside of Jupyter. No errors display when the code is run. I've looked through the handful of SO posts relating to this same issue, but I've not been able to apply the same solutions successfully.
I wasn't sure how to create a toy problem out of this, so in addition to the code sample below, the full code (including the regression code and corresponding data) can be found at my github here (code: Regression&Plotting.ipynb, data: pred_data.csv, historical_data.csv, features_created.pkd.)
import pandas as pd
import datetime
from bokeh.io import curdoc, output_notebook, output_file
from bokeh.layouts import row, column
from bokeh.models import Select, DataRange1d, ColumnDataSource
from bokeh.plotting import figure
#Must be run from the command line
def get_historical_data(src_hist, drug_id):
historical_data = src_hist.loc[src_hist['ndc'] == drug_id]
historical_data.drop(['Unnamed: 0', 'date'], inplace = True, axis = 1)#.dropna()
historical_data['date'] = pd.to_datetime(historical_data[['year', 'month', 'day']], infer_datetime_format=True)
historical_data = historical_data.set_index(['date'])
historical_data.sort_index(inplace = True)
# csd_historical = ColumnDataSource(historical_data)
return historical_data
def get_prediction_data(src_test, drug_id):
#Assign the new date
#Write a new dataframe with values for the new dates
df_pred = src_test.loc[src_test['ndc'] == drug_id].copy()
df_pred.loc[:, 'year'] = input_date.year
df_pred.loc[:, 'month'] = input_date.month
df_pred.loc[:, 'day'] = input_date.day
df_pred.drop(['Unnamed: 0', 'date'], inplace = True, axis = 1)
prediction = lin_model.predict(df_pred)
prediction_data = pd.DataFrame({'drug_id': prediction[0][0], 'predictions': prediction[0][1], 'date': pd.to_datetime(df_pred[['year', 'month', 'day']], infer_datetime_format=True, errors = 'coerce')})
prediction_data = prediction_data.set_index(['date'])
prediction_data.sort_index(inplace = True)
# csd_prediction = ColumnDataSource(prediction_data)
return prediction_data
def make_plot(historical_data, prediction_data, title):
#Historical Data
plot = figure(plot_width=800, plot_height = 800, x_axis_type = 'datetime',
toolbar_location = 'below')
plot.xaxis.axis_label = 'Time'
plot.yaxis.axis_label = 'Price ($)'
plot.axis.axis_label_text_font_style = 'bold'
plot.x_range = DataRange1d(range_padding = 0.0)
plot.grid.grid_line_alpha = 0.3
plot.title.text = title
plot.line(x = 'date', y='nadac_per_unit', source = historical_data, line_color = 'blue', ) #plot historical data
plot.line(x = 'date', y='predictions', source = prediction_data, line_color = 'red') #plot prediction data (line from last date/price point to date, price point for input_date above)
return plot
def update_plot(attrname, old, new):
ver = vselect.value
new_hist_source = get_historical_data(src_hist, ver) #calls the function above to get the data instead of handling it here on its own
historical_data.data = ColumnDataSource.from_df(new_hist_source)
# new_pred_source = get_prediction_data(src_pred, ver)
# prediction_data.data = new_pred_source.data
#Import data source
src_hist = pd.read_csv('data/historical_data.csv')
src_pred = pd.read_csv('data/pred_data.csv')
#Prep for default view
#Initialize plot with ID number
ver = 781593600
#Set the prediction date
input_date = datetime.datetime(2020, 3, 31) #Make this selectable in future
#Select-menu options
menu_options = src_pred['ndc'].astype(str) #already contains unique values
#Create select (dropdown) menu
vselect = Select(value=str(ver), title='Drug ID', options=sorted((menu_options)))
#Prep datasets for plotting
historical_data = get_historical_data(src_hist, ver)
prediction_data = get_prediction_data(src_pred, ver)
#Create a new plot with the source data
plot = make_plot(historical_data, prediction_data, "Drug Prices")
#Update the plot every time 'vselect' is changed'
vselect.on_change('value', update_plot)
controls = row(vselect)
curdoc().add_root(row(plot, controls))
UPDATED: ERRORS:
1) No errors show up in Jupyter Notebook.
2) CLI shows a UserWarning: Pandas doesn't allow columns to be careated via a new attribute name, referencing `historical_data.data = ColumnDatasource.from_df(new_hist_source).
Ultimately, the plot should have a line for historical data, and another line or dot for predicted data derived from sklearn. It also has a dropdown menu to select each item to plot (one at a time).
Your update_plot is a no-op that does not actually make any changes to Bokeh model state, which is what is necessary to change a Bokeh plot. Changing Bokeh model state means assigning a new value to a property on a Bokeh object. Typically, to update a plot, you would compute a new data dict and then set an existing CDS from it:
source.data = new_data # plain python dict
Or, if you want to update from a DataFame:
source.data = ColumnDataSource.from_df(new_df)
As an aside, don't assign the .data from one CDS to another:
source.data = other_source.data # BAD
By contrast, your update_plot computes some new data and then throws it away. Note there is never any purpose to returning anything at all from any Bokeh callback. The callbacks are called by Bokeh library code, which does not expect or use any return values.
Lastly, I don't think any of those last JS console errors were generated by BokehJS.
Being rather new to programming with python I tried to language detect segments of text in pandas data frame.
So first I made a function for the 'langdetect' package
import pandas as pd
from langdetect import detect
def language_detect(x):
lang = detect(x)
print(lang)
My second step would be to feed in the data frame for processing. All the segments that need detecting are in separate rows in the dataframe under the same column header.
result = [language_detect(x) for x in df['column_name']]
df['l_detect'] = pd.append(result)
In the output I see the texts being recognized properly.
But when I try to print result.
it returns me with only the value for every entry 'none'
So my questions are:
why do I get 'none' when the the print output from the function has the right values
How can I attach this to my current data frame, since when I try to append it I get 'none' on
every field as well.
Thanks in advance.
The problem is that result is empty because your function language_detect() doesn't return anything (it is only printing the results).
import pandas as pd
from langdetect import detect
lst = [('this is a test', 1), ('what language is this?', 4), ('stackoverflow is a website', 23)]
df = pd.DataFrame(lst, columns = ['text', 'something'])
def language_detect(x):
lang = detect(x)
print(lang)
result = [language_detect(x) for x in df['text']]
result
#Output:[None, None, None]
Just give it a return value:
def language_detect(x):
lang = detect(x)
return lang
df['l_detect'] = df['text'].apply(language_detect)
df.head()
#Output:
# text something l_detect
#0 this is a test 1 en
#1 what language is this? 4 en
#2 stackoverflow is a website 23 en
and it will work as expected.
First time posting in stackoverflow. Hope you guys are doing well.
Recently, I am trying to create a filterable Bokeh graph to graph a US map base on the filter an user select. However, when I try to fit a geometry to ColumnDataSource. It is giving me an error: TypeError: Object of type Polygon is not JSON serializable when I run show(figure)
The code below shows how I want to update geometry to ColumnDataSource and I get an error
# <------- This is where the graph starts------->
# reset the graph
reset_output()
# import data
data = gpd.read_file("/Users/xxxx/Desktop/cb_2015_us_state_500k/cb_2015_us_state_500k.shp", encoding="utf-8")
data1 = data[~data.STUSPS.isin(['AK','AS', 'GU', 'HI', 'PR','MP', 'VI'])]
data2 = data[data.STUSPS.isin(['TX', 'UT'])]
# get a list of unique value
unique_state = sorted(list(data2.NAME.unique()))
select = Select(title="State", options=unique_state)
# get data into ColumnDataSource
source=ColumnDataSource(ColumnDataSource.from_df(data2.loc[:]))
# crate filtered dataframe
filteredSource = ColumnDataSource(data=dict(STUSPS=[],NAME=[],ALAND=[]))
columns = [TableColumn(field="NAME",title="NAME",sortable=True),
TableColumn(field="STUSPS",title="STUSPS",sortable=True),
TableColumn(field="ALAND",title="ALAND",sortable=True),
TableColumn(field="geometry",title="geometry",sortable=True)]
data_table=DataTable(source=filteredSource,columns=columns, width=800 )
# <---- Call back starts ---->
callback = CustomJS(args=dict(source=source,
filteredSource=filteredSource,
data_table=data_table), code="""
var data = source.data;
var f = cb_obj.value;
var d2 = filteredSource.data;
d2['STUSPS']=[]
d2['NAME']=[]
d2['ALAND']=[]
d2['geometry']=[]
for(i = 0; i < data['NAME'].length;i++){
if(data['NAME'][i]==f){
d2['STUSPS'].push(data['STUSPS'][i])
d2['NAME'].push(data['NAME'][i])
d2['ALAND'].push(data['ALAND'][i])
d2['geometry'].push(data['geometry'][i])
}
}
filteredSource.change.emit()
// trigger change on datatable
data_table.change.emit()
""")
select.js_on_change('value',callback)
layout = column(widgetbox(select, data_table))
# output_file("filter.html", title="filter example")
show(layout)
Afterwards, I saw an example directly fitting shape file to dictionary or dataframe which may solve the problem. Here is the link:
Bokeh Mapping Counties
However, when I using the code to graph it. it is giving me
ValueError: Out of range float values are not JSON compliant
This is the code I run:
import shapefile
import itertools
shp = open("/Users/xxxx/Desktop/cb_2015_us_state_500k/cb_2015_us_state_500k.shp", "rb")
dbf = open("/Users/xxxx/Desktop/cb_2015_us_state_500k/cb_2015_us_state_500k.dbf", "rb")
sf = shapefile.Reader(shp=shp, dbf=dbf)
lats = []
lons = []
ct_name = []
st_id = []
ct_state_name = []
for shprec in sf.shapeRecords():
st_id.append(int(shprec.record[0]))
ct_name.append(shprec.record[5])
ct_state_name.append(shprec.record[4])
lat, lon = map(list, zip(*shprec.shape.points))
indices = shprec.shape.parts.tolist()
lat = [lat[i:j] + [float('NaN')] for i, j in zip(indices, indices[1:]+[None])]
lon = [lon[i:j] + [float('NaN')] for i, j in zip(indices, indices[1:]+[None])]
lat = list(itertools.chain.from_iterable(lat))
lon = list(itertools.chain.from_iterable(lon))
lats.append(lat)
lons.append(lon)
map_data = pd.DataFrame({'x': lats, 'y': lons, 'state': st_id, 'county_name': ct_name, 'ct_state_name': ct_state_name})
map_data_m = map_data[map_data.ct_state_name.isin(['NJ'])]
source = ColumnDataSource(map_data_m)
TOOLS="pan,wheel_zoom,box_zoom,reset,hover,save"
p = figure(title="Title", tools=TOOLS,
x_axis_location=None, y_axis_location=None)
p.grid.grid_line_color = None
p.patches('x', 'y', source=source,
fill_color='color', fill_alpha=0.7,
line_color="white", line_width=0.5)
show(p)
anyone who is able to help me to resolve either of the question? I have been stuck for few days. Thanks a lot!
Could you check if there are no infinite numbers in the dataset?
Reference: https://stackoverflow.com/a/47085304/2908623
There is an example here for how to create a multi-colored text title.
However, I want to apply this to a plot that already has a figure in it.
For example, if I apply it to this (same code as with the example minus a few extras and with another figure)...:
plt.rcdefaults()
import matplotlib.pyplot as plt
%matplotlib inline
from matplotlib import transforms
fig = plt.figure(figsize=(4,3), dpi=300)
def rainbow_text(x,y,ls,lc,**kw):
t = plt.gca().transData
fig = plt.gcf()
plt.show()
#horizontal version
for s,c in zip(ls,lc):
text = plt.text(x,y," "+s+" ",color=c, transform=t, **kw)
text.draw(fig.canvas.get_renderer())
ex = text.get_window_extent()
t = transforms.offset_copy(text._transform, x=ex.width, units='dots')
plt.figure()
rainbow_text(0.5,0.5,"all unicorns poop rainbows ! ! !".split(),
['red', 'orange', 'brown', 'green', 'blue', 'purple', 'black'],
size=40)
...the result is 2 plots with the title enlarged.
This sort of makes sense to me because I'm using plt. two times.
But how do I integrate it so that it only refers to the first instance of plt. in creating the title?
Also, about this line:
t = transforms.offset_copy(text._transform, x=ex.width, units='dots')
I notice it can alter the spacing between words, but when I play with the values of x, results are not predictable (spacing is inconsistent between words).
How can I meaningfully adjust that value?
And finally, where it says "units='dots'", what are the other options? Are 'dots' 1/72nd of an inch (and is that the default for Matplotlib?)?
How can I convert units from dots to inches?
Thanks in advance!
In fact the bounding box of the text comes in units unlike the ones used, for example, in scatterplot. Text is a different kind of object that gets somehow redraw if you resize the window or change the ratio. By having a stabilized window you can ask the coordinates of the bounding box in plot units and build your colored text that way:
a = "all unicorns poop rainbows ! ! !".split()
c = ['red', 'orange', 'brown', 'green', 'blue', 'purple', 'black']
f = plt.figure(figsize=(4,3), dpi=120)
ax = f.add_subplot(111)
r = f.canvas.get_renderer()
space = 0.1
w = 0.5
counter = 0
for i in a:
t = ax.text(w, 1.2, a[counter],color=c[counter],fontsize=12,ha='left')
transf = ax.transData.inverted()
bb = t.get_window_extent(renderer=f.canvas.renderer)
bb = bb.transformed(transf)
w = w + bb.xmax-bb.xmin + space
counter = counter + 1
plt.ylim(0.5,2.5)
plt.xlim(0.6,1.6)
plt.show()
, which results in:
This, however, is still not ideal since you need to keep controlling the size of your plot axis to obtain the correct spaces between words. This is somewhat arbitrary but if you manage to do your program with such a control it's feasible to use plot units to achieve your intended purpose.
ORIGINAL POST:
plt. is just the call to the library. In truth you are creating an instance of plt.figure in the global scope (so it can be seen in locally in the function). Due to this you are overwriting the figure because you use the same name for the variable (so it's just one single instance in the end). To solve this try controlling the names of your figure instances. For example:
import matplotlib.pyplot as plt
#%matplotlib inline
from matplotlib import transforms
fig = plt.figure(figsize=(4,3), dpi=300)
#plt.show(fig)
def rainbow_text(x,y,ls,lc,**kw):
t = plt.gca().transData
figlocal = plt.gcf()
#horizontal version
for s,c in zip(ls,lc):
text = plt.text(x,y," "+s+" ",color=c, transform=t, **kw)
text.draw(figlocal.canvas.get_renderer())
ex = text.get_window_extent()
t = transforms.offset_copy(text._transform, x=ex.width, units='dots')
plt.show(figlocal) #plt.show((figlocal,fig))
#plt.figure()
rainbow_text(0.5,0.5,"all unicorns poop rainbows ! ! !".split(),
['red', 'orange', 'brown', 'green', 'blue', 'purple', 'black'],
size=40,)
I've commented several instructions but notice I give a different name for the figure local to the function (figlocal). Also notice that in my examples of show I control directly which figure should be shown.
As for your other questions notice you can use other units as can be seen in the function documentation:
Return a new transform with an added offset.
args:
trans is any transform
kwargs:
fig is the current figure; it can be None if units are 'dots'
x, y give the offset
units is 'inches', 'points' or 'dots'
EDIT: Apparently there's some kind of problem with the extents of the bounding box for text that does not give the correct width of the word and thus the space between words is not stable. My advise is to use the latex functionality of Matplotlib to write the colors in the same string (so only one call of plt.text). You can do it like this:
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('pgf')
from matplotlib import rc
rc('text',usetex=True)
rc('text.latex', preamble=r'\usepackage{color}')
a = "all unicorns poop rainbows ! ! !".split()
c = ['red', 'orange', 'brown', 'green', 'blue', 'purple', 'black']
st = ''
for i in range(len(a)):
st = st + r'\textcolor{'+c[i]+'}{'+a[i]+'}'
plt.text(0.5,0.5,st)
plt.show()
This however is not an ideal solution. The reason is that you need to have Latex installed, including the necessary packages (notice I'm using the color package). Take a look at Yann answer in this question: Partial coloring of text in matplotlib
#armatita: I think your answer actually does what I need. I thought I needed display coordinates instead, but it looks like I can just use axis 1 coordinates, if that's what this is (I'm planning on using multiple axes via subplot2grid). Here's an example:
import matplotlib.pyplot as plt
%matplotlib inline
dpi=300
f_width=4
f_height=3
f = plt.figure(figsize=(f_width,f_height), dpi=dpi)
ax1 = plt.subplot2grid((100,115), (0,0), rowspan=95, colspan=25)
ax2 = plt.subplot2grid((100,115), (0,30), rowspan=95, colspan=20)
ax3 = plt.subplot2grid((100,115), (0,55), rowspan=95, colspan=35)
ax4 = plt.subplot2grid((100,115), (0,95), rowspan=95, colspan=20)
r = f.canvas.get_renderer()
t = ax1.text(.5, 1.1, 'a lot of text here',fontsize=12,ha='left')
space=0.1
w=.5
transf = ax1.transData.inverted()
bb = t.get_window_extent(renderer=f.canvas.renderer)
bb = bb.transformed(transf)
e = ax1.text(.5+bb.width+space, 1.1, 'text',fontsize=12,ha='left')
print(bb)
plt.show()
I'm not sure what you mean about controlling the axis size, though. Are you referring to using the code in different environments or exporting the image in different sizes? I plan on having the image used in the same environment and in the same size (per instance of using this approach), so I think it will be okay. Does my logic make sense? I have a weak grasp on what's really going on, so I hope so. I would use it with a function (via splitting the text) like you did, but there are cases where I need to split on other characters (i.e. when a word in parentheses should be colored, but not the parentheses). Maybe I can just put a delimiter in there like ','? I think I need a different form of .split() because it didn't work when I tried it.
At any rate, if I can implement this across all of my charts, it will save me countless hours. Thank you so much!
Here is an example where there are 2 plots and 2 instances of using the function for posterity:
import matplotlib.pyplot as plt
%matplotlib inline
dpi=300
f_width=4
f_height=3
f = plt.figure(figsize=(f_width,f_height), dpi=dpi)
ax1 = plt.subplot2grid((100,60), (0,0), rowspan=95, colspan=30)
ax2 = plt.subplot2grid((100,60), (0,30), rowspan=95, colspan=30)
f=f #Name for figure
string = str("Group 1 ,vs. ,Group 2 (,sub1,) and (,sub2,)").split(',')
color = ['black','red','black','green','black','blue','black']
xpos = .5
ypos = 1.2
axis=ax1
#No need to include space if incuded between delimiters above
#space = 0.1
def colortext(f,string,color,xpos,ypos,axis):
#f=figure object name (i.e. fig, f, figure)
r = f.canvas.get_renderer()
counter = 0
for i in string:
t = axis.text(xpos, ypos, string[counter],color=color[counter],fontsize=12,ha='left')
transf = axis.transData.inverted()
bb = t.get_window_extent(renderer=f.canvas.renderer)
bb = bb.transformed(transf)
xpos = xpos + bb.xmax-bb.xmin
counter = counter + 1
colortext(f,string,color,xpos,ypos,axis)
string2 = str("Group 1 part 2 ,vs. ,Group 2 (,sub1,) and (,sub2,)").split(',')
ypos2=1.1
colortext(f,string2,color,xpos,ypos2,axis)
plt.show()