python-plotly multiple lines in same graph with same Y axis - python-3.x
I have a csv file that looks like this:
time,price,m1,m2,m3,m4,m5,m6,m7,m8,buy/sell
10.30.01,102,105,100.5,103.5,110,100.9,103.02,111,105.0204,
10.30.02,103,104.5,101,104,110.2,101.4,104.03,110.5,104.5204,
10.30.03,104,104,101.5,104.5,110.4,101.9,105.04,110,104.0204,
10.30.04,105,103.5,102,105,110.6,102.4,106.05,109.5,103.5204,
10.30.05,106,103,102.5,105.5,110.8,102.9,107.06,109,103.0204,
10.30.06,107,102.5,103,106,111,103.4,108.07,108.5,102.5204,
10.30.07,108,102,103.5,106.5,111.2,103.9,109.08,108,102.0204,
10.30.08,109,101.5,104,107,111.4,104.4,110.09,107.5,101.5204,BUY
10.30.09,110,101,104.5,107.5,111.6,104.9,111.1,107,101.0204,
10.30.10,111,100.5,105,108,111.8,105.4,112.11,106.5,100.5204,
10.30.11,112,101,105.5,108.5,112,105.9,113.12,106,101.0204,
10.30.12,113,101.5,106,109,112.2,106.4,114.13,105.5,101.5204,SELL
10.30.13,114,102,106.5,109.5,112.4,106.9,115.14,105,102.0204,
10.30.14,115,102.5,107,110,112.6,107.4,116.15,104.5,102.5204,
10.30.15,116,103,107.5,110.5,112.8,107.9,117.16,104,103.0204,BUY
10.30.16,117,103.5,108,111,113,108.4,118.17,103.5,103.5204,
I want to take time in x-axis and price,m1,m2,m3,m4,m5,m6,m7,m8 in Y axis, since its the same range all are in same y-axis as line graphs. and buy/sell column in the same graph as scatter plot. How to do this with plotly ?
sorry for the simple question (if it is one), I tried a lot couldn't crack it. thank you in advance
A great resource for Scatter plot related questions is Plotly's documentation on scatter plots.
Plotting all of the columns price,m1,m2,m3,m4,m5,m6,m7,m8 can be done by looping through a list, and adding each of these columns as a trace.
Then I would recommend that you draw vertical lines in the Scatter plot for each time with BUY or SELL, by iterating through the non-null entries in the buy/sell column and using a shape to create a vertical line. You can also add an arrow and text pointing to each line using an annotation.
import pandas as pd
import plotly.graph_objects as go
from plotly.subplots import make_subplots
df = pd.read_csv("buysell.csv")
fig = go.Figure()
cols = ['price','m1','m2','m3','m4','m5','m6','m7','m8']
for col in cols:
fig.add_trace(go.Scatter(
x=df['time'],
y=df[col],
name=col
))
# iterate over any rows with 'BUY' or 'SELL'
for index, row in df.dropna(subset=['buy/sell']).iterrows():
fig.add_shape(
type='line',
x0=row['time'],
y0=0,
x1=row['time'],
y1=1,
yref='paper',
line=dict(
color="red",
width=1,
dash="dot",
)
)
df_max, df_min = df[cols].max().max(), df[cols].min().min()
fig.add_annotation(
x=row['time'],
y=df_max,
text=row['buy/sell'],
showarrow=True,
arrowhead=4,
)
fig.show()
Related
Seaborn / Matplotlib: Subplots depending on one column
I have a Dataframe and based on its data, I draw lineplots for it. The code currently looks as simple as that: ax = sns.lineplot(x='datapoints', y='mean', hue='index', data=df) sns.move_legend(ax, "upper left", bbox_to_anchor=(1, 1)) Now, there actually is a column, called "klinger", which has 8 different values and I would like to get a plot consisting of eight subplots (4x2) for it, all sharing just one legend. Is that an easy thing to do? Currently, I generate sub-dfs by filtering and just draw eight diagrams and cut them together with a graphic tool, but this can't be the solution
You can get what you are looking for with sns.relplot and kind='line'.
Use col='klinger' to plot subplots as many as you need, col_wrap=4 will help to obtain 4x2 shape, and col_order=klinger_categories will select which categories you want to plot.
import numpy as np
import pandas as pd
import seaborn as sns
number = 100
klinger_categories = ['a','b','c','d','e','f','g','h']
data = {'datapoints': np.arange(number),
'mean': np.random.normal(0,1,size=number),
'index': np.random.choice(np.arange(2),size=number),
'klinger': np.random.choice(klinger_categories,size=number),
}
df = pd.DataFrame(data)
sns.relplot(
data=df, x='datapoints', y='mean', hue='index', kind='line',
col='klinger', col_wrap=4, col_order=klinger_categories
)
How to plot a histogram with plot.hist for continous data in a dataframe in pandas?
In this data set I need to plot,pH as the x-column which is having continuous data and need to group it together the pH axis as per the quality value and plot the histogram. In many of the resources I referred I found solutions for using random data generated. I tried this piece of code.
plt.hist(, density=True, bins=1)
plt.ylabel('quality')
plt.xlabel('pH');
Where I eliminated the random generated data, but I received and error
File "<ipython-input-16-9afc718b5558>", line 1
plt.hist(, density=True, bins=1)
^
SyntaxError: invalid syntax
What is the proper way to plot my data?I want to feed into the histogram not randomly generated data, but data found in the data set.
Your Error
The immediate problem in your code is the missing data to the plt.hist() command.
plt.hist(, density=True, bins=1)
should be something like:
plt.hist(data_table['pH'], density=True, bins=1)
Seaborn histplot
But this doesn't get the plot broken down by quality. The answer by Mr.T looks correct, but I'd also suggest seaborn which works with "melted" data like you have. The histplot command should give you what you want:
import seaborn as sns
sns.histplot(data=df, x="pH", hue="quality", palette="Dark2", element='step')
Assuming the table you posted is in a pandas.DataFrame named df with columns "pH" and "quality", you get something like:
The palette (Dark2) can can be any matplotlib colormap.
Subplots
If the overlaid histograms are too hard to see, an option is to do facets or small multiples. To do this with pandas and matplotlib:
# group dataframe by quality values
data_by_qual = df.groupby('quality')
# create a sub plot for each quality group
fig, axes = plt.subplots(nrows=len(data_by_qual),
figsize=[6,12],
sharex=True)
fig.subplots_adjust(hspace=.5)
# loop over axes and quality groups together
for ax, (quality, qual_data) in zip(axes, data_by_qual):
ax.hist(qual_data['pH'], bins=10)
ax.set_title(f"quality = {quality}")
ax.set_xlabel('pH')
Altair Facets
The plotting library altair can do this for you:
import altair as alt
alt.Chart(df).mark_bar().encode(
alt.X("pH:Q", bin=True),
y='count()',
).facet(row='quality')
Several possibilities here to represent multiple histograms. All have in common that the data have to be transformed from long to wide format - meaning, each category is in its own column:
import matplotlib.pyplot as plt
import pandas as pd
#test data generation
import numpy as np
np.random.seed(123)
n=300
df = pd.DataFrame({"A": np.random.randint(1, 100, n), "pH": 3*np.random.rand(n), "quality": np.random.choice([3, 4, 5, 6], n)})
df.pH += df.quality
#instead of this block you have to read here your stored data, e.g.,
#df = pd.read_csv("my_data_file.csv")
#check that it read the correct data
#print(df.dtypes)
#print(df.head(10))
#bringing the columns in the required wide format
plot_df = df.pivot(columns="quality")["pH"]
bin_nr=5
#creating three subplots for different ways to present the same histograms
fig, (ax1, ax2, ax3) = plt.subplots(3, 1, figsize=(6, 12))
ax1.hist(plot_df, bins=bin_nr, density=True, histtype="bar", label=plot_df.columns)
ax1.legend()
ax1.set_title("Basically bar graphs")
plot_df.plot.hist(stacked=True, bins=bin_nr, density=True, ax=ax2)
ax2.set_title("Stacked histograms")
plot_df.plot.hist(alpha=0.5, bins=bin_nr, density=True, ax=ax3)
ax3.set_title("Overlay histograms")
plt.show()
Sample output:
It is not clear, though, what you intended to do with just one bin and why your y-axis was labeled "quality" when this axis represents the frequency in a histogram.
Gantt Chart for USGS Hydrology Data with Python?
I have a compiled a dataframe that contains USGS streamflow data at several different streamgages. Now I want to create a Gantt chart similar to this. Currently, my data has columns as site names and a date index as rows.
Here is a sample of my data.
The problem with the Gantt chart example I linked is that my data has gaps between the start and end dates that would normally define the horizontal time-lines. Many of the examples I found only account for the start and end date, but not missing values that may be in between. How do I account for the gaps where there is no data (blanks or nan in those slots for values) for some of the sites?
First, I have a plot that shows where the missing data is.
import missingno as msno
msno.bar(dfp)
Now, I want time on the x-axis and a horizontal line on the y-axis that tracks when the sites contain data at those times. I know how to do this the brute force way, which would mean manually picking out the start and end dates where there is valid data (which I made up below).
from datetime import datetime
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.dates as dt
df=[('RIO GRANDE AT EMBUDO, NM','2015-7-22','2015-12-7'),
('RIO GRANDE AT EMBUDO, NM','2016-1-22','2016-8-5'),
('RIO GRANDE DEL RANCHO NEAR TALPA, NM','2014-12-10','2015-12-14'),
('RIO GRANDE DEL RANCHO NEAR TALPA, NM','2017-1-10','2017-11-25'),
('RIO GRANDE AT OTOWI BRIDGE, NM','2015-8-17','2017-8-21'),
('RIO GRANDE BLW TAOS JUNCTION BRIDGE NEAR TAOS, NM','2015-9-1','2016-6-1'),
('RIO GRANDE NEAR CERRO, NM','2016-1-2','2016-3-15'),
]
df=pd.DataFrame(data=df)
df.columns = ['A', 'Beg', 'End']
df['Beg'] = pd.to_datetime(df['Beg'])
df['End'] = pd.to_datetime(df['End'])
fig = plt.figure(figsize=(10,8))
ax = fig.add_subplot(111)
ax = ax.xaxis_date()
ax = plt.hlines(df['A'], dt.date2num(df['Beg']), dt.date2num(df['End']))
How do I make a figure (like the one shown above) with the dataframe I provided as an example? Ideally I want to avoid the brute force method.
Please note: values of zero are considered valid data points.
Thank you in advance for your feedback!
Find date ranges of non-null data
2020-02-12 Edit to clarify logic in loop
df = pd.read_excel('Downloads/output.xlsx', index_col='date')
Make sure the dates are in order:
df.sort_index(inplace=True)
Loop thru the data and find the edges of the good data ranges. Get the corresponding index values and the name of the gauge and collect them all in a list:
# Looping feels like defeat. However, I'm not clever enough to avoid it
good_ranges = []
for i in df:
col = df[i]
gauge_name = col.name
# Start of good data block defined by a number preceeded by a NaN
start_mark = (col.notnull() & col.shift().isnull())
start = col[start_mark].index
# End of good data block defined by a number followed by a Nan
end_mark = (col.notnull() & col.shift(-1).isnull())
end = col[end_mark].index
for s, e in zip(start, end):
good_ranges.append((gauge_name, s, e))
good_ranges = pd.DataFrame(good_ranges, columns=['gauge', 'start', 'end'])
Plotting
Nothing new here. Copied pretty much straight from your question:
fig = plt.figure(figsize=(10,8))
ax = fig.add_subplot(111)
ax = ax.xaxis_date()
ax = plt.hlines(good_ranges['gauge'],
dt.date2num(good_ranges['start']),
dt.date2num(good_ranges['end']))
fig.tight_layout()
Here's an approach that you could use, it's a bit hacky so perhaps some else will produce a better solution but it should produce your desired output. First use pd.where to replace non NaN values with an integer which will later determine the position of the lines on y-axis later, I do this row by row so that all data which belongs together will be at the same height. If you want to increase the spacing between the lines of the gantt chart you can add a number to i, I've provided an example in the comments in the code block below.
The y-labels and their positions are produced in the data munging steps, so this method will work regardless of the number of columns and will position the labels correctly when you change the spacing described above.
This approach returns matplotlib.pyplot.axes and matplotlib.pyplot.Figure object, so you can adjust the asthetics of the chart to suit your purposes (i.e. change the thickness of the lines, colours etc.). Link to docs.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
df = pd.read_excel('output.xlsx')
dates = pd.to_datetime(df.date)
df.index = dates
df = df.drop('date', axis=1)
new_rows = [df[s].where(df[s].isna(), i) for i, s in enumerate(df, 1)]
# To increase spacing between lines add a number to i, eg. below:
# [df[s].where(df[s].isna(), i+3) for i, s in enumerate(df, 1)]
new_df = pd.DataFrame(new_rows)
### Plotting ###
fig, ax = plt.subplots() # Create axes object to pass to pandas df.plot()
ax = new_df.transpose().plot(figsize=(40,10), ax=ax, legend=False, fontsize=20)
list_of_sites = new_df.transpose().columns.to_list() # For y tick labels
x_tick_location = new_df.iloc[:, 0].values # For y tick positions
ax.set_yticks(x_tick_location) # Place ticks in correct positions
ax.set_yticklabels(list_of_sites) # Update labels to site names
MatPlotLib Plot last few items differently
I'm exploring MatPlotLib and would like to know if it is possible to show last few items in a dataset differently.
Example: If my dataset contains 100 numbers, I want to display last 5 items in different color.
So far I could do it with one last record using annotate, but want to show last few items dotted with 'red' color as against the blue line.
I could finally achieve this by changing few things in my code.
Below is what I have done.
Let me know in case there is a better way. :)
series_df = pd.read_csv('my_data.csv')
series_df = series_df.fillna(0)
series_df = series_df.sort_values(['Date'], ascending=True)
# Created a new DataFrame for last 5 items series_df2
plt.plot(series_df["Date"],series_df["Values"],color="red", marker='+')
plt.plot(series_df2["Date"],series_df2["Values"],color="blue", marker='+')
You should add some minimal code example or a figure with the desired output to make your question clear. It seems you want to highlight some of the last few points with a marker. You can achieve this by calling plot() twice: import numpy as np import matplotlib.pyplot as plt N = 50 x = np.arange(N) y = np.random.rand(N) plt.figure() plt.plot(x, y) plt.plot(x[-5:], y[-5:], ls='', c='tab:red', marker='.', ms=10)
Using "hue" for a Seaborn visual: how to get legend in one graph?
I created a scatter plot in seaborn using seaborn.relplot, but am having trouble putting the legend all in one graph. When I do this simple way, everything works fine: import pandas as pd import numpy as np from scipy import stats import matplotlib.pyplot as plt import seaborn as sns df2 = df[df.ln_amt_000s < 700] sns.relplot(x='ln_amt_000s', y='hud_med_fm_inc', hue='outcome', size='outcome', legend='brief', ax=ax, data=df2) The result is a scatter plot as desired, with the legend on the right hand side. However, when I try to generate a matplotlib figure and axes objects ahead of time to specify the figure dimensions I run into problems: a4_dims = (10, 10) # generating a matplotlib figure and axes objects ahead of time to specify figure dimensions df2 = df[df.ln_amt_000s < 700] fig, ax = plt.subplots(figsize = a4_dims) sns.relplot(x='ln_amt_000s', y='hud_med_fm_inc', hue='outcome', size='outcome', legend='brief', ax=ax, data=df2) The result is two graphs -- one that has the scatter plots as expected but missing the legend, and another one below it that is all blank except for the legend on the right hand side. How do I fix this such? My desired result is one graph where I can specify the figure dimensions and have the legend at the bottom in two rows, below the x-axis (if that is too difficult, or not supported, then the default legend position to the right on the same graph would work too)? I know the problem lies with "ax=ax", and in the way I am specifying the dimensions as matplotlib figure, but I'd like to know specifically why this causes a problem so I can learn from this. Thank you for your time.
The issue is that sns.relplot is a "Figure-level interface for drawing relational plots onto a FacetGrid" (see the API page). With a simple sns.scatterplot (the default type of plot used by sns.relplot), your code works (changed to use reproducible data):
df = pd.read_csv("https://vincentarelbundock.github.io/Rdatasets/csv/datasets/iris.csv", index_col=0)
fig, ax = plt.subplots(figsize = (5,5))
sns.scatterplot(x = 'Sepal.Length', y = 'Sepal.Width',
hue = 'Species', legend = 'brief',
ax=ax, data = df)
plt.show()
Further edits to legend
Seaborn's legends are a bit finicky. Some tweaks you may want to employ:
Remove the default seaborn title, which is actually a legend entry, by getting and slicing the handles and labels
Set a new title that is actually a title
Move the location and make use of bbox_to_anchor to move outside the plot area (note that the bbox parameters need some tweaking depending on your plot size)
Specify the number of columns
fig, ax = plt.subplots(figsize = (5,5))
sns.scatterplot(x = 'Sepal.Length', y = 'Sepal.Width',
hue = 'Species', legend = 'brief',
ax=ax, data = df)
handles, labels = ax.get_legend_handles_labels()
ax.legend(handles=handles[1:], labels=labels[1:], loc=8,
ncol=2, bbox_to_anchor=[0.5,-.3,0,0])
plt.show()