Pandas Matplotlib Line Graph - python-3.x

Given the following data frame:
import pandas as pd
import numpy as np
df = pd.DataFrame(
{'YYYYMM':[201603,201503,201403,201303,201603,201503,201403,201303],
'Count':[5,6,2,7,4,7,8,9],
'Group':['A','A','A','A','B','B','B','B']})
df
Count Group YYYYMM
0 5 A 201603
1 6 A 201503
2 2 A 201403
3 7 A 201303
4 4 B 201603
5 7 B 201503
6 8 B 201403
7 9 B 201303
I need to generate a line graph with one line per group with a summary table at the bottom. Something like this:
I need each instance of 'YYYYMM' to be treated like a year by Pandas/Matplotlib.
So far, this seems to help, but I'm not sure if it will do the trick:
df['YYYYMM']=df['YYYYMM'].astype(str).str[:-2].astype(np.int64)
Then, I did this to pivot the data:
t=df.pivot_table(df,index=['YYYYMM'],columns=['Group'],aggfunc=np.sum)
Count
Group A B
YYYYMM
2013 7 9
2014 2 8
2015 6 7
2016 5 4
Then, I tried to plot it:
import matplotlib.pyplot as plt
%matplotlib inline
fig, ax = plt.subplots(1,1)
t.plot(table=t,ax=ax)
...and this happened:
I'd like to do the following:
remove all lines (borders) from the table at the bottom
remove the jumbled text in the table
remove the x axis tick labels (it should just show the years for tick labels)
I can clean up the rest myself (remove legend and borders, etc..).
Thanks in advance!

I may not have fully understood what you mean by 1., since you are showing the table lines in your reference. I have also not understood whether you want to transpose the table.
What you may be looking for is:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
df = pd.DataFrame(
{'YYYYMM':[201603,201503,201403,201303,201603,201503,201403,201303],
'Count':[5,6,2,7,4,7,8,9],
'Group':['A','A','A','A','B','B','B','B']})
df['YYYYMM']=df['YYYYMM'].astype(str).str[:-2].astype(int)
t=pd.pivot_table(df, values='Count', index='YYYYMM',columns='Group',aggfunc=np.sum)
t.index.name = None
fig, ax = plt.subplots(1,1)
t.plot(table=t,ax=ax)
ax.xaxis.set_major_formatter(plt.NullFormatter())
plt.tick_params(
axis='x', # changes apply to the x-axis
which='both', # both major and minor ticks are affected
bottom='off', # ticks along the bottom edge are off
top='off', # ticks along the top edge are off
labelbottom='off') # labels along the bottom edge are off
plt.show()

Related

Convert Dataframe to display in matplotlib line chart with dates

I am facing an issue while plotting graph in matplotlib as I am unable to convert data exactly to give inputs to matplotlib
Here is my data
date,GOOG,AAPL,FB,BABA,AMZN,GE,AMD,WMT,BAC,GM,T,UAA,SHLD,XOM,RRC,BBY,MA,PFE,JPM,SBUX
1989-12-29,,0.117203,,,,0.352438,3.9375,3.48607,1.752478,,2.365775,,,1.766756,,0.166287,,0.110818,1.827968,
1990-01-02,,0.123853,,,,0.364733,4.125,3.660858,1.766686,,2.398184,,,1.766756,,0.173216,,0.113209,1.835617,
1990-01-03,,0.124684,,,,0.36405,4.0,3.660858,1.780897,,2.356516,,,1.749088,,0.194001,,0.113608,1.896803,
1990-01-04,,0.1251,,,,0.362001,3.9375,3.641439,1.743005,,2.403821,,,1.731422,,0.190537,,0.115402,1.904452,
1990-01-05,,0.125516,,,,0.358586,3.8125,3.602595,1.705114,,2.287973,,,1.722587,,0.190537,,0.114405,1.9121,
1990-01-08,,0.126347,,,,0.360635,3.8125,3.651146,1.714586,,2.326588,,,1.749088,,0.17668,,0.113409,1.9121,
1990-01-09,,0.1251,,,,0.353122,3.875,3.55404,1.714586,,2.273493,,,1.713754,,0.17668,,0.111017,1.850914,
1990-01-10,,0.119697,,,,0.353805,3.8125,3.55404,1.681432,,2.210742,,,1.722587,,0.173216,,0.11301,1.843264,
1990-01-11,,0.11471,,,,0.353805,3.875,3.592883,1.667222,,2.23005,,,1.731422,,0.169751,,0.111814,1.82032,
I have converted it as following dataframe
AAPL
2016 0.333945
2017 0.330923
2018 0.321857
2019 0.312790
<class 'pandas.core.frame.DataFrame'>
by using following code:
import pandas as pd
df = pd.read_csv("portfolio.txt")
companyname = "AAPL"
frames = df.loc[:, df.columns.str.startswith(companyname)]
l1 = frames.loc['2015-6-1':'2019-6-10']
print(l1)
print(type(l1))
plt.plot(li1, label="Company Past Information")
plt.xlabel('Risk Aversion')
plt.ylabel('Optimal Investment Portfolio')
plt.title('Optimal Investment Portfolio For Low, Medium & High')
plt.legend()
plt.show()
After plotting to matplotlib I getting output correctly for which data is existed.
But for which data is not available graph is plotting wrongly.
GOOG
2016 NaN
2017 NaN
2018 NaN
2019 NaN
Due to this I am unable to plot graph correctly
Please help out of this
Thanks in advance
If you're reading you data in from a .csv using pandas you can:
import pandas as pd
df = pd.csv_read(your_csv, parse_dates=[0]) # 0 means your dates are in the first column
Otherwise you can convert your data column to datatime using:
import pandas as pd
df['date'] = pd.to_datetime(df['date'])
When using matplotlib then you can:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot(df.iloc[:, 0], df.loc[:, some_column])
plt.show()

How to create a scatter plot where values are across multiple columns?

I have a dataframe in Pandas in which the rows are observations at different times and each column is a size bin where the values represent the number of particles observed for that size bin. So it looks like the following:
bin1 bin2 bin3 bin4 bin5
Time1 50 200 30 40 5
Time2 60 60 40 420 700
Time3 34 200 30 67 43
I would like to use plotly/cufflinks to create a scatterplot in which the x axis will be each size bin, and the y axis will be the values in each size bin. There will be three colors, one for each observation.
As I'm more experienced in Matlab, I tried indexing the values using iloc (note the example below is just trying to plot one observation):
df.iplot(kind="scatter",theme="white",x=df.columns, y=df.iloc[1,:])
But I just get a key error: 0 message.
Is it possible to use indexing when choosing x and y values in Pandas?
Rather than indexing, I think you need to better understand how pandas and matplotlib interact each other.
Let's go by steps for your case:
As the pandas.DataFrame.plot documentation says, the plotted series is a column. You have the series in the row, so you need to transpose your dataframe.
To create a scatterplot, you need both x and y coordinates in different columns, but you are missing the x column, so you also need to create a column with the x values in the transposed dataframe.
Apparently pandas does not change color by default with consecutive calls to plot (matplotlib does it), so you need to pick a color map and pass a color argument, otherwise all points will have the same color.
Here a working example:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
#Here I copied you data in a data.txt text file and import it in pandas as a csv.
#You may have a different way to get your data.
df = pd.read_csv('data.txt', sep='\s+', engine='python')
#I assume to have a column named 'time' which is set as the index, as you show in your post.
df.set_index('time')
tdf = df.transpose() #transpose the dataframe
#Drop the time column from the trasponsed dataframe. time is not a data to be plotted.
tdf = tdf.drop('time')
#Creating x values, I go for 1 to 5 but they can be different.
tdf['xval'] = np.arange(1, len(tdf)+1)
#Choose a colormap and making a list of colors to be used.
colormap = plt.cm.rainbow
colors = [colormap(i) for i in np.linspace(0, 1, len(tdf))]
#Make an empty plot, the columns will be added to the axes in the loop.
fig, axes = plt.subplots(1, 1)
for i, cl in enumerate([datacol for datacol in tdf.columns if datacol != 'xval']):
tdf.plot(x='xval', y=cl, kind="scatter", ax=axes, color=colors[i])
plt.show()
This plots the following image:
Here a tutorial on picking colors in matplotlib.

Difficulty grouping barchart using Python, Pandas and Matplotlib

I am having difficulty getting plot.bar to group the bars together the way I have them grouped in the dataframe. The dataframe returns the grouped data correctly, however, the bar graph is providing a separate bar for every line int he dataframe. Ideally, everything in my code below should group 3-6 bars together for each department (Dept X should have bars grouped together for each type, then count of true/false as the Y axis).
Dataframe:
dname Type purchased
Dept X 0 False 141
True 270
1 False 2020
True 2604
2 False 2023
True 1047
Code:
import psycopg2
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
##connection and query data removed
df = pd.merge(df_departments[["id", "dname"]], df_widgets[["department", "widgetid", "purchased","Type"]], how='inner', left_on='id', right_on='department')
df.set_index(['dname'], inplace=True)
dx=df.groupby(['dname', 'Type','purchased'])['widgetid'].size()
dx.plot.bar(x='dname', y='widgetid', rot=90)
I can't be sure without a more reproducible example, but try unstacking the innermost level of the MultiIndex of dx before plotting:
dx.unstack().plot.bar(x='dname', y='widgetid', rot=90)
I expect this to work because when plotting a DataFrame, each column becomes a legend entry and each row becomes a category on the horizontal axis.

labeling data points with dataframe including empty cells

I have an Excel sheet like this:
A B C D
3 1 2 8
4 2 2 8
5 3 2 9
2 9
6 4 2 7
Now I am trying to plot 'B' over 'C' and label the data points with the entrys of 'A'. It should show me the points 1/2, 2/2, 3/2 and 4/2 with the corresponding labels.
import matplotlib.pyplot as plt
import pandas as pd
import os
df = pd.read_excel(os.path.join(os.path.dirname(__file__), "./Datenbank/Test.xlsx"))
fig, ax = plt.subplots()
df.plot('B', 'C', kind='scatter', ax=ax)
df[['B','C','A']].apply(lambda x: ax.text(*x),axis=1);
plt.show()
Unfortunately I am getting this:
with the Error:
ValueError: posx and posy should be finite values
As you can see it did not label the last data point. I know it is because of the empty cells in the sheet but i cannot avoid them. There is just no measurement data at this positions.
I already searched for a solution here:
Annotate data points while plotting from Pandas DataFrame
but it did not solve my problem.
So, is there a way to still label the last data point?
P.S.: The excel sheet is just an example. So keep in mind in reality there are many empty cells at different positions.
You can simply trash the invalid data rows from df before plotting them
df = df[df['B'].notnull()]

Is it possible to explicitly set order the stacks in a matplotlib stackplot?

I want to explicitly set the order of the stacks in a Matplotlib stackplot. Here is an example:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
np.random.seed(1)
df = pd.DataFrame(np.random.randint(0,100,size=(100,4)),columns=list('ABCD'))
df.plot(kind='area',stacked=True,figsize=(20,10));
This produces the following image:
The last row of the dataframe from:
df.tail(1)
is:
A B C D
99 16 30 84 57
Here is what I want to achieve:
I want to re-order the plot of the stacks such that the stacks are plotted from the bottom up A, B, D, C i.e. the columns ordered from the bottom up, by the order of their increasing values in the last row of the df.
So far, I have tried re-ordering explicitly the columns in the df before plotting:
df[['A','B','D','C']].plot(kind='area',stacked=True,figsize=(20,10))
but this produces exactly the same chart as above.
Thank you for any help here!
The graphs are not the same. Look at the areas beneath the red graph for a particle x. The shapes for those graphs are different for the green and blue shaded areas.
And now,
df[['A','B','D','C']].plot(kind='area',stacked=True,figsize=(20,10))

Resources