I am attempting to run a for loop in order to plot multiple scatter plots. For the code that I have, I only get one plot at the end. How to go about generating the correct row x column plots to save?
I have checked out some of the answers given here and here, but it does not work for me. Is there a more optimum way to generate these plots?
Here is my code:
from sklearn.datasets import make_classification
import seaborn as sns
import numpy as np
from matplotlib import pyplot as plt
import pandas as pd
# Generate noisy Data
num_trainsamples = 500
num_testsamples = 50
X_train,y_train = make_classification(n_samples=num_trainsamples,
n_features=240,
n_informative=9,
n_redundant=0,
n_repeated=0,
n_classes=10,
n_clusters_per_class=1,
class_sep=9,
flip_y=0.2,
#weights=[0.5,0.5],
random_state=17)
n_components=2
n_neighbours=[1, 5, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30]
local_connectivity=2
min_dist=0.15
target_names = ['t1', 't2', 't3', 't4', 't5', 't6', 't7', 't8', 't9', 't10']
plt.figure(figsize=(15,15))
for i in range(0, len(n_neighbours)):
plt.subplot(3,5,i+1)
plt.clf()
plt.scatter(
X_train[:, 0],
X_train[:, 1],
s = 20,
c=y_train,
cmap=plt.cm.nipy_spectral,
edgecolor="k",
linewidths=0.75,
label=y_train,
alpha=0.45,
)
plt.title(f'n_components = {n_components}, n_neighbors = {n_neighbours[i]}, local_conn = {local_connectivity}, min_dist = {min_dist}')
cbar = plt.colorbar(boundaries=np.arange(11)-0.5)
cbar.set_ticks(np.arange(10))
cbar.set_ticklabels(target_names)
The reason for you seeing just one plot is the line plt.clf(). This command tells matplotlib to clear current figure. So, each time you loop through the code, it clear the previous figure and so, you see just the last one. Commenting that line will give you the below figure, which is what I think you are looking for...
PLOT
Related
I'm using a csv file to divide into 2 clusters. The blue one is the benign URL and the red one is the malicious URL. The problem is my instructor want to know the closest point of two clusters and the distance between them. I tried to use the distance between two centroids but it didn't work!
Thank you for your help!
Here the code and and the clusters:
from sklearn.cluster import KMeans
from sklearn.metrics.pairwise import euclidean_distances
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = pd.read_csv("major_combined.csv")
X_mal = data.iloc[:,[1, 5, 6, 11, 13, 15, 19, 20, 23]]
kmeans_mal = KMeans(n_clusters=2, init='random', max_iter=300, n_init=10, random_state = 0)
y_kmeans_mal = kmeans_mal.fit_predict(X_mal)
plt.figure(figsize=(10,10))
plt.scatter(X_mal.iloc[y_kmeans_mal == 0,4], X_mal.iloc[y_kmeans_mal == 0,8], s= 100, color = 'blue', label ='0')
plt.scatter(X_mal.iloc[y_kmeans_mal == 1,4], X_mal.iloc[y_kmeans_mal == 1,8], s= 100, color = 'red', label ='1')
plt.scatter(kmeans_mal.cluster_centers_[:,4], kmeans_mal.cluster_centers_[:,8], s= 300, color = 'yellow', label ='Centroid')
plt.title('k-means clustering')
plt.xlabel('document')
plt.ylabel('window')
plt.legend()
plt.grid()
plt.show()
in my code, if I call pred and test object I get these results.
Pred = array([16.88414476, 33.73226078, 75.357018 , 26.79480124, 60.49103328])
test = array([20, 27, 69, 30, 62], dtype=int64)
I apply:
plt.scatter(pred,test)
How I plot both pred and test results on the graph?
so please help! , how to find the desired output.
Scatter plot would take the given two values as x and y values of the plot.
If you want to plot both of them as separate data, use plt.plot
import matplotlib.pyplot as plt
import numpy as np
Pred = np.array([16.88414476, 33.73226078, 75.357018 , 26.79480124, 60.49103328])
test = np.array([20, 27, 69, 30, 62])
plt.plot(Pred)
plt.plot(test, linestyle='--')
Use could also use the pandas plot functionality
pd.DataFrame({'pred': Pred, 'test': test}).plot()
I think it is not possible to plot 2 arrays in scatter but you can do it in a plot
from matplotlib import pyplot as plt
import numpy as np
Pred = np.array([16.88414476, 33.73226078, 75.357018 , 26.79480124, 60.49103328])
test = np.array([20, 27, 69, 30, 62])
plt.plot(Pred, label='Pred Label')
plt.plot(test, label='Test Label')
plt.legend() # To Show the the labels' names
I have a dataset like
x = 3,4,6,77,3
y = 8,5,2,5,5
labels = "null","exit","power","smile","null"
Then I use
from matplotlib import pyplot as plt
plt.scatter(x,y)
colorbar = plt.colorbar(labels)
plt.show()
to make a scatter plot, but cannot make colorbar showing labels as its colors.
How to get this?
I'm not sure, if it's a good idea to do that for scatter plots in general (you have the same description for different data points, maybe just use some legend here?), but I guess a specific solution to what you have in mind, might be the following:
from matplotlib import pyplot as plt
# Data
x = [3, 4, 6, 77, 3]
y = [8, 5, 2, 5, 5]
labels = ('null', 'exit', 'power', 'smile', 'null')
# Customize colormap and scatter plot
cm = plt.cm.get_cmap('hsv')
sc = plt.scatter(x, y, c=range(5), cmap=cm)
cbar = plt.colorbar(sc, ticks=range(5))
cbar.ax.set_yticklabels(labels)
plt.show()
This will result in such an output:
The code combines this Matplotlib demo and this SO answer.
Hope that helps!
EDIT: Incorporating the comments, I can only think of some kind of label color dictionary, generating a custom colormap from the colors, and before plotting explicitly grabbing the proper color indices from the labels.
Here's the updated code (I added some additional colors and data points to check scalability):
from matplotlib import pyplot as plt
from matplotlib.colors import LinearSegmentedColormap
import numpy as np
# Color information; create custom colormap
label_color_dict = {'null': '#FF0000',
'exit': '#00FF00',
'power': '#0000FF',
'smile': '#FF00FF',
'addon': '#AAAAAA',
'addon2': '#444444'}
all_labels = list(label_color_dict.keys())
all_colors = list(label_color_dict.values())
n_colors = len(all_colors)
cm = LinearSegmentedColormap.from_list('custom_colormap', all_colors, N=n_colors)
# Data
x = [3, 4, 6, 77, 3, 10, 40]
y = [8, 5, 2, 5, 5, 4, 7]
labels = ('null', 'exit', 'power', 'smile', 'null', 'addon', 'addon2')
# Get indices from color list for given labels
color_idx = [all_colors.index(label_color_dict[label]) for label in labels]
# Customize colorbar and plot
sc = plt.scatter(x, y, c=color_idx, cmap=cm)
c_ticks = np.arange(n_colors) * (n_colors / (n_colors + 1)) + (2 / n_colors)
cbar = plt.colorbar(sc, ticks=c_ticks)
cbar.ax.set_yticklabels(all_labels)
plt.show()
And, the new output:
Finding the correct middle point of each color segment is (still) not good, but I'll leave this optimization to you.
I've got the following simple script that plots a graph:
import matplotlib.pyplot as plt
import numpy as np
T = np.array([6, 7, 8, 9, 10, 11, 12])
power = np.array([1.53E+03, 5.92E+02, 2.04E+02, 7.24E+01, 2.72E+01, 1.10E+01, 4.70E+00])
plt.plot(T,power)
plt.show()
As it is now, the line goes straight from point to point which looks ok, but could be better in my opinion. What I want is to smooth the line between the points. In Gnuplot I would have plotted with smooth cplines.
Is there an easy way to do this in PyPlot? I've found some tutorials, but they all seem rather complex.
You could use scipy.interpolate.spline to smooth out your data yourself:
from scipy.interpolate import spline
# 300 represents number of points to make between T.min and T.max
xnew = np.linspace(T.min(), T.max(), 300)
power_smooth = spline(T, power, xnew)
plt.plot(xnew,power_smooth)
plt.show()
spline is deprecated in scipy 0.19.0, use BSpline class instead.
Switching from spline to BSpline isn't a straightforward copy/paste and requires a little tweaking:
from scipy.interpolate import make_interp_spline, BSpline
# 300 represents number of points to make between T.min and T.max
xnew = np.linspace(T.min(), T.max(), 300)
spl = make_interp_spline(T, power, k=3) # type: BSpline
power_smooth = spl(xnew)
plt.plot(xnew, power_smooth)
plt.show()
Before:
After:
For this example spline works well, but if the function is not smooth inherently and you want to have smoothed version you can also try:
from scipy.ndimage.filters import gaussian_filter1d
ysmoothed = gaussian_filter1d(y, sigma=2)
plt.plot(x, ysmoothed)
plt.show()
if you increase sigma you can get a more smoothed function.
Proceed with caution with this one. It modifies the original values and may not be what you want.
See the scipy.interpolate documentation for some examples.
The following example demonstrates its use, for linear and cubic spline interpolation:
import matplotlib.pyplot as plt
import numpy as np
from scipy.interpolate import interp1d
# Define x, y, and xnew to resample at.
x = np.linspace(0, 10, num=11, endpoint=True)
y = np.cos(-x**2/9.0)
xnew = np.linspace(0, 10, num=41, endpoint=True)
# Define interpolators.
f_linear = interp1d(x, y)
f_cubic = interp1d(x, y, kind='cubic')
# Plot.
plt.plot(x, y, 'o', label='data')
plt.plot(xnew, f_linear(xnew), '-', label='linear')
plt.plot(xnew, f_cubic(xnew), '--', label='cubic')
plt.legend(loc='best')
plt.show()
Slightly modified for increased readability.
One of the easiest implementations I found was to use that Exponential Moving Average the Tensorboard uses:
def smooth(scalars: List[float], weight: float) -> List[float]: # Weight between 0 and 1
last = scalars[0] # First value in the plot (first timestep)
smoothed = list()
for point in scalars:
smoothed_val = last * weight + (1 - weight) * point # Calculate smoothed value
smoothed.append(smoothed_val) # Save it
last = smoothed_val # Anchor the last smoothed value
return smoothed
ax.plot(x_labels, smooth(train_data, .9), x_labels, train_data)
I presume you mean curve-fitting and not anti-aliasing from the context of your question. PyPlot doesn't have any built-in support for this, but you can easily implement some basic curve-fitting yourself, like the code seen here, or if you're using GuiQwt it has a curve fitting module. (You could probably also steal the code from SciPy to do this as well).
Here is a simple solution for dates:
from scipy.interpolate import make_interp_spline
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as dates
from datetime import datetime
data = {
datetime(2016, 9, 26, 0, 0): 26060, datetime(2016, 9, 27, 0, 0): 23243,
datetime(2016, 9, 28, 0, 0): 22534, datetime(2016, 9, 29, 0, 0): 22841,
datetime(2016, 9, 30, 0, 0): 22441, datetime(2016, 10, 1, 0, 0): 23248
}
#create data
date_np = np.array(list(data.keys()))
value_np = np.array(list(data.values()))
date_num = dates.date2num(date_np)
# smooth
date_num_smooth = np.linspace(date_num.min(), date_num.max(), 100)
spl = make_interp_spline(date_num, value_np, k=3)
value_np_smooth = spl(date_num_smooth)
# print
plt.plot(date_np, value_np)
plt.plot(dates.num2date(date_num_smooth), value_np_smooth)
plt.show()
It's worth your time looking at seaborn for plotting smoothed lines.
The seaborn lmplot function will plot data and regression model fits.
The following illustrates both polynomial and lowess fits:
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
T = np.array([6, 7, 8, 9, 10, 11, 12])
power = np.array([1.53E+03, 5.92E+02, 2.04E+02, 7.24E+01, 2.72E+01, 1.10E+01, 4.70E+00])
df = pd.DataFrame(data = {'T': T, 'power': power})
sns.lmplot(x='T', y='power', data=df, ci=None, order=4, truncate=False)
sns.lmplot(x='T', y='power', data=df, ci=None, lowess=True, truncate=False)
The order = 4 polynomial fit is overfitting this toy dataset. I don't show it here but order = 2 and order = 3 gave worse results.
The lowess = True fit is underfitting this tiny dataset but may give better results on larger datasets.
Check the seaborn regression tutorial for more examples.
Another way to go, which slightly modifies the function depending on the parameters you use:
from statsmodels.nonparametric.smoothers_lowess import lowess
def smoothing(x, y):
lowess_frac = 0.15 # size of data (%) for estimation =~ smoothing window
lowess_it = 0
x_smooth = x
y_smooth = lowess(y, x, is_sorted=False, frac=lowess_frac, it=lowess_it, return_sorted=False)
return x_smooth, y_smooth
That was better suited than other answers for my specific application case.
Given the following data frame:
import pandas as pd
import numpy as np
df=pd.DataFrame({'A':['A','B','C','D','E','F','G','H','I','J','K','L','M','N'],
'B':[20,25,39,43,32,17,40, 40, 34, 56, 76, 23, 54, 34]})
I'd like to create a bubble chart where each y-tick label is the same color as its respective dot. The code below works great if I only had say 4 rows of data and 4 colors in my color list. However, for some reason, when I have more than 9 or so rows of data (and colors in my color list), it only takes the first 9 elements of colors in the l.set_color(i) line. Any thoughts as to why this occurs? Is it a limitation of zip when iterating? Related to the data frame?
import matplotlib.pyplot as plt
import matplotlib.ticker as mtick
labels=df.A[::-1]
vals=df.B[::-1]
ind=np.arange(len(labels))
colors1=['r','g','b','c','y','y','y','g','b','c','y','y','y','g']
fig, ax = plt.subplots(1, 1, figsize = (6,4))
for i in ind:
plt.plot(vals[i],i,marker='o',markeredgecolor='none', markersize=17, alpha=.5, linestyle='none', color=colors1[i])
ax.tick_params(axis='x',which='both',bottom='on',top='off',color='grey',labelcolor='grey')
ax.tick_params(axis='y',which='both',left='off',right='off',color='grey',labelcolor='grey')
ax.spines['top'].set_visible(False);ax.spines['right'].set_visible(False);
ax.spines['bottom'].set_visible(False);ax.spines['left'].set_visible(False)
ax.set_xlim([0,50])
ax.set_ylim([min(ind)-1,max(ind)+1])
fontcols=colors1[::-1]
for l,i in zip(ax.yaxis.get_ticklabels(),fontcols):
l.set_color(i)
l.set_fontsize(11)
print(l,i) #This shows that only 9 members are being colored for some reason
plt.yticks(ind,labels,fontsize=14)
plt.show()
Thanks in advance!
You just need to set the yticks before you try and set the colours. As it is, matplotlib creates 9 ticks by default, you set their colours, then you tell it you want 14 ticks after. With just a little reordering, it all works:
import matplotlib.pyplot as plt
import matplotlib.ticker as mtick
import pandas as pd
import numpy as np
df=pd.DataFrame({'A':['A','B','C','D','E','F','G','H','I','J','K','L','M','N'],
'B':[20,25,39,43,32,17,40, 40, 34, 56, 76, 23, 54, 34]})
labels=df.A[::-1]
vals=df.B[::-1]
ind=np.arange(len(labels))
colors1=['r','g','b','c','y','y','y','g','b','c','y','y','y','g']
fig, ax = plt.subplots(1, 1, figsize = (6,4))
for i in ind:
plt.plot(vals[i],i,marker='o',markeredgecolor='none', markersize=17, alpha=.5, linestyle='none', color=colors1[i])
ax.tick_params(axis='x',which='both',bottom='on',top='off',color='grey',labelcolor='grey')
ax.tick_params(axis='y',which='both',left='off',right='off',color='grey',labelcolor='grey')
ax.spines['top'].set_visible(False);ax.spines['right'].set_visible(False);
ax.spines['bottom'].set_visible(False);ax.spines['left'].set_visible(False)
ax.set_xlim([0,80]) # I increased this to fit all your data in
ax.set_ylim([min(ind)-1,max(ind)+1])
fontcols=colors1 # NOTE: you don't need to reverse this
plt.yticks(ind,labels,fontsize=14)
for l,i in zip(ax.yaxis.get_ticklabels(),fontcols):
l.set_color(i)
l.set_fontsize(11)
print(l,i)
plt.show()
Also note, you don't need to reverse the colour list before setting the tick colours