Put a smooth line through data points (with filters) - python-3.x

I am trying to apply a filter to the peak datapoints of my impulse plot and smoothen them out but it doesn't seem to work. Required file signal.csv
scipy savgol_filter
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import find_peaks, savgol_filter
df = pd.read_csv('signal.csv')
df.plot(grid = 1,
c = (0,0,255/255),
linewidth = 0.5,
figsize = (10,5),
legend = False,
xlim = [df.index[0], df.index[-1]],
ylim = 0)
plt.xlabel('Zeit / ms')
plt.ylabel('UHF-Signal / mV')
plt.title('UHF')
x = df.T.to_numpy()[1]
peaks, _ = find_peaks(x, distance = 150, height = 4)
sgf = savgol_filter(peaks, 51, 3)
plt.plot(sgf, x[peaks], c = 'orange')
plt.plot(peaks, x[peaks], 'o', c = 'red')
plt.show()
scipy butter filter
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from scipy.signal import find_peaks, butter, filtfilt
df = pd.read_csv('signal.csv')
df.plot(grid = 1,
c = (0,0,255/255),
linewidth = 0.5,
figsize = (10,5),
legend = False,
xlim = [df.index[0], df.index[-1]],
ylim = 0)
plt.xlabel('Zeit / ms')
plt.ylabel('UHF-Signal / mV')
plt.title('UHF')
x = df['1'].values
peaks, _ = find_peaks(x, distance = 150, height = 4)
c, e = butter(10, 0.3)
z = filtfilt(c, e, peaks)
plt.plot(z, x[peaks], c = 'orange')
plt.plot(peaks, x[peaks], 'o', c = 'red')
plt.show()
As you can see the result is the same. How can I smoothen out the orange line? I want something like this:
Thanks in advance

You are smoothing the wrong variable. peaks are indices into x (which really are heights / y-values, which makes everything a bit confusing). Substituting
sgf = savgol_filter(x[peaks], 5, 3)
plt.plot(peaks, sgf, c = 'orange', linewidth=3)
for the corresponding lines in your code yields the following plot:
The fit is not great but neither of the methods you are using will deal with the sharp transition around x=2000 very well. I would try a Kalman filter next, or -- if the decay constant for all of your exponentials is the same -- try to fit the exponentials directly to the data using non-negative deconvolution as discussed here.

Related

Why don't the axes extend by plt.xlim(-1.5, 1.5)?

I'm trying to extend the plot by plt.xlim(-1.5, 1.5) and plt.ylim(-1.5, 1.5). Could you please explain why the the range of the plot is not as expected?
import pandas as pd
from sklearn import preprocessing
from sklearn import decomposition
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')
import numpy as np
# Change the image format to svg for better quality
%config InlineBackend.figure_format = 'svg'
decathlon = pd.read_csv("https://raw.githubusercontent.com/leanhdung1994/Deep-Learning/main/decathlon.txt", sep='\t')
decathlon_scaled = decathlon.copy()
decathlon_scaled.iloc[:, 0:10] = preprocessing.scale(decathlon.iloc[:, 0:10])
pca_scaled = decomposition.PCA(n_components = 10).fit(decathlon_scaled.iloc[:, 0:10])
decathlon_scaled_pca = pca_scaled.transform(decathlon_scaled.iloc[:, 0:10])
decathlon_scaled_pca_nor = decathlon_scaled_pca / np.sqrt((decathlon_scaled_pca ** 2).sum(axis = 0))
decathlon_scaled_nor = decathlon_scaled.iloc[:, 0:10] / np.sqrt((decathlon_scaled.iloc[:, 0:10] ** 2).sum(axis = 0))
decathlon_corr_circle = decathlon_scaled_pca_nor.T.dot(decathlon_scaled_nor)
decathlon_corr_circle
tmp = np.transpose(decathlon_corr_circle)[:, 0:2]
tmp = pd.DataFrame(tmp)
tmp.index = decathlon.columns[0:10]
fig = plt.figure(figsize = 1 * np.array(plt.rcParams['figure.figsize'])) # This is to have bigger plot
ax = sns.scatterplot(data = tmp,
x = tmp[0], y = tmp[1])
for i in range(10):
plt.arrow(0, 0, tmp[0][i], tmp[1][i],
color = 'orange', head_width = 0.025, length_includes_head = True)
circle = plt.Circle((0, 0), 1, color='g', fill=False)
ax.add_artist(circle)
plt.xlim(-1.5, 1.5)
plt.ylim(-1.5, 1.5)
plt.axis('equal')
The problem is that using plt.axis('equal') is equivalent to using ax.set_aspect('equal', adjustable='datalim'). That adjustable='datalim' is modifying the axis limits, even if you don't want it to.
Using the object-oriented approach for all of the last 3 lines of code is one way to solve this problem, since the default value of adjustable is box, not datalim. box means the shape of the axes will be changed to enforce the equal aspect ratio, compared to datalim which will keep the axes the same size, but change the axis limits. Using the state-machine version, plt.axis('equal'), doesn't allow you to set it to box, so the object-oriented approach is the best option to give you more control.
Change
plt.xlim(-1.5, 1.5)
plt.ylim(-1.5, 1.5)
plt.axis('equal')
to
ax.set_xlim(-1.5, 1.5)
ax.set_ylim(-1.5, 1.5)
ax.set_aspect('equal')

My cmap won't separate around 0, what exactly is wrong?

I'm trying to plot out a contour plot with a colour scheme that has one colour for positive values and one colour for negative values. I tried using the answer in Colorplot that distinguishes between positive and negative values but to no avail. I've attached a snippet of code below, with this code the cmap doesn't separate Red and Blue at 0 but at -40. What exactly is wrong?
import numpy as np
from matplotlib.colors import BoundaryNorm
from matplotlib.ticker import MaxNLocator
Vr=1438.7228 #some constants
Va=626.8932
dr=3.11
da=1.55
def func(x,y):
repulsive = Vr*np.log( ((y+x)**2 + dr**2)/((y-x)**2 + dr**2) )
attractive = Va*np.log( ((y+x)**2 + da**2)/((y-x)**2 + da**2) )
return (1.0/(4.0*x*y))*(repulsive-attractive)
x = np.linspace(1e-4,10,100)
y = np.linspace(1e-4,10,100)
X, Y = np.meshgrid(x, y)
Z = func(X,Y)
levels = MaxNLocator(nbins=15).tick_values(Z.min(), Z.max())
cmap = plt.get_cmap('RdBu')
norm = BoundaryNorm(levels, ncolors=cmap.N, clip=True)
sc = plt.contourf(X, Y, Z, norm=norm, cmap=cmap)
plt.colorbar(sc)
plt.show()

Plotting colorbar in Python 3

I am trying to color the errorbar points based on the color from an array. But getting an error. My code is shown below:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.cm import ScalarMappable, coolwarm as cmap
from matplotlib.colors import Normalize
fig = plt.figure(1)
sp = fig.add_subplot(1, 1, 1)
sp.set_xlabel(r'$x$')
sp.set_ylabel(r'$y$')
x = np.random.rand(10)
y = np.random.rand(10)
M = np.logspace(9, 10, 10)
norm = Normalize(vmin=8, vmax=11,clip=False) # controls the min and max of the colorbar
smap = ScalarMappable(cmap=cmap, norm=norm)
for xi, yi, Mi in zip(x, y, M):
c = cmap(norm(np.log10(Mi))) # make sure to color by log of mass, not mass
sp.errorbar(
xi,
yi,
yerr=[[.1], [.1]],
xerr=[[.1], [.1]],
ecolor=c,
marker='o',
mec=c,
mfc=c
)
cb = plt.colorbar(smap)
cb.set_label(r'$\log_{10}M$')
I am getting the following error:
TypeError: You must first set_array for mappable
For matplotlib < 3.1, you need to set an array - which can be empty
sm = ScalarMappable(cmap=cmap, norm=norm)
sm.set_array([])
fig.colorbar(sm)
For matplotlib >= 3.1, this is not necessary any more.
sm = ScalarMappable(cmap=cmap, norm=norm)
fig.colorbar(sm)

Draw curves with triple colors and width by using matplotlib and LineCollection [duplicate]

The figure above is a great artwork showing the wind speed, wind direction and temperature simultaneously. detailedly:
The X axes represent the date
The Y axes shows the wind direction(Southern, western, etc)
The variant widths of the line were stand for the wind speed through timeseries
The variant colors of the line were stand for the atmospheric temperature
This simple figure visualized 3 different attribute without redundancy.
So, I really want to reproduce similar plot in matplotlib.
My attempt now
## Reference 1 http://stackoverflow.com/questions/19390895/matplotlib-plot-with-variable-line-width
## Reference 2 http://stackoverflow.com/questions/17240694/python-how-to-plot-one-line-in-different-colors
def plot_colourline(x,y,c):
c = plt.cm.jet((c-np.min(c))/(np.max(c)-np.min(c)))
lwidths=1+x[:-1]
ax = plt.gca()
for i in np.arange(len(x)-1):
ax.plot([x[i],x[i+1]], [y[i],y[i+1]], c=c[i],linewidth = lwidths[i])# = lwidths[i])
return
x=np.linspace(0,4*math.pi,100)
y=np.cos(x)
lwidths=1+x[:-1]
fig = plt.figure(1, figsize=(5,5))
ax = fig.add_subplot(111)
plot_colourline(x,y,prop)
ax.set_xlim(0,4*math.pi)
ax.set_ylim(-1.1,1.1)
Does someone has a more interested way to achieve this? Any advice would be appreciate!
Using as inspiration another question.
One option would be to use fill_between. But perhaps not in the way it was intended. Instead of using it to create your line, use it to mask everything that is not the line. Under it you can have a pcolormesh or contourf (for example) to map color any way you want.
Look, for instance, at this example:
import matplotlib.pyplot as plt
import numpy as np
from scipy.interpolate import interp1d
def windline(x,y,deviation,color):
y1 = y-deviation/2
y2 = y+deviation/2
tol = (y2.max()-y1.min())*0.05
X, Y = np.meshgrid(np.linspace(x.min(), x.max(), 100), np.linspace(y1.min()-tol, y2.max()+tol, 100))
Z = X.copy()
for i in range(Z.shape[0]):
Z[i,:] = c
#plt.pcolormesh(X, Y, Z)
plt.contourf(X, Y, Z, cmap='seismic')
plt.fill_between(x, y2, y2=np.ones(x.shape)*(y2.max()+tol), color='w')
plt.fill_between(x, np.ones(x.shape) * (y1.min() - tol), y2=y1, color='w')
plt.xlim(x.min(), x.max())
plt.ylim(y1.min()-tol, y2.max()+tol)
plt.show()
x = np.arange(100)
yo = np.random.randint(20, 60, 21)
y = interp1d(np.arange(0, 101, 5), yo, kind='cubic')(x)
dv = np.random.randint(2, 10, 21)
d = interp1d(np.arange(0, 101, 5), dv, kind='cubic')(x)
co = np.random.randint(20, 60, 21)
c = interp1d(np.arange(0, 101, 5), co, kind='cubic')(x)
windline(x, y, d, c)
, which results in this:
The function windline accepts as arguments numpy arrays with x, y , a deviation (like a thickness value per x value), and color array for color mapping. I think it can be greatly improved by messing around with other details but the principle, although not perfect, should be solid.
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
x = np.linspace(0,4*np.pi,10000) # x data
y = np.cos(x) # y data
r = np.piecewise(x, [x < 2*np.pi, x >= 2*np.pi], [lambda x: 1-x/(2*np.pi), 0]) # red
g = np.piecewise(x, [x < 2*np.pi, x >= 2*np.pi], [lambda x: x/(2*np.pi), lambda x: -x/(2*np.pi)+2]) # green
b = np.piecewise(x, [x < 2*np.pi, x >= 2*np.pi], [0, lambda x: x/(2*np.pi)-1]) # blue
a = np.ones(10000) # alpha
w = x # width
fig, ax = plt.subplots(2)
ax[0].plot(x, r, color='r')
ax[0].plot(x, g, color='g')
ax[0].plot(x, b, color='b')
# mysterious parts
points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
# mysterious parts
rgba = list(zip(r,g,b,a))
lc = LineCollection(segments, linewidths=w, colors=rgba)
ax[1].add_collection(lc)
ax[1].set_xlim(0,4*np.pi)
ax[1].set_ylim(-1.1,1.1)
fig.show()
I notice this is what I suffered.

plt.subplot_adjust() not working correctly

I am making some density plots like so:
import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import r2_score
import matplotlib
from scipy import stats
import matplotlib.gridspec as gridspec
from mpl_toolkits.axes_grid1.inset_locator import InsetPosition
from matplotlib.ticker import FormatStrFormatter
import matplotlib.cm as cm
from scipy.ndimage.filters import gaussian_filter
import random
matplotlib.rcParams.update({'font.size': 16})
matplotlib.rcParams['xtick.direction'] = 'in'
matplotlib.rcParams['ytick.direction'] = 'in'
x = random.sample(range(1, 10001), 1000)
y = random.sample(range(1, 10001), 1000)
def myplot(x, y, s, bins=1000):
heatmap, xedges, yedges = np.histogram2d(x, y, bins=bins)
heatmap = gaussian_filter(heatmap, sigma=s)
extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]
return heatmap.T, extent
cmap = cm.YlOrRd
fig, (ax, ax1, cax) = plt.subplots(ncols = 3, figsize = (15, 5),
gridspec_kw={"width_ratios":[1,1, 0.5]})
img, extent = myplot(x, y, 20)
im = ax.imshow(img, extent = extent, origin = 'lower', cmap = cmap)
ax.text(0.05, 0.92, '$R^2$ = {}'.format(np.round(r2_score(x, y), 2)), fontsize=14, color = 'k', transform = ax.transAxes)
ax.plot(ax.get_xlim(), ax.get_ylim(), ls="--", c=".3")
ax.set_xlabel("Black Sky")
ax.set_ylabel("Blue Sky")
img2, extent2 = myplot(x, y, 20)
ax1.imshow(img2, extent = extent2, origin = 'lower', cmap = cmap)
ax1.text(0.05, 0.92, '$R^2$ = {}'.format(np.round(r2_score(x, y), 2)), fontsize=14, color = 'k', transform = ax1.transAxes)
ax1.axes.get_yaxis().set_visible(False)
ax1.yaxis.set_ticks([])
ax1.plot(ax1.get_xlim(), ax1.get_ylim(), ls="--", c=".3")
ax1.set_xlabel("White Sky")
ip = InsetPosition(ax1, [1.05,0,0.05,1])
cax.set_axes_locator(ip)
fig.colorbar(im, cax=cax, ax=[ax,ax1], use_gridspec = True)
plt.subplots_adjust(wspace=0.1, hspace=0)
which gives me a plot like this:
No matter what I change wspace to the plot stays the same. I think this is because when I turn of the y-axis in ax1 I am just making the text blank instead of removing the y-axis all together. Is there a way to do this so that I can make the width spacing between the figures closer together?
As commented, wspace sets the minimal distance between plots. This distance may be larger in case of equal aspect axes. Then it will depend on the figure size, figure aspect and image aspect.
A. Use automatic aspect
You may set aspect = "auto" in your imshow plots,
ax.imshow(..., aspect = "auto")
B. Adjust the subplot parameters
You may set the left or right subplot parameter to something smaller. E.g.
plt.subplots_adjust(wspace=0.0, hspace=0, right=0.7)
C. Adjust the figure size
Using a smaller figure width, which is closer to the actual image aspect will also reduce whitespace around the figure.
E.g, making the figure only 11 inches wide and using 5% padding on the right,
plt.subplots(..., figsize = (11, 5))
plt.subplots_adjust(wspace=0.0, hspace=0, right=.95)

Resources