I am trying to perform the scatter plot of X and Y matrices, each of size 54x365, with the following code on MATLAB. The data was extracted from excel.
clc
clear
A = xlsread('Test_data.xlsx', 'Sheet 1', 'F3:NF56');
B = xlsread('Test_data.xlsx', 'Sheet 2', 'F3:NF56');
scatter (A,B)
Although they are of similar size, MATLAB produces the following statement:
Error using scatter (line 44)
X and Y must be vectors of the same length.
Error in Untitled2 (line 11)
scatter(A,B)
Note the following:
A = [ A, B, C, D, E ;
F, G, H, I, J ]
B = [ a, b, c, d, e ;
f, g, h, i, j ]
The variables (A,a), (B,b) and so on are plotted so as to produce a scatter plot.
I need help to perform the scatter plot. Thank you.
Reshaping the arrays as row vectors may allow the scatter() function to plot the data. Here the arrays are reshaped to have dimensions that are 1 by Number_Of_Values in each array.
%Generating random test data%
A = rand(54,365);
B = rand(54,365);
%Reshaping to allow plotting%
Number_Of_Values = numel(A);
A = reshape(A,[1 Number_Of_Values]);
B = reshape(B,[1 Number_Of_Values]);
scatter(A,B);
Ran using MATLAB R2019b
Related
I am attempting to translate a MATLAB function to Python from Timothy Sauer,
Numerical Analysis Second Edition, page 546, Program 12.8. The original function
receives a square matrix and returns a matrix with the same eigenvalues but in
Upper Hessenberg form. The original function creates Householder reflectors to produce zeros in the
offdiagonals of the matrix and performs similarity transformations on the original matrix to
get it to upper hessenberg form.
My Python translation succeeds only in obtaining the eigenvalues for 3x3 matrices
but not for 4x4 matrices. Would anyone know the cause of the error? I pasted my code with success and failing cases below. Thank you.
import numpy as np
import math
norm = lambda v:math.sqrt(np.sum(v**2))
def upper_hessenberg(A):
'''
Translated from Timothy Sauer, Numerical Analysis Second Edition, page 546, Program 12.8
Input: Square Matrix, A
Output: B, a Similar Matrix with Same Eigenvalues as A except in Upper Hessenberg form
V, a matrix containing the reflectors used to produce zeros in the off diagonals
'''
rows, columns = A.shape
B = A[:,:].astype(np.float) #will store the similar matrix
V = np.zeros(shape=(rows,columns),dtype=float) #will store the reflectors
for column in range(columns-2): #start from the 1st column end at the third to last column
row = column
x = B[row+1: ,column] #decapitate the column
reflection_of_x = np.zeros(len(x)) #first entry is the norm, followed by 0s
if abs(norm(x)) <= np.finfo(float).eps: #if there are already 0s inthe offdiagonals skip this column
continue
reflection_of_x[0] = norm(x)
v = reflection_of_x - x # v, (the difference vector) represents the line connecting the original column to the reflection of the column (see Timothy Sauer Num Analysis 2nd Edition Figure 4.11 Householder reflector)
v = v/norm(v) #normalize to length of 1 (unit vector)
V[:len(v), column] = v #save the reflector in an upper triangular matrix called V
#verify with x-2*(x # v * v) should equal a vector with all zeros except the leading entry
column_projections = np.outer(v , v # B[row+1:, column:]) #project each col onto difference vector
B[row+1:, column:] = B[row+1:, column:] - (2 * column_projections)
row_projections = np.outer(v, B[row:, column + 1:] # v).T #project each row onto difference vector
B[row:, column + 1:] = B[row:, column + 1:] - (2 * row_projections)
return V, B
# Algorithm succeeds only with 3x3 matrices
eigvectors = np.array([
[1,3,2],
[4,5,6],
[7,8,9],
])
eigvalues = np.array([
[4,0,0],
[0,3,0],
[0,0,2]
])
M = eigvectors # eigvalues # np.linalg.inv(eigvectors)
print("The expected eigvals :", np.linalg.eigvals(M))
V,B = upper_hessenberg(M)
print("For 3x3 matrices, The function successfully produces these eigvals",np.linalg.eigvals(B))
#But with 4x4 matrices it fails
eigvectors = np.array([
[1,3,2,4],
[4,5,6,2],
[7,8,9,5],
[5,2,7,8]
])
eigvalues = np.array([
[4,0,0,0],
[0,3,0,0],
[0,0,2,0],
[0,0,0,1]
])
M = eigvectors # eigvalues # np.linalg.inv(eigvectors)
print("The expected eigvals :", np.linalg.eigvals(M))
V,B = upper_hessenberg(M)
print("For 4x4 matrices, The function fails to obtain correct eigvals",np.linalg.eigvals(B))
Your error is that you try to be too efficient. While the last rows are indeed increasingly reduced with leading zeros, this is not the case for the last columns. So in row_projections you need to remove the limiter row:, change to B[:, column + 1:].
You are using the unstable variant of the "improved" Householder reflector. The older version would use the larger of x_refl - x and x_refl + x by setting reflection_of_x[0] = -np.sign(x[0])*norm(x) (or remove all minus signs there).
The stable variant of the improved reflector would use the binomial trick in the normalization of x_refl - x if this difference becomes too small.
x_refl - x = [ norm(x) - x[0], - x[1:] ]
= [ norm(x[1:])^2/(norm(x) + x[0]), - x[1:] ]
(x_refl - x)/norm(x_refl - x)
[ norm(x[1:]), - (norm(x)+x[0])*(x[1:]/norm(x[1:])) ]
= -----------------------------------------------------
sqrt(2*norm(x)*(norm(x)+x[0]))
While the parts may have wildly different scales, no catastrophic cancellation happens for x[0]>0.
See the discussion about the same algorithm from Golub/van Loan 4th ed. in for further details and opinions and the code from that book.
I'm trying the following to convey the value in a list to a list of [R,G,B] values.
data = range(0,6)
minima = min(data)
maxima = max(data)
norm = matplotlib.colors.Normalize(vmin=minima, vmax=maxima, clip=True)
mapper = cm.ScalarMappable(norm=norm, cmap=cm.Greys_r)
node_color = []
for d in data:
node_color.append(mapper.to_rgba(d))
The above returns a 4th dimension A. I would like to know if there is a way to obtain only RGB values.
mapper.to_rgba(d) returns a tuple of the form (r, g, b, a). You can directly assign the result to a 4-tuple as r, g, b, a = mapper.to_rgba(d). And then create a triple as (r, g, b) to be stored in a list.
mapper.to_rgba also works when it gets a list or array as parameter, so calling mapper.to_rgba(data) directly gets the list of all rgba-tuples. Via a list comprehension, a new list of rgb-triples can be created:
import matplotlib
from matplotlib import cm
data = range(0, 6)
norm = matplotlib.colors.Normalize(vmin=min(data), vmax=max(data), clip=True)
mapper = cm.ScalarMappable(norm=norm, cmap=cm.Greys_r)
node_color = [(r, g, b) for r, g, b, a in mapper.to_rgba(data)]
PS: The above code gives r, g and b values between 0 and 1. Depending on the application, integer values from 0 to 255 could be needed:
node_color = [(r, g, b) for r, g, b, a in mapper.to_rgba(data, bytes=True)]
I am grouping a dataframe via a certain column, and attempting to pcolormesh each grouped dataframe to its own subplot, ensuring that all subplots have the same bins.
My code is as follows:
abins, rbins = np.arange(0,25,1),np.arange(0,70,1)
A, R = np.meshgrid(np.deg2rad(15*abins),rbins)
fig, ax = plt.subplots(2,4,subplot_kw=dict(projection="polar"), figsize=(20,10),sharey=True)
k=0
for name, df in bigdf.groupby('sv'):
pc = ax.reshape(-1)[k].pcolormesh(A,R,df.groupby([pd.cut(df.el, rbins), pd.cut(df.mlt, abins)])['vtec'].mean().unstack(), \
cmap='RdBu_r')
k+=1
However, when I run I get the following error:
TypeError: Dimensions of C (58, 16) are incompatible with X (25) and/or Y (70); see help(pcolormesh)
I believe that there is something going on with A and R, since when I run
abins, rbins = np.arange(0,25,1),np.arange(0,70,1)
A, R = np.meshgrid(np.deg2rad(15*abins),rbins)
fig, ax = plt.subplots(2,4,subplot_kw=dict(projection="polar"), figsize=(20,10),sharey=True)
k=0
for name, df in bigdf.groupby('sv'):
pc = ax.reshape(-1)[k].pcolormesh(df.groupby([pd.cut(df.el, rbins), pd.cut(df.mlt, abins)])['vtec'].mean().unstack(), \
cmap='RdBu_r')
k+=1
without A and R, I get the following result:
but I require A and R so that I have my desired bins. Alternatively something could be going on in the pd.cut, but I do not know much about this function.
Any ideas? I can provide the dataframe if necessary, but for reproducability, column el, and mlt can be random values within rbins and abins, respectively. And vtec can just be a random set of values.
EDIT: Here is a subsection of the dataframe to reproduce the problem https://drive.google.com/open?id=1qEn6i2zU6sblY9kwfMPrDRY8rXS9i0Gj
I believe the error is occurring due to pd.cut() discarding desired bins when no values fall in them. Is there a way to stop it from doing this?
I managed to fix the issue using pd.MultiIndex.from_product and reindex to force the pd.groupby after pd.cut to keep empty bins, even if there are no data to populate them:
abins, rbins = np.arange(0,25,1),np.arange(0,70,1)
A, R = np.meshgrid(np.deg2rad(15*abins),rbins)
fig, ax = plt.subplots(2,4,subplot_kw=dict(projection="polar"), figsize=(20,10),sharey=True)
k=0
for name, df in bigdf.groupby('sv'):
idx = pd.MultiIndex.from_product([rbins, abins])
pc = ax.reshape(-1)[k].pcolormesh(A,R,df.groupby([pd.cut(df.el, rbins), pd.cut(df.mlt, abins)])['vtec'].mean().reindex(idx, fill_value=np.nan).unstack(), \
cmap='RdBu_r')
k+=1
Which gives the desired output:
As the title suggests, I'm trying to fit a piecewise equation to a large data set. The equations I would like to fit to my data are as follows:
y(x) = b, when x < c
else:
y(x) = b + exp(a(x-c)) - 1, when x >= c
There are multiple answers to how such an issue can be addressed, but as a Python beginner I can't figure out how to apply them to my problem:
Curve fit with a piecewise function?
Conditional curve fit with scipy?
The problem is that all variables (a,b and c) have to be calculated by the fitting algorithm.
Thank you for your help!
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
# Reduced Dataset
y = np.array([23.032, 21.765, 20.525, 21.856, 21.592, 20.754, 20.345, 20.534,
23.502, 21.725, 20.126, 21.381, 20.217, 21.553, 21.176, 20.976,
20.723, 20.401, 22.898, 22.02 , 21.09 , 22.543, 22.584, 22.799,
20.623, 20.529, 20.921, 22.505, 22.793, 20.845, 20.584, 22.026,
20.621, 23.316, 22.748, 20.253, 21.218, 23.422, 23.79 , 21.371,
24.318, 22.484, 24.775, 23.773, 25.623, 23.204, 25.729, 26.861,
27.268, 27.436, 29.471, 31.836, 34.034, 34.057, 35.674, 41.512,
48.249])
x = np.array([3756., 3759., 3762., 3765., 3768., 3771., 3774., 3777., 3780.,
3783., 3786., 3789., 3792., 3795., 3798., 3801., 3804., 3807.,
3810., 3813., 3816., 3819., 3822., 3825., 3828., 3831., 3834.,
3837., 3840., 3843., 3846., 3849., 3852., 3855., 3858., 3861.,
3864., 3867., 3870., 3873., 3876., 3879., 3882., 3885., 3888.,
3891., 3894., 3897., 3900., 3903., 3906., 3909., 3912., 3915.,
3918., 3921., 3924.])
# Simple exponential function without conditions (works so far)
def exponential_fit(x,a,b,c):
return b + np.exp(a*(x-c))
popt, pcov = curve_fit(exponential_fit, x, y, p0 = [0.1, 20,3800])
plt.plot(x, y, 'bo')
plt.plot(x, exponential_fit(x, *popt), 'r-')
plt.show()
You should change your function to something like
def exponential_fit(x, a, b, c):
if x >= c:
return b + np.exp(a*(x-c))-1
else:
return b
Edit: As chaosink pointed out in the comments, this approach no longer works as the the above function assumes that x is a scalar. However, curve_fit evaluates the function for array-like x. Consequently, one should use vectorised operations instead, see here for more details. To do so, one can either use
def exponential_fit(x, a, b, c):
return np.where(x >= c, b + np.exp(a*(x-c))-1, b)
or chaosink's suggestion in the comments:
def exponential_fit(x, a, b, c):
mask = (x >= c)
return mask * (b + np.exp(a*(x-c)) - 1) + ~mask * b
Both give:
I had some list:
x = [1,2,3,4]
y = [1.4,2.5,3.2,4.5]
And I named each node:
a = (1,1.4)
b = (2,2.5)
c = (3,3.2)
d = (4,4.5)
I want x,y to be a scatter plot and to have some connection
a to c
b to c and d
And my code is like:
for t in np.arange(5):
plt.axvline(t)
plt.scatter([1,2,3,4],[1.4,2.5,3.2,4.5])
plt.plot([1,2,3],[1.4,2.5,3.2])
plt.show()
But it connect all the node together
How to make a connection to only a to c and b to c and d and also mark abcd on my scatter plot?