I've just recently started using basemap, and i've encountered a problem.
When i run :
import numpy as np
import matplotlib.pyplot as plt
import netCDF4 as nc
import glob
from mpl_toolkits.basemap import Basemap
import warnings #note that I did not include all of my code, so some imports may be unnecessary.
%matplotlib inline
datapath = "/somedirectory/www.ncei.noaa.gov/data/avhrr-land-leaf-area-index-and-fapar/access/2001" #I've downloaded the data from here as well, I've averaged over every month.
datafiles = glob.glob(datapath+"/"+"avg_LAI_fAPAR*.nc")
data = [None]*len(datafiles)
month = [None]*len(datafiles)
lats = [None] * len(datafiles)
lons = [None] * len(datafiles)
fAPAR =[None] * len(datafiles)
LAI =[None] * len(datafiles)
for number, file in enumerate(datafiles):
month[number] = file[-4:-2]
data[number] = nc.Dataset(file,format = 'NETCDF4')
lats[number] = data[number]["latitude"]
lons[number] = data[number]["longitude"]
fAPAR[number] = data[number]["FAPAR"]
LAI[number] = data[number]["LAI"]
m = Basemap(width=5000000,height=3500000,
resolution='l',\
lat_ts=40)
for k in range(1): #only do one loop, it takes a long time to run on my machine. Idea is that is should be able to loop through all months in 2001
print(k)
plt.figure()#figsize=(20,10))
lon, lat = np.meshgrid(lons[k], lats[k])
xi, yi = m(lon,lat)
cs = m.pcolor(xi,yi,np.squeeze(LAI[k][0])) #note that the first dimension [k] comes from the for-loop, the second [0] is the temporal part of the LAI (avereged out in a bash script).
m.drawcoastlines()
m.drawcountries()
cbar = m.colorbar(cs, location='bottom', pad="10%")
plt.title('LAI on {}'.format(month[k]))`
The plot comes out empty, so nothing plotted (only white space). The data is masked, but also if I unmasked the data (i.e. replace the masked data by a nan), the plot did not show anything. The np.nanmean(LAI[0]) is about 1, but the regular mean is about -90, as the fill value is -100.
The data seems to work with NCview in linux.
I work on python 3.6, with the latest packages installed. Does anyone know where the problem might be?
Thanks in advance!
Related
I have been following lectures of MIT open course on Application of Mathematics in Finance. I am trying to code out the concepts for better understanding.
According to lectures(from what I understand), if random variable X is normally distributed then exp(X) is log-normally distributed and vice versa (please correct me if I am wrong here).
Here is what I tried:
I have list of integers that are normally distributed:
import numpy as np
import matplotlib.pyplot as plt
from math import sqrt
X = np.array(l)
mu = np.mean(X)
sigma = np.std(X)
count, bins, ignored = plt.hist(X, 35, density=True)
plt.plot(bins, 1/(sigma * np.sqrt(2 * np.pi)) * np.exp( - (bins - mu)**2 / (2 * sigma**2)
),linewidth=2, color='r')
plt.show()
Output:
Normally distributed curve
Now I want to get log-normal distribution from above data, here is what I have tried:
import numpy as np
import matplotlib.pyplot as plt
from math import sqrt
X = np.array(l)
ln = []
for x in X:
val = np.e**x
ln.append(val)
X_ln = np.array(ln)
X_ln = np.array(X_ln) / np.min(X_ln)
mu = np.mean(X_ln)
sigma = np.std(X_ln)
count, bins, ignored = plt.hist(X_ln, 10, density=True)
x = np.linspace(min(bins), max(bins), 10000)
pdf = (np.exp(-(np.log(x) - mu)**2 / (2 * sigma**2)) / (x * sigma * np.sqrt(2 * np.pi)))
plt.plot(x, pdf, color='r', linewidth=2)
plt.show()
Output :
Not so clean Output
I know there is a better way to do this, but I can't figure out how. Any suggestions would be highly appreciated.
Here are couple of references:
Log normal distribution in Python
MIT lecture notes(topic-1.1)
In case this is relevant, here is a list of elements I am trying to process:
List of elements
Update 1:
I have normalized X before adding values to ln. This fixed the distribution of histogram, however, I can't seem to fix to get red line to show log-normal distribution. Also the new histogram distribution is not very different from normal distribution. I can't think of any suitable reason for that.
This is the block of code I have added:
def normalize(v):
norm=np.linalg.norm(v, ord=1)
if norm==0:
norm=np.finfo(v.dtype).eps
return v/norm
X = np.array(l)
X = normalize(X)
New Output:
Slightly better result
I want to plot on the same graph one function (data coming from an array) and the line between (xmin,ymin) to (xmax,ymax) of data
import numpy as np
import matplotlib.pyplot as plt
exo_data = np.loadtxt('./exoplanets.dat')
au2m = 149597870700
day2seconds = 86400
M_Jupiter = 1.898e27
M_Sun = 1.989e30
G = 6.674e11
R=exo_data[:,0]*au2m
T=exo_data[:,1]*day2seconds
Mp=exo_data[:,2]*M_Jupiter
Ms=exo_data[:,3]*M_Sun
Mstar=np.mean(Ms)
C=(mth.pi*4)/G*Mstar
x=R**3
y=T**2
xmin=np.min(R)
ymin=np.min(T)
xmax=np.max(R)
ymax=np.max(T)
plt.plot([xmin,ymin], [xmax,ymax], color='red')
plt.plot(x,y, 'ro', color='blue')
plt.show
plt.close
I have only the first plot in spyder console, not the second one
Expected this :
I'm not sure if it's going to work for you, I suggest you to try the following
plt.plot([xmin,ymin], [xmax,ymax], x, y,'ro', color = 'red')
plt.show
Try to run the script not in the spider console, but in the editor to the left of it. The console processes command by command, therefore, it will process one plot at the time and print it in the same way.
Create a file code.py with your script, and then run it. You can do this with the play button in spyder, or by going to a console and typing python code.py.
Proof are doing everything right. As stated here. This should work:
from numpy import *
import math
import matplotlib.pyplot as plt
t = linspace(0, 2*math.pi, 400)
a = sin(t)
b = cos(t)
c = a + b
plt.plot(t, a, 'r') # plotting t, a separately
plt.plot(t, b, 'b') # plotting t, b separately
plt.plot(t, c, 'g') # plotting t, c separately
plt.show()
And should produce this.
Finally it works... It's my fault. I declared G with e11 and it must be e-11
I am trying to minimize noise from the CSI values. I generated some fake CSI values and added some random error along with it.
I am giving the code
import numpy as np
#import tkinter as Tk
import scipy as sp
import matplotlib.pyplot as plt
from scipy import signal
from scipy.fftpack import fft
#Generate fake CSI values
csi_values =np.array([0 + 0j]*64)
print(csi_values)
csi_values[13] = .5 + .2j
csi_values[12] = .75 + .25j
csi_values[11] = 1 + .35j
csi_values[10] = 1.5 + .5j
print(csi_values[10:14:1])
print(csi_values)
fft_value = fft(csi_values)
print(fft_value)
print(len(fft_value))
#print(fft_value)
real_csi= abs(csi_values)
mean_csi= np.mean(real_csi)
print(real_csi)
print(mean_csi)
x_axis = np.linspace(0.0, 63,64)
print(x_axis.shape,fft_value.shape)
plt.plot(abs(fft_value))
plt.grid()
plt.show()
#CSI Subcarrier selection based on average amplitude of all subcarriers
csi_truncated = csi_values[abs(csi_values) > mean_csi]
print(csi_truncated)
#Removal of noise using median filter
csi_error= csi_values + np.random.rand(64)
print(csi_error)
trim_csi = sp.signal.medfilt(csi_error,5)
However I am getting the following error
File "python_check.py", line 54, in <module>
trim_csi = sp.signal.medfilt(csi_error,5)
File "/home/avishek/venv3/lib/python3.5/site-packages/scipy/signal/signaltools.py", line 893, in medfilt
return sigtools._order_filterND(volume, domain, order)
ValueError: order_filterND not available for this type
What is the reason for this error? I am using the correct packages too I guess
Still got the same problem with newest scipy/python.
For other people coming here: You can switch to this function
scipy.ndimage.filters.median_filter
which does the same but works without this error.
this is a bit detailed, but help appreciated. It's a slightly annoying feature of seaborn that regplot can't handle datetime axes. This is especially so when you want to use the lowess parameter, which estimates local means to give a curve (as opposed to line) that tracks changes in a plot. (This functionality is available in R, for instance.)
This is a problem for me because I have a linear plot that I'd like to smooth out but without using a rolling mean. This is my plot:
To solve this problem, I took the index of my dataframe as the x-axis, and used statsmodels to calculate the lowess line as follows:
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import statsmodels.api as sm
rated = pd.read_csv('filepath.csv')
rated['linear'] = rated.index
x = rated['linear']
y = rated['trust_num']
lowess = sm.nonparametric.lowess(y, x, frac=.2)
low_y = [i[1] for i in lowess]
low_x = [i[0] for i in lowess]
rated['low_y'] = low_y
rated['low_x'] = low_x
rated['low_y'] = low_y
rated['low_x'] = low_x
h = sns.lineplot(x = 'linear', y = 'trust_num', data = rated)
h = sns.lineplot(x = 'linear', y = 'low_y', data = rated, color = 'r')
This produces exactly what I'd like:
The final step comes with assigning dates to the x-axis, which I do as follows:
labels = [i for i in rated['date']]
h.set_xticklabels(labels)
The result is a clustered x-axis, as below:
Fair enough, this is a common plotting problem. So I try to rotate my xtick labels:
plt.xticks(rotation=45)
But it makes no difference. Can anyone advise how I might declutter the axis? Seems a pain to nearly get there and fall at a seemingly simple problem!
This seems like it should be an easy fix but I can't get it to work. I would like 40°N to display in the attached plot, but setting the labels argument in drawparallels to [1,0,1,1] isn't doing the trick. That should plot the parallels lables where they intersect the left, top and bottom of the plot according to the documentation. I would also like for 0° to once again show up in the bottom right corner. Any idea of how I can fix those 2 issues?
from netCDF4 import Dataset as NetCDFFile
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.basemap import Basemap
from mpl_toolkits.basemap import addcyclic
nc = NetCDFFile('C:/myfile.nc')
lat = nc.variables['lat'][:]
lon = nc.variables['lon'][:]
time = nc.variables['time'][:]
olr = nc.variables['olr'][:]
olr,lon = addcyclic(olr,lon)
map = Basemap(llcrnrlon=0.,llcrnrlat=-40.,urcrnrlon=360.,urcrnrlat=40.,resolution='l')
lons,lats = np.meshgrid(lon,lat)
x,y = map(lons,lats)
levels = np.arange(-19.5,20.0,0.5)
levels = levels[levels!=0]
ticks = np.arange(-20.0,20.0,4.0)
cs = map.contourf(x,y,olr[0],levels, cmap='bwr')
cbar = plt.colorbar(cs, orientation='horizontal', cmap='bwr', spacing='proportional', ticks=ticks)
cbar.set_label('Outgoing Longwave Radiation Anomalies $\mathregular{(W/m^2)}$')
map.drawcoastlines()
map.drawparallels(np.arange(-40,40,20),labels=[1,0,1,1], linewidth=0.5, fontsize=7)
map.drawmeridians(np.arange(0,360,40),labels=[1,1,0,1], linewidth=0.5, fontsize=7)
The first part of the question is easy. In order for the label to show up, you have to actually draw the parallel, but np.arange(-40,40,20) does not include 40. So, if you change that statement to np.arange(-40,41,20) your 40N label will show up.
The second part should in principle be solvable in the same way, but Basemap apparently uses the modulo of the longitudes to compute the position of the labels, so just using np.arange(0,361,40) when drawing the meridians will result in two 0 labels on top of each other. However, we can capture the labels that drawmeridians generates and manually change the position of the second 0 label. The labels are stored in a dictionary, so they are easy to deal with. To compute the x position of the last label, I compute the difference in x-position between the first and the second label, multiply that with the amount of meridians to be drawn (360/40) and add the x-position of the first label.
Here the complete example:
import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.basemap import Basemap
map = Basemap(llcrnrlon=0.,llcrnrlat=-40.,urcrnrlon=360.,urcrnrlat=40.,resolution='l')
map.drawcoastlines()
yticks = map.drawparallels(
np.arange(-40,41,20),labels=[1,0,1,1], linewidth=0.5, fontsize=7
)
xticks = map.drawmeridians(
np.arange(0,361,40),labels=[1,1,0,1], linewidth=0.5, fontsize=7
)
first_pos = xticks[0][1][0].get_position()
second_pos = xticks[40][1][0].get_position()
last_x = first_pos[0]+(second_pos[0]-first_pos[0])*360/40
xticks[360][1][0].set_position((last_x,first_pos[1]))
plt.show()
Here the resulting plot:
Hope this helps.