I have a datasat that has a specific _FillValue defined, this is the ncdump output:
Driver: netCDF/Network Common Data Format
Files: data.nc
Size is 2688, 2016
Origin = (89.988090807592116,30.011910669975187)
Pixel Size = (0.023818384815780,-0.023821339950372)
Subdatasets:
SUBDATASET_1_NAME=NETCDF:"data.nc":data
SUBDATASET_1_DESC=[2016x2688] toa_brightness_temperature (16-bit integer)
Corner Coordinates:
Upper Left ( 89.9880908, 30.0119107)
Lower Left ( 89.9880908, -18.0119107)
Upper Right ( 154.0119092, 30.0119107)
Lower Right ( 154.0119092, -18.0119107)
Center ( 122.0000000, 6.0000000)
Band 1 Block=1344x1008 Type=Int16, ColorInterp=Undefined
NoData Value=-32768
Unit Type: K
Offset: 244.291473388672, Scale:0.0018558754818514
Metadata:
add_offset=244.29147
coordinates=time
keywords=Infra-red, brightness temperature
long_name=toa_brightness_temperature
NETCDF_VARNAME=data
scale_factor=0.0018558755
standard_name=toa_brightness_temperature
units=K
_FillValue=-32768
I can see the _FillValue above, but is it possible to save this as a variable in Python?
So this should work in netcdf4-python:
from netCDF4 import Dataset
nc = Dataset('data.nc')
var = nc.variables['data']
fill_value = var._FillValue
Related
I am trying to add vertical lines indicating the average of datasets in a layered histogram in Altair (based on their example). My attempt below is failing:
base = alt.Chart(outcomes)
bar = base.transform_fold(
['Push','Dealer Win','Player Win','Ace High Push'],
as_=['Outcome','Outcomes out of 1000']
).mark_bar(
opacity=0.3,
binSpacing=0
).encode(
alt.X('Outcomes out of 1000:Q', bin=alt.Bin(maxbins=100)),
alt.Y('count()', stack=None),
alt.Color('Outcome:N')
)
rule = base.transform_fold(
['Push','Dealer Win','Player Win','Ace High Push'],
as_=['Count','Outcome']
).mark_rule(
color='red'
).encode(
alt.X('mean(Outcome):Q'),
size=alt.value(2)
)
bar + rule
which results in:
When I do just bar though the layered histogram renders just fine:
Basically what I'm looking for is:
Thanks🙏
Update (less than an hour after original post):
Thanks #debbes for the speedy guidance! I was able to use your example to get this working via:
base = alt.Chart(outcomes).transform_fold(
['Push','Dealer Win','Player Win','Ace High Push'],
as_=['Outcome','Outcomes out of 1000']
).transform_bin(
field='Outcomes out of 1000',
as_='bin_meas',
bin=alt.Bin(maxbins=100)
).encode(
color='Outcome:N'
)
hist = base.mark_bar(
opacity=0.3,
binSpacing=0
).encode(
alt.X('bin_meas:Q'),
alt.Y('count()', stack=None)
)
rule = base.mark_rule(
size=2
).encode(
alt.X('mean(Outcomes out of 1000):Q')
)
hist + rule
which results in:
In this case you have to use the transform_bin instead of doing the binning in the X encoding:
base = alt.Chart(source).transform_fold(
['Trial A', 'Trial B', 'Trial C'],
as_=['Experiment', 'Measurement']
).transform_bin(
field='Measurement',
as_='bin_meas',
bin=alt.Bin(maxbins=100)
).encode(
color='Experiment:N'
)
hist = base.mark_bar(opacity=0.3,binSpacing=0).encode(
alt.X('bin_meas:Q'),
alt.Y('count()', stack=None),
)
rule = base.mark_rule(size=2).encode(alt.X('mean(Measurement):Q'),)
hist + rule
I am working on electrophysiological data which is in .abf format.
I want to obtain the hyperpolarization depth as indicated above in the figure. This is what I have done so far;
import matplotlib.pyplot as plt
import pyabf
import pandas as pd
abf = pyabf.ABF("test.abf")
abf.setSweep(10) # I can access a given sweep. Here sweep 10
df = pd.DataFrame({'time': abf.sweepX, 'current':abf.sweepY})
df1 = df.loc[15650:15800]
df1.plot(x='time', y='current')
I am thinking to apply change in derivative to find the first point of interest (x1,y1) and then lower point (x2,y2), but it looks complex. I would appreciate if someone give some hint or procedure.
The dataset as follow,
time current
0.7825 -63.323975
0.78255 -63.171387
0.7826 -62.89673
0.78265 -62.713623
0.7827 -62.469482
0.78275 -62.37793
0.7828 -62.10327
0.78285 -61.950684
0.7829 -61.76758
0.78295 -61.584473
0.783 -61.401367
0.78305 -61.24878
0.7831 -61.035156
0.78315 -60.85205
0.7832 -60.72998
0.78325 -60.516357
0.7833 -60.455322
0.78335 -60.2417
0.7834 -60.08911
0.78345 -59.96704
0.7835 -59.814453
0.78355 -59.661865
0.7836 -59.509277
0.78365 -59.417725
0.7837 -59.23462
0.78375 -59.11255
0.7838 -58.95996
0.78385 -58.86841
0.7839 -58.685303
0.78395 -58.59375
0.784 -58.441162
0.78405 -58.34961
0.7841 -58.19702
0.78415 -58.044434
0.7842 -57.922363
0.78425 -57.769775
0.7843 -57.678223
0.78435 -57.434082
0.7844 -57.34253
0.78445 -56.9458
0.7845 -56.274414
0.78455 -54.96216
0.7846 -53.253174
0.78465 -51.208496
0.7847 -48.950195
0.78475 -46.325684
0.7848 -43.09082
0.78485 -38.42163
0.7849 -31.036377
0.78495 -22.033691
0.785 -13.397217
0.78505 -6.072998
0.7851 -0.61035156
0.78515 2.7160645
0.7852 3.9367676
0.78525 3.4179688
0.7853 1.3427734
0.78535 -1.4953613
0.7854 -5.0964355
0.78545 -9.185791
0.7855 -13.641357
0.78555 -18.249512
0.7856 -23.132324
0.78565 -27.98462
0.7857 -32.714844
0.78575 -37.261963
0.7858 -41.47339
0.78585 -45.22705
0.7859 -48.553467
0.78595 -51.54419
0.786 -53.985596
0.78605 -56.18286
0.7861 -58.013916
0.78615 -59.539795
0.7862 -60.760498
0.78625 -61.88965
0.7863 -62.652588
0.78635 -63.323975
0.7864 -63.934326
0.78645 -64.2395
0.7865 -64.60571
0.78655 -64.78882
0.7866 -65.00244
0.78665 -64.971924
0.7867 -65.093994
0.78675 -65.03296
0.7868 -64.971924
0.78685 -64.819336
0.7869 -64.78882
0.78695 -64.66675
0.787 -64.48364
0.78705 -64.42261
0.7871 -64.2395
0.78715 -64.11743
0.7872 -63.964844
0.78725 -63.842773
0.7873 -63.659668
0.78735 -63.568115
0.7874 -63.446045
0.78745 -63.26294
0.7875 -63.171387
0.78755 -62.98828
0.7876 -62.89673
0.78765 -62.74414
0.7877 -62.713623
0.78775 -62.530518
0.7878 -62.438965
0.78785 -62.37793
0.7879 -62.25586
0.78795 -62.164307
0.788 -62.042236
0.78805 -62.01172
0.7881 -61.88965
0.78815 -61.88965
0.7882 -61.73706
0.78825 -61.706543
0.7883 -61.645508
0.78835 -61.61499
0.7884 -61.523438
0.78845 -61.462402
0.7885 -61.431885
0.78855 -61.340332
0.7886 -61.37085
0.78865 -61.279297
0.7887 -61.279297
0.78875 -61.157227
0.7888 -61.187744
0.78885 -61.09619
0.7889 -61.157227
0.78895 -61.12671
0.789 -61.09619
0.78905 -61.12671
0.7891 -61.00464
0.78915 -61.00464
0.7892 -60.97412
0.78925 -60.97412
0.7893 -60.943604
0.78935 -61.00464
0.7894 -60.913086
0.78945 -60.97412
0.7895 -60.943604
0.78955 -60.913086
0.7896 -60.943604
0.78965 -60.85205
0.7897 -60.85205
0.78975 -60.821533
0.7898 -60.88257
0.78985 -60.88257
0.7899 -60.913086
0.78995 -60.88257
0.79 -60.913086
We can plot the difference in current between consecutive points (which essentially is to a constant factor the derivative, since times are evenly spaced). First chart shows the actual diffs. Based on this we can set some threshold, such as 0.3, and apply it to filter the main DataFrame. The filtered values are shown in orange on the second chart:
fig, ax = plt.subplots(2, figsize=(8,8))
# plot derivative
df['current'].diff().plot(ax=ax[0])
# current
threshold = 0.4
df['filtered'] = df.loc[df['current'].diff().abs() > threshold]
df.plot(ax=ax[1])
# add spans
x = df['filtered'].dropna()
ax[1].axhspan(x.iloc[0], x.iloc[-1], alpha=0.3, edgecolor='skyblue', facecolor="none", hatch='////')
ax[1].axvspan(x.index.min(), x.index.max(), alpha=0.3, edgecolor='orange', facecolor="none", hatch='\\\\')
Output:
If you're interested in range values, you can dropna values in the filtered subset and find min and max from the index:
print('min', df['filtered'].dropna().index.min())
print('max', df['filtered'].dropna().index.max())
Output:
min 0.78445
max 0.7865
For the value of the gap you can use:
abs(df['filtered'].dropna().iloc[-1] - df['filtered'].dropna().iloc[0])
Output:
7.6599100000000035
Note: We can alternatively also get left edges of these spans as points where diff in the point is lower than the threshold and diff in the next point is higher than the threshold, and similarly for the right edges. This would also work in case we have multiple peaks:
threshold = 0.3
x = df['current'].diff().abs()
spanA = df.loc[(x < threshold) & (x.shift(-1) >= threshold)]
spanB = df.loc[(x >= threshold) & (x.shift(-1) < threshold)]
print(spanA)
current
time
0.7844 -57.34253
print(spanB)
current
time
0.7865 -64.60571
I am trying to segment 3d tomographs of porous networks in python. I am able to calculate the distance map with ndimage.distance_transform_edt and the peaks with feature.peak_local_max. when I apply the watershed algorithm a get an acceptable result, but the markers of the peaks are not located at the visible peaks, see image, of the distance map
Thanks in advance
Here the code a is the image
D = ndimage.distance_transform_edt(a)
localMax = feature.peak_local_max(D, indices=False, min_distance=50,
labels=a)
localMax2 = feature.peak_local_max(D, indices=True, min_distance=50,
labels=a)
markers = ndimage.label(localMax, structure=np.ones((3,3,3)))[0]
labels = morphology.watershed(-D,markers,mask=a)
I found a way:
i had to exclude the borders and apply a threshold
D = ndimage.distance_transform_edt(a)
localMax = feature.peak_local_max(D, indices=False, min_distance=30,
labels=a,threshold_abs=9,exclude_border=1)
localMax2 = feature.peak_local_max(D, indices=True, min_distance=30,
labels=a,threshold_abs=9,exclude_border=1)
#markers = ndimage.label(localMax, structure=np.ones((3,3,3)))[0]
markers = ndimage.label(localMax, structure=np.ones((3,3,3)))[0]
labels = morphology.watershed(-D,markers,mask=a)
regions=measure.regionprops(labels,intensity_image=a)
I am creating a Solar Array Simulator with python to simulate the voltage and current being created with different angles and intensities of sunlight as a satellite orbits around the earth. I have a very simple program that outputs the voltage and current just with the angle (no orbiting parameters yet). However, I need it to communicate the outputs generated with a E4350B model solar array simulator through a serial port, and I don't know where to start. I have installed pip and used that to install PySerial but do not know what to do from there. How do I communicate the voltage and amp outputs to the simulator through COM ports? Here is what I have for my program that runs the simulator.
from math import sin,radians,pi
import time
'''Needed information:
Voc (per one cell) = 2,680mV
Vmp (per one cell = 2,325mV
Isc (per one cell) = 453mA
Imp (per one cell) = 434mA
angular velocity = .05d/s'''
#Timer eclipse set for total eclipse tim ein seconds
timeEclipse = 1800
#total eclipse time = 30 mins
#Timer sun set for how many seconds it takes to change one degree
timeSun = 20
#Total sun exposer time = 60 mins
#Find the Vmp, Voc, Imp, and Isc from Beta angles 0 - 180
def Exposure():
tSun = timeSun
for x in range(0,181):
angle = sin(radians(x))
Voc = 2680 * angle
Vmp = 2325 * angle
#Amps are going to be a function of voltage
Isc = 453 * angle
Imp = 434 * angle
#
print('At angle ',x,' Vmp = ',Vmp,
'mV, Voc = ',Voc,'mV, Isc = ',Isc,'mA, and Imp = ',Imp,'mA')
time.sleep(tSun)
#Simulate time during eclipse
#Outputs nothing
def Eclipse():
tEclipse = timeEclipse
time.sleep(tEclipse)
#Run loop through Exposure and eclipse
def Run():
run = True
while(run):
Exposure()
Eclipse()
P.S. For anybody who dabbles in a little bit of physics, I need a way to find the current as a function of the voltage at every angle.
I want to plot a simplified heatmap that is not so difficult to edit with the scalar vector graphics program I am using (inkscape). The original heatmap as produced below contains lots of rectangles, and I wonder if they could be merged together in the different sectors to simplify the output pdf file:
nentries=100000
ci=rainbow(nentries)
set.seed=1
mean=10
## Generate some data (4 factors)
i = data.frame(
a=round(abs(rnorm(nentries,mean-2))),
b=round(abs(rnorm(nentries,mean-1))),
c=round(abs(rnorm(nentries,mean+1))),
d=round(abs(rnorm(nentries,mean+2)))
)
minvalue = 10
# Discretise values to 1 or 0
m0 = matrix(as.numeric(i>minvalue),nrow=nrow(i))
# Remove rows with all zeros
m = m0[rowSums(m0)>0,]
# Reorder with 1,1,1,1 on top
ms =m[order(as.vector(m %*% matrix(2^((ncol(m)-1):0),ncol=1)), decreasing=TRUE),]
rowci = rainbow(nrow(ms))
colci = rainbow(ncol(ms))
colnames(ms)=LETTERS[1:4]
limits=c(which(!duplicated(ms)),nrow(ms))
l=length(limits)
toname=round((limits[-l]+ limits[-1])/2)
freq=(limits[-1]-limits[-l])/nrow(ms)
rn=rep("", nrow(ms))
for(i in toname) rn[i]=paste(colnames(ms)[which(ms[i,]==1)],collapse="")
rn[toname]=paste(rn[toname], ": ", sprintf( "%.5f", freq ), "%")
heatmap(ms,
Rowv=NA,
labRow=rn,
keep.dendro = FALSE,
col=c("black","red"),
RowSideColors=rowci,
ColSideColors=colci,
)
dev.copy2pdf(file="/tmp/file.pdf")
Why don't you try RSvgDevice? Using it you could save your image as svg file, which is much convenient to Inkscape than pdf
I use the Cairo package for producing svg. It's incredibly easy. Here is a much simpler plot than the one you have in your example:
require(Cairo)
CairoSVG(file = "tmp.svg", width = 6, height = 6)
plot(1:10)
dev.off()
Upon opening in Inkscape, you can ungroup the elements and edit as you like.
Example (point moved, swirl added):
I don't think we (the internet) are being clear enough on this one.
Let me just start off with a successful export example
png("heatmap.png") #Ruby dev's think of this as kind of like opening a `File.open("asdfsd") do |f|` block
heatmap(sample_matrix, Rowv=NA, Colv=NA, col=terrain.colors(256), scale="column", margins=c(5,10))
dev.off()
The dev.off() bit, in my mind, reminds me of an end call to a ruby block or method, in that, the last line of the "nested" or enclosed (between png() and dev.off()) code's output is what gets dumped into the png file.
For example, if you ran this code:
png("heatmap4.png")
heatmap(sample_matrix, Rowv=NA, Colv=NA, col=terrain.colors(32), scale="column", margins=c(5,15))
heatmap(sample_matrix, Rowv=NA, Colv=NA, col=greenred(32), scale="column", margins=c(5,15))
dev.off()
it would output the 2nd (greenred color scheme, I just tested it) heatmap to the heatmap4.png file, just like how a ruby method returns its last line by default