I have infile.txt file with multiple columns and rows like this:
infile.txt
2020 01 13 00 28.5833 77.2000 979 0 282.6 284.3 285.4 0 0
2020 01 13 00 28.5833 77.2000 925 469.578 290.4 296.9 297.7 3.6 5.1
2020 01 13 00 28.5833 77.2000 909 613.987 290.8 298.8 299.5 4.7 3.3
2020 01 13 00 28.5833 77.2000 850 1169.4 288 301.6 303.1 9.3 0
2020 01 13 00 28.5833 77.2000 700 2776.28 279 308.9 309.6 0 7.1
2020 01 13 00 28.5833 77.2000 500 5561.01 258.1 314.6 314.8 14.2 11.9
and, I want to perform some column-based calculation as follows:
awk '{R=0; if($12) R=(('$g'/'$theta_vs')*($11-'$theta_vs')*($8-'$z_s'))/(($12^2)+($13^2)); print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,R }' > test.txt
This work perfect and keep R=0 when $12!=0 but this provides an output value 0 when $12==0 as follows:
outputfile:
2020 01 13 00 28.5833 77.2000 979 0 282.6 284.3 285.4 0 0 0
2020 01 13 00 28.5833 77.2000 925 469.578 290.4 296.9 297.7 3.6 5.1 5.08926
2020 01 13 00 28.5833 77.2000 909 613.987 290.8 298.8 299.5 4.7 3.3 9.01363
2020 01 13 00 28.5833 77.2000 850 1169.4 288 301.6 303.1 9.3 0 8.21755
2020 01 13 00 28.5833 77.2000 700 2776.28 279 308.9 309.6 0 7.1 0
2020 01 13 00 28.5833 77.2000 500 5561.01 258.1 314.6 314.8 14.2 11.9 16.3555
I want to keep R=0 when both $12 && $13 is 0.
How can I make it?
Thank you
Change
if ($12)
to
if ($12 || $13)
This will assign R if either of them is non-zero, and leave it at 0 if both of them are zero.
Could you please try following.
awk -v G="$g" -v theta="$theta_vs" -v z="$z_s" '{R=0; if($12 || $13) R=((G/theta)*($11-theta)*($8-z))/(($12^2)+($13^2)); print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,R }' Input_file
In case your lines have only 13 fields then as per Ed sir's suggestion adding following.
awk -v G="$g" -v theta="$theta_vs" -v z="$z_s" '{R=0; if($12 || $13) R=((G/theta)*($11-theta)*($8-z))/(($12^2)+($13^2)); print $0,R }' Input_file
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 4 months ago.
Improve this question
I have a time series data wherein measurement values from different sensors have been captured asynchronously and concatenated into same ascii file. The sensor values have been captured at the same time instance.
The values are white space separated.
Original file looks like below.
2022 281 08 48 14 876 10 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 14 876 10 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 14 876 10 NOTSAMPLED NOTSAMPLED 1.00
2022 281 08 48 15 391 11 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 15 391 11 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 15 391 11 NOTSAMPLED NOTSAMPLED 1.00
2022 281 08 48 15 896 12 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 15 896 12 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 15 896 12 NOTSAMPLED NOTSAMPLED 1.00
Now I need to replace the string NOTSAMPLED with the previous instance sensor value as mentioned below and also merge sensor values spread across multiple rows and columns into single row having same time.
2022 281 08 48 14 876 10 1.00 0.0 1.0
2022 281 08 48 15 391 11 1.00 0.0 1.0
2022 281 08 48 15 896 12 1.00 0.0 1.0
Similarly if input data is
2022 281 08 48 14 876 10 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 14 876 10 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 14 880 10 NOTSAMPLED NOTSAMPLED 10.00
2022 281 08 48 15 391 11 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 15 391 11 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 15 395 11 NOTSAMPLED NOTSAMPLED 11.00
2022 281 08 48 15 896 12 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 15 896 12 NOTSAMPLED 0.00 NOTSAMPLED
2022 281 08 48 15 900 12 NOTSAMPLED NOTSAMPLED 12.00
then my expected output is
2022 281 08 48 14 876 10 1.00 0.00 NOTSAMPLED
2022 281 08 48 14 880 10 1.00 0.00 10.00
2022 281 08 48 15 391 11 1.00 0.00 10.00
2022 281 08 48 15 395 11 1.00 0.00 11.00
2022 281 08 48 15 896 12 1.00 0.00 11.00
2022 281 08 48 15 900 12 1.00 0.00 12.00
How can it be achieved using sed/awk or any other bash shell scripting commands ?
I tried the following script.
#! /bin/bash
inp_filename=$1
awk '
NR == 1 { split($0, filldown) }
{
for (i = 6; i <= NF; i++)
if ($i != "NOTSAMPLED")
filldown[i] = $i
else
$i = filldown[i]
print
}
' $inp_filename`
But the result is
2022 281 08 48 14 876 10 1.00 NOTSAMPLED NOTSAMPLED
2022 281 08 48 14 876 10 1.00 0.00 NOTSAMPLED
2022 281 08 48 14 876 10 1.00 0.00 1.00
2022 281 08 48 15 391 11 1.00 0.00 NOTSAMPLED
2022 281 08 48 15 391 11 1.00 0.00 NOTSAMPLED
2022 281 08 48 15 391 11 1.00 0.00 1.00
2022 281 08 48 15 896 12 1.00 0.00 NOTSAMPLED
2022 281 08 48 15 896 12 1.00 0.00 NOTSAMPLED
2022 281 08 48 15 896 12 1.00 0.00 1.00
One awk idea:
awk '
function print_values( j) {
if (FNR>1) {
printf "%s", ts_prev # print previous date/time stamp
for (j=8;j<=NF;j++) # loop through sensor values and append to current line of output
printf "%s%s", OFS, (values[j]=="" ? "NOTSAMPLED" : values[j])
print "" # terminate current line of output
}
ts_prev = ts_curr
}
{ gsub(/\r/,"") # per comment from OP, need to remove windows/dow line endings
ts_curr=$1 # save date/time stamp of current line
for (i=2;i<=7;i++)
ts_curr = ts_curr FS $i
if (ts_curr != ts_prev) # if this is a new date/time stamp then ...
print_values() # print previous date/time stamp and associated sensor values
for (i=8;i<=NF;i++) # loop through values and ...
if ($i != "NOTSAMPLED") # if a valid value then ...
values[i]=$i # save the value
}
END { print_values() } # flush last date/time stamp to stdout
' sensor.dat
This generates:
2022 281 08 48 14 876 10 1.00 0.00 NOTSAMPLED
2022 281 08 48 14 880 10 1.00 0.00 10.00
2022 281 08 48 15 391 11 1.00 0.00 10.00
2022 281 08 48 15 395 11 1.00 0.00 11.00
2022 281 08 48 15 896 12 1.00 0.00 11.00
2022 281 08 48 15 900 12 1.00 0.00 12.00
This will fill "NOTSAMPLED" with the valid sample value of the same column in the preceding row. The first few (3) rows remain unchanged, as there is no preceding sampled value.
awk -v ns=NOTSAMPLED '
BEGIN {a[0]=ns; a[1]=ns; a[2]=ns}
{
for (i=0; i<3; ++i) {
if ($(NF-i) == ns) {
$(NF-i) = a[2-i]
}
else {
a[2-i] = $(NF-i)
}
}
print
}' myfile
This fills fields with no preceding value (near the start) with the closest proceeding value. It parses the file twice - once to gets the first three sampled values and again to fill in the same manner as the previous example.
awk -v n="NOTSAMPLED" '
FNR==NR && filled != 3 {
for (i=0; i<3; ++i) {
if ($(NF-i) != n && a[2-i] == "") {
a[2-i] = $(NF-i)
++filled
}
}
nextline
}
FNR!=NR {
for (i=0; i<3; ++i) {
if ($(NF-i) == n) {
$(NF-i) = a[2-i]
}
else {
a[2-i] = $(NF-i)
}
}
print
}' myfile myfile
The output delimits all columns with a single space. If you want the longer whitespace, just change print to printf with an appropriate format string (like printf "...%20s%20s%20s\n", ...,$8,$9,$10)
Explanation:
Scanning each row, keep an array of the most recent valid samples for each column.
Replace NOTSAMPLED, with the value from the array. If a field is valid, instead update the corresponding array element.
I recently dug out an old book of mine, The Hawaiian Computer Mystery, published in 1985. There is a fragment of code in BASIC on page 81,
1 For N = 7 to 77
2 Print N, SQR(N) - INT (SQR [N] )
3 Next N
4 End
I can sort of see what it should do, but I can't get it to run. There's apparently an error in the second line, but I can't figure out what.
Assuming that you must find the digits after the decimal point of the square root of a number, than the issue is with the square brackets - they must be round. The following code:
1 For N = 7 to 77
2 Print N, SQR(N) - INT (SQR (N) )
3 Next N
4 End
(blank line in the end)
will produce the following result:
7 .64575124
8 .8284271
9 0
10 .1622777
11 .31662488
12 .46410155
13 .60555124
14 .7416575
15 .87298346
16 0
17 1.23105526E-1
18 .2426405
19 .35889912
20 .47213602
21 .5825758
22 .69041586
23 .7958317
24 .89897966
25 0
26 .09901953
27 .19615221
28 .29150248
29 .38516474
30 .47722578
31 .5677643
32 .65685415
33 .7445626
34 .8309517
35 .91608
36 0
37 .08276272
38 .16441393
39 .24499798
40 .3245554
41 .40312433
42 .48074055
43 .5574384
44 .63324976
45 .7082038
46 .78233004
47 .8556547
48 .9282031
49 0
50 .07106781
51 .14142847
52 .21110249
53 .28010988
54 .34846926
55 .41619825
56 .483315
57 .54983425
58 .6157732
59 .68114567
60 .7459669
61 .8102498
62 .8740077
63 .93725395
64 0
65 6.2257767E-2
66 .1240387
67 .18535233
68 .24621105
69 .30662346
70 .36660004
71 .42614937
72 .485281
73 .5440035
74 .60232544
75 .6602545
76 .71779823
77 .77496433
I am working with a imx6 dual light digi board. I want to add Ad7879 touch support. I 've been followed the next steps:
-First I changed the kernel config file adding support fo touchscreen and ad7879. In kernel config file I 've made the following changes:
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_AD7879=y
CONFIG_TOUCHSCREEN_AD7879_I2C=y
At the first I configured ad7879 as module doing:
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_AD7879=m
CONFIG_TOUCHSCREEN_AD7879_I2C=m
After that I made the device initialization in the device tree.
In imx6qdl-ccimx6sbc.dtsi file I put the next:
&i2c3 {
ad7879#2c
{
compatible = "adi,ad7879-1";
reg = <0x2c>;
interrupt-parent = <&gpio6>;
interrupts = <15 IRQ_TYPE_EDGE_FALLING>;
touchscreen-max-pressure = <4096>;
adi,resistance-plate-x = <120>;
adi,first-conversion-delay = /bits/ 8 <3>;
adi,acquisition-time = /bits/ 8 <1>;
adi,median-filter-size = /bits/ 8 <2>;
adi,averaging = /bits/ 8 <1>;
adi,conversion-interval = /bits/ 8 <255>;
};
The ad7879 interrupt controller is conected to the imx6 in EXP_I2C_IRQ_N pin that is the GPIO_6_15, for that reason I put interrupt-parent = <&gpio6> and interrupts = <15 IRQ_TYPE_EDGE_FALLING>;.
In imx6qdl-ccimx6sbc.dts file I put:
&i2c3 {
...
ad7879#2c {
status ="okay";
};
...
};
Then I followed with compile the linux image and device tree.
In the imx6 with the linux image that I configures the ad7879 as module I loaded the ad7879 modules doing this:
root:~> modprobe ad7879
root:~> modprobe ad7879-i2c
but there is not log messages abaout the ad7879 and there is no ad7879 node asociated.
If I check the device node to chek the device node:
root#ccimx6sbc: ls -la /dev/input/
drwxr-xr-x 4 root root 180 Jan 1 2000 .
drwxr-xr-x 14 root root 3480 May 16 14:49 ..
drwxr-xr-x 2 root root 80 Jan 1 2000 by-id
drwxr-xr-x 2 root root 120 Jan 1 2000 by-path
crw-rw---- 1 root input 13, 64 Jan 1 2000 event0
crw-rw---- 1 root input 13, 65 Jan 1 2000 event1
crw-rw---- 1 root input 13, 66 Jan 1 2000 event2
crw-rw---- 1 root input 13, 63 Jan 1 2000 mice
crw-rw---- 1 root input 13, 32 Jan 1 2000 mouse0
root#ccimx6sbc:# cat /sys/class/input/input0/name
da9063-onkey
root#ccimx6sbc:# cat /sys/class/input/input1/name
Genius 4D Scroll Mouse
root#ccimx6sbc:# cat /sys/class/input/input2/name
sgtl5000-audio Headphone Jack
You can see that there is no node asociated.
root#ccimx6sbc:# cat /proc/interrupts
CPU0 CPU1
29: 140884 13795 GIC 29 twd
34: 406 0 GIC 34 sdma
35: 0 0 GIC 35 VPU_JPG_IRQ
37: 1 0 GIC 37 2400000.ipu
38: 12 0 GIC 38 2400000.ipu
42: 28 0 GIC 42
44: 0 0 GIC 44 VPU_CODEC_IRQ
50: 0 0 GIC 50 vdoa
51: 0 0 GIC 51 rtc alarm
54: 52 0 GIC 54 mmc3
55: 52 0 GIC 55 mmc1
57: 3580 0 GIC 57 mmc0
61: 353 0 GIC 61 21f0000.serial
63: 0 0 GIC 63 2008000.ecspi
69: 472 0 GIC 69 21a4000.i2c
70: 1367 0 GIC 70 21a8000.i2c
72: 109 0 GIC 72 2184200.usb
75: 0 0 GIC 75 2184000.usb
79: 0 0 GIC 79 202c000.ssi
81: 0 0 GIC 81 imx_thermal
87: 167 0 GIC 87 i.MX Timer Tick
112: 0 0 GIC 112 20bc000.wdog
134: 0 0 GIC 134 mipi_dsi
137: 466 0 GIC 137 2101000.jr0
138: 0 0 GIC 138 2102000.jr1
139: 0 0 GIC 139 mmdc_1
144: 0 0 GIC 144 mmdc_1
147: 0 0 GIC 147 20e0000.hdmi_video
150: 62234 0 GIC 150 2188000.ethernet
151: 0 0 GIC 151 2188000.ethernet
192: 0 0 gpio-mxc 0 headphone detect
364: 1 0 gpio-mxc 12 da9063-irq
413: 0 1 da9063-irq 3 HWMON
414: 0 0 da9063-irq 0 ONKEY
415: 0 0 da9063-irq 1 ALARM
IPI0: 0 0 CPU wakeup interrupts
IPI1: 0 58 Timer broadcast interrupts
IPI2: 1405 2387 Rescheduling interrupts
IPI3: 0 0 Function call interrupts
IPI4: 19 33 Single function call interrupts
IPI5: 0 0 CPU stop interrupts
IPI6: 426 476 IRQ work interrupts
IPI7: 0 0 completion interrupts
Err: 0
You can se that there is not interrupts asociated to ad7879.
There is no difference when I build the linux kernel with this configuration:
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_AD7879=y
CONFIG_TOUCHSCREEN_AD7879_I2C=y
I did run dmesg and there is no log entries associated with ad7879.
In the imx6 linux I did the following commands:
>root#ccimx6sbc:i2cdetect 2
with this response:
WARNING! This program can confuse your I2C bus, cause data loss and worse!
I will probe file /dev/i2c-2.
I will probe address range 0x03-0x77.
Continue? [Y/n] y
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- UU -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- 2c -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: UU -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
If you can see the identifier of the device is detected (0x2c).
Then I made:
>root#ccimx6sbc: i2cdump -r 0-0x40 2 0x2c
With this response:
No size specified (using byte-data access)
WARNING! This program can confuse your I2C bus, cause data loss and worse!
I will probe file /dev/i2c-2, address 0x2c, mode byte
Probe range limited to 0x00-0x40.
Continue? [Y/n] y
0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef
00: 00 00 40 00 00 00 00 00 00 00 00 00 00 00 03 00 ..#...........?.
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
40: 00
You can se in 0x02 and 0x0e are two of the default values for the registers of ad7879 and they are correct. So I concluded that the controller is well connected.
I don't if I missing something.
I have a bash script as below:
day=(58 34 107 91 43 39 41 76 37 47 70 74 56 19 95 38 48 96 50 76 89 79 46 105 26 88 69 87 23 82 99 77 114 52 87 63 33 52 57 45 48 49 55 60 34 107 48 40 25 20 16)
year=(1952 1953 1954 1955 1956 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004)
for dom in $day; do
for yrs in $year; do
ncks -O -d time,$dom imdJJAS$yrs.nc ac_$yrs.nc
done
done
Basically i am trying to extract the time dimension for each year using the NCO ncks command, the scripts run but the outputs are incorrect. For year 1951, it succesfully extracted the 58th time value, but from 1952 onwards, it extracts the last value in the day array (16), which is incorrect.
ive tried setting {$day[a]} since its an array, but if i used this, for all years in extracts the last value in the array instead.
I am not too sure what im doing wrong, ive looked through quite a few post regarding this, but it doest seem to be working.
Id appreciate any help.
Cheers!
$array by itself will expand to the first element in the array. To expand to the full array you should use ${array[#]}:
for dom in "${day[#]}"; do
for yrs in "${year[#]}"; do
ncks -O -d "time,${dom}" "imdJJAS${yrs}.nc" "ac_${yrs}.nc"
done
done
I also quoted your variable expansions and changed $dom and $yrs to ${dom} and ${yrs}. The later is done to prevent mistakenly referring to an undefined variable $dom_abc is not the same as ${dom}_abc
If I understand your intention correctly, you are trying to use corresponding values from both arrays. In that case you need a numerical index. for VAR in ARRAY iterates over all values of the array.
-53 45
-54 43
-55 42
-56 41
-57 40
-59 37
-61 35
-61 36
-62 34
-64 33
-65 31
-65 32
-67 30
-68 29
-69 28
-72 25
-73 23
-73 24
-74 22
-76 20
-76 22
-78 20
-79 18
-80 17
-81 16
In the above you will see that at -61 occurs twice and so do some other values. I want to just create a new file without any duplicates. So the new file should have either -61 35 or -61 36 ...
How do I do that?! I tried using sort using uniq but that didn't work.
Assuming your data is in a file called input
cat input | sort -u -n
When doing a numeric (-n) sort along with unique (-u), the duplicate checking is achieved.
If you can guarantee the length of the first field,
sort | uniq --check-chars=4
will do the trick.
Otherwise, try awk:
awk '{ if (FNR == 1 || last != $1) print; last = $1; }'