How to keep values from a messy table based on string characteristics - string

Question

I have a really difficult file.asv (values separated with #) that contains lines with un-matching columns.
Example:
name#age#city#lat#long
eric#paris#4.4283333333333331e+01#-1.0550000000000000e+02
dan#43#berlin#3.1366000000000000e+01#-1.0371500000000000e+02
london##2.5250000000000000e+01#1.0538333000000000e+02
Latitude and Longitude values are pretty consistent. They have 22 or 23 characters (depending on the positive (absent) and negative sign), and always with scientific notation. I would like to keep only latitude and longitude from each line.
Expected output:
lat#long
4.4283333333333331e+01#-1.0550000000000000e+02
3.1366000000000000e+01#-1.0371500000000000e+02
2.5250000000000000e+01#1.0538333000000000e+02
Headers are not totally necessary, I can add them later. I could also work with separated latitude and longitude outputs, and then paste them together. Any sed or awk command I could use?

Answer 1

Use this awk:
awk 'BEGIN{OFS=FS="#"} {print $(NF-1),$NF}' file
Here,
OFS - Output Field Separator
FS - Input Field Separator
NF - Number of Fields
Assuming that latitude and longitude always be a last fields. $NF and $(NF-1) will print last two fields.
Test:
$ awk 'BEGIN{OFS=FS="#"} {print $(NF-1),$NF}' file
lat#long
4.4283333333333331e+01#-1.0550000000000000e+02
3.1366000000000000e+01#-1.0371500000000000e+02
2.5250000000000000e+01#1.0538333000000000e+02

Answer 2

Simple grep would do, assuming -o option is present
$ grep -o '[^#]*#[^#]*$' file.asv
lat#long
4.4283333333333331e+01#-1.0550000000000000e+02
3.1366000000000000e+01#-1.0371500000000000e+02
2.5250000000000000e+01#1.0538333000000000e+02

Answer 3

Trying to select fields using regular expression
$ cat ll.awk
function rep(c, n, ans) { # repeat `c' `n' times
while (n--) ans = ans c
return ans
}
function build_re( d, s, os) { # build a regexp `r'
d = "[0-9]" # a digit
s = "[+-]" # sign
os = s "?" # optional sign
r = os d "[.]" rep(d, 16) "e" s d d # adjust here
r = "^" r "$" # match entire string
}
function process( sep, line, i) {
for (i = 1; i <= NF; i ++ ) {
if (!($i ~ r)) continue # skip fields
line = line sep $i; sep = FS
}
if (length(line)) print line
}
BEGIN {
build_re()
FS = "#"
}
{ # call on every line of input
process()
}
Usage:
$ awk -f ll.awk file.txt
4.4283333333333331e+01#-1.0550000000000000e+02
3.1366000000000000e+01#-1.0371500000000000e+02
2.5250000000000000e+01#1.0538333000000000e+02

Answer 4

One more in Gnu awk using gensub:
$ awk '{print gensub(/(.+)((#[^#]+){2})$/,"\\2","g",$0)}' file
#lat#long
#4.4283333333333331e+01#-1.0550000000000000e+02
#3.1366000000000000e+01#-1.0371500000000000e+02
#2.5250000000000000e+01#1.0538333000000000e+02