Im begining with bash and I want to find my ip in a .txt file analyzing it.
This is an example of part of the file:
"Direc. inet:192.****** Difus.:"
The path I think on is searching all the text between "inet:" and " ". My biggest approach until now is getting the entire line with "grep inet:" but I can't figure out how to get just the ip not the entire line with the ip.
Thank you!
Perl to the rescue:
perl -ne 'print $1, "\n" if /inet:([^ ]+)/'
-n reads the input line by line;
[^ ] matches a character that isn't a space
+ means the character must be present one or more times
(...) creates a capture group, the first capture group is referenced as $1
Since you're on Linux, you can take advantage of GNU grep's -o and -P options:
grep -oP '(?<= inet:)[^ ]+' file.txt
Example:
$ grep -oP '(?<= inet:)[^ ]+' <<<'Direc. inet:192.****** Difus.:'
192.******
-o tells grep to only output the matching part(s) of each line.
-P activates support for PCREs (Perl-compatible regular expressions), which support look-behind assertions such as (?<= inet:), which allow a sub-expression (inet:, in this case) to participate in matching, without being captured (returned) as part of the matched string.
[^ ]+ then simply captures everything after inet: up to the first space char. (character set [^ ] matches any char. that is not (^) a space, and + matches this set 1 or more times).
Try combination of awk and grep. Below solution may help Link 1 .
Lin 2
Related
I am trying to fetch the output of a variable stored in a file in another shell script.
Example:
cat abc.log
var1=2
var2=2
var3=25
I am writing a script to fetch the value of var3.
Thank you in advance.
awk -F= '$1 ~ /^[[:space:]]*var3/ { print $2 }' abc.log
Set the field delimiter to = and then where the line contains "var3", print the second field.
Alternatively, you could:
source abc.log
and then:
echo $var3
Using sed you can isolate 25 with particularity with:
sed -n '/^[[:space:]]*var3=/s/^[^=]*=//p' file
Explanation
This is the general substitution form s/find/replace/ with a matching expression preceding it. The total form is /match/s/find/replace/. The option -n suppresses the normal printing of pattern-space and the p at the end tells sed to print the line where the match and substitution took place. Specifically,
/match/ locates a line with any number of preceding whitespace characters followed by var3=. The POSIX [:space:] character class matches any whitespace,
the /find/ is all characters anchored from the '^' beginning that are not the [^=] character and then match the literal '=' character, and finally
the /replace/ is the empty-string leaving the 25 alone which is printed.
Example Use/Output
$ sed -n '/^[[:space:]]*var3=/s/^[^=]*=//p' file
25
A grep one-liner, if your grep has support for Perl-compatible regular expressions (the -P option; not all greps support that)
grep -Po '^\s*var3=\K.*' abc.log
or,
grep -Po '^\s*var3=\K.*' abc.log | tail -n1
in order to get the last value of the var3, if multiple var3s is a possibility.
I haven't found anything that clearly answers my question. Although very close, I think...
I have a file with a line:
# Skipsdata for serienummer 1158
I want to extract the 4 digit number at the end and put it into a variable, this number changes from file to file so I can't just search for "1158". But the "# Skipsdata for serienummer" always remains the same.
I believe that either grep, sed or awk may be the answer but I'm not 100 % clear on their usage.
Using Awk as
numberRequired=$(awk '/# Skipsdata for serienummer/{print $NF}' file)
printf "%s\n" "$numberRequired"
1158
You can use grep with the -o switch, which prints only the matched part instead of the whole line.
Print all numbers at the end of lines from file yourFile
grep -Po '\d+$' yourFile
Print all four digit numbers at the end of lines like described in your question:
grep -Po '^# Skipsdata for serienummer \K\d{4}$' yourFile
-P enables perl style regexes which support \d and especially \K.
\d matches any digit (0-9).
\d{4} matches exactly four digits.
\K lets grep forget the previously matched part, such that only the part afterwards is printed.
There are multiple ways to find your number. Assuming the input data is in a file called inputfile:
mynumber=$(sed -n 's/# Skipsdata for serienummer //p' <inputfile) will print only the number and ignore all the other lines;
mynumber=$(grep '^# Skipsdata for serienummer' inputfile | cut -d ' ' -f 5) will filter the relevant lines first, then only output the 5th field (the number)
I have a log which got like this :
.....client connection.....remote=/xxx.xxx.xxx.xxx]].......
I need to extract all lines in the log which contain the above,and print just the ip after remote=.. This would be something in the pattern :
grep "client connection" xxx.log | sed -e ....
Using grep:
grep -oP '(?<=remote=/)[^\]]+' file
o is to extract only the pattern, instead of entire line.
P is to match perl like regex. In this case, we are using "negative look behind". It will try to match set of characters which is not "]" which is preceeded by remote=/
grep -oP 'client connection.*remote=/\K.*?(?=])' input
Prints anything between remote=/ and closest ] on the lines which contain client connection.
Or by using sed back referencing: Here the line is divided into three parts/groups which are later referred by \1 \2 or \3. Each group is enclosed by ( and ). Here IP address belongs to 2nd group, so whole line is replaced by 2nd group which is IP address.
sed -r '/client connection/ s_(^.*remote=/)(.*?)]](.*)_\2_g' input
Or using awk :
awk -F'/|]]' '/client connection/{print $2}' input
Try this:
grep 'client connection' test.txt | awk -F'[/\\]]' '{print $2}'
Test case
test.txt
---------
abcd
.....client connection.....remote=/10.20.30.40]].......
abcs
.....client connection.....remote=/11.20.30.40]].......
.....client connection.....remote=/12.20.30.40]].......
Result
10.20.30.40
11.20.30.40
12.20.30.40
Explanation
grep will shortlist the results to only lines matching client connection. awk uses -F flag for delimiter to split text. We ask awk to use / and ] delimiters to split text. In order to use more than one delimiter, we place the delimiters in [ and ]. For example, to split text by = and :, we'd do [=:].
However, in our case, one of the delimiters is ] since my intent is to extract IP specifically from /x.x.x.x] by spitting the text with / and ]. So we escape it ]. The IP is the 2nd item from the splitting.
A more robust way, improved over this answer would be to also use GNU grep in PCRE mode with -P for perl style regEx match, but matching both the patterns as suggested in the question.
grep -oP "client connection.*remote=/\K(\d{1,3}\.){3}\d{1,3}" file
10.20.30.40
11.20.30.40
12.20.30.40
Here, client connection.*remote matches both the patterns in the lines and extracts IP from the file. The \K is a PCRE syntax to ignore strings up to that point and print only the capture group following it.
(\d{1,3}\.){3}\d{1,3}
To match the IP i.e. 3 groups of digits separated by dots of length from 1 to 3 followed by 4th octet.
I have a file like
key\0value\n
akey\0value\n
key2\0value\n
I have to create a script that take as argument a word. I have to return every lines having a key exactly the same than the argument.
I tried
grep -aF "$key\x0"
but grep seems to do not understand the \x0 (\0 same result). Futhermore, I have to check that the line begins with "$key\0"
I only can use sed grep and tr and other no maching commands
To have the \0 taken into account try :
grep -Pa "^key\x0"
it works for me.
Using sed
sed will work:
$ sed -n '/^key1\x00/p' file
key1value
The use of \x00 to represent a hex character is a GNU extension to sed. Since this question is tagged linux, that is not a problem.
Since the null character does not display well, one might (or might not) want to improve the display with something like this:
$ sed -n 's/^\(akey\)\x00/\1-->/p' file
akey-->value
Using sed with keys that contain special characters
If the key itself can contain sed or shell active characters, then we must escape them first and then run sed against the input file:
#!/bin/bash
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
sed -n "$script" file
To use this script, simply supply the key as the first argument on the command line, enclosed in single-quotes, of course, to prevent shell processing.
To see how it works, let's look at the pieces in turn:
sed 's:[]\[^$.*/]:\\&:g' <<<"$1"
This puts a backslash escape in front of all sed-active characters.
printf -v script '/^%s\\x00/p' "$(sed 's:[]\[^$.*/]:\\&:g' <<<"$1")"
This creates a sed command using the escaped key and stores it in the shell variable script.
sed -n "$script" file
This runs sed using the shell variable script as the sed command.
Using awk
The question states that awk is not an acceptable tool. For completeness, though, here is an awk solution:
$ awk -F'\x00' -v k=key1 '$1 == k' file
key1value
Explanation:
-F'\x00'
awk divides the input up into records (lines) and divides the records up into fields. Here, we set the field separator to the null character. Consequently, the first field, denoted $1, is the key.
-v k=key1
This creates an awk variable, called k, and sets it to the key that we are looking for.
$1 == k
This statement looks for records (lines) for which the first field matches our specified key. If a match is found, the line is printed.
I have long file with the following list:
/drivers/isdn/hardware/eicon/message.c//add_b1()
/drivers/media/video/saa7134/saa7134-dvb.c//dvb_init()
/sound/pci/ac97/ac97_codec.c//snd_ac97_mixer_build()
/drivers/s390/char/tape_34xx.c//tape_34xx_unit_check()
(PROBLEM)/drivers/video/sis/init301.c//SiS_GetCRT2Data301()
/drivers/scsi/sg.c//sg_ioctl()
/fs/ntfs/file.c//ntfs_prepare_pages_for_non_resident_write()
/drivers/net/tg3.c//tg3_reset_hw()
/arch/cris/arch-v32/drivers/cryptocop.c//cryptocop_setup_dma_list()
/drivers/media/video/pvrusb2/pvrusb2-v4l2.c//pvr2_v4l2_do_ioctl()
/drivers/video/aty/atyfb_base.c//aty_init()
/block/compat_ioctl.c//compat_blkdev_driver_ioctl()
....
It contains all the functions in the kernel code. The notation is file//function.
I want to copy some 100 files from the kernel directory to another directory, so I want to strip every line from the function name, leaving just the filename.
It's super-easy in python, any idea how to write a 1-liner in the bash prompt that does the trick?
Thanks,
Udi
cat "func_list" | sed "s#//.*##" > "file_list"
Didn't run it :)
You can use pure Bash:
while read -r line; do echo "${line%//*}"; done < funclist.txt
Edit:
The syntax of the echo command is doing the same thing as the sed command in Eugene's answer: deleting the "//" and everything that comes after.
Broken down:
"echo ${line}" is the same as "echo $line"
the "%" deletes the pattern that follows it if it matches the trailing portion of the parameter
"%" makes the shortest possible match, "%%" makes the longest possible
"//*" is the pattern to match, "*" is similar to sed's ".*"
See the Parameter Expansion section of the Bash man page for more information, including:
using ${parameter#word} for matching the beginning of a parameter
${parameter/pattern/string} to do sed-style replacements
${parameter:offset:length} to retrieve substrings
etc.
here's a one liner in (g)awk
awk -F"//" '{print $1}' file
Here's one using cut and rev
cat file | rev | cut -d'/' -f2-| rev