This question already has answers here:
How to delete duplicate lines in a file without sorting it in Unix
(9 answers)
Closed 4 years ago.
Text file:
1 1
2 2
3 3
1 1
I want to catch 1 1 as duplicated
Your question is not quite clear, but you can filter out duplicate lines with uniq:
sort file.txt | uniq
or simply
sort -u file.txt
(thanks RobEarl)
You can also print only repeating lines with
sort file.txt | uniq -d
One way using GNU awk:
awk 'array[$0]++' file.txt
Results:
1 1
You can use it easily:
sort -u file.txt
OR
awk '!x[$0]++' file.txt
Related
This question already has answers here:
How do I delete a matching line, the line above and the one below it, using sed?
(7 answers)
Closed 1 year ago.
I have a file named file.txt and it contains several lines containing string "NaN". How can I delete lines containing "Nan" and one line before and after it. I know sed －i '/pattern/d' file.txt can delete the matched line, but how can I delete neaby lines of the matched line.
Best regards
This is a most inelegant solution, but works. Perhaps, it will anger a UNIX guru to come and provide a real answer.
grep 'Nan' -n -C 1 target_file.txt | awk -F '[-:]' '{print $2}' | sed '2d' | paste -d, - - | sed 's/$/d/' > del_lines.sed && sed -f del_lines.sed target_file.txt > output_file.txt
fyi, The word "perticular" is misspelled, its "par-" like in golf!
This question already has answers here:
How to find the last field using 'cut'
(14 answers)
Closed 4 years ago.
Hello everyone i want to know to cut the last field with separator :
without knowing how many fields that i have any ideas please .
is there any option for command cut .
You can revert the string and then print 1st character. Itself cut can't work from backwards.
echo "Your string ABC" | rev | cut -c 1
Awk is the right tool for this. Try :
ls -lh | awk '{ print $NF }'
This question already has answers here:
How to use sed/grep to extract text between two words?
(14 answers)
Closed 4 years ago.
how to print text between two specific words using awk, sed ?
$ ofed_info | awk '/MLNX_OFED_LINUX/{print}'
MLNX_OFED_LINUX-4.1-1.0.2.0 (OFED-4.1-1.0.2):
$
Output required:-
4.1-1.0.2.0
Following awk may help you here.(considering that your input to awk will be same as shown sample only)
your_command | awk '{sub(/[^-]*/,"");sub(/ .*/,"");sub(/-/,"");print}'
Solution 2nd: With sed solution now.
your_command | sed 's/\([^-]*\)-\([^ ]*\).*/\2/'
Solution 3rd: Using awk's match utility:
your_command | awk 'match($0,/[0-9]+\.[0-9]+\-[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/){print substr($0,RSTART,RLENGTH)}'
You may use this sed:
echo 'MLNX_OFED_LINUX-4.1-1.0.2.0 (OFED-4.1-1.0.2):' |
sed -E 's/^[^-]*-| .*//g'
4.1-1.0.2.0
This sed command removes text till first hyphen from start or text starting with space towards end.
Try this:
ofed_info | sed -n 's/^MLNX_OFED_LINUX-\([^ ]\+\).*/\1/p'
The sed command only selects lines starting with the keyword and prints the version attached to it.
This question already has answers here:
Sort files numerically in bash
(3 answers)
Closed 8 years ago.
I've been trying to sort my filenames with commands similar to ls -1 | sort -n -t "_" -k1 but just can't get it to work. Please help.
I have:
10_filename
11_filename
12_filename
1_filename
2_filename
I want to get:
1_filename
2_filename
...
10_filename
11_filename
Please try following this will solve the issue
ls -1v
-v It sorts on basis of file version versions
Try this,
ls -1 *\_filename | sort -n
or
ls -1 | sort -n
ls -1 | sort -t '_' +1 +0n
below, a bit heavy but working if sort does not accept field order and using simple string sort.
ls -1 | sed 's/^\([0-9]*\)_\(.*\)/000\1_\1_\2/;s/^0*\([0-9]\{3\}\)/\1/;s/\([0-9]\{1,\}_[0-9]\{1,\}_\)\(.*\)/\2_\1/' | sort -n | sed 's/\(.*\)_[0-9]\{1,\}_\([0-9]\{1,\}\)_$/\2_\1/'
This question already has answers here:
Sort a text file by line length including spaces
(13 answers)
Closed 9 years ago.
I'm new to linux and checking to see if there is any sort command available for below scenario. I have a file containing lines like below.
this is a 10
this is 5
this 40
this is a boy in 3
this is a boy 6
I would like to sort it like
this 40
this is 5
this is a 10
this is a boy 6
this is a boy in 3
Any help is appreciated!
Assuming all of your data is in a text file called testfile this answer should work:
cat testfile | awk '{ print length, $0 }' | sort -n | cut -d" " -f2-
It comes from this question: Sort a text file by line length including spaces
If you don't care about blank lines you can run
$ sort <filename>
this 40
this is 5
this is a 10
this is a boy 6
this is a boy in 3
sort can read a file and organize the output. If you want to remove the empty lines, one option you have is sed
$ sort test.txt | sed '/^$/d'
this 40
this is 5
this is a 10
this is a boy 6
this is a boy in 3