Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I want to change a text file that contains for example :
a 1
b 2
a 3
b 4
to:
a b
1 2
3 4
any idea how I can accomplish that? I am not familiar with awk but I think it is the way to go?
I'm assuming the input is always two columns, the first column contains
the column headers of the output repeated over and over, and that the
output may contain one or more columns.
$ cat t.awk
{ sep = (FNR % n == 0) ? "\n" : " " }
NR==FNR { printf $1 sep; if (sep == "\n") nextfile; next; }
{ printf $2 sep }
The number of output columns is set with -v when invoking awk. Note also
that the input file needs to be provided twice on the command line as the
script does a quick initial pass to print out the output column headers
before starting over to print the data (too lazy to deal with arrays).
Two-column output:
$ cat file.txt
a 1
b 2
a 3
b 4
a 5
b 6
$ awk -v n=2 -f t.awk file.txt file.txt
a b
1 2
3 4
5 6
Three-column output:
$ cat file2.txt
a 1
b 2
c 3
a X
b Y
c Z
a #
b #
c $
$ awk -v n=3 -f t.awk file2.txt file2.txt
a b c
1 2 3
X Y Z
# # $
Related
This question already has answers here:
combining columns of 2 files using shell script
(1 answer)
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 1 year ago.
I am looking for a simple line of code (if possible) to simply merge two files column wise and save the final result into a new file.
Edited to response to the first answer #heitor:
By using paste file1.csv file2.csv, What happened is:
For instance file 1:
A B
1 2
file2:
C D
3 4
By doing paste -d , file1.csv file2.csv >output.csv I got
A B
C D
1 2
3 4
not
A B C D
1 2 3 4
By doing cat file1.csv file2.csv I got
A B
1 2
C D
3 4
Neither of them is what I want. Any idea?
Any idea?
Use paste -d , to merge the two files and > to redirect the command output to another file:
$ paste -d , file1.csv file2.csv > output.csv
E.g.:
$ cat file1.csv
A,B
$ cat file2.csv
C,D
$ paste -d , file1.csv file2.csv > output.csv
$ cat output.csv
A,B,C,D
-d , tells paste to use , as the delimiter to join the columns.
> tells the shell to write the output of the paste command to the file output.csv
Indeed using paste is pretty simple,
$ cat file1.csv
A B
1 2
$ cat file2.csv
C D
3 4
$ paste -d " " file1.csv file2.csv
A B C D
1 2 3 4
With the -d option I replaced the default tab character with a space.
Edit:
In case you want to redirect that to another file then,
paste -d " " file1.csv file2.csv > file3.csv
$ cat file3.csv
A B C D
1 2 3 4
This question already has answers here:
How to delete rows from a csv file based on a list values from another file?
(3 answers)
Closed 2 years ago.
Now I have two files as follows:
$ cat file1.txt
john 12 65 0
Nico 3 5 1
king 9 5 2
lee 9 15 0
$ cat file2.txt
Nico
king
Now I would like to remove each line which contains a name fron the second file in its first column.
Ideal result:
john 12 65 0
lee 9 15 0
Could anyone tell me how to do that? I have tried the code like this:
for i in 'less file2.txt'; do sed "/$i/d" file1.txt; done
But it does not work properly.
You don't need to iterate it, you just need to use grep with-v option to invert match and -w to force pattern to match only WHOLE words
grep -wvf file2.txt file1.txt
This job suites awk:
awk 'NR == FNR {a[$1]; next} !($1 in a)' file2.txt file1.txt
john 12 65 0
lee 9 15 0
Details:
NR == FNR { # While processing the first file
a[$1] # store the first field in an array a
next # move to next line
}
!($1 in a) # while processing the second file
# if first field doesn't exist in array a then print
This question already has answers here:
Find duplicate lines in a file and count how many time each line was duplicated?
(7 answers)
Closed 7 years ago.
I have file file.txt which look like this
a
b
b
c
c
c
I want to know the command to which get file.txt as input and produces the output
a 1
b 2
c 3
I think uniq is the command you are looking for. The output of uniq -c is a little different from your format, but this can be fixed easily.
$ uniq -c file.txt
1 a
2 b
3 c
If you want to count the occurrences you can use uniq with -c.
If the file is not sorted you have to use sort first
$ sort file.txt | uniq -c
1 a
2 b
3 c
If you really need the line first followed by the count, swap the columns with awk
$ sort file.txt | uniq -c | awk '{ print $2 " " $1}'
a 1
b 2
c 3
You can use this awk:
awk '!seen[$0]++{ print $0, (++c) }' file
a 1
b 2
c 3
seen is an array that holds only uniq items by incrementing to 1 first time an index is populated. In the action we are printing the record and an incrementing counter.
Update: Based on comment below if intent is to get a repeat count in 2nd column then use this awk command:
awk 'seen[$0]++{} END{ for (i in seen) print i, seen[i] }' file
a 1
b 2
c 3
I've got an input file (input.txt) like this:
name value1 value2
A 3 1
B 7 4
C 2 9
E 5 2
And another file with a list of names (names.txt) like so:
B
C
Using grep -f, I can get all the lines with names "B" and "C"
grep -wFf names.txt input.txt
to get
B 7 4
C 2 9
However, I want to keep the header at the top of the output file, and also rename the column name "name" with "ID". And using grep, to keep the rows with names B and C, the output should be:
**ID** value1 value2
B 7 4
C 2 9
I'm thinking awk should be able to accomplish this, but being new to awk I'm not sure how to approach this. Help appreciated!
While it is certainly possible to do this in awk, the fastest way to solve your actual problem is to simply prepend the header you want in front of the grep output.
echo **ID** value1 value2 > Output.txt && grep -wFf names.txt input.txt >> Output.txt
Update Since the OP has multiple files, we can modify the above line to pull the first line out of the input file instead.
head -n 1 input.txt | sed 's/name/ID/' > Output.txt && grep -wFf names.txt input.txt >> Output.txt
Here is how to do it with awk
awk 'FNR==NR {a[$1];next} FNR==1 {$1="ID";print} {for (i in a) if ($1==i) print}' name input
ID value1 value2
B 7 4
C 2 9
Store the names in an array a
Then test filed #1 if it contains data in array a
I want to write shell script which function as below description
Cat file1 and file2 which have x number of rows and column(Equal number of rows and column). Both files where created already.
Script should add(sum value of each element) each row's column element from file1 and file2 and generate output as file3.
file1-:
10,10,10,10
11,11,11,11
file2-:
5,5,5,5
11,11,11
file3 would have output-:
15,15,15,15
22,22,22,22
Since you seem to know x, the number of columns, you can simply sum up explicitly. For example, with x=4:
--- script.sh ---
#!/bin/bash
while true; do
read -r c1 c2 c3 c4 <&3
read -r d1 d2 d3 d4 <&4
if [ -z "$c1" -o -z "$d1" ]; then
break
fi
echo "$(($c1 + $d1)) $(($c2 + $d2)) $(($c3 + $d3)) $(($c4 + $d4))" >>3.txt
done 3<1.txt 4<2.txt
Here's a sample run:
$ ./script.sh && cat 1.txt 2.txt 3.txt
1 2 3 4
5 6 7 8
9 9 9 9
1 1 1 1
1 1 1 1
1 1 1 1
2 3 4 5
6 7 8 9
10 10 10 10
I know you asked for a shell script, but I find this kind of task very easy to achieve using python.
So in case it helps anyone, here's a quick python script. This script supports an arbitrary number of input files (one or more):
#! python
import sys
if len(sys.argv) <= 1:
raise RuntimeError('usage: %s file1 file2' % sys.argv[0])
for lines in zip(sys.args[1:]):
print sum( float(line.strip()) for line in lines )