I have two files which has following contents
File1
Line1file1
Line2file1
line3file1
line4file1
File2
Line1file2
Line2file2
line3file2
line4file2
I want to have these file's content merged to file3 as
File3
Line1file1
Line1file2
Line2file1
Line2file2
line3file1
line3file2
line4file1
line4file2
How do I merge the files consecutively from one file and another file in bash?
Thanks
You can always use paste command.
paste -d"\n" File1 File2 > File3
$ cat file1
Line1file1
Line2file1
line3file1
line4file1
$ cat file2
Line1file2
Line2file2
line3file2
line4file2
$ paste -d '\n' file1 file2 > file3
$ cat file3
Line1file1
Line1file2
Line2file1
Line2file2
line3file1
line3file2
line4file1
line4file2
paste is the way to go for this, but this alternative can be a useful approach if you ever need to add extra conditions or don't want to end up with blank lines when one file has more lines than the other or anything else that makes it a more complicated problem:
$ awk -v OFS='\t' '{print FNR, NR, $0}' file1 file2 | sort -n | cut -f3-
Line1file1
Line1file2
Line2file1
Line2file2
line3file1
line3file2
line4file1
line4file2
In Linux:
grep -En '.?' File1 File2 | sed -r 's/^[^:]+:([^:]+):(.*)$/\1 \2/g' \
| sort -n | cut -d' ' -f2- > File3
If you're on OS X, use -E instead of -r for the sed command. The idea is this:
Use grep to number the lines of each file.
Use sed to drop the file name and put the line number into a space-separated column.
Use sort -n to sort by the line number, which is stable and preserves file order.
Drop the line number with cut and redirect to the file.
Edit: Using paste is much simpler but will result in blank lines if one of your files is longer than the other, this method will only continue with lines from the longer file.
while read line1 && read -u 3 line2
do
printf "$line1\n" >> File3
printf "$line2\n" >> File3
done < File1 3<File2
you can use file descriptors, to read from two files and print each line to the out file.
Related
I used head -3 to extract headers from some files that I needed to show header data I did this:
head -3 file1 file2 file3
and head -3 * works also.
I thought sed 3 file1 file2 file3 would work but it only gives the first file's output and not the others. I then tried sed -n '1,2p' file1 file2 file3. Again only the first file produced any output. I also tried with a wildcard sed -n '1,2p' filename* same result only the first file's output.
Everything I read seems like it should work. sed *filesnames*.
Thanks in advance
Assuming GNU sed as question is tagged linux. From GNU sed manual
-s
--separate By default, sed will consider the files specified on the command line as a single continuous long stream. This GNU sed
extension allows the user to consider them as separate files: range
addresses (such as ‘/abc/,/def/’) are not allowed to span several
files, line numbers are relative to the start of each file, $ refers
to the last line of each file, and files invoked from the R commands
are rewound at the start of each file.
Example:
$ cat file1
foo
bar
$ cat file2
123
456
$ sed -n '1p' file1 file2
foo
$ sed -n '3p' file1 file2
123
$ sed -sn '1p' file1 file2
foo
123
When using -i, the -s option is implied
$ sed -i '1chello' file1 file2
$ cat file1
hello
bar
$ cat file2
hello
456
Both files has lines of string and numeric data minimum of 2000 lines.
How to add non duplicate data from file2.txt to file1.txt.
Basically file2 has the new data lines but we also want to ensure we are not adding duplicate lines to file1.txt.
File1.txt > this is the main data file
File2.txt > this file has the new data we want to add to file1
thanks,
Sort the two files together with the -u option to remove duplicates.
sort -u File1.txt File2.txt > NewFile.txt && mv NewFile.txt File1.txt
Another option if the file is sorted, just to have some choice (and I like comm :) )
comm --check-order --output-delimiter='' -13 File1.txt File2.txt >> File1.txt
use awk:
awk '!a[$0]++' File1.txt File2.txt
You can use grep, like this:
# grep those lines from file2 which are not in file1
grep -vFf file1 file2 > new_file2
# append the results to file1
cat new_file2 >> file1
I have two files
File1.txt:
docker/registry:2.4.2
docker/rethinkdb:latest
docker/swarm:1.0.0
File2.txt:
docker/registry:2.4.1
docker/rethinkdb:1.0.0
docker/swarm:1.0.0
The output should be:
docker/registry:2.4.2
docker/rethinkdb:latest
In other words, every line in File1 that doesn't exist in File2 should be part of the output.
I have tried doing the following but it is not working.
diff File1.txt File2.txt
You could just use grep for it:
$ grep -v -f file2.txt file1.txt
docker/registry:2.4.2
docker/rethinkdb:latest
If there are lots of rows in the files I'd probably use #user000001 solution.
With awk you can do:
awk 'NR==FNR{a[$0];next}!($0 in a)' file2 file1
With comm:
comm -23 <(sort File1.txt) <(sort File2.txt)
I want to compare two files and redirect the difference between the two files to third one.
file1:
/opt/a/a.sql
/opt/b/b.sql
/opt/c/c.sql
In case any file has # before /opt/c/c.sql, it should skip #
file2:
/opt/c/c.sql
/opt/a/a.sql
I want to get the difference between the two files. In this case, /opt/b/b.sql should be stored in a different file. Can anyone help me to achieve the above scenarios?
file1
$ cat file1 #both file1 and file2 may contain spaces which are ignored
/opt/a/a.sql
/opt/b/b.sql
/opt/c/c.sql
/opt/h/m.sql
file2
$ cat file2
/opt/c/c.sql
/opt/a/a.sql
Do
awk 'NR==FNR{line[$1];next}
{if(!($1 in line)){if($0!=""){print}}}
' file2 file1 > file3
file3
$ cat file3
/opt/b/b.sql
/opt/h/m.sql
Notes:
The order of files passed to awk is important here, pass the file to check - file2 here - first followed by the master file -file1.
Check awk documentation to understand what is done here.
You can use some tools like cat, sed, sort and uniq.
The main observation is this: if the line is in both files then it is not unique in cat file1 file2.
Furthermore in cat file1 file2| sort, all doubles are in sequence. Using uniq -u we get unique lines and have this pipe:
cat file1 file2 | sort | uniq -u
Using sed to remove leading whitespace, empty and comment lines, we get this final pipe:
cat file1 file2 | sed -r 's/^[ \t]+//; /^#/ d; /^$/ d;' | sort | uniq -u > file3
I have about 50 files in a directory
Have
File1: 1|2|3
File2: 3|4|5
File3: A|B|C
WANT
File1: A|1|2|3
File2: A|3|4|5
File3: A|A|B|C
I'll appreciate if anyone can solve it with awk command. I'm open to other solutions in linux. Also, I want to run it once an perform edits on all files in a directory.
The solution (see below) I have will require me to run it on each file one at a time and I don't think that's efficient
awk '{print "A|"$0}' File1
Try the below sed command,
sed -i 's/^/A|/' file1 file2 file3
To make it work on all the files in the current directory,
sed -i 's/^/A|/' *
With GNU awk for -i inplace:
gawk -i inplace '{print "A|"$0}' file1 file2 file3