Awk to print each file - linux

I have about 50 files in a directory
Have
File1: 1|2|3
File2: 3|4|5
File3: A|B|C
WANT
File1: A|1|2|3
File2: A|3|4|5
File3: A|A|B|C
I'll appreciate if anyone can solve it with awk command. I'm open to other solutions in linux. Also, I want to run it once an perform edits on all files in a directory.
The solution (see below) I have will require me to run it on each file one at a time and I don't think that's efficient
awk '{print "A|"$0}' File1


Try the below sed command,
sed -i 's/^/A|/' file1 file2 file3
To make it work on all the files in the current directory,
sed -i 's/^/A|/' *


With GNU awk for -i inplace:
gawk -i inplace '{print "A|"$0}' file1 file2 file3

Related

How to combine multiple files in linux with delimiter seperation?

I'm trying to combine multiple files to 1 file using cat command.
However I wish to add a separation line like "----" in between the file contents.
Is there a way we can achieve this with cat or any other tool?
cat file1 file2 file3 file4 > newfile

you can use the following command for combining multiple files with --- delimiter.
awk 'FNR==1 && NR!=1 {print "---"}{print}' file1 file2 > newfile
command is copied from this post of Unix stack excahnge
https://unix.stackexchange.com/questions/163782/combine-two-text-files-with-adding-some-separator-between

Using Sed to extract the headers in multiple files

I used head -3 to extract headers from some files that I needed to show header data I did this:
head -3 file1 file2 file3
and head -3 * works also.
I thought sed 3 file1 file2 file3 would work but it only gives the first file's output and not the others. I then tried sed -n '1,2p' file1 file2 file3. Again only the first file produced any output. I also tried with a wildcard sed -n '1,2p' filename* same result only the first file's output.
Everything I read seems like it should work. sed *filesnames*.
Thanks in advance

Assuming GNU sed as question is tagged linux. From GNU sed manual
-s
--separate By default, sed will consider the files specified on the command line as a single continuous long stream. This GNU sed
extension allows the user to consider them as separate files: range
addresses (such as ‘/abc/,/def/’) are not allowed to span several
files, line numbers are relative to the start of each file, $ refers
to the last line of each file, and files invoked from the R commands
are rewound at the start of each file.
Example:
$ cat file1
foo
bar
$ cat file2
123
456
$ sed -n '1p' file1 file2
foo
$ sed -n '3p' file1 file2
123
$ sed -sn '1p' file1 file2
foo
123
When using -i, the -s option is implied
$ sed -i '1chello' file1 file2
$ cat file1
hello
bar
$ cat file2
hello
456

Getting values in a file which is not present in another file

I have two files
File1.txt:
docker/registry:2.4.2
docker/rethinkdb:latest
docker/swarm:1.0.0
File2.txt:
docker/registry:2.4.1
docker/rethinkdb:1.0.0
docker/swarm:1.0.0
The output should be:
docker/registry:2.4.2
docker/rethinkdb:latest
In other words, every line in File1 that doesn't exist in File2 should be part of the output.
I have tried doing the following but it is not working.
diff File1.txt File2.txt

You could just use grep for it:
$ grep -v -f file2.txt file1.txt
docker/registry:2.4.2
docker/rethinkdb:latest
If there are lots of rows in the files I'd probably use #user000001 solution.

With awk you can do:
awk 'NR==FNR{a[$0];next}!($0 in a)' file2 file1

With comm:
comm -23 <(sort File1.txt) <(sort File2.txt)

shell script to compare two files and write the difference to third file

I want to compare two files and redirect the difference between the two files to third one.
file1:
/opt/a/a.sql
/opt/b/b.sql
/opt/c/c.sql
In case any file has # before /opt/c/c.sql, it should skip #
file2:
/opt/c/c.sql
/opt/a/a.sql
I want to get the difference between the two files. In this case, /opt/b/b.sql should be stored in a different file. Can anyone help me to achieve the above scenarios?

file1
$ cat file1 #both file1 and file2 may contain spaces which are ignored
/opt/a/a.sql
/opt/b/b.sql
/opt/c/c.sql
/opt/h/m.sql
file2
$ cat file2
/opt/c/c.sql
/opt/a/a.sql
Do
awk 'NR==FNR{line[$1];next}
{if(!($1 in line)){if($0!=""){print}}}
' file2 file1 > file3
file3
$ cat file3
/opt/b/b.sql
/opt/h/m.sql
Notes:
The order of files passed to awk is important here, pass the file to check - file2 here - first followed by the master file -file1.
Check awk documentation to understand what is done here.

You can use some tools like cat, sed, sort and uniq.
The main observation is this: if the line is in both files then it is not unique in cat file1 file2.
Furthermore in cat file1 file2| sort, all doubles are in sequence. Using uniq -u we get unique lines and have this pipe:
cat file1 file2 | sort | uniq -u
Using sed to remove leading whitespace, empty and comment lines, we get this final pipe:
cat file1 file2 | sed -r 's/^[ \t]+//; /^#/ d; /^$/ d;' | sort | uniq -u > file3

How to search for lines from one file in another in linux

I have file1 and file2. I need to search and print out if any line in file1 exists in file2. If not exists, I need to know there is no entry exists by any way. How can I achieve this in linux command ?

Just need grep:
grep -f file1 file2
If you want process file1 as fixed string, not regex pattern, parameter -F need be added.
grep -F -f file1 file2

grep can do this task.
grep -f File1 File2
However other commands like diff and cmp can also be used.

Resources