I am running a script that has been working fine. However, yesterday, I got a couple errors. These errors are after several loops of the script:
sed: cant read file3.txt: No such file or directory
grep: file3.txt: No such file or directory
grep: file3.txt: No such file or directory
sed: cant read file3.txt: No such file or directory
grep: file3.txt: No such file or directory
Keep in mind, these errors do not happen consistently. It's occurring once in a while somewhere near this part of the script. File3.txt is the file not being found:
cat file1.txt | while read LINE; do grep -m 1 $LINE file2.txt >> file3.txt; done
sed -i 's/string//g' file3.txt
grep 'string' file3.txt | cut -d '|' -f1-2 > file4.txt
grep -v 'string' file3.txt | cut -d '|' -f1-2 >> file5.txt
sed -i 's/string//' file3.txt
grep -Fvf file3.txt file1.txt > file6.txt
Now, I'm thinking that since file3.txt is being appended, or later operated on by SED, sometimes the next command starts too soon and it can't find the file? Should I put a wait command in between?
I have looked up many pages with this error, but was unable to find anything:
cat file_name | grep "something" results "cat: grep: No such file or directory" in shell scripting
Pipe multiple commands to a single command with no EOF signal wait
grep command works in command line, but not in bash script: get no such file or directory erro
https://serverfault.com/questions/169539/sed-cant-find-a-file-that-obviously-exists
"No such file or directory" but it exists
If you think that putting a wait or sleep command will help, please let me know. Or, if you think there's a better solution, that would be great too. I'm running on Cygwin terminal. Any insight is greatly appreciated.
Instead of redirecting to file3.txt inside the while loop, redirect the whole loop. Then the file will be created even if the loop never runs because the input file is empty.
while read LINE; do
grep -m 1 $LINE file2.txt
done < file1.txt > file3.txt
If file1.txt is ever empty then file3.txt won't be created.
Also do grep -m 1 $LINE file2.txt will cause problems if there are crucial characters (space is the easiest of them).
Let's assume that the $LINE variable contains more than one word separated by spaces: hello world.
Now the command looks like this: grep -m 1 hello world file2.txt - grep interpretation will look something like this: let's find all hello in file named world and file named file2.txt in current folder.
Using "$LINE" instead of $LINE will lead you to a whole different scenario.
Look at the difference between the following two:
grep -m 1 $LINE file2.txt
grep -m 1 "$LINE" file2.txt
Related
I am trying to make a shell script which prints out the last modification dates of the following files.
Somehow the script just prints out an empty line
"modified" is a file which contains the names and the modification dates of the files in the following format(delimiter='#'):
>modified
for i in file1 file2 file3
do
echo $i#`stat --printf='%y\n' $i`>>modified
done
Having created that file, I'm trying to search it like:
for i in file1 file2 file3
do
var=`grep -w "$i" modified | cut -d'#' -f2`
echo $var
done
As mentioned by Charles, there's no reason to create that modified file for that (unless you are planning to use that file for another purpose).
Also, you can give different arguments to your stat command, as in:
stat --printf='%y\n' file1 file2 file3
This gives exactly the same output as what you're aiming for.
This question already has answers here:
How can I use a file in a command and redirect output to the same file without truncating it?
(14 answers)
Closed 3 years ago.
I'm testing basic bash command on Ubuntu, when I put on terminal
wc -l < file1.txt > file2.txt
all lines from file1 are read by wc and the number is saved to file2.txt.
However if I use the similar command wc -l < file1.txt > file1.txt but I try to save it in the same > file1 the output is always 0.
What is the reason of this behavior ?
When you write the command wc -l < file1.txt > file1.txt, your shell will first read the command as a whole and redirect the standard input, output from/to file1.txt.
Then when a file is open for writing using >, if a file exists it will be overwritten and an empty file is created. the process (your wc) will then start its execution and it will write in it until the end of the execution.
Your wc -l will however have to read the file and count the number of lines in it, as the file is empty (overwritten) it will just dump in it:
0 file1.txt
To prove this look at what happens in appending mode using >>
$ cat file1.txt
a
b
b
$ wc -l file1.txt
3 file1.txt
$ wc -l file1.txt >> file1.txt
$ cat file1.txt
a
b
b
3 file1.txt
Your file content is intact and wc does properly read the number of lines before appending the result to the file.
Note:
In general, it is never recommended to write and read in the same file except if you know exactly what you are doing. Some commands have an inline option to modify the file during the run of the command and it is highly recommended to use them, as the file will not be corrupted even if the command crash in the middle of the execution.
Its because redirections have higher priority than other commands.
Here the file1 will be emptied before any operations can be performed on it
Try >> on file1.txt instead of >. >> will append to file unlike > overwriting whole file. Try -
wc -l < file1.txt >> file1.txt
I am looking to compare file1.txt contents with the last n file contents of file2.txt Can someone help in identifying this logic using anything in shell script.
Example if file1.txt has 10 lines, the last 10 lines of file2.txt should be compared for difference.
With bash's process substitution <() and command substitution $().
diff file1.txt <(tail -n $(wc -l < file1.txt) file2.txt)
echo 'Sum'
awk '!found[$1]++' file.txt file2.txt | sort -n
echo 'A\B:'
sort file2.txt file2.txt file.txt | uniq -u
It just prints like 3 errors, file not found, command not found, 'ort wrong option', 'niq wrong option', whatever 'niq' and 'ort' are. Does it have some syntax errors or i need to install some packages? It all worked fine on cygwin on Windows.
I have file.txt with names one per line as shown below:
ABCB8
ABCC12
ABCC3
ABCC4
AHR
ALDH4A1
ALDH5A1
....
I want to grep each of these from an input.txt file.
Manually i do this one at a time as
grep "ABCB8" input.txt > output.txt
Could someone help to automatically grep all the strings in file.txt from input.txt and write it to output.txt.
You can use the -f flag as described in Bash, Linux, Need to remove lines from one file based on matching content from another file
grep -o -f file.txt input.txt > output.txt
Flag
-f FILE, --file=FILE:
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-o, --only-matching:
Print only the matched (non-empty) parts of a matching line, with
each such part on a separate output line.
for line in `cat text.txt`; do grep $line input.txt >> output.txt; done
Contents of text.txt:
ABCB8
ABCC12
ABCC3
ABCC4
AHR
ALDH4A1
ALDH5A1
Edit:
A safer solution with while read:
cat text.txt | while read line; do grep "$line" input.txt >> output.txt; done
Edit 2:
Sample text.txt:
ABCB8
ABCB8XY
ABCC12
Sample input.txt:
You were hired to do a job; we expect you to do it.
You were hired because ABCB8 you kick ass;
we expect you to kick ass.
ABCB8XY You were hired because you can commit to a rational deadline and meet it;
ABCC12 we'll expect you to do that too.
You're not someone who needs a middle manager tracking your mouse clicks
If You don't care about the order of lines, the quick workaround would be to pipe the solution through a sort | uniq:
cat text.txt | while read line; do grep "$line" input.txt >> output.txt; done; cat output.txt | sort | uniq > output2.txt
The result is then in output.txt.
Edit 3:
cat text.txt | while read line; do grep "\<${line}\>" input.txt >> output.txt; done
Is that fine?