I am new to the Linux Command Line and truth be told, I am a little intimidated be all the commands. I would like to add a line between find. What commands are there for? There is a lot out there about files but nothing about the content.
cd /applbeh/cr/acr/xfb/send/
find Mt940 -type f -iname '*MT940*' | xargs -r rm -v
find Mt940 -type f -iname '*OVK*' | xargs -r rm -v
Thanks
A oneliner to add a line to a file is:
echo 'line' >> file
With line (without the quotes), the content of the line you wish to add and file the name of the file you wish to write to.
>> is an IO redirection that means append, it means you add the given output (echo 'line') thus line to the file.
Related
So I've a file called allTextFiles.txt in which it has paths of a all regular text file.
eg:
./test/file1.txt
./test/file2.txt
./test/file3.txt
My task is to formulate shell command such that command CAT will go through all of these paths and display content of each file.
Is it possible ?
Using xargs with the allTextFiles.txt
You can use the command cat to list the content and xargs (Xargs command documentation) to process every line of the file. For example:
cat allTextFiles.txt | xargs cat
kk #./test/file1.txt
jj #./test/file2.txt
yy #./test/file3.txt
Using find command without allTextFiles.txt
You can use the command find (find command documentation) to go trough the main folder and search recursively. Once you find a file, to use the cat command.
You achieve the purpose of showing all the content of txt files recursively with only one command.
find . -type f -name "*.txt" -exec cat {} \;
kk #./test/file1.txt
jj #./test/file2.txt
yy #./test/file3.txt
Where the . means current directory. -type f only filters files. -name "*.txt" filters only files ending with .txt extension and the -exec part is where you process all the files found.
Does it cover your need?
You can use a loop; something like
while IFS= read -r file; do
cat "$file"
done < allTextFiles.txt
See Bash FAQ 001 for more discussion on reading a file a line at a time (Much of the content applies to all shells, not just bash).
If you are using bash:
readarray -t files < allTextFiles.txt
cat "${files[#]}"
is an alternative that avoids the explicit loop.
I have a line to be added to 3rd line of all files in directory. What's the commandline to do this operation
Lets say I want to add "color #c2451 " to 3rd line of files in Class directory
Try and use the find cmd piped into xargs and the sed cmd.
You'd have to cd into the directory first with the files.
find . -type f -name '*' | xargs sed -i "3i color #c2451"
Add text to file at certain line in Linux
Change multiple files
In my hierarchy of directories I have many text files called STATUS.txt. These text files each contain one keyword such as COMPLETE, WAITING, FUTURE or OPEN. I wish to execute a shell command of the following form:
./mycommand OPEN
which will list all the directories that contain a file called STATUS.txt, where this file contains the text "OPEN"
In future I will want to extend this script so that the directories returned are sorted. Sorting will determined by a numeric value stored the file PRIORITY.txt, which lives in the same directories as STATUS.txt. However, this can wait until my competence level improves. For the time being I am happy to list the directories in any order.
I have searched Stack Overflow for the following, but to no avail:
unix filter by file contents
linux filter by file contents
shell traverse directory file contents
bash traverse directory file contents
shell traverse directory find
bash traverse directory find
linux file contents directory
unix file contents directory
linux find name contents
unix find name contents
shell read file show directory
bash read file show directory
bash directory search
shell directory search
I have tried the following shell commands:
This helps me identify all the directories that contain STATUS.txt
$ find ./ -name STATUS.txt
This reads STATUS.txt for every directory that contains it
$ find ./ -name STATUS.txt | xargs -I{} cat {}
This doesn't return any text, I was hoping it would return the name of each directory
$ find . -type d | while read d; do if [ -f STATUS.txt ]; then echo "${d}"; fi; done
... or the other way around:
find . -name "STATUS.txt" -exec grep -lF "OPEN" \{} +
If you want to wrap that in a script, a good starting point might be:
#!/bin/sh
[ $# -ne 1 ] && echo "One argument required" >&2 && exit 2
find . -name "STATUS.txt" -exec grep -lF "$1" \{} +
As pointed out by #BroSlow, if you are looking for directories containing the matching STATUS.txt files, this might be more what you are looking for:
fgrep --include='STATUS.txt' -rl 'OPEN' | xargs -L 1 dirname
Or better
fgrep --include='STATUS.txt' -rl 'OPEN' |
sed -e 's|^[^/]*$|./&|' -e 's|/[^/]*$||'
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# simulate `xargs -L 1 dirname` using `sed`
# (no trailing `\`; returns `.` for path without dir part)
Maybe you can try this:
grep -rl "OPEN" . --include='STATUS.txt'| sed 's/STATUS.txt//'
where grep -r means recursive , -l means only list the files matching, '.' is the directory location. You can pipe it to sed to remove the file name.
You can then wrap this in a bash script file where you can pass in keywords such as 'OPEN', 'FUTURE' as an argument.
#!/bin/bash
grep -rl "$1" . --include='STATUS.txt'| sed 's/STATUS.txt//'
Try something like this
find -type f -name "STATUS.txt" -exec grep -q "OPEN" {} \; -exec dirname {} \;
or in a script
#!/bin/bash
(($#==1)) || { echo "Usage: $0 <pattern>" && exit 1; }
find -type f -name "STATUS.txt" -exec grep -q "$1" {} \; -exec dirname {} \;
You could use grep and awk instead of find:
grep -r OPEN * | awk '{split($1, path, ":"); print path[1]}' | xargs -I{} dirname {}
The above grep will list all files containing "OPEN" recursively inside you dir structure. The result will be something like:
dir_1/subdir_1/STATUS.txt:OPEN
dir_2/subdir_2/STATUS.txt:OPEN
dir_2/subdir_3/STATUS.txt:OPEN
Then the awk script will split this output at the colon and print the first part of it (the dir path).
dir_1/subdir_1/STATUS.txt
dir_2/subdir_2/STATUS.txt
dir_2/subdir_3/STATUS.txt
The dirname will then return only the directory path, not the file name, which I suppose it what you want.
I'd consider using Perl or Python if you want to evolve this further, though, as it might get messier if you want to add priorities and sorting.
Taking up the accepted answer, it does not output a sorted and unique directory list. At the end of the "find" command, add:
| sort -u
or:
| sort | uniq
to get the unique list of the directories.
Credits go to Get unique list of all directories which contain a file whose name contains a string.
IMHO you should write a Python script which:
Examines your directory structure and finds all files named STATUS.txt.
For each found file:
reads the file and executes mycommand depending on what the file contains.
If you want to extend the script later with sorting, you can find all the interesting files first, save them to a list, sort the list and execute the commands on the sorted list.
Hint: http://pythonadventures.wordpress.com/2011/03/26/traversing-a-directory-recursively/
The following bash script finds a .txt file from the given directory path, then changes one word (change mountain to sea) from the .txt file
#!/bin/bash
FILE=`find /home/abc/Documents/2011.11.* -type f -name "abc.txt"`
sed -e 's/mountain/sea/g' $FILE
The output I am getting is ok in this case.
My problem is if I want to give the directory path as command line argument then it is not working. Suppose, I modify my bash script to:
#!/bin/bash
FILE=`find $1 -type f -name "abc.txt"`
sed -e 's/mountain/sea/g' $FILE
and invoke it like:
./test.sh /home/abc/Documents/2011.11.*
Error is:
./test.sh: line 2: /home/abc/Documents/2011.11.10/abc.txt: Permission denied
Can anybody suggest how to access directory path as command line argument?
Your first line should be:
FILE=`find "$#" -type f -name "abc.txt"`
The wildcard will be expanded before calling the script, so you need to use "$#" to get all the directories that it expands to and pass these as the arguments to find.
You don't need to pass .* to your script.
Have your script like this:
#!/bin/bash
# some sanity checks here
path="$1"
find "$path".* -type f -name "abc.txt" -exec sed -i.bak 's/mountain/sea/g' '{}' \;
And run it like:
./test.sh "/home/abc/Documents/2011.11"
PS: See how sed can be invoked directly from find itself using -exec option.
I need to traverse a directory so starting in one directory and going deeper into difference sub directories. However I also need to be able to have access to each individual file to modify the file. Is there already a command to do this or will I have to write a script? Could someone provide some code to help me with this task? Thanks.
The find command is just the tool for that. Its -exec flag or -print0 in combination with xargs -0 allows fine-grained control over what to do with each file.
Example: Replace all foo's by bar's in all files in /tmp and subdirectories.
find /tmp -type f -exec sed -i -e 's/foo/bar/' '{}' ';'
for i in `find` ; do
if [ -d $i ] ; then do something with a directory ; fi
if [ -f $i ] ; then do something with a file etc. ; fi
done
This will return the whole tree (recursively) in the current directory in a list that the loop will go through.
This can be easily achieved by mixing find, xargs, sed (or other file modification command).
For example:
$ find /path/to/base/dir -type f -name '*.properties' | xargs sed -ie '/^#/d'
This will filter all files with file extension .properties.
The xargs command will feed the file path generated by find command into the sed command.
The sed command will delete all lines start with # in the files (feed by xargs).
Command combination in this way is very flexible.
For example, find command have different parameters so you can filter by user name, file size, file path (eg: under /test/ subfolder), file modification time.
Another dimension of flexibility is how and what to change in your file. For ex, sed command allows you to make changes on file in applying substitution (specify via regular expressions). Similarly, you can use gzip to compress the file. And so on ...
You would usually use the find command. On Linux, you have the GNU version, of course. It has many extra (and useful) options. Both will allow you to execute a command (eg a shell script) on the files as they are found.
The exact details of how to make changes to the file depend on the change you want to make to the file. That is probably best scripted, with find running the script:
POSIX or GNU:
find . -type f -exec your_script '{}' +
This will run your script once for a group of files with those names provided as arguments. If you want to do it one file at a time, replace the + with ';' (or \;).
I am assuming SearchMe is the example directory name you need to traverse completely.
I am also assuming, since it was not specified, the files you want to modify are all text file. Is this correct?
In such scenario I would suggest using the command:
find SearchMe -type f -exec vi {} \;
If you are not familiar with vi editor, just use another one (nano, emacs, kate, kwrite, gedit, etc.) and it should work as well.
Bash 4+
shopt -s globstar
for file in **
do
if [ -f "$file" ];then
# do some processing to your file here
# where the find command can't do conveniently
fi
done