Sed and grep in multiple files - linux

I want to use "sed" and "grep" to search and replace in multiples files, excluding some directories.
I run this command:
$ grep -RnI --exclude-dir={node_modules,install,build} 'chaine1' /projets/ | sed -i 's/chaine1/chaine2/'
I get this message:
sed: pas de fichier d'entrée
I also tried with these two commands:
$ grep -RnI --exclude-dir={node_modules,install,build} 'chaine1' . | xargs -0 sed -i 's/chaine2/chaine2/'
$ grep -RnI --exclude-dir={node_modules,install,build} 'chaine2' . -exec sed -i 's/chaine1/chaine2/g' {} \;
But,it doesn't work!!
Could you help me please?
Thanks in advance.

You want find with -exec. Don't bother running grep, sed will only change lines containing your pattern anyway.
find \( -name node_modules -o -name install -o -name build \) -prune \
-o -type f -exec sed -i 's/chaine1/chaine2/' {} +

First, the direct outputs of grep command are not file paths. They look like this {file_path}:{line_no}:{content}. So the first thing you need to do is to extract file paths. We can do this use cut command or use -l option of grep.
# This will print {file_path}
$ echo {file_path}:{line_no}:{content} | cut -f 1 -d ":"
# This is a better solution, because it only prints each file once even though
# the grep pattern appears at many lines of a file.
$ grep -RlI --exclude-dir={node_modules,install,build} "chaine1" /projets/
Second, sed -i does not read from stdin. We can use xargs to read each file path from stdin and then pass it to sed as its argument. You have already done this.
The complete command like this:
$ grep -RlI --exclude-dir={node_modules,install,build} "chaine1" /projets/ | xargs -i sed -i 's/chaine1/chaine2/' {}
Edit: Thanks to #EdMorton's comment, I dig into find. My previous solutions will dig into files not in exclusive directories once by grep, and then process files containing pattern string for another time by sed. However, we can first use find to filter files according to their path names, and then use sed to process files only once.
My find solution is almost the same as #knittl's, but with bug fixed. Besides, I try to explain why it gets the similar results with grep. Because I still not find how to skip binary files like -I option of grep.
$ find \( \( -name node_modules -o -name install -o -name build \) -prune -type f \
-o -type f \) -exec echo {} +
or
find \( \( -name node_modules -o -name install -o -name build \) -prune \
-o -type f \) -type f -exec echo {} +
\( -name pat1 -o -name pat2 \) gives paths matching pat1 or pat2 (include files and directories), where -o means logical or. -prune ignores a directory and the files under it. They combine to achieve similar function with exclude-dir in grep.
-type f gives paths of regular files.

Related

find and sed doesn't work in centos 7

I'm trying to find and replace a word in my entire project and I tried below versions of find and sed in centos 7 but nothing works.
find ./ -name "*.php" -exec sed -i '' s/mysql_/mysqli_/g \;
find ./ -name "*.php" -exec sed -i '' s/mysql_/mysqli_/g {} \;
find ./ -name "*.php" -exec sed -i '' 's/mysql_/mysqli_/g' {} \;
find ./ -name "*.php" -ls | xargs sed -i '' 's/mysql_/mysqli_/g'
sed: can't read s/mysql_/mysqli_/g: No such file or directory
All above commands giving me this error in loop even though I'm running these commands from the root of my project. Permissions are all correct. If I simply use find command alone it's working
find ./ -name "*.php" -ls (This Works)
I tried solutions available in stackoverflow but nothing works.
The fist pair of quotes in sed aren't necessary, try:
find ./ -name "*.php" -exec sed -i s/mysql_/mysqli_/g {} \;
The syntax is either -i'prefix' or --in-place='prefix', not -i 'prefix', since you added an space between the prefix and the argument, it's making sed use the prefix (empty string) argument as the regex and use the actual regex as a filename argument, which obviously won't find.
That's why you are getting the can't read s/mysql_/mysqli_/g: No such file or directory error.

How to pipe the results of 'find' to mv in Linux

How do I pipe the results of a 'find' (in Linux) to be moved to a different directory? This is what I have so far.
find ./ -name '*article*' | mv ../backup
but its not yet right (I get an error missing file argument, because I didn't specify a file, because I was trying to get it from the pipe)
find ./ -name '*article*' -exec mv {} ../backup \;
OR
find ./ -name '*article*' | xargs -I '{}' mv {} ../backup
xargs is commonly used for this, and mv on Linux has a -t option to facilitate that.
find ./ -name '*article*' | xargs mv -t ../backup
If your find supports -exec ... \+ you could equivalently do
find ./ -name '*article*' -exec mv -t ../backup {} \+
The -t option is a GNU extension, so it is not portable to systems which do not have GNU coreutils (though every proper Linux I have seen has that, with the possible exception of Busybox). For complete POSIX portability, it's of course possible to roll your own replacement, maybe something like
find ./ -name '*article*' -exec sh -c 'mv "$#" "$0"' ../backup {} \+
where we shamelessly abuse the convenient fact that the first argument after sh -c 'commands' ends up as the "script name" parameter in $0 so that we don't even need to shift it.
Probably see also https://mywiki.wooledge.org/BashFAQ/020
I found this really useful having thousands of files in one folder:
ls -U | head -10000 | egrep '\.png$' | xargs -I '{}' mv {} ./png
To move all pngs in first 10000 files to subfolder png
mv $(find . -name '*article*') ../backup
Here are a few solutions.
find . -type f -newermt "2019-01-01" ! -newermt "2019-05-01" \
-exec mv {} path \;**
or
find path -type f -newermt "2019-01-01" ! -newermt "2019-05-01" \
-exec mv {} path \;
or
find /Directory/filebox/ -type f -newermt "2019-01-01" \
! -newermt "2019-05-01" -exec mv {} ../filemove/ \;
The backslash + newline is just for legibility; you can equivalently use a single long line.
xargs is your buddy here (When you have multiple actions to take)!
And using it the way I have shown will give great control to you as well.
find ./ -name '*article*' | xargs -n1 sh -c "mv {} <path/to/target/directory>"
Explanation:
-n1
Number of lines to consider for each operation ahead
sh -c
The shell command to execute giving it the lines as per previous condition
"mv {} /target/path"
The move command will take two arguments-
1) The line(s) from operation 1, i.e. {}, value substitutes automatically
2) The target path for move command, as specified
Note: the "Double Quotes" are specified to allow any number of spaces or arguments for the shell command which receives arguments from xargs

In Linux terminal, how to delete all files in a directory except one or two

In a Linux terminal, how to delete all files from a folder except one or two?
For example.
I have 100 image files in a directory and one .txt file.
I want to delete all files except that .txt file.
From within the directory, list the files, filter out all not containing 'file-to-keep', and remove all files left on the list.
ls | grep -v 'file-to-keep' | xargs rm
To avoid issues with spaces in filenames (remember to never use spaces in filenames), use find and -0 option.
find 'path' -maxdepth 1 -not -name 'file-to-keep' -print0 | xargs -0 rm
Or mixing both, use grep option -z to manage the -print0 names from find
In general, using an inverted pattern search with grep should do the job. As you didn't define any pattern, I'd just give you a general code example:
ls -1 | grep -v 'name_of_file_to_keep.txt' | xargs rm -f
The ls -1 lists one file per line, so that grep can search line by line. grep -v is the inverted flag. So any pattern matched will NOT be deleted.
For multiple files, you may use egrep:
ls -1 | grep -E -v 'not_file1.txt|not_file2.txt' | xargs rm -f
Update after question was updated:
I assume you are willing to delete all files except files in the current folder that do not end with .txt. So this should work too:
find . -maxdepth 1 -type f -not -name "*.txt" -exec rm -f {} \;
find supports a -delete option so you do not need to -exec. You can also pass multiple sets of -not -name somefile -not -name otherfile
user#host$ ls
1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt josh.pdf keepme
user#host$ find . -maxdepth 1 -type f -not -name keepme -not -name 8.txt -delete
user#host$ ls
8.txt keepme
Use the not modifier to remove file(s) or pattern(s) you don't want to delete, you can modify the 1 passed to -maxdepth to specify how many sub directories deep you want to delete files from
find . -maxdepth 1 -not -name "*.txt" -exec rm -f {} \;
You can also do:
find -maxdepth 1 \! -name "*.txt" -exec rm -f {} \;
In bash, you can use:
$ shopt -s extglob # Enable extended pattern matching features
$ rm !(*.txt) # Delete all files except .txt files

How to grep lines that end with .c or .cpp?

I have a file as below, I want to grep for lines having .c or .cpp extension. I have tried using cat file|grep ".c" grep but I am getting all types of extensions as output. Please shed some light on this. Thanks in advance.
file contents are below:
/dir/a/b/cds/main.c
/dir/a/f/cmdss/file.cpp
/dir/a/b/cds/main.h
/dir/a/f/cmdss/file.hpp
/dir/a/b/cdys/main_abc.c
/dir/a/f/cmfs/file_123.cpp
grep supports regular expressions.
$ grep -E '\.(c|cpp)$' input
-E means 'Interpret PATTERN as an extended regular expression'
\. means a dot .
() is a group
c|cpp is an alternative
$ is the lineend
$ grep -E '\.cp{2}?' testfile1
/dir/a/b/cds/main.c
/dir/a/f/cmdss/file.cpp
/dir/a/b/cdys/main_abc.c
/dir/a/f/cmfs/file_123.cpp
$
May be this variant will useful. Here p{2} mean 'symbol p meet 2 times after symbol c'
Also you can use --include parameter like below
grep --include \*.hpp --include \*.cpp your_search_pattern
The Android framework defines a bash function extensions named cgrep, it goes recursively in the project directory, and it's much faster than using grep -r.
Usage:
cgrep <expession to find>
it greps only C/C++ header and source files.
function cgrep()
{
find . -name .repo -prune -o -name .git -prune -o -type f \( -name '*.c ' -o -name '*.cc' -o -name '*.cpp' -o -name '*.h' \) -print0 | xargs -0 gre p --color -n "$#"
}
You can paste this in you .bashrc file, or use the inline directly in shell.

How to change encoding in many files?

I try this:
find . -exec iconv -f iso8859-2 -t utf-8 {} \;
but output goes to the screen, not to the same file. How to do it?
Try this:
find . -type f -print -exec iconv -f iso8859-2 -t utf-8 -o {}.converted {} \; -exec mv {}.converted {} \;
It will use temp file with '.converted' suffix (extension) and then will move it to original name, so be careful if you have files with '.converted' suffixes (I don't think you have).
Also this script is not safe for filenames containing spaces, so for more safety you should double-quote: "{}" instead of {} and "{}.converted" instead of {}.converted
read about enconv.
If you need to convert to your current terminal encoding you can do it like that:
find . -exec enconv -L czech {}\;
Or exactly what you wanted:
find . -exec enconv -L czech -x utf8 {}\;
I found this method worked well for me, especially where I had multiple file encodings and multiple file extensions.
Create a vim script called script.vim:
set bomb
set fileencoding=utf-8
wq
Then run the script on the file extensions you wish to target:
find . -type f \( -iname "*.html" -o -iname "*.htm" -o -iname "*.php" -o -iname "*.css" -o -iname "*.less" -o -iname "*.js" \) -exec vim -S script.vim {} \;
No one proposed a way to automatically detect encoding and recode.
Here is an example to recode to UTF-8 all HTM/HTML files from master branch of a GIT.
git ls-tree master -r --name-only | grep htm | xargs -n1 -I{} bash -c 'recode "$(file -b --mime-encoding {})..utf-8" {}'

Resources