Please help me with advice
I need to go through the files in this $tmp directory and make substitutions in the lines according to the rule described in sed
I trained on one file and this sed worked it out normally.
But when I perform a replacement in for , I get some nonsense on the output
for file in $tmp/*.txt; do
find $tmp/ -type f -name "*.txt" -exec sed -i -e 's/, /\n''/g; s/"//g; s/$/\x0A/; s/:[ ]/;/g; s/\n''/;\n''/g; s/^[ t]*//' {} \;
done
How can I fix it?
Thanks…
UP
I did it.
find - It was really unnecessary
#Rob Sweet Thanks!
You don't need the for loop AND the find command. Both essentially do the same thing, which is to iterate through all the files in /tmp that match *.txt.
When you don't need to edit files in a subdir of /tmp, than give sed all files in once.
sed -i your_command *.txt
Off topic:
I don't understand your sed command. Why two single quotes? When you want to replace a newline, consider option -z.
Related
I wonder if there is a one-liner for batch processing a set of files in one folder and redirect the results into a different folder.
I tried something like this:
find input_dir/ -name "PATTERN" | xargs -I {} sed 's:foo:bar:g' > output_dir/{}
For example, input_dir/ has file A, B, C and my hoped result is to have processed files A, B, C in output_dir/, with the same file names.
my hope was to use {} to replace file names and to build the output file paths, but this didn't work.
Anyone knows how to fix this? Or other better ways of doing so?
Thanks!
My technique for this is to write a shell script that does the job, and then run it via find. For example, your actions could be written into a script munger.sh:
#!/bin/sh
for file in "$#"
do
output="output_dir/$(basename "$file")"
sed -e 's:foo:bar:g' "$file" > "$output"
done
The find command becomes:
find input_dir -name "PATTERN" -exec sh munger.sh {} +
This runs the script with the file names as arguments, bundling conveniently large number of file names into a single invocation of the shell script. If you're not going to need it again, you can simply remove munger.sh when you're done.
Yes, you can do all sorts of contortions to execute the command the way you want (perhaps using find … -exec bash -c "the script to be executed" arg0 {} +) but it is often harder than writing a relatively simple script and using it and throwing it away. There tend to be fewer problems with quoting, for example, when you run an explicit script than when you try to write the script on the command line. If you find yourself fighting with single quotes, double quotes and backslashes (or back-quotes), then it is time to use a simple script as shown.
Using GNU Parallel it looks like this:
find input_dir/ -name "PATTERN" | parallel sed s:foo:bar:g {} '>' output_dir/{/}
If the sed command has special chars, then you need to quote those double:
find input_dir/ -name "PATTERN" | parallel sed 's:foo.\*:bar:g' {} '>' output_dir/{/}
In two steps:
find input_dir/ -name "PATTERN" -exec cp -t output_dir/ {} +
than
sed 's:foo:bar:g' -i output_dir/*
or, if output_dir could contain files not matching "PATTERN":
find output_dir -name "PATTERN" -exec sed -e 's:foo:bar:g' -i {} +
I need a script that will do the following, but have no idea how to approach the idea.
Recursively scan a directory /home/root/multicraft/servers finding all files named "server.properties"
Edit the file replacing "view-distance=(a number here)" with "view-distance=4"
How would I go about doing this?
You can use find to recursively go through directories. It can execute a command via -exec. To edit the file, you can use sed:
find /home/root/multicraft/servers \
-name server.properties \
-exec sed -i~ 's/view-distance=[0-9]\+/view-distance=4/' {} \;
try this:
find /home/root/multicraft/servers -name "server.properties"|xargs sed -ri 's/(view-distance)=[0-9]+/\1=4/'
making backup before trying it.
I am trying to find the whole source code for occurrences of, say, "MY_NAME" and want to replace it with, say, "YOUR_NAME". I already know the files and the line numbers where they occur and i want to make a patch for the same so that anyone running the patch can do the same. Can anyone please help?
You can do it by console. Just use find to locate destination files, and then you can declare what you want to replace with what sentence. In example:
find -name '*' | xargs perl -pi -e 's/MY_NAME/YOUR_NAME/g'
It might be easier to do a sed command, and then generate a patch.
sed -e '12s/MY_NAME/YOUR_NAME/g;32s/MY_NAME/YOUR_NAME/g' file > file2
This will replace MY_NAME with YOUR_NAME on lines 12 and 32, and save the output into file2.
You can also generate a sed script if there are many changes:
#!/bin/sed -f
12s/MY_NAME/YOUR_NAME/g
32s/MY_NAME/YOUR_NAME/g
Then, for applying to many files, you should use find:
find -type f '(' -iname "*.c" -or -iname "*.h" ')' -exec "./script.sed" '{}' \;
Hope this helps =)
Use the command diff to create a patch-file that can then be distributed and applied with the patch-command.
man diff Will give you a lot of information on the process.
How do I convert all EOL (dos->unix) of all files in a directory and sub-directories recursively without dos2unix? (I do not have it and cannot install it.)
Is there a way to do it using tr -d '\r' and pipes? If so, how?
For all files in current directory you can do it with a Perl one-liner: perl -pi -e 's/\r\n/\n/g' * (stolen from here)
EDIT: And with a small modification you can do subdirectory recursion:
find | xargs perl -pi -e 's/\r\n/\n/g'
You can use sed's -i flag to change the files in-place:
find . -type f -exec sed -i 's/\x0d//g' {} \+
If I were you, I would keep the files around to make sure the operation went okay. Then you can delete the temporary files when you get done. This can be done like so:
find . -type f -exec sed -i'.OLD' 's/\x0d//g' {} \+
find . -type f -name '*.OLD' -delete
Do you have sane file names and directory names without spaces, etc in them?
If so, it is not too hard. If you've got to deal with arbitrary names containing newlines and spaces, etc, then you have to work harder than this.
tmp=${TMPDIR:-/tmp}/crlf.$$
trap "rm -f $tmp.?; exit 1" 0 1 2 3 13 15
find . -type f -print |
while read name
do
tr -d '\015' < $name > $tmp.1
mv $tmp.1 $name
done
rm -f $tmp.?
trap 0
exit 0
The trap stuff ensures you don't get temporary files left around. There other tricks you can pull, with more random names for your temporary file names. You don't normally need them unless you work in a hostile environment.
You can also use the editor in batch mode.
find . -type f -exec bash -c 'echo -ne "%s/\\\r//\nx\n" | ex "{}" ' \;
If \r isn't followed by \n (maybe the case in files of Tim Pote):
deleting \r (using tr -d) may remove newlines
replacing \r with \n may not cause double / triple newlines
Maybe Tim Pote could verify the points above for the files he mentioned.
This removes carriage returns from all files in the current directory and all subdirectories, and should work on most Unix-like OSs:
grep -lIUre '\r' | xargs sed -i 's/\r//'
If its done in widows:
try to run the command in git bash:
$ find | xargs perl -pi -e 's/\r\n/\n/g'
It can show some Can't do inplace edit: type a message so ignore it
How do I replace every occurrence of a string with another string below my current directory?
Example: I want to replace every occurrence of www.fubar.com with www.fubar.ftw.com in every file under my current directory.
From research so far I have come up with
sed -i 's/www.fubar.com/www.fubar.ftw.com/g' *.php
You're on the right track, use find to locate the files, then sed to edit them, for example:
find . -name '*.php' -exec sed -i -e 's/www.fubar.com/www.fubar.ftw.com/g' {} \;
Notes
The . means current directory - i.e. in this case, search in and below the current directory.
For some versions of sed you need to specify an extension for the -i option, which is used for backup files.
The -exec option is followed by the command to be applied to the files found, and is terminated by a semicolon, which must be escaped, otherwise the shell consumes it before it is passed to find.
Solution using find, args and sed:
find . -name '*.php' -print0 | xargs -0 sed -i 's/www.fubar.com/www.fubar.ftw.com/g'
A pure bash solution
#!/bin/bash
shopt -s nullglob
for file in *.php
do
while read -r line
do
echo "${line/www.fubar.com/www.fubar.ftw.com}"
done < "$file" > tempo && mv tempo "$file"
done
A more efficient * alternative to the currently accepted solution:
`grep "www.fubar.com" . -lr | xargs sed -i 's/www.fubar.com/www.fubar.ftw.com/g'
This avoids the inefficiency of the find . -exec method, which needlessly runs a sed in-place replacement over all files below your current directory regardless of if they contain the string you're looking for or not, by instead using grep -lr. This gets just the files containing the string you want to replace which you can then pipe to xargs sed -i to perform the in-place replacement on just those files.
* : I used time to make a cursory comparison of my method with the accepted solution (adapted for my own use case); The find . -exec-style method took 3.624s to run on my machine and my above proposed solution took 0.156s, so roughly 23x faster for my use case.
If there are no subfolders, a simpler to remember way is
replace "www.fubar.com" "www.fubar.ftw.com" -- *
where * can also be a list of files
from the manual:
Invoke replace in one of the following ways:
shell> replace from to [from to] ... -- file_name [file_name] ...
shell> replace from to [from to] ... < file_name
If you have hidden files with a dot you can add those to * with
shopt -s dotglob
If you only have one depth of subfolders you can use */* instead of *
replace "www.fubar.com" "www.fubar.ftw.com" -- */*
When using ZSH as your shell you can do:
sed -i 's/www.fubar.com/www.fubar.ftw.com/g' **/*.php