I have a lot of files named like activity_unpublish_39x39.png, abc_29x29.png and etc.
I want to convert the name to activity_unpublish.png (remove _39x39)
and abc.png (remove _29x29).
Could anyone tell me how I can achieve that?
It would be better working on Mac OS X.
The following small shell script should work on Linux and also on Mac OS. Note that it's working in the current folder, further you have to change pat and suf to your needs (here suf="\.png" and pat="_[0-9]+x[0-9]+$suf" to work with your given example).
It uses sed with -E which is undocumented in the manpage. It's the option to go in Mac OS which is known as -r in Linux. In Linux it is also existent but as said not documented:
#!/bin/sh
suf="\.png"
pat="_[0-9]+x[0-9]+$suf"
for f in *; do
if [[ $f =~ $pat ]]; then
newName=$(echo "$f" | sed -E "s/$pat/$suf/g")
mv "$f" "$newName"
fi
done
I got the answer from my kind colleague.
Use this shell script.
#!/bin/sh
for file in *_[0-9]*x[0-9]*.png
do
mv $file $(echo $file | sed 's/_[0-9]*x[0-9]*//')
done
Related
sorry for that odd title. I didn't know how to word it the right way.
I'm trying to write a script to filter my wiki files to those got directories with the same name and the ones without. I'll elaborate further.
here is my file system:
what I need to do is print a list of those files which have directories in their name and another one of those without.
So my ultimate goal is getting:
with dirs:
Docs
Eng
Python
RHEL
To_do_list
articals
without dirs:
orphan.txt
orphan2.txt
orphan3.txt
I managed to get those files with dirs. Here is me code:
getname () {
file=$( basename "$1" )
file2=${file%%.*}
echo $file2
}
for d in Mywiki/* ; do
if [[ -f $d ]]; then
file=$(getname $d)
for x in Mywiki/* ; do
dir=$(getname $x)
if [[ -d $x ]] && [ $dir == $file ]; then
echo $dir
fi
done
fi
done
but stuck with getting those without. if this is the wrong way of doing this please clarify the right one.
any help appreciated. Thanks.
Here's a quick attempt.
for file in Mywiki/*.txt; do
nodir=${file##*/}
test -d "${file%.txt}" && printf "%s\n" "$nodir" >&3 || printf "%s\n" "$nodir"
done >with 3>without
This shamelessly uses standard output for the non-orphans. Maybe more robustly open another separate file descriptor for that.
Also notice how everything needs to be quoted unless you specifically require the shell to do whitespace tokenization and wildcard expansion on the value of a token. Here's the scoop on that.
That may not be the most efficient way of doing it, but you could take all files, remove the extension, and the check if there isn't a directory with that name.
Like this (untested code):
for file in Mywiki/* ; do
if [ -f "$d" ]; then
dirname=$(getname "$d")
if [ ! -d "Mywiki/$dirname" ]; then
echo "$file"
fi
fi
done
To List all the files in current dir
list1=`ls -p | grep -v /`
To List all the files in current dir without extension
list2=`ls -p | grep -v / | sed 's/\.[a-z]*//g'`
To List all the directories in current dir
list3=`ls -d */ | sed -e "s/\///g"`
Now you can get the desired directory listing using intersection of list2 and list3. Intersection of two lists in Bash
I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
I am working on a project for one of my professors and he asked me to sort a couple hundred .fits images based on their header files (specifically what star they are images of) I think that grep would be the best way to do this however I can't seam to figure out how to use grep based on the header.
I am entering:
ls | imhead *.fits | grep -E -r "PG\ 1104+243" *
to just list them out for now, once they are listed I know how to copy them into a directory.
I am new to using grep so I am unsure as to where my error lies? any help would be greatly appreciated! Thanks!
Assuming that imghead will extract the headers of the .fits as txt, you can use a simple shell script to do it:
script.sh
#!/bin/bash
grep "$1" "$2" > /dev/null 2>&1 && echo "$2"
Note that the + is a special character if you use extended regular expression, meaning if you pass the -E as in the question. A simple grep without any options should do the trick here.
Use find to exec the script on every *.fits file in the current folder:
find -maxdepth 1 -name '*.fits' -exec ./script.sh 'PG 1104+243' {} \;
If you are going to copy/move/alter or do something with the files you find, you might be better off, in terms of complexity and ease of quoting, using a loop like this:
#!/bin/bash
find . -name \*.fits -print0 | while read -d '' -r file; do
echo Checking file: $file
imhead "$file" | grep -q 'PG 1104+243'
if [ $? -eq 0 ]; then
echo Object matches: $file
fi
done
I apologize if this is a trivial question. I am learning how to use linux bash and this little task is giving me a headache...
So I need to write a script, let's call it count.sh. I want that: for each file in the working directory, prints the filename, the number of lines, and the number of words to the console:
test.txt 100 1023
someOtherfiles 10 233
So far, I know that the following gives me all the files names in the directory. And thanks for all who helped me, I get this working version:
for f in *; do
echo -n "$f"
cat "$f" | wc -wl
done
I would really appreciate your help! Thanks ahead!
P.s. If you know great resources (links for tutorials) for learning about script and you are willing to share it with me. I think I really need to know these basics. Thanks again!
If you must have the file name as the first field in your output, try this:
for f in *; do
if [ -f "$f" ]; then
echo -n "$f"
cat "$f" | wc -wl
fi
done
for f in *; do
if [[ -f $f ]]; then
echo "$f $(wc -wl < "$f")"
fi
done
[[ -f $f ]] processes only files (excludes subdirectories) and also handles the case where the directory is empty (in which case * is (by default) left unexpanded, i.e. assigned to $f as is).
echo "$f $(wc -wl < "$f")" uses command substitution ($( ... )) to directly include the output from the enclosed command in the output string passed to echo.
Note that the reason that < is used to direct the content of file $f to wc via stdin is that wc would otherwise append the name of the input file to its output (thanks, #R Sahu).
I originally had a set of images of the form image_001.jpg, image_002.jpg, ...
I went through them and removed several. Now I'd like to rename the leftover files back to image_001.jpg, image_002.jpg, ...
Is there a Linux command that will do this neatly? I'm familiar with rename but can't see anything to order file names like this. I'm thinking that since ls *.jpg lists the files in order (with gaps), the solution would be to pass the output of that into a bash loop or something?
If I understand right, you have e.g. image_001.jpg, image_003.jpg, image_005.jpg, and you want to rename to image_001.jpg, image_002.jpg, image_003.jpg.
EDIT: This is modified to put the temp file in the current directory. As Stephan202 noted, this can make a significant difference if temp is on a different filesystem. To avoid hitting the temp file in the loop, it now goes through image*
i=1; temp=$(mktemp -p .); for file in image*
do
mv "$file" $temp;
mv $temp $(printf "image_%0.3d.jpg" $i)
i=$((i + 1))
done
A simple loop (test with echo, execute with mv):
I=1
for F in *; do
echo "$F" `printf image_%03d.jpg $I`
#mv "$F" `printf image_%03d.jpg $I` 2>/dev/null || true
I=$((I + 1))
done
(I added 2>/dev/null || true to suppress warnings about identical source and target files. If this is not to your liking, go with Matthew Flaschen's answer.)
Some good answers here already; but some rely on hiding errors which is not a good idea (that assumes mv will only error because of a condition that is expected - what about all the other reaons mv might error?).
Moreover, it can be done a little shorter and should be better quoted:
for file in *; do
printf -vsequenceImage 'image_%03d.jpg' "$((++i))"
[[ -e $sequenceImage ]] || \
mv "$file" "$sequenceImage"
done
Also note that you shouldn't capitalize your variables in bash scripts.
Try the following script:
numerate.sh
This code snipped should do the job:
./numerate.sh -d <your image folder> -b <start number> -L 3 -p image_ -s .jpg -o numerically -r
This does the reverse of what you are asking (taking files of the form *.jpg.001 and converting them to *.001.jpg), but can easily be modified for your purpose:
for file in *
do
if [[ "$file" =~ "(.*)\.([[:alpha:]]+)\.([[:digit:]]{3,})$" ]]
then
mv "${BASH_REMATCH[0]}" "${BASH_REMATCH[1]}.${BASH_REMATCH[3]}.${BASH_REMATCH[2]}"
fi
done
I was going to suggest something like the above using a for loop, an iterator, cut -f1 -d "_", then mv i i.iterator. It looks like it's already covered other ways, though.