Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Closed 8 years ago.
Improve this question
How do I create a bash file that creates a symbolic link (linux) for each file moved from Folder A to Folder B.
However, this would be done selecting the 150 biggest current files from Folder A.
Can probably write it in a one liner, but easier in a simple bash script like
#!/bin/bash
FolderA="/some/folder/to/copy/from/"
FolderB="/some/folder/to/copy/to/"
while read -r size file; do
mv -iv "$file" "$FolderB"
ln -s "${FolderB}${file##*/}" "$file"
done < <(find "$FolderA" -maxdepth 1 -type f -printf '%s %p\n'| sort -rn | head -n150)
Note ${file##*/} removes everything before the last /, per
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce a pattern just as in pathname expansion. If the
pattern matches the beginning of the value of parameter, then the result of the expansion is the expanded value of
parameter with the shortest matching pattern (the ``#'' case) or the longest matching pattern (the ``##'' case)
deleted. If parameter is # or *, the pattern removal operation is applied to each positional parameter in turn, and
the expansion is the resultant list. If parameter is an array variable subscripted with # or *, the pattern removal
operation is applied to each member of the array in turn, and the expansion is the resultant list.
Also, it may seem like a good idea to just do for file in $(command), but process substitution and while/read works better in general to avoid word-splitting issues like splitting up files with spaces, etc...
As with any task, break it up into smaller pieces, and things will fall into place.
Select the biggest 150 files from FolderA
This can be done with du, sort, and awk, the output of which you stuff into an array:
du -h /path/to/FolderA/* | sort -h | head -n150 | awk '{print $2}'
Move files from FolderA into FolderB
Take the list from the last command, and iterate through it:
for file in ${myarray[#]}; do mv "$file" /path/to/FolderB/; done
Make a symlink to the new location
Again, just iterate through the list:
for file in ${myarray[#]; do ln -s "$file" "/path/to/FolderB/${file/*FolderA\/}"; done
Related
This question already has answers here:
Rename multiple files based on pattern in Unix
(24 answers)
Closed 5 years ago.
Write a simple script that will automatically rename a number of files. As an example we want the file *001.jpg renamed to user defined string + 001.jpg (ex: MyVacation20110725_001.jpg) The usage for this script is to get the digital camera photos to have file names that make some sense.
I need to write a shell script for this. Can someone suggest how to begin?
An example to help you get off the ground.
for f in *.jpg; do mv "$f" "$(echo "$f" | sed s/IMG/VACATION/)"; done
In this example, I am assuming that all your image files contain the string IMG and you want to replace IMG with VACATION.
The shell automatically evaluates *.jpg to all the matching files.
The second argument of mv (the new name of the file) is the output of the sed command that replaces IMG with VACATION.
If your filenames include whitespace pay careful attention to the "$f" notation. You need the double-quotes to preserve the whitespace.
You can use rename utility to rename multiple files by a pattern. For example following command will prepend string MyVacation2011_ to all the files with jpg extension.
rename 's/^/MyVacation2011_/g' *.jpg
or
rename <pattern> <replacement> <file-list>
this example, I am assuming that all your image files begin with "IMG" and you want to replace "IMG" with "VACATION"
solution : first identified all jpg files and then replace keyword
find . -name '*jpg' -exec bash -c 'echo mv $0 ${0/IMG/VACATION}' {} \;
for file in *.jpg ; do mv $file ${file//IMG/myVacation} ; done
Again assuming that all your image files have the string "IMG" and you want to replace "IMG" with "myVacation".
With bash you can directly convert the string with parameter expansion.
Example: if the file is IMG_327.jpg, the mv command will be executed as if you do mv IMG_327.jpg myVacation_327.jpg. And this will be done for each file found in the directory matching *.jpg.
IMG_001.jpg -> myVacation_001.jpg
IMG_002.jpg -> myVacation_002.jpg
IMG_1023.jpg -> myVacation_1023.jpg
etcetera...
find . -type f |
sed -n "s/\(.*\)factory\.py$/& \1service\.py/p" |
xargs -p -n 2 mv
eg will rename all files in the cwd with names ending in "factory.py" to be replaced with names ending in "service.py"
explanation:
In the sed cmd, the -n flag will suppress normal behavior of echoing input to output after the s/// command is applied, and the p option on s/// will force writing to output if a substitution is made. Since a sub will only be made on match, sed will only have output for files ending in "factory.py"
In the s/// replacement string, we use "& " to interpolate the entire matching string, followed by a space character, into the replacement. Because of this, it's vital that our RE matches the entire filename. after the space char, we use "\1service.py" to interpolate the string we gulped before "factory.py", followed by "service.py", replacing it. So for more complex transformations youll have to change the args to s/// (with an re still matching the entire filename)
Example output:
foo_factory.py foo_service.py
bar_factory.py bar_service.py
We use xargs with -n 2 to consume the output of sed 2 delimited strings at a time, passing these to mv (i also put the -p option in there so you can feel safe when running this). voila.
NOTE: If you are facing more complicated file and folder scenarios, this post explains find (and some alternatives) in greater detail.
Another option is:
for i in *001.jpg
do
echo "mv $i yourstring${i#*001.jpg}"
done
remove echo after you have it right.
Parameter substitution with # will keep only the last part, so you can change its name.
Can't comment on Susam Pal's answer but if you're dealing with spaces, I'd surround with quotes:
for f in *.jpg; do mv "$f" "`echo $f | sed s/\ /\-/g`"; done;
You can try this:
for file in *.jpg;
do
mv $file $somestring_${file:((-7))}
done
You can see "parameter expansion" in man bash to understand the above better.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I've been seeing a ton of answers that show how to add to the beginning or end of a file, but I'm looking to replace numbers in the middle of a file. I have, for example:
ShowName - S00E01 - Episode Name.mkv
ShowName - S00E02 - Episode Name.mkv
ShowName - S00E03 - Episode Name.mkv
I want to change the E01-E03 part to be something like E20 to E22, or similar, without modifying the rest of the filename.
Couldn't figure out how to do this using linux's "rename" call, any other suggestions?
Linux utility rename is just a simple tool. A more advanced tool using a regular expression is perl-rename, which usually installed separately. However it still will not solve your problem.
For anything more complicated, i usually resort to writing a small bash for loop.
E.g. this script should work for your problem:
# for every file ending with .mkv
for f in *.mkv; do
# transform the filename using sed, so that character '|' character will separate episode number from the lest of the filename (so it can be extracted)
# e.g.
# 'ShowName - S00E01 - Episode Name.mkv' will be
# 'ShowName - S00E|01| - Episode Name.mkv'
# Then read such string to three variables:
# prefix enum and suffix splitting on '|' character
IFS='|' read -r prefix enum suffix < <(sed 's/\(.*S[0-9][0-9]E\)\([0-9][0-9]\)\(.*\)/\1|\2|\3/' <<<"$f");
# newfilename consist of prefix, calculated episode number and the rest of the filename
# i assumed you want to add 19 to episode number
# it may be also a good idea to move files to another directory, to avoid unintentional overwriting of existing files
# you may also consider using -n/--no-clobber or --backup options to mv
newf="another_directory/${prefix}$(printf "%02d" "$((enum-1+20))")${suffix}"
# move "$f" to "$newf"
# filenames have special characters (spaces), so remember about qoutes
echo "'$f' -> '$newf'"
mv -v "$f" "$newf"
done
Use some other tools like grep to help you:
for f in *.mkv; do
NUM=$(echo "$f" | grep -Po '(?<=E)[0-9]{2}')
NEWNUM=$((NUM+20))
fn=${f/E${NUM}/E${NEWNUM}}
mv "$f" "$fn"
done
The rest should be easily done with shell's builtin string replacement functionality.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am writing a bash script to check if an array element is in a file.
For example:
I have an array of errors errors=("1234" "5678" "9999")
I have a file that contains patterns of strings
123400 452612 9999A0 1010EB
I am looking to loop over the file that contains the errors and check to see if any of the array elements matches any string pattern in the file. If it does then give me back the exact array pattern it matched in the file for further processing.
Any ideas on how I can do this?
Here's a way where you only need to invoke grep once:
$ grep -oFf <(printf "%s\n" "${errors[#]}") file
1234
9999
The -f option is to specify a file that contains the pattern. I use a process substitution to "contain" the patterns, one per line.
The -F option specifies plain-text matching: I assume your "errors" array won't contain regular expressions.
Sounds like you just want a loop:
for error in "${errors[#]}"; do
if grep -qE "(^| )$error( |\$)" file; then
# $error was found in the file
fi
done
This matches the error preceded by the start of the line or a space, and followed by a space or the end of the line.
I made an effort to not match appearances of the errors within substrings but if you don't care, then you could change the grep command to this:
grep -qF "$error" file
This will return success if the error string occurs anywhere on the line.
The script goes like this,
#/bin/bash
errors=("1234" "5678" "9999")
for error in "${errors[#]}"
do
grep -o "$error" file
done
For a sample file,
$ cat file
123400 452612 9999A0 1010EB
The script produces an output
$ ./script.sh
1234
9999
meaning the above two keys from the array have matched in the file. The -o flag in grep is to identify only the matching parts from the array. An excerpt from the man grep page.
-o, --only-matching
Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.
This question already has answers here:
How can I remove the extension of a filename in a shell script?
(15 answers)
Closed 6 years ago.
I'm new to scripting concept.. I have a requirement to rename multiple files in a directory like filename.sh.x into filename.sh
First I tried to get the file names in a particular directory.. so i followed the below scripting code
for entry in PathToThedirectory/*sh.x
do
echo $entry
done
& the above code listed down all the file names with full path..
But my expected o/p is : to get file names alone like abc.sh.x,
so that I can proceed with the split string mechanism to perform rename
operation easily...
help me to solve this ... Thanks in advance
First approach trying to follow OP suggestions:
for i in my/path/*.py.x
do
basename=$(basename "$i")
mv my/path/"$basename" my/path/"${basename%.*}"
done
And maybe, you can simplify it:
for i in my/path/*.py.x
do
mv "$i" "${i%.*}";
done
Documentation regarding this kind of operation (parameter expansion): https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
In particular:
${parameter%word} : The word is expanded to produce a pattern just as in filename expansion. If the pattern matches a trailing portion of the expanded value of parameter, then the result of the expansion is the value of parameter with the shortest matching pattern (the ‘%’ case) or the longest matching pattern (the ‘%%’ case) deleted
So, ${i%.*} means:
Take $i
Match .* at the end of its value (. being a literal character)
Remove the shortest matching pattern
Look into prename (installed together with the perl package on ubuntu).
Then you can just do something like:
prename 's/\.x$//' *.sh.x
In ksh you can do this:
for $file in $(ls $path)
do
new_file=$(basename $path/$file .x)
mv ${path}/${file} ${path}/${new_file}
done
This should do the trick:
for file in *.sh.x;
do
mv "$file " "${file /.sh.x}";
done
Running this rename command from the root directory should work:
rename 's/\.sh\.x$/.sh/' *.sh.x
for i in ls -la /path|grep -v ^d|awk '{print $NF}'
do
echo "basename $i"
done
it will give u the base name of all files or you can try below
find /path -type f -exec basename {} \;
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
I am currently working on a C project that contains over 50 .h and .c files. I would like to know if there is a quick way to search for certain lines of code (like ctrl+f for a window for example) without having to actually search each file one by one.
Thank you in advance
On Linux/Unix there's a command line tool called grep you can use it to search multiple files for a string. For examples if I wanted to search for strcpy in all files:
~/sandbox$ grep -rs "strcpy"*
test.c: strcpy(OSDMenu.name,"OSD MENU");
-r gives searches recursivly so you get all the files in all directories (from the current one) searched. -s ignores warnings, in case you run into non-readable files.
Now if you wanted to search for something custom, and you can't remember the case there's options like -i to allow for case insenstive searches.
~/sandbox$ grep -rsi "myint" *
test.c: int myInt = 5;
test.c: int MYINT = 10;
You can also use regular expressions in case you forgot exactly what you were looking for was called (indeed the name, 'grep' comes from the sed command g/re/p -- global/regular expression/print:
~/sandbox$ grep -rsi "my.*" *
test.c: int myInt = 5;
test.c: int MYINT = 10;
test.c: float myfloat = 10.9;
install cygwin if you aren't using *nix and use find/grep, e.g.
find . -name '*\.[ch]' | xargs grep -n 'myfuncname'
In fact, I made this a little script findinsrc that can be called with findinsrc path1, [path2, ...] pattern. The central line, after checking arguments etc, is
find "${#:1:$#-1}" -type f \( -iname '*.c' -o -iname '*.cpp' -o -iname '*.h' -o -iname '*.hpp' \) -print0 | xargs -0 grep -in "${#:$#}"
"${#:1:$#-1}" are the positional parameters 1 .. n-1, that is, the path(s), supplied as the starting points for find. "${#:$#}" is the last parameter, the pattern supplied to grep.
the -o "option" to find is a logical OR combining the search criteria; because the "default" combination of options is AND, all the ORs must be parenthesized for correct evaluation. Because parentheses have special meaning to the shell, they must be escaped so that they are passed through to find as command line arguments.
-print0 instructs find to separate its output items not with a newline or space but with a null character which cannot appear in path names; this way, there is a clear distinction between whitespace in a path ("My Pictures" nonsense) and separation between paths.
-iname is a case insensitive search, in case files are ending in .CPP etc.
xargs -0 is there specifically to digest find -print0 output: xargs will separate arguments read from stdin at null bytes, not at whitespace.
grep -in: -i instructs grep to perform a case insensitive search (which suits my bad memory and is catered exactly to this "find the bloody function no matter the capitalization you know what I mean" use case). The -n prints the line number, in addition to the file name, where the match occurred.
I have similar scripts findinmake, whre the find pattern includes regular Makefiles, CMakeLists.txt and a proprietary file name; and findinscripts that looks through bat, cmd and sh files. That seemed easier than introducing options to a generic script.
You can use grep to search through the files using the terminal/command line.
grep -R "string_to_search" .
-R to be recursive, search in all sub directories too
Then string you want
Then is the location, . for the current directory
On windows you can use findstr which will find files that contain strings that either exactly match or regular expression match the specified string / pattern.
findstr /?
from the command line will give you the usage. It can also recurse subdirectories (/s).
If you're using a text editor and the shell, then you can use shell tools like grep.
grep -R "some pattern" directory
However you should consider using an IDE such as Eclipse (it's not just for Java), Netbeans (there is a C plugin) or KDevelop. IDEs have keyboard shortcuts for things like "find everywhere the highlighted function is called".
Or of course there's Emacs...