I would like to ask some help regarding my script below:
#!/bin/bash
year=$(date +%Y)
lastyear=$((year-1))
month=$(date +%m)
log="$lastyear$month"
mkdir -p "/root/temp/$lastyear"
mkdir -p "/root/temp/$lastyear/$month"
cd /root
mv -f "*$log*" "/root/temp/$lastyear/$month"
Error Prompt is
mv: cannot stat `*201602*': No such file or directory
My target was to move all the files that has the specific "201602" string in file name to a specific location.
Sample logs files is OUTXXX-201602XXX, INXXX-201602XXX, 201602XXXX.
This will be implemented through crontab, because there are about 500k+ log files to be transferred and using find will receive a too much argument error T_T.
Any suggestions will help!
You have to put globbing characters outside of quotes for them to be interpreted.
Try instead:
mv -f *"$log"* …
The shell will still keep the resulting matches as one word, so this is OK even if the filenames have spaces.
Related
I am making a custom command that moves or duplicates a file to a wastebasket directory instead of deleting it. I am trying to make a directory if it already isn't there, make a duplicate if a file has already been executed on, and simply move it if it doesn't. The issue is that I keep getting a no such file or directory error regardless of where I place the wastebasket directory. Do note that simply moving or copying the file with base linux commands work fine, and that being in root doesn't fix the issue. What steps should I take?
#!/bin/bash
set -x
mkdir -p /home/WASTEBASKIT #This makes a wastebasket directory if it doesn't already exist.
if test -e "$1"; then
if test -e /home/WASTEBASKIT/"$1"; then #Checking for duplicate files.
cp "$1" "/home/WASTEBASKIT/$1.$$"
else
mv "$1" "/home/WASTEBASKIT"
fi
else
printf '%s\n' "File not found." #Error if a file is not there.
fi
Here are the results: ++ mkdir -p /home/WASTEBASKIT
++ test -e config.sh
++ test -e /home/WASTEBASKIT/config.sh
++ cp config.sh.945 ' /home/WASTEBASKIT'
cp: cannot stat 'config.sh.945': No such file or directory
cp config.sh.945 ' /home/WASTEBASKIT'
cp: cannot stat 'config.sh.945': No such file or directory
The problem is on this line:
cp "$1" "$1.$$" "/home/WASTEBASKIT"
You try to copy two files into /home/WASTEBASKIT, namely $1 and $1.$$. The latter does not exist.
Change it to:
cp "$1" "/home/WASTEBASKIT/$1.$$"
I suggest that you instead create a unique file since process numbers aren't unique, so instead of the copy above, do something like:
newfile=$(mktemp "WASTEBASKIT/$1.XXXXXXX")
cp -p "$1" "$newfile"
You can then list all the copies with ls -t WASTEBASKIT to get them in historical order, newest first - or with ls -tr WASTEBASKIT to get the oldest first.
Also note: printf'%s\n' "File not found." will likely generate an error like printf%s\n: command not found.... You need to insert a space between the command printf and the argument '%s\n'.
The moving part is also wrong since you have a space before /home. It should be:
mv "$1" /home/WASTEBASKIT
mv "$1" " /home/WASTEBASKIT"
First issue: spaces matter. If you have previously created the /home/WASTEBASKIT directory, and then execute that copy command above, it will not copy the file into that directory - you will most likely end up with a file in your home directory called spaceWASTEBASKIT (unless you already have a directory of that name, including the leading space) in which case it will go into that directory.
Either way, it won't go where you want it to go.
Secondly, the command below is not doing what you seem to think. It will try to copy two files to the directory, the second of which probably does not even exist (config.sh.945 in your case):
cp "$1" "$1.$$" "/home/WASTEBASKIT"
If you want to create a "uniquely" versioned file so as to not overwrite an existing one, that would be:
mv "$1" "/home/WASTEBASKIT/$1.$$"
Note the quotes around the word "uniquely" since there's no guarantee $1.$$ may not also exist in the wastebasket - the PIDs do eventually wrap around at some point, and also do so on reboot.
I suspect a better approach (though still not bullet-proof) would be just to prefix every file with the date and time so that:
you can sort duplicates to establish the order of creation; and
sans date changes, the date/time won't give you possible duplicates (unless you're doing it more then once per second).
That approach would be something like:
mv "$1" "/home/WASTEBASKIT/$(date -u +%Y%m%d_%H%M%S)$1"
or, making duplicates even less likely:
mv "$1" "/home/WASTEBASKIT/$(date -u +%Y%m%d_%H%M%S)_${RANDOM}_$1"
cat paste_output.txt | while read -r file_name path_name file;
do mkdir -p -- "$path_name";
wget "$file_name";
mv "$file" "$path_name";
done;
Hi! I have this piece of code that reads field by field from the file specified. What I am trying to do here is I am creating a directory that is specified in second field and then I am downloading file specified in first field and then after having that file downloaded I am that file in the directory specified in second field.
Output: I am getting the desired directory structure and files downloaded however files are downloading in the directory I am executing the commands from.
How to move files in the desired directories?
You can use the -P flag of wget to put the file in the target directory.
If the directory doesn't exist, it will create it,
so this also let's you save on the mkdir.
while read -r file_name path_name file; do
wget -P "$path_name" "$file_name"
done < paste_output.txt
I made some other improvements to the script:
The cat is useless, input redirection is better
The semicolons at end of lines are unnecessary
It's good to indent the body of loops, for readability
This probably quite basic but I have spent whole day finding an answer without much success.
I have an executable script that resides in ~/Desktop/shell/myScript.sh
I want a single line command to run this script from my terminal that outputs to a new directory in same directory where the script is located no matter what my present working directory is.
I was using:
mkdir -p tmp &&
./Desktop/shell/myScript.sh|grep '18x18'|cut -d":" -f1 > tmp/myList.txt
But it creates new directory in present working directory and not on the target location.
Any help would be appreciated.
Thanks!
You could solve it in one line if you pre-define a variable:
export LOC=$HOME/Desktop/shell
Then you can say
mkdir -p $LOC/tmp && $LOC/myScript.sh | grep '18x18' | cut -d":" -f1 > $LOC/tmp/myList.txt
But if you're doing this repeatedly it might be better long-term to wrap myScript.sh so that it creates the directory, and redirects the output, for you. The grep and cut parameters, as well as the output file name, would be passed as command-line arguments and options to the wrapper.
How about this:
SCRIPTDIR="./Desktop/shell/" ; mkdir "$SCRIPTDIR/tmp" ; "$SCRIPTDIR/myScript.sh" | grep '18x18' | cut -d ":" -f 1 > "$SCRIPTDIR/tmp/myList.txt"
In your case you have to give the path to the script anyway. If you put the script in the path where it is automatically searched, e.g. $HOME/bin, and you can just type myScript.sh without the directory prefix, you can use SCRIPTDIR=$( dirname $( which myScript.sh ) ).
Mixing directories with binaries and data files is usually a bad idea. For temporary files /tmp is the place to go. Consider that your script might become famous and get installed by the administrator in /usr/bin and run by several people at the same time. For this reason, try to think mktemp.
YOUR SCRIPT CAN DO THIS FOR YOU WITH SOME CODES
Instead of doing this manually from the command line and who knows where you will move your script and put it. add the following codes
[1] Find your script directory location using dirname
script_directory=`dirname $0`
The above code will find your script directory and save it in a variable.
[2] Create your "tmp" folder in your script directory
mkdir "$script_directory/tmp 2> /dev/null"
The above code will make a directory called "tmp" in your script directory. If the directory exist, mkdir will not overwrite any existing directory using this command line and gave an error. I hide all errors by "2> /dev/null"
[3] Open your script and modify it using "cut" and then redirect the output to a new file
cat "$0"|grep '18x18'|cut -d":" -f1 > "$script_directory"/tmp/myList.txt
I want to write a script that do specific thing:
I have a txt file e.g.
from1/from2/from3/apple.file;/to1/to2/to3;some not important stuff
from1/from2/banana.file;/to1/to5;some not important stuff
from1/from10/plum.file;/to1//to5/to100;some not important stuff
Now i want to copy file from each line (e.g. apple.file), from original directory tree to new, non existing directories, after first semicolon (;).
I try few code examples from similar questions, but nothing works fine and I'm too weak in bash scripting, to find errors.
Please help :)
need to add some conditions:
file not only need to be copy, but also rename. Example line in file.txt:
from1/from2/from3/apple.file;to1/to2/to3/juice.file;some1
from1/from2/banana.file;to1/to5/fresh.file;something different from above
so apple.file need to be copy and rename to juice.file and put in to1/to2/to3/juice.file
I think thaht cp will also rename file but
mkdir -p "$to"
from answer below will create full folder path with juice.file as folder
In addidtion after second semicolon in each line will be something different, so how to cut it off?
Thanks for all help
EDIT: There will be no spaces in input txt file.
Try this code..
cat file | while IFS=';' read from to some_not_important_stuff
do
to=${to:1} # strip off leading space
mkdir -p "$to" # create parent for 'to' if not existing yet
cp -i "$from" "$to" # option -i to get a warning when it would overwrite something
done
Using awk
(run the awk command first and confirm the output is fine, then add |sh to do the copy)
awk -F";" '{printf "cp %s %s\n",$1,$2}' file |sh
Using shell (get updated that need manually create folder, base on alfe's
while IFS=';' read from to X
do
mkdir -p $to
cp $from $to
done < file
I had this same problem and used tar to solve it! Posted here:
tmpfile=/tmp/myfile.tar
files="/some/folder/file1.txt /some/other/folder/file2.txt"
targetfolder=/home/you/somefolder
tar --file="$tmpfile" "$files"
tar --extract --file="$tmpfile" --directory="$targetfolder"
In this case, tar will automatically create all (sub)folders for you! Best,
Nabi
I know my topic is a little confusing, but here is what I want to do.
I have a file which I would like to create a link to in my home directory ~/bin, but when I execute the file that is symbolically linked, the file requires another file in its directory. Therefore, it fails to run because it cannot find the other file. What can I do?
Well, you have two simple solutions.
edit the shell script to point to the absolute path of the file, not just the the basename.
./path/to/file.sh
VS
file.sh
so something like this should do what your after. sed -i 's|file.sh|./path/to/file.sh|g' ~/bin/script.sh it searches your symlinked file, script.sh in this case, and replaces the call to file.sh to ./path/to/file.sh. note you often see sed use /'s. but it can use just about anything as a delimiter, if you wish to use /'s here you will need to escape them. /. you may want to consider escaping the . (period) as well, but in this case its not necessary. If you are new to sed realize that the -i flag means it will edit the file in place. Lastly, realize its a simple search and replace operation and you may chose to do it by hand.
The second way is to create a ln -s to the file as you did with the other file so there exists a symbolic link between both files.
ln -s /far/off/script.sh ~/bin/script.sh
and
ln -s /far/off/file.sh ~/bin/file.sh
more on symlinking
I would rather create a script file in ~/bin/` that calls your executable from the appropriate directory.
Here is an example using /sbin/ifconfig:
$ cat > ~/bin/file
#!/bin/bash
file=/sbin/ifconfig
cd `dirname $file`
`basename $file`
(ctr+d)
$ chmod +x ~/bin/file
$ file
Here you should see the output of ifconfig but the point is: its get executed from the /sbin directory. So if ifconfig had dependencies it would work properly. Just replace /sbin/ifconfig with your absolute path.
Alternatively, you can modify your script as
pushd ~/bin
##### your script here
popd
Combination of readlink and dirname will get the actual directory of the script:
my_dir=$(dirname "$(readlink -f "$0")")
source "$my_dir/other_file"