I have created a script that goes through specific subdirectories of files and tells me how many files are in each sub-directory that start with s. My problem occurs when it is searching and the sub-directory has failed to be created. For some reason, when the sub-directory that this script is searching for does not exist, it replaces the output with another previously created variable????
I am writing this in bash for linux.
I am looking at the following subdirectories...
participantdirectory/EmotMRI
participantdirectory/EmotMRI/firstfour/
participantdirectory/T1
So, this is the output I should get, when the subdirectory exists and everything is ok. It is the same for all files (if it is correct).
/home/orkney_01/jsiegel/ruth_data/participants/analysis2/1206681446/20090303/14693
16 in firstfour
776 in EmotMRI folder
2 files in T1 folder
For a directory which does not have a subdirectory created, I get this output...
bash: cd: /home/orkney_01/jsiegel/ruth_data/participants/analysis2/2102770508/20090210 /14616/EmotMRI/firstfour/: No such file or directory
/home/orkney_01/jsiegel/ruth_data/participants/analysis2/2102770508/20090210/14616
776 in firstfour
114 in EmotMRI folder
2 files in T1 folder
I think that, because firstfour is a subdirectory of EmotMRI, when firstfour folder hasn't been created, it substitutes the scan numbers in EmotMRI for this answer? The number of scans in EmotMRI (in this instance is correct). Here is my script below. If this is happening, how do I stop it from doing this?
for d in $(cat /home/orkney_01/jsiegel/ruth_data/lists/full_participant_list_location_may20)
do
if [ -d "$d" ]
then
gr="failed"
er="failed"
fr="failed"
cd $d/EmotMRI/firstfour/
gr=$(ls s*| wc -l)
echo " "
echo "$d"
echo "$gr in firstfour"
cd $d/EmotMRI/
er=$(ls s*| wc -l)
echo "$er in EmotMRI folder"
cd $d/T1/
fr=$(ls s*| wc -l)
echo "$fr files in T1 folder"
cd $d/EmotMRI
else
echo "$d is currently not available in directory"
fi
done
cd /home/orkney_01/jsiegel/ruth_data/
echo "Check complete"
I know you will probably have many improvements on this script, I am very new to linux. Thanks for your help,
Currently, you set gr to the output of ls s* | wc -l regardless of whether you successfully change your working directory. When that cd fails, it leaves you in whatever directory you were in previously.
You can combine your cd command into your other commands to set gr:
gr=$(cd $d/EmotMRI/firstfour/ && ls s* | wc -l || echo failed)
This way, if you successfully cd into the subdirectory, gr will be set to the output of the commands after &&. Otherwise, gr will be set to the output of the command after the ||. You can do the same thing with er and fr.
You are getting error messages that you shoud fix. Cd is failing because you are not allowed to change into a non-existent directory. Your shell will just stay in the directory it was in. It looks like you know how to test for directory existence, so you should just do that more to avoid trying to go into non-existent directories.
Related
I was wondering what does the ls -A option do on it's own and why does it work in the code below when checking if a directory is empty or not... ?
I can't find any other answers online and the man pages I don't understand what it means when I read the documentation.
Thanks
#!/bin/bash
FILE=""
DIR="/empty_dir"
# init
# look for empty dir
if [ "$(ls -A $DIR)" ]; then
echo "Take action $DIR is not Empty"
else
echo "$DIR is Empty"
fi
# rest of the logic
Nothing related to bash. It tells ls to show all "hidden" files (those whose names start with .) except for . and .. (special directory entries representing the current directory and the parent directory, respectively).
Compare with ls -a, which shows all hidden files including . and ... The presence of those two entries in the output would make the directory appear to be not empty, even though it really is.
I have some csv files in location A like this
abc1.csv,
abc2.csv,
abc3.csv
I have a cron job which runs every 30 mins and in each execution I want to copy only 1 file(which shouldn't be repeated) and placed it in location B
I had though of 2 ways of doing this
1)I will pick the first file in the list of files and copy it to location B and will delete it once copied.Problem with this is I am not sure when the file will get copied completely and if i delete before its completed copied it can be an issue
2)I will have a temp folder.So i will copy the file from location A to location B and also keep it in temp location.In next iteration, when I pick the file from list of files I will compare its existence in the temp file location.If it exists I will move to next file .I think this will be more time consuming etc.
Please suggest if there is any other better way
you can use this bash script for your use case:
source="/path/to/.csv/directory"
dest="/path/to/destination/directory"
cd $source
for file in *.csv
do
if [ ! -f $dest/"$file" ]
then
cp -v $file $dest
break
fi
done
You can ensure you move the already copied file with:
cp abc1.csv destination/ && mv abc1.csv.done
(here you can make your logic to find only *.csv files, and not take into account *.done files.. that have been already processed by your script... or use any suffix you want..
if the cp does not succeed, nothing after that will get executed, so the file will not be moved.
You can also replace mv with rm to delete it:
cp abc1.csv destination/ && rm -f abc1.csv
Further more, you can add to the above commands error messages in case you want to be informed if the cp failed:
cp abc1.csv destination/ && mv abc1.csv.done || echo "copy of file abc1.csv failed"
And get informed via CRON/email output
Finally I took some idea from both the opted solution.Here is the final script
source="/path/to/.csv/directory"
dest="/path/to/destination/directory"
cd $source
for file in *.csv
do
if [ ! -f $dest/"$file" ]
then
cp -v $file $dest || echo "copy of file $file failed"
rm -f $file
break
fi
done
/tmp/target_dir IN_MODIFY,IN_CREATE,IN_MOVED_TO /tmp/script.sh $#
contents of script.sh
echo $1 > /tmp/script.log
on executing
cp -r some_dir /tmp/target_dir
contents of /tmp/script.log
/tmp/target_dir
on executing
cp some_file /tmp/target_dir/some_dir
contents of /tmp/script.log
/tmp/target_dir/some_dir
here instead of watched directory directory on which work is done is echoed.
As far as i understand from here $# is used to display watched system path.
As far as i understand from here $# is used to display watched system path.
Stated there is: When monitoring a directory, the events marked with an asterisk (*) above can occur for files in the directory, in which case the name field in the returned event data identifies the name of the file within the directory.
instead of watched directory directory on which work is done is echoed.
The directory on which work is done is exactly the file within the directory as documented. (Copying some_file into /tmp/target_dir/some_dir has modified /tmp/target_dir/some_dir.)
I'm trying to create a command that allows me move a file to another directory. For example when I enter "move file1" in the command line, it should move the file "file1" to another directory. I know it can simply done as mv file1 /path/to/destination, But I want to create a new command. I'm kind of new Linux user, please help me.
This is what I tried:
Created an alias for move='/home/bin/move.sh'
So, now when I type move in the command line, it goes and execute move.sh script
Started writing a shell script move.sh as :
#!/bin/bash
mv "$2" "/path/to/destination"
I'm not knowing how to proceed. The whole process might be wrong too. Please help me solve this.
Thanks in advance
Create a function:
move () { mv -t /path/to/destination "$#" ;}
put it in ~/.bashrc to make it permanent.
Now run it as :
move /source /files
From here you can read on how to select arguments.
From here you can read more on how to check for number of arguments.
move.sh
#!/bin/bash
if (( $# < 2 )); then
# TODO: print usage
exit 1
fi
mv "$1" "$2"
Then you will need to make it executable.
chmod u+x move.sh
You can remove the .sh part. It wont change anything.
mv move.sh move
And then you should be able to call the file
move asd /home/
Just make sure that the alias calls the correct file.
If you want to make life easier delete the alias and place the file in the /bin/ directory
cp move /bin/
Good luck.
I'm having problems moving files into a folder after I create it in a shell script.
My script looks like:
#!/bin/bash
echo -e "Processing\033[36m" $1 "\033[0mwith the German script";
if [ ! -d ${1%.dat} ]; then
echo -e "making directory\033[33m" ${1%.dat} "\033[0msince it didn't exist...";
mkdir ${1%.dat};
fi
...processing occurs here... (irrelevant to issue)
if [ -d ${1%.dat} ]; then
mv useragents_$1 /${1%.dat}/useragents_$1;
mv summary_$1 /${1%.dat}/summary_$1;
more /${1%.dat}/useragents_$1;
else
echo -e "\033[31mERROR: cannot move files to folder.\033[0m";
fi
As you can see I create the folder if it doesn't exist in the top section and then if it exists I move the files into that folder in the bottom section, the problem is that it doesn't create the folder in time to move the files in (I'm assuming) so when it reaches the lower code, I only get the ERROR.
I tried using, sleep 5, but it only slows down the script and has no effect on the ERROR.
I would really appreciate some advice.
Errors below:
mv: cannot move `useragents_100_stns2_stns6.dat' to `/100_stns2_stns6/useragents_100_stns2_stns6.dat': No such file or directory
mv: cannot move `summary_100_stns2_stns6.dat' to `/100_stns2_stns6/summary_100_stns2_stns6.dat': No such file or directory
/100_stns2_stns6/useragents_100_stns2_stns6.dat: No such file or directory
Pass 1
Your check:
if [ ! -d ${1%.dat} ]; then
should be:
if [ -d ${1%.dat} ]; then
You created the directory; if it is a directory, move stuff into it.
Typo in question
Pass 2
You create:
mkdir ${1%.dat}
You try to move files:
mv useragents_$1 /${1%.dat}/useragents_$1;
Note the leading slash in the move compared to the create. Make those consistent.
Are you sure of this part ? It uses a root directory.
/${1%.dat}/summary_$1;
You probably want to do this instead:
${1%.dat}/summary_$1;
It allows you to move the file into the directory IN your current directory.