I would like to zip a directory, but I need to use a full path and I need that the zip files starts in the directory, not in /.
That question was solved here: Command to zip a directory using a specific directory as the root
The answer would be basically to use this:
If I want to zip full/path/to/directory/ in a file named myFile.zip, I would need to use the command
cd /full/path/to/ && zip -r myFile.zip directory/
Now, my question: is this threadsafe? without && (2 commands) would be unsafe, is && enough to make it threadsafe?
Saying command1 && command2 implies that command2 would be executed only if command1 exits with a return code of zero. If /full/path/to/ doesn't exist than the zip command wouldn't be executed.
Regarding the thread safety part, if you have multiple processes trying to create a file with the same name in a given directory, chances are that it'll lead to unexpected results.
Related
I have a shell script myautoappupgrade.sh where I automate a process of application upgrade. The script has to be run on few different servers. Unfortunately, the application is located in slightly different directory on each server - the number for parent directory varies between 1-20. How I can modify the script, so that the directory can be replaced by some sort of variable? I don't want to edit the script for each server because there are many directory queries in the automation script.
example:
cd /ae1/apps/myapp/upgradefiles/
unzip file.zip
./install.sh
the directory slightly changes on another server:
cd /ae2/apps/myapp/upgradefiles/
unzip file.zip
./install.sh
and another..
cd /ae3/apps/myapp/upgradefiles/
unzip file.zip
./install.sh
Try something like this:
#!/bin/bash
num=$1
cd /ae${num}/apps/myapp/upgradefiles/file.zip
unzip file.zip
./install.sh
Then call the script with the number as first argument:
myautoappupgrade.sh 1
The simple and obvious solution is to not hard-code the directory at all. Modify the script so it accepts the parent directory as an argument, or just cd into the parent directory before running the script.
Perhaps something like this:
while read server dir; do
ssh "$server" "cd '$dir' && unzip apps/myapp/upgradefiles/file.zip/file.zip && ./install.sh"
done <<\:
ernie /ae1
bert /ae2
cookiemonster /home/cmonster/anN
:
It would probably be even better if you unzipped into a temporary directory, but hopefully this should get you moving in the right direction.
Of course, if you can be sure that /ae[0-1] is always there and there is only one match,
cd /ae[0-9]/apps/myapp/upgradefiles/file.zip
would do what you are asking.
(Do you really have a file named file.zip inside a directory also named file.zip? I'm guessing actually take away the file.zip from the end of the cd path.)
By simply using:
cd /ae*/apps/myapp/upgradefiles/
The * will expand any character.
I have dozens of scripts, all in different directories. (exported/expanded Talend jobs)
At this moment each job has 1 or 2 scripts, starting with the same lines, most important one:
CD ***path-to-script***
and several lines to set the Java path and start the job.
I want to create a script, which will be ran from all these scripts.
e.g.:
/scripts/talend.sh
And in all talend scripts, the first line will run /scripts/talend.sh, some examples of where these scripts are ran from:
/talend-job1_0.1/talend-job1_0.1/talend-job1/talend-job1.sh
/talend-task2_0.1/talend-task2_0.1/talend-task2/talend-task2.sh
/talend-job3_0.1/talend-job3_0.1/talend-job3/talend-job3.sh
How can I determine where the /scripts/talend.sh is started from, so I can CD to that path from within /scripts/talend.sh.
The Talend scripts are not run from within the directory itself, but from a cronjob, or a different users home directory.
EDIT:
The question was marked as duplicate, but Getting the source directory of a Bash script from within is not answering my question 100%.
Problem is:
- The basic script is being called from different scripts
- Those different scripts can be run from command line, with, and with or without a symbolic link.
- The $0, the $BASH_SOURCE and the pwd all do some things, but no solution mentioned covers all the difficulties.
Example:
/scripts/talend.sh
In this script I want to configure the $PATH and $HOME_PATH of Java, and CD to the place where the Talend job is placed. (It's a package, so that script MUST be run from that location).
Paths to the jobs are, for example:
/u/talend/talendjob1/sub../../talendjob1.sh
/u/talend/talendjob2/sub../../talendjob2.sh
/u/talend/talendjob3/sub../../talendjob3.sh
Multiple jobs are run from a TMS application. This application cannot run these scripts with the whol name (to long, name can only be 6 long), so in a different location I have symbolic links:
/u/tms/links/p00001 -> /u/talend/talendjob1/sub../../talendjob1.sh
/u/tms/links/p00002 -> /u/talend/talendjob1/sub../../talendjob2.sh
/u/tms/links/p00003 -> /u/talend/talendjob1/sub../../talendjob3.sh
/u/tms/links/p00004 -> /u/talend/talendjob1/sub../../talendjob4.sh
I think you get an overview of the complexity and why I want only one basic talend script, where I can leave all basic stuff. But I only can do that, if I know the source of the Talend script, because there I have to be to start that talend job.
These answers (beyond the first) are specific to Linux, but should be very robust there -- working with directory names containing spaces, literal newlines, wildcard characters, etc.
To change to your own source directory (a FAQ covered elsewhere):
cd "$(basename "$BASH_SOURCE")"
To change to your parent process's current directory:
cd "/proc/$PPID/cwd"
If you want to change to the directory passed as the first command-line argument to your parent process:
{ IFS= read -r -d '' _ && IFS= read -r -d '' argv1; } <"/proc/$PPID/cmdline"
cd "$argv1"
That said, personally, I'd just export the job directory to the environment variable in the parent process, and read that environment variable in the children. Much, much simpler, more portable, more accurate, and compliant with best process.
You can store pwd in a variable and then cd to it when you want to go back
This works for me:
In
/scripts/talend.sh
do
cd ${1%/*}
${1%/*} will strip off everything after the last / effectively providing a dirname for $1, which is the path to the script that calls this one.
and than call the script with the line:
/scripts/talend.sh $0.
Calling the script with $0 passes the name of the current script as an argument to the child which as shown above can be used to cd to the correct directory.
When you source /scripts/talend.sh the current directory is unchanged:
The scripts
# cat /scripts/talend.sh
echo "Talend: $(pwd)"
# cat /talend-job1_0.1/talend-job1_0.1/talend-job1/talend-job1.sh
echo Job1
. /scripts/talend.sh
Executing job1
# cd /talend-job1_0.1/talend-job1_0.1
# talend-job1/talend-job1.sh
Job1
Talend: /talend-job1_0.1/talend-job1_0.1
When you want to see the dir where the calling script is in, see get dir of script.
EDIT:
When you want to have the path of the callling script (talend-job1.sh) without having to cd to that dir first, you should get the dir of the script (see link above) and source talend.sh:
# cat /scripts/talend.sh
cd "$( dirname "${BASH_SOURCE[0]}" )"
echo "Talend: $(pwd)"
In talend.sh get the name of the calling script and then the directory:
parent_cmd=$(ps -o args= $PPID)
set -- $parent_cmd
parent_cmd=$(dirname $2)
Update: as pointed by Charles Duffy in the comments below this will cause havoc when used with paths containing white-space or glob patterns.
If procfs is available you could read the content of /proc/$PPID/cmdline or if portability is a concern do a better parsing of the args.
In /scripts/talend.sh:
cd "$(dirname "$0")"
Or:
cd "$(dirname "$BASH_SOURCE")"
Another one is:
cd "$(dirname "$_")"
#This must be the first line of your script after the shebang line
#Otherwise don't use it
Note: The most reliable of the above is $BASH_SOURCE
I'm searching for just one command — nothing with && or | — that creates a directory and then immediately changes your current directory to the newly-created directory. (This is a question someone got for his exams of "linux-usage", he made a new command that did that, but that didn't give him the points.) This is on a debian server if that matters.
I believe you are looking for this:
mkdir project1 && cd "$_"
define a bash function for that purpose in your $HOME/.bashrc e.g.
function mkdcd () {
mkdir "$1" && cd "$1"
}
then type mkdcd foodir in your interactive shell
So stricto sensu, what you want to achieve is impossible without a shell function containing some && (or at least a ; ) ... In other words, the purpose of the exercise was to make you understand why functions (or aliases) are useful in a shell....
PS it should be a function, not a script (if it was a script, the cd would affect only the [sub-] shell running the script, not the interactive parent shell); it is impossible to make a single command or executable (not a shell function) which would change the directory of the invoking interactive parent shell (because each process has its own current directory, and you can only change the current directory of your own process, not of the invoking shell process).
PPS. In Posix shells you should remove the functionkeyword, and have the first line be mkdcd() {
For oh-my-zsh users: take 'directory_name'
Reference: Official oh-my-zsh github wiki
Putting the following into your .bash_profile (or equivalent) will give you a mkcd command that'll do what you need:
# mkdir, cd into it
mkcd () {
mkdir -p "$*"
cd "$*"
}
This article explains it in more detail
I don't think this is possible but to all people wondering what is the easiest way to do that (that I know of) which doesn't require you to create your own script is:
mkdir /myNewDir/
cd !$
This way you don't need to write the name of the new directory twice.
!$ retrieves the last ($) argument of the last command (!).
(There are more useful shortcuts like that, like !!, !* or !startOfACommandInHistory. Search on the net for more information)
Sadly mkdir /myNewDir/ && cd !$ doesn't work: it retrieves the last of argument of the previous command, not the last one of the mkdir command.
Maybe I'm not fully understanding the question, but
>mkdir temp ; cd temp
makes the temp directory and then changes into that directory.
mkdir temp ; cd temp ; mv ../temp ../myname
You can alias like this:
alias mkcd 'mkdir temp ; cd temp ; mv ../temp ../'
You did not say if you want to name the directory yourself.
cd `mktemp -d`
Will create a temp directory and change into it.
Maybe you can use some shell script.
First line in shell script will create the directory and second line will change to created directory.
I need to write a shell script to run as a cron task, or preferably on creation of a file in a certain folder.
I have an incoming and an outgoing folder (they will be used to log mail). There will be files created with codes as follows...
bmo-001-012-dfd-11 for outgoing and 012-dfd-003-11 for incoming. I need to filter the project/client code (012-dfd) and then place it in a folder in the specific project folder.
Project folders are located in /projects and follow the format 012-dfd. I need to create symbolic links inside the incoming or outgoing folders of the projects, that leads to the correct file in the general incoming and outgoing folders.
/incoming/012-dfd-003-11.pdf -> /projects/012-dfd/incoming/012-dfd-003-11.pdf
/outgoing/bmo-001-012-dfd-11.pdf -> /projects/012-dfd/outgoing/bmo-001-012-dfd-11.pdf
So my questions
How would I make my script run when a file is added to either incoming or outgoing folder
Additionally, is there any associated disadvantages with running upon file modification compared with running as cron task every 5 mins
How would I get the filename of recent (since script last run) files
How would I extract the code from the filename
How would I use the code to create a symlink in the desired folder
EDIT: What I ended up doing...
while inotifywait outgoing; do find -L . -type l -delete; ls outgoing | php -R '
if(
preg_match("/^\w{3}-\d{3}-(\d{3}-\w{3})-\d{2}(.+)$/", $argn, $m)
&& $m[1] && (file_exists("projects/$m[1]/outgoing/$argn") != TRUE)
){
`ln -s $(pwd)/outgoing/$argn projects/$m[1]/outgoing/$argn;`;
}
'; done;
This works quite well - cleaning up deleted symlinks also (with find -L . -type l -delete) but I would prefer to do it without the overhead of calling php. I just don't know bash well enough yet.
Some near-answers for your task breakdown:
On linux, use inotify, possibly through one of its command-line tools, or script language bindings.
See above
Assuming the project name can be extracted thinking positionally from your examples (meaning not only does the project name follows a strict 7-character format, but what precedes it in the outgoing file also does):
echo `basename /incoming/012-dfd-003-11.pdf` | cut -c 1-7
012-dfd
echo `basename /outgoing/bmo-001-012-dfd-11.pdf`| cut -c 9-15
012-dfd
mkdir -p /projects/$i/incoming/ creates directory /projects/012-dfd/incoming/ if i = 012-dfd,
ln -s /incoming/foo /projects/$i/incoming/foo creates a symbolic link from the latter argument, to the preexisting, former file /incoming/foo.
How would I make my script run when a file is added to either incoming or outgoing folder
Additionally, is there any associated disadvantages with running upon file modification compared with running as cron task
every 5 mins
If a 5 minutes delay isn't an issue, I would go for the cron job (it's easier and -IMHO- more flexible)
How would I get the filename of recent (since script last run) files
If your script runs every 5 minutes, then you can tell that all the files created in between now (and now - 5 minutes) are newso, using the command ls or find you can list those files.
How would I extract the code from the filename
You can use the sed command
How would I use the code to create a symlink in the desired folder
Once you have the desired file names, you can usen ln -s command to create the symbolic link
I'm writing a PHP script that downloads a series of generated files (using wget) into a directory, and then zips then up, using the zip command.
The downloads work perfectly, and the zipping mostly works. I run the command:
zip -r /var/www/oraviewer/rgn_download/download/fcst_20100318_0319.zip /var/www/oraviewer/rgn_download/download/fcst_20100318_0319
which yields a zip file with all the downloaded files, but it contains the full /var/www/oraviewer/rgn_download/download/ directories, before reaching the fcst_20100318_0319/ directory.
I'm probably just missing a flag, or something small, from the zip command, but how do I get it to use fcst_20100318_0319/ as the root directory?
I don't think zip has a flag to do that. I think the only way is something like:
cd /var/www/oraviewer/rgn_download/download/ && \
zip -r fcst_20100318_0319.zip fcst_20100318_0319
(The backslash is just for clarity, you can remove it and put everything on one line.)
Since PHP is executing the command in a subshell, it won't change your current directory.
I have also get it worked by using this command
exec('cd '.$_SERVER['DOCUMENT_ROOT'].' && zip -r com.zip "./"');
cd /home/public_html/site/upload/ && zip -r sub_upload.zip sub_upload/
Use the -j or --junk-paths option in your zip command.
From the zip man page:
-j
--junk-paths
Store just the name of a saved file (junk the path), and do not store
directory names. By default, zip will store the full path (relative
to the current directory).