Disclaimer: I am very new to Bash scripting (and Linux in general), so forgive me for a stupid question.
A friend of mine gave me a script which makes a backup copy of certain files onto Dropbox. Here's the code in full:
#!/bin/sh
DATE=`date +%Y-%m-%d`
tarname='backup-'$DATE'.tar.gz'
cd ~/
directoriesToBack='.bashrc Desktop/School/ Desktop/Research\ Project'
tar -X ~/Desktop/My\ Programs/scripts/crons/exclude.txt -zcvf $tarname $directoriesToBack
mv $tarname ~/Dropbox
The variable directoriesToBack obviously contains the directories to be copied. Exclude.txt is a text file of files which are not to be backed up.
If I try to run this script, I get an error because of Desktop/Research Project: my computer looks for the directory Desktop/Research instead. I've tried to use double quotes instead of single quotes, and to replace \ with an ordinary space, but these tries didn't work. Does anyone know how I can make a backup of a directory with spaces in its name?
Don't try to do this with strings. It will not work and it will cause pain. See I'm trying to put a command in a variable, but the complex cases always fail! for various details and discussion.
Use an array instead.
#!/bin/bash
DATE=$(date +%Y-%m-%d)
tarname=backup-$DATE.tar.gz
cd ~/
directoriesToBack=(.bashrc Desktop/School "Desktop/Research Project")
tar -X ~/Desktop/My\ Programs/scripts/crons/exclude.txt -zcvf "$tarname" "${directoriesToBack[#]}"
I also fixed the quoting of variables/etc. and used $() instead of backticks for the date command execution (as $() can be nested and generally has better semantics and behaviour).
Please run the script and show the EXACT error message. I suspect that what is going wrong is not what you think it is. I suspect that the envar directoriesToBack is not what you think it is.
cd Desktop/"Research Project" (With Quotation marks)
You'll find that a lot of code in many languages use Quotes to signify a space.
Related
I want to replace double spaces with one space in the filenames of a lot of photos. These photos are located in directory /foto and it's subfolders. How to do this? For example "photo 1.jpg" needs to become "photo 1.jpg"
The best way is to use commandline, because it's on CloudLinux server. (and it is over 50GB of photos). I searched here on Stackoverflow, also Google to find the command I need. I guess rename is the one to use, or mv.
The only things I found were commands about replacing space and replacing other symbols, but not about double (multiple) spaces.
find -iname \*.* | rename -v "s/\s{2}/ /g"
This is the final command which helped me out. I used perl rename, see answer by Gilles
Use this, using Perl's rename :
rename 's/\s{2}/ /g' files*
Remove -n switch when the output looks good.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (GNU)
$ file "$(readlink -f "$(type -p rename)")"
and you have a result that contains Perl script, ASCII text executable and not containing ELF, then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
Replace /path/to/rename to the path of your perl rename executable.
If you don't have this command, search your package manager to install it or do it manually (no deps...)
This tool was originally written by Larry Wall, the Perl's dad.
I have a RedHat linux box and I had written a script in the past to move files from one location to another with a specific text in the body of the file.
I typically only write scripts once a year so every year I forget more and more... That being said,
Last year I wrote this script and used it and it worked.
For some reason, I can not get it to work today and I know it's a simple issue and I shouldn't even be asking for help but for some reason I'm just not looking at it correctly today.
Here is the script.
ls -1 /var/text.old | while read file
do
grep -q "to.move" $file && mv $file /var/text.old/TBD
done
I'm listing all the files inside the /var/text.old directory.
I'm reading each file
then I'm grep'ing for "to.move" and holing the results
then I'm moving the resulting found files to the folder /var/text.old/TBD
I am an admin and I have rights to the above files and folders.
I can see the data in each file
I can mv them manually
I have use pwd to grab the correct spelling of the directory.
If anyone can just help me to see what the heck I'm missing here that would really make my day.
Thanks in advance.
UPDATE:
The files I need to move do not have Whitespaces.
The Error I'm getting is as follows:
grep: 9829563.msg: No such file or directory
NOTE: the file "982953.msg" is one of the files I need to move.
Also note: I'm getting this error for every file in the directory that I'm listing.
You didn't post any error, but I'm gonna take a guess and say that you have a filename with a space or special shell character.
Let's say you have 3 files, and ls -1 gives us:
hello
world
hey there
Now, while splits on the value of the special $IFS variable, which is set to <space><tab><newline> by default.
So instead of looping of 3 values like you expect (hello, world, and hey there), you loop over 4 values (hello, world, hey, and there).
To fix this, we can do 2 things:
Set IFS to only a newline:
IFS="
"
ls -1 /var/text.old | while read file
...
In general, I like setting IFS to a newline at the start of the script, since I consider this to be slightly "safer", but opinions on this probably vary.
But much better is to not parse the output of ls, and use for:
for file in /var/text.old/*`; do
This won't fork any external processes (piping to ls to while starts 2), and behaves "less surprising" in other ways. See here for some examples.
The second problem is that you're not quoting $file. You should always quote pathnames with double quoted: "$file" for the same reasons. If $file has a space (or a special shell character, such as *, the meaning of your command changes:
file=hey\ *
mv $file /var/text.old/TBD
Becomes:
mv hey * /var/text.old/TBD
Which is obviously very different from what you intended! What you intended was:
mv "hey *" /var/text.old/TBD
I am building a source file with some alias to executable files (these are working just fine) and assigning directories to variables in order to get to the directory quicker, with less typing. For example, if I source example.source:
#!/usr/bin/bash
mydir="/path/to/some/dir"
I can get to /path/to/some/dir with
cd $mydir
However, I am not being able to use tab complete to navigate through other sub-directories like I would do by typing the complete path. I mean, if I use the tab key to complete the variable I get cd $mydir but not cd $mydir/ (I have to delete the last space character and manually type the slash / to see the next sub-directories). Hope this is an understandable question. Is there any workaround for this?
EDIT: the linux distribution I'm using is Slackware Linux 3.2.31.c x86_64 GenuineIntel GNU/Linux
EDIT2: GNU bash, version 4.2.37(2)-release
Apparently this feature is starting to be implemented in bash 4.3, release 26-Feb-2014 09:25.
Reading the NEWS file in bash 4.3 I found this:
i. The word completion code checks whether or not a filename
containing a
shell variable expands to a directory name and appends `/' to the word
as appropriate. The same code expands shell variables in command names
when performing command completion.
Unfortunately I cannot do a de novo installation of bash (because I'm working on a server) but I hope this can help others.
If I understand your question, then I believe it can be solved by putting this at the top of your example.source. This will list your contents every-time that you cd.
#!/usr/bin/bash
# Make cd change directories and then list the contents
function cd() {
builtin cd $*;
ls;
}
mydir="/path/to/some/dir"
cd $mydir
My other suggestion is to try to put cd within your alias. Something like this:
mydir="cd /path/to/some/dir"
$mydir
I am very much a beginner at this and have searched for answers my question but have not found any that I understand how to implement. Any help would be greatly appreciated.
I have a script:
FILE$=`ls ~/Desktop/File_Converted/`
mkdir /tmp/$FILE
mv ~/Desktop/File_Converted/* /tmp/$FILE/
So I can use Applescript to say when a file is dropped into this desktop folder, create a temp directory, move the file there and the do other stuff. I then delete the temp directory. This is fine as far as it goes, but the problem is that if another file is dropped into File_Converted directory before I am doing doing stuff to the file I am currently working with it will change the value of the $FILE variable before the script has completed operating on the current file.
What I'd like to do is use a variable set up where the variable is, say, $FILE1. I check to see if $FILE1 is defined and, if not, use it. If it is defined, then try $FILE2, etc... In the end, when I am done, I want to reclaim the variable so $FILE1 get set back to null again and the next file dropped into the File_Converted folder can use it again.
Any help would be greatly appreciated. I'm new to this so I don't know where to begin.
Thanks!
Dan
Your question is a little difficult to parse, but I think you're not really understanding shell globs or looping constructs. The globs are expanded based on what's there now, not what might be there earlier or later.
DIR=$(mktemp -d)
mv ~/Desktop/File_Converted/* "$DIR"
cd "$DIR"
for file in *; do
: # whatever you want to do to "$file"
done
You don't need a LIFO -- multiple copies of the script run for different events won't have conflict over their variable names. What they will conflict on is shared temporary directories, and you should use mktemp -d to create a temporary directory with a new, unique, and guaranteed-nonconflicting name every time your script is run.
tempdir=$(mktemp -t -d mytemp.XXXXXX)
mv ~/Desktop/File_Converted/* "$tempdir"
cd "$tempdir"
for f in *; do
...whatever...
done
What you describe is a classic race condition, in which it is not clear that one operation will finish before a conflicting operation starts. These are not easy to handle, but you will learn so much about scripting and programming by handling them that it is well worth the effort to do so, even just for learning's sake.
I would recommend that you start by reviewing the lockfile or flock manpage. Try some experiments. It looks as though you probably have the right aptitude for this, for you are asking exactly the right questions.
By the way, I suspect that you want to kill the $ in
FILE$=`ls ~/Desktop/File_Converted/`
Incidentally, #CharlesDuffy correctly observes that "using ls in scripts is indicative of something being done wrong in and of itself. See mywiki.wooledge.org/ParsingLs and mywiki.wooledge.org/BashPitfalls." One suspects that the suggested lockfile exercise will clear up both points, though it will probably take you several hours to work through it.
I'm trying to use something along the lines of
unexpand -t 4 *.php
but am unsure how to write this command to do what I want.
Weirdly,
unexpand -t 4 file.php > file.php
gives me an empty file. (i.e. overwriting file.php with nothing)
I can specify multiple files okay, but don't know how to then overwrite each file.
I could use my IDE, but there are ~67000 instances of to be replaced over 200 files, and this will take a while.
I expect that the answers to my question(s) will be standard unix fare, but I'm still learning...
You can very seldom use output redirection to replace the input. Replacing works with commands that support it internally (since they then do the basic steps themselves). From the shell level, it's far better to work in two steps, like so:
Do the operation on foo, creating foo.tmp
Move (rename) foo.tmp to foo, overwriting the original
This will be fast. It will require a bit more disk space, but if you do both steps before continuing to the next file, you will only need as much extra space as the largest single file, this should not be a problem.
Sketch script:
for a in *.php
do
unexpand -t 4 $a >$a-notab
mv $a-notab $a
done
You could do better (error-checking, and so on), but that is the basic outline.
Here's the command I used:
for p in $(find . -iname "*.js")
do
unexpand -t 4 $(dirname $p)/"$(basename $p)" > $(dirname $p)/"$(basename $p)-tab"
mv $(dirname $p)/"$(basename $p)-tab" $(dirname $p)/"$(basename $p)"
done
This version changes all files within the directory hierarchy rooted at the current working directory.
In my case, I only wanted to make this change to .js files; you can omit the iname clause from find if you wish, or use different args to cast your net differently.
My version wraps filenames in quotes, but it doesn't use quotes around 'interesting' directory names that appear in the paths of matching files.
To get it all on one line, add a semi after lines 1, 3, & 4.
This is potentially dangerous, so make a backup or use git before running the command. If you're using git, you can verify that only whitespace was changed with git diff -w.