Well, I'm a linux newbie, and I'm having an issue with a simple bash script.
I've got a program that adds to a log file while it's running. Over time that log file gets huge. I'd like to create a startup script which will rename and move the log file before each run, effectively creating separate log files for each run of the program. Here's what I've got so far:
pastebin
DATE=$(date +"%Y%m%d%H%M")
mv server.log logs/$DATE.log
echo program
When run, I see this:
: command not found
program
When I cd to the logs directory and run dir, I see this:
201111211437\r.log\r
What's going on? I'm assuming there's some syntax issue I'm missing, but I can't seem to figure it out.
UPDATE: Thanks to shellter's comment below, I've found the problem to be due to the fact that I'm editing the .sh file in Notepad++ in windows, and then sending via ftp to the server, where I run the file via ssh. After running dos2unix on the file, it works.
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
mv server.log logs/$(date -d "today" +"%Y%m%d%H%M").log
The few lines you posted from your script look okay to me. It's probably something a bit deeper.
You need to find which line is giving you this error. Add set -xv to the top of your script. This will print out the line number and the command that's being executed to STDERR. This will help you identify where in your script you're getting this particular error.
BTW, do you have a shebang at the top of your script? When I see something like this, I normally expect its an issue with the Shebang. For example, if you had #! /bin/bash on top, but your bash interpreter is located in /usr/bin/bash, you'll see this error.
EDIT
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
Two ways:
Select the Edit->EOL Conversion->Unix Format menu item when you edit a file. Once it has the correct line endings, Notepad++ will keep them.
To make sure all new files have the correct line endings, go to the Settings->Preferences menu item, and pull up the Preferences dialog box. Select the New Document/Default Directory tab. Under New Document and Format, select the Unix radio button. Click the Close button.
A single line method within bash works like this.
[some out put] >$(date "+%Y.%m.%d-%H.%M.%S").ver
will create a file with a timestamp name with ver extension.
A working file listing snap shot to a date stamp file name as follows can show it working.
find . -type f -exec ls -la {} \; | cut -d ' ' -f 6- >$(date "+%Y.%m.%d-%H.%M.%S").ver
Of course
cat somefile.log > $(date "+%Y.%m.%d-%H.%M.%S").ver
or even simpler
ls > $(date "+%Y.%m.%d-%H.%M.%S").ver
I use this command for simple rotate a file:
mv output.log `date +%F`-output.log
In local folder I have 2019-09-25-output.log
Well, it's not a direct answer to your question, but there's a tool in GNU/Linux whose job is to rotate log files on regular basis, keeping old ones zipped up to a certain limit. It's logrotate
You can write your scripts in notepad but just make sure you convert them
using this ->
$ sed -i 's/\r$//' yourscripthere
I use it all they time when I'm working in cygwin and it works. Hope this helps
First, thanks for the answers above! They lead to my solution.
I added this alias to my .bashrc file:
alias now='date +%Y-%m-%d-%H.%M.%S'
Now when I want to put a time stamp on a file such as a build log I can do this:
mvn clean install | tee build-$(now).log
and I get a file name like:
build-2021-02-04-03.12.12.log
Related
We have a linux server and for some transactions it is keeping the log files only for the last 10 days. After than the file gets deleted. I want to copy these files to another location using a script. I searched google but couldn't satisfactory result. I'm new to Linux also.
Can someone please guide me if this can be achieved and how ?
You can use the previous answer by nissim abehcera in a sh script:
cp -R SOURCE_DIRECTORY DESTINATION_DIRECTORY
Just paste the bash commands in a text file, name it file.sh and make sure it is executable:
chmod +x file.sh
You can just run the script and it will do whatever you wrote in there.
I seem to be having an issue.
I'm trying to write a batch file that uses Linux commands such as rm, mv, clear, and cat within a Windows batch file, but the catch is I can't seem to figure out what I need to do in order for the Windows command line to recognize that when I type in mv I want it to move a file for me, or rm to remove a file of course.
So far all I have figured out is that I could possibly use __DOSKEY__ but it doesn't work in batch files or with parameters (doh!). Thus, all I have gotten so far is:
#echo off
mv dummy.txt
Now my question is how do I get the Windows command line to recognize that mv = move ? Everytime I run the file it just gives me a blank command line.
I know this may sound stupid but my experience is more on the Linux side of the command line than the Windows side, and any help would be greatly appreciated!
Internal cmd commands cannot be aliased. For external commands, you could create hardlinks, but "move" for example is an internal command.
I used logrotate to create a log file that gets rolled over every day at 03:00; so the file that is created has the following format:
userlog.%Y-%m-%d
the script then zips the file as well so the end result is userlog.%Y-%m-%d.gz .... an actual file name is userlog.2015-09-09.gz.
I am writing a shell script that will copy the file created every day and will then unzip it, run a search on the log, extract results, format them, and then email them, and then delete the copied zip file.
I can get everything to work smoothly but I cannot copy the file using the method that it was created in.
I am trying to run the following:
sudo cp userlog.%Y-%d-%m.gz /home/local/temp
but the command does not execute. My understanding is that the current date, month, and year should be substituted into the variable fields in the file name.
If you could please correct my approach and understanding of this concept. Or please let me know if you feel this should work. Or if there is an alternate approach to call a file created in this fashion then kindly advise.
Presently I am having to write the filename into the script manually every morning and I would very much like to escape this hardship.
You need a simple command substitution.
sudo cp userlog.$(date +%Y-%d-%m).gz /home/local/temp
%Y by itself is just a static string; but when passed as an argument to the date command, it specifies the four-digit year for the given date. (Default is today's date.)
Regardless of your problems with this particular command, you should certainly not need to edit the script file every day. A very fundamental feature of scripts is the parametrization of arguments. If your current script looks like
#!/bin/bash
gzgrep -c "failure" /path/to/userlog.2015-09-11.gz
sudo cp /path/to/userlog.2015-09-11.gz /home/local/temp
then it can easily be refactored to read the date as a command-line parameter:
#!/bin/bash
: ${1?"Syntax: $0 yyyy-mm-dd"}
gzgrep -c "failure" /path/to/userlog."$1".gz
sudo cp /path/to/userlog."$1".gz /home/local/temp
If you saved this as userlog somewhere in your PATH, the solution to your original problem would now be simply
userlog $(date +%Y-%m-%d)
which of course could be saved as a script, too, or as a shell function in your personal .bash_profile or similar. But if this is mandatory daily routine, you'll probably prefer to add it as a cron job to run every morning (even if you are away). Cron will send you an email with any output. (Do notice that cron does not necessarily have the same PATH etc as your interactive login shell, though.)
I have a RedHat linux box and I had written a script in the past to move files from one location to another with a specific text in the body of the file.
I typically only write scripts once a year so every year I forget more and more... That being said,
Last year I wrote this script and used it and it worked.
For some reason, I can not get it to work today and I know it's a simple issue and I shouldn't even be asking for help but for some reason I'm just not looking at it correctly today.
Here is the script.
ls -1 /var/text.old | while read file
do
grep -q "to.move" $file && mv $file /var/text.old/TBD
done
I'm listing all the files inside the /var/text.old directory.
I'm reading each file
then I'm grep'ing for "to.move" and holing the results
then I'm moving the resulting found files to the folder /var/text.old/TBD
I am an admin and I have rights to the above files and folders.
I can see the data in each file
I can mv them manually
I have use pwd to grab the correct spelling of the directory.
If anyone can just help me to see what the heck I'm missing here that would really make my day.
Thanks in advance.
UPDATE:
The files I need to move do not have Whitespaces.
The Error I'm getting is as follows:
grep: 9829563.msg: No such file or directory
NOTE: the file "982953.msg" is one of the files I need to move.
Also note: I'm getting this error for every file in the directory that I'm listing.
You didn't post any error, but I'm gonna take a guess and say that you have a filename with a space or special shell character.
Let's say you have 3 files, and ls -1 gives us:
hello
world
hey there
Now, while splits on the value of the special $IFS variable, which is set to <space><tab><newline> by default.
So instead of looping of 3 values like you expect (hello, world, and hey there), you loop over 4 values (hello, world, hey, and there).
To fix this, we can do 2 things:
Set IFS to only a newline:
IFS="
"
ls -1 /var/text.old | while read file
...
In general, I like setting IFS to a newline at the start of the script, since I consider this to be slightly "safer", but opinions on this probably vary.
But much better is to not parse the output of ls, and use for:
for file in /var/text.old/*`; do
This won't fork any external processes (piping to ls to while starts 2), and behaves "less surprising" in other ways. See here for some examples.
The second problem is that you're not quoting $file. You should always quote pathnames with double quoted: "$file" for the same reasons. If $file has a space (or a special shell character, such as *, the meaning of your command changes:
file=hey\ *
mv $file /var/text.old/TBD
Becomes:
mv hey * /var/text.old/TBD
Which is obviously very different from what you intended! What you intended was:
mv "hey *" /var/text.old/TBD
I'm in my first semester of BASH Scripting and I am having some difficulty. I've taken other programming courses like C++ or Java but the syntax of Bash is killing me. I'd love some advice on this problem. I need to do the following:
Extract Today's data from /var/log/secure file
Check to see if I have a directory called 'mylogs'
If I don't - then create one
Check to see if you already have a file matching the current day, month and hour
in the ‘mylogs’ directory.
If you do, echo to the screen “File exists, nothing written to my log”, and
exit. If it doesn’t exist then write today’s data from /var/log/secure to your
‘mylog-month-day-hour’ file. Example (February, 4th at 2pm) output:
mylog-02-04-14
I just need help with the syntax portion of the script.
Thanks - I'd love any websites helping out in BASH as well.
Extract Today's data from /var/log/secure file
You could do this ...
grep "^Feb 24" /var/log/secure
Check to see if I have a directory called 'mylogs' and If I don't - then create one
You can do this ...
test -d mylogs || mkdir mylogs
Check to see if you already have a file matching the current day, month and hour in the ‘mylogs’ directory. (Assuming file names are of the format DDMMHH)
test -e mylogs/`date +%d%m%H` && echo "I already have a file"
If you do, echo to the screen “File exists, nothing written to my log”, and exit. If it doesn’t exist then write today’s data from /var/log/secure to your ‘mylog-month-day-hour’ file. Example (February, 4th at 2pm) output: mylog-02-04-14
Eh you should get the idea by now. You can tackle this one now I think ;) A helpful command to know is man -k <keyword>
First read some bash basics. Then, there are links describing your particular problem.
http://www.cyberciti.biz/faq/linux-howto-unzip-files-in-root-directory/
Check if a directory exists in a shell script
How to use Bash to create a folder if it doesn't already exist?
http://www.electrictoolbox.com/test-file-exists-bash-shell/
http://www.cyberciti.biz/tips/howto-linux-unix-write-to-syslog.html