I have as iperf.sh shell script in multiple sub servers that runs at every " 1,14,28,42,50 * * * * " and pings the iperf server to check bandwidth , is there any way to randomize this cron or setting up a shell script that sleeps and runs at random time...?
[ Note : The issue that i am facing with this classic cron system is all sub-servers are running the iperf.sh script at the same time and my main-Iperf server is getting high cpu utilization which is resulting to improper ping data. ]
Thanks In Advance.
You can add a randomized wait period at the start of your script (or even in the crontab itself, as suggested in the comments).
I recommend GNU shuf which will be more portable than $RANDOM (since not all shells will support it, e.g. dash won't).
sleep $(shuf -i5-20 -n1)
# Rest of script
You can experiment with the range of random wait periods (5 to 20 seconds in this example).
Related
I have a function in a bash script that executes a long process called runBatch. Basically runBatch takes a file as an argument and loads the contents into a db. (runBatch is just a wrapper function for a database command that loads the content of the file)
My function has a loop that looks something like the below, where I am currently recording start time and elapsed time for the process to variables.
for batchFile in `ls $batchFilesDir`
do
echo "Batch file is $batchFile"
START_TIME=$(($(date +%s%N)/1000000))
runBatch $batchFile
ELAPSED_TIME=$(($(($(date +%s%N)/1000000))-START_TIME))
IN_SECONDS=$(awk "BEGIN {printf \"%.2f\",${ELAPSED_TIME}/1000}")
done
Then I am writing some information on each batch (such as time, etc.) to a table in a html page I am generating.
How would I go about recording the highest memory/cpu usage while the runBatch is running, along with the time, etc?
Any help appreciated.
Edit: I managed to get this done. I added a wrapper script around this script that runs this script in the background. I pass it's PID with $! to another script in the wrapper script that monitors the processes CPU and Memory usage with top every second. I compile everything into a html page at the end when the PID is no longer alive. Cheers for the pointers.
You should be able to get the PID of the process using $!,
runBatch $batchFile &
myPID=$!
and then you can run a top -b -p $myPID to print out a ticking summary of CPU.
Memory:
cat /proc/meminfo
Next grep whatever you want,
Cpu, it is more complicated - /proc/stat expained
Average load:
cat /proc/loadavg
For timing "runBatch" use
time runBatch
like
time sleep 10
Once you've got the pid of your process (e.g. like answered here) you can use (with watch(1) & cat(1) or grep(1)) the proc(5) file system, e.g.
watch cat /proc/$myPID/stat
(or use /proc/$myPID/status or /proc/$myPID/statm, or /proc/$myPID/maps for the address space, etc...)
BTW, to run batch jobs you should consider batch (and you might look into crontab(5) to run things periodically)
I am trying to create some capacity planning reports and one of the requrements is to have info on Memory usage for a few Unix Servers.
Now my knowledge of Unix is very low. I usually just log on and run a few scripts.
But for this report I need to gather VMStat data and produce reports based on previous the previous weeks data broken down by hour which is an average of Vmstat data taken every 10 seconds.
So first question: is VMStat logging on by default and if so what location on the server is the data output to?
If not how can I set this up?
Thanks
vmstat is a command that you run.
To generate one week of Virtual Memory stats spaced out at ten second intervals (less the last one) is 60,479 10 second intervals
So the command you want is:
nohup vmstat 10 604879 > myvmstatfile.dat &
This will make a very big file myvmstatfile.dat
EDIT: RobKielty (The & will put this job in the background, the nohup will prevent the task from hanging up when you logout of the command shell. If you ran this command it would be prudent to monitor the disk partition to which this file was being written to. Use df -h /path/to/directory/where/outputfile/resides to monitor the disk space usage.)
I have no idea what you need to do with the data, so I can't help you there.
Create a crontab entry (crontab -e) like this
0 0 * * 0 /path/to/my/vmstat_script.sh
The file vmstat_script.sh will contain the follow bash script commands.
#!/bin/bash
# vmstat_script.sh
vmstat 10 604879 > myvmstatfile.dat
mv myvmstatfile.dat myvmstatfile.dat.`date +%Y-%m-%d`
This will create one file per week with a name like myvmstatfile.dat.2012-07-01
The command I use for monitoring the Linux vm metrics is below:
nohup vmstat 10 720| (while read; do echo "$(date +%d-%m-%Y" "%H:%M:%S) $REPLY"; done) >> nameofLogfile.log
Here nohup is used for running the process in background.
It will run for 2 hours with interval of 10 secs.
This is the best command for generating graphs and reports as timestamp will also be included in logs along with different metrics, so that we can filter the logs accordingly.
I have to write a script where I take a tcpdump on my machine and on a remote machine "simultaneously". That is the beginning of capture (0th second) should be simultaneous, so that I can compare the two tcpdumps in my analysis.
Is there a way I can achieve this?
If you just need approximate time (e.g. with a margin of error in range of, say, 200ms), then just make sure both machines have the same time (e.g. via NTP) and then use e.g. cron to run both commands at the same time.
If you want this to be more often, you might want to use at command instead of cron. You can do some simple date arithmetics, e.g. see this:
Bash date/time arithmetic
or sleep until the specified time:
Bash: Sleep until a specific time/date
in both scripts (i.e. local and remote), then run the local command and run the command on the remote machine using ssh.
If you are OK to use e.g. Python, you can make the use of datetime module, e.g. see this:
http://www.doughellmann.com/PyMOTW/datetime/
The idea is pretty much this:
Take current time
Calculate target time - add some cushion seconds (e.g. 10 seconds)
Run both scripts with that time as the parameter (one locally, one remotely with ssh)
Sleep until that time in both scripts - if you cannot ssh in 10 seconds or even worse if it takes more than 10 seconds to run local script, you have more serious problems than this one :)
Run tcpdump in both scripts - they should be pretty much synced up (with some tolerance, but I don't think it will ever go over 50ms on any recent system)
Hope this helps.
Here's something I wrote just now to synchronise multiple test clients:
#!/usr/bin/python
import time
import sys
now = time.time()
mod = float(sys.argv[1])
until = now - now % mod + mod
print "sleeping until", until
while True:
delta = until - time.time()
if delta <= 0:
print "done sleeping ", time.time()
break
time.sleep(delta / 2)
This script sleeps until next "rounded" or "sharp" time.
A simple use case is to run ./sleep.py 10; ./test_client1.py in one terminal and ./sleep.py 10; ./test_client2.py in another.
You want to make sure clocks on your machines are synchronised.
Alternatively, use one of these options in tcpdump, use something that gives you full timestamp.
-t
Don't print a timestamp on each dump line.
-tt
Print an unformatted timestamp on each dump line.
-ttt
Print a delta (micro-second resolution) between current and previous line on each dump line.
-tttt
Print a timestamp in default format proceeded by date on each dump line.
-ttttt
Print a delta (micro-second resolution) between current and first line on each dump line.
Finally you could run something like execnet to start commands on multiple machines at (almost) the same time.
Does anybody know how to see the processes for all users using top command in Cygwin (part of procps library under System).
I know this can be done in *nix but I am struggling in Cygwin. I have tried using pslist but it does not behave in a putty SSH console.
I need to have a solution where I can see a top like dialog using SSH. I do not have any NTLM SSO access to the Win2k3 guest at all so ssh is the only way in.
top only displays Cygwin processes. ps -W will list Windows processes as well.
Manytimes the command "tasklist" gets the job done more effectively. It built into windows, just make sure your System32 folder is part of your bash profile PATH. There is also procps itself. You should also try using mintty for your terminal. You could always try attaching any of these task apps to screen, and or using watch to poll the information.
It seems you can do something like:
wmic process get ProcessId,Name,UserModeTime,KernelModeTime /EVERY:1
The User and Kernel mode times there seem to be expressed in 1/10,000,000th of second.
You should be able to post-process that output to get the CPU-usage per second.
Here using cygwin's perl:
wmic process get ProcessId,Name,UserModeTime,KernelModeTime /EVERY:1 |
perl -lne '
if (/\S/) {
my ($k,$c,$p,$u) = split /\s{2,}/;
$n{"$p\t$c"}=$k+$u;
} else {
my %c;
for my $k (keys %n) {
$c{$k} = $n{$k} - $o{$k} if defined $o{$k}
}
print "$_\t" . $c{$_}/1e5 for (sort {$c{$b}<=>$c{$a}} keys %c)[0..20];
%o = %n; %n = undef; print ""
}'
Outputs something like:
0 System Idle Process 588.12377
2196 sh.exe 107.00075
248 svchost.exe 85.80055
7140 explorer.exe 26.52017
[...]
every second.
Note that if the System Idle Process shows just under 800% on an idle system, that's because your system has 8 CPU cores (well at least 8 threads) as that counts the CPU time of all CPUs.
Also note that the EVERY:1 above is a lie. wmic doesn't seem to give that output every second. More likely, it sleeps roughly 1 second between each report and doesn't compensate for the time it takes to compute the report. So in practice, it will run every 1 second and a bit which means those percentages are not very accurate and slightly overestimated.
i want to write a script in debian , which will make an application stop at a certain system time . Before anyone thinks its a homework , it is not . My internet connection allows unlimited download only in a certain time period , so i would like to stop the torrent application once the time period is over with .
Cron (http://en.wikipedia.org/wiki/Cron) should be your friend here. Add 2 cron jobs to your crontab, one to start the script and one to stop it.
Check into crontab -e - you want something like this in there:
0 2 * * * /path/to/yourtorrentclient start
0 6 * * * /path/to/yourtorrentclient stop
if you want to turn it on at 02:00 every day, and turn it off at 06:00 every day, and assuming yourtorrentclient responds to arguments start and stop. Rewrite as needed.