I am trying to figure out why bash autocompletion on filesystem is slow on my PC. My Linux machine is connected to an AD through PAM and I am suspecting bash is trying to query a network mount (which is slow since it queries PAM) every time I use TAB for autocomplete.
I have tried set -x and when I do autocomplete on /var the slowest operation is the following line:
[[ /var == ~* ]]
Also, the following line takes a few seconds to execute in bash when I am connected to the network whereas it returns immediately if it is not connected:
TEMP=~*
I would like to know what bash is trying to expand ~* to or find a workaround.
trying running it with strace
for example
strace echo $FOO
if the system is accessing your mount, you will know
Related
I have a script that monitors a specific server, giving me the disk usage, CPU usage, etc. I am using 2 Ubuntu VMs: I run the script on the server using SSH (ssh user#ip < script.sh from the first VM), and I want to make it show values in real time, so I tried 2 approaches I found on here:
1. while loop with clear
The first approach is using a while loop with "clear" to make the script run multiple times, giving new values every time and clearing the previous output like so:
while true
do
clear;
# bunch of code
done
The problem here is that it doesn't clear the terminal, it just keeps printing the new results one after another.
2. watch
The second approach uses watch:
watch -n 1 Script.sh
This works fine on the local machine (to monitor the current machine where the script is), but I can't find a way to make it run via SSH. Something like
ssh user#ip < 'watch -n 1 script.sh'
works in principle, but requires that the script be present on the server, which I want to avoid. Is there any way to run watch for the remote execution (via SSH) of a script that is present on the local machine?
For your second approach (using watch), what you can do instead is to run watch locally (from within the first VM) with an SSH command and piped-in script like this:
watch -n 1 'ssh user#ip < script.sh'
The drawback of this is that it will reconnect in each watch iteration (i.e., once a second), which some server configurations might not allow. See here for how to let SSH re-use the same connection for serial ssh runs.
But if what you want to do is to monitor servers, what I really recommend is to use a monitoring system like 'telegraf'.
I am looking for a good way monitor and log mounts on a CentOS 6.5 box. Since I am new to Linux shell scripting I am somewhat at a loss as to if there is something that is already around and proven which I could just plug in or is there a good method I should direct my research toward to build my own.
In the end what I am hoping to have running is a check of each of the 9 mounts on the server to confirm they are up and working. If there is an issue I would like to log the information to a file, possibly email out the info, and check the next mount. 5-10 minutes later I would like to run it again. I know that probably this isn't needed but we are trying to gather evidence if there is an issue or show to a vendor that what they are saying is the issue is not a problem.
This shell script will test each mountpoint and send mail to root if any of them is not mounted:
#!/bin/bash
while sleep 10m;
do
status=$(for mnt in /mnt/disk1 /mnt/disk2 /mnt/disk3; do mountpoint -q "$mnt" || echo "$mnt missing"; done)
[ "$status" ] && echo "$status" | mail root -s "Missing mount"
done
My intention here is not to give a complete turn-key solution but, instead, to give you a starting point for your research.
To make this fit your precise needs, you will need to learn about bash and shell scripts, cron jobs, and other of Unix's very useful tools.
How it works
#!/bin/bash
This announces that this is a bash script.
while sleep 10m; do
This repeats the commands in the loop once every 10 minutes.
status=$(for mnt in /mnt/disk1 /mnt/disk2 /mnt/disk3; do mountpoint -q "$mnt" || echo "$mnt missing"; done)
This cycles through mount points /mnt/disk1, /mnt/disk2, and /mnt/disk3 and tests that each one is mounted. If it isn't, a message is created and stored in the shell variable status.
You will want to replace /mnt/disk1 /mnt/disk2 /mnt/disk3 with your list of mount points, whatever they are.
This uses the command mountpoint which is standard on modern linux versions. It is part of the util-linux package. It might be missing on old installations.
[ "$status" ] && echo "$status" | mail root -s "Missing mount"
If status contains any messages, they will be mailed to root with the subject line Missing mount.
There are a few different versions of the mail command. You may need to adjust the argument list to work with the version on your system.
done
This marks the end of the while loop.
Notes
The above script uses a while loop that runs the tests every ten minutes. If you are familiar with the cron system, you may want to use that to run the commands every 10 minutes instead of the while loop.
I am trying to transfer a large dataset (768 Gigs) from one remote machine to another using bash on ubuntu 16.04. The problem I appear to be having is that I use rsync and the machine will transfer for a few hours and then quit when the connection inevitably gets interrupted. So suppose Im on machine A and the remote servers are machines B and C (all machines using ubuntu 16.04). I ssh to machine B and use this command:
nohup rsync -P -r -e ssh /path/to/files/on/machine_B user#machine_C:directory &
note that I have the authorized key setup so no password is required between machines B and C
A few hours later I get the following in the nohup file:
sending incremental filelist
file_1.bam
90,310,583,648 100% 36.44MB/s 0:39:23 (xfr#4, to-chk=5/10)
file_2.bam
79,976,321,885 100% 93.25MB/s 0:13:37 (xfr#3, to-chk=6/10)
file_3.bam
88,958,959,616 88% 12.50MB/s 0:15:28 rsync error: unexplained error (code 129) at rsync.c(632) [sender=3.1.1]
rsync: [sender] write error: Broken pipe (32)
I used nohup because I though it would keep running even if there was a hangup. I have not tried sh -c and I have not tried running the command from machine A because at this point whatever I try would be guesswork, ideas would be appreciated.
for those that are interested I also tried running the following script with the nohup command on machine B.
script:
chomp( my #files = `ls /path/to/files/on/machineB/*` );
foreach ( #files ) { system("scp $_ user#machineC:destination/"); }
I still got truncated files.
at the moment the following command appears to be working:
nohup rsync -P --append -r -e ssh /path/to/files/on/machine_B user#machine_C:directory &
you just have to check the nohup file for a broken pipe error and re-enter the command if necessary.
I had the same problem and solved it in multiple steps:
First I made sure that I ran all commands on tmux terminals. This adds a layer of safety on top of nohup, as it keeps connections alive: https://en.wikipedia.org/wiki/Tmux
I combined the rsync command with the while command to enforce that the copy is attempted an infinite number of times even if the pipe breaks:
while ! rsync <your_source> <your_destination>; do echo "Rsync failed. Retrying ..."; done
This approach is brute force and it will work if for each attempt, rsync manages to copy at least a few files. Eventually, even with wasteful repeats and multiple failures, all the files will be copied and my command above will exit gratefully.
Per the title, I'm getting the following warning when I try to scp through an ssh tunnel. In my case, I cannot scp directly to foo because port 1234 on device foo is being forwarded to another machine bar on a private network (and bar is the machine that is giving me a tunnel to 192.168.1.23).
$ # -f and -N don't matter and are only to run this example in one terminal
$ ssh -f -N -p 1234 userA#foo -L3333:192.168.1.23:22
$ scp -P 3333 foo.py ubuntu#localhost:
ubuntu#localhost's password:
stty: standard input: Inappropriate ioctl for device
foo.py 100% 1829 1.8KB/s 00:00
Does anyone know why I might be getting this warning about Inappropriate ioctl for device?
I got the exact same problem when I included the following line on my ~/.bashrc:
stty -ixon
The purpose of this line was to allow the use of Ctrl-s in reverse search of bash.
This gmane link has a solution: (original link dead) => Web Archive version of gmane link
'stty' applies to ttys, which you have for interactive login sessions.
.kshrc is executed for all sessions, including ones where stdin isn't
a tty. The solution, other than moving it to your .profile, is to
make execution conditional on it being an interactive shell.
There are several ways to check for interecative shell. The following solves the problem for bash:
[[ $- == *i* ]] && stty -ixon
Got the same issue while executing the script remotely. After many tries didn't get any luck to solve this error. Then got an article to run a shell script through ssh. This was an issue related to ssh, not any other command. ssh -t "command" -t will allocate a pseudo TTY to the ssh and this error won't come.
at the end i created a blank .cshrc file ( for ubuntu 18.04). worked
So I'm ssh'ing into a router that has several VM's. It is setup using LDAP so that each VM has the same files, settings, etc. However they have different cores allocated, different libraries and packages installed. Instead of logging into each VM individually and running the command, I want to automate it by putting the script in .bashrc.
So what I have so far:
export LD_LIBRARY_PATH=/lhome/username
# .so files are in ~/ to avoid permission denied problems
output=$(cat /proc/cpuinfo | grep "^cpu cores" | uniq | tail -c 2)
current=server_name
if [[ `hostname-s` != $current ]]; then
ssh $current
fi
/path/to/program --hostname $(echo $(hostname -s)) --threads $((output*2))
Each VM, upon logging in, will execute this script, so I have to check if the current VM has the hostname to avoid an SSH loop. The idea is to run the program, then exit back out to the origin to resume the script. The problem is of course that the process will die upon logging out.
It's been suggested to me to use TMUX on an array of the hostnames, but I would have no idea on how to approach this.
You could install clusterSSH, set up a list of hostnames, and execute things from the terminal windows opened. You may use screen/tmux/nohup to allow processes started to keep running, even after logout.
Yet, if you still want to play around with scripting, you may install tmux, and use:
while read host; do
scp "script_to_run_remotely" ${host}:~/
ssh ${host} tmux new-session -d '~/script_to_run_remotely'\; detach
done < hostlist
Note: hostlist should be a list of hostnames, one per line.