Trying to use SCP to copy multiple files from remote to local using script - linux

So I'll start with the fact that I'm relatively new to linux scripting, so if I am going about it the wrong way, let me know.
I am creating a script that is meant to copy logs from many different hosts onto the local machine depending on user input.
One of the functions I am writing requires the use of scp. Each time you use the scp command at a particular remote host, you have to enter your password. So to save time for the user, I want to copy any file that the particular host may have on it that the user wants.
I know I can do this using scp user#Remoteipaddress:'directory/file1 directory/file2' local/machine/directory
I have it running (what I feel is too many, so if there is a better way let me know) a bunch of loops.
The portion with the scp command is my main issue. Code looks fine if I quote it and echo it. I can even copy and paste the echoed result and it will work, but if I let the script do it I receive bash: -c: line 0: unexpected EOF while looking for matching `''
edit: $app is a static number created in another portion of program
added a couple things that seemed to be missing. I'm trying to piece together from multiple areas of program without making it more messy than it already is
#assigns different remote host paths do array variable
until [ $scriptCounter == $app ]
do
scpScript[$scriptCounter]="user#${ipAddress[$ipCounter]}:'"
((++ipCounter))
((++scriptCounter))
done
#$app value gets set by another function - typically 3 if that matters
scpCount=0
DayCounter=0
ipScriptCounter=0
until [ $Count == $app ]
do
((++scpCount))
mkdir ~/MyDocuments/Logs/$3/app$scpCount
echo "Creating ~/MyDocuments/Logs/${3}/app${scpCount}"
#there is one log for each day, $totalDiffDays is the total amount of days
#$DayCounter is set and gets marked up everytime it goes through loop until
it matches total days
until [ $DayCounter == $totalDiffDays ]
do
scpPath[$DayCounter]="/var/log/docker/theLog*${datePath[$DayCounter]}*"
noSpaceSCP[$DayCounter]=${scpPath[$DayCounter]//[[:blank:]]/}
((++DayCounter))
done
fullSCPscript[$scpCount]="${scpScript[$ipScriptCounter]}${noSpaceSCP[*]}'"
#this portion I have an issue with.
scp ${fullSCPscript[$scpCount]} ~/MyDocuments/Logs/$3/app$scpCount
#this ups the array counter for my ipaddress array
((++ipScriptCounter))
#How im zeroing out the $DayCounter so it will run through again for other
nodes but with different IP address
until [ $DayCounter == "0" ]
do
((--DayCounter))
done
done
example output i get when I echo the line with the scp command
scp user#10.10.200.100:'/var/log/docker/theLog*2018-07-26* /var/log/docker/theLog*2018-07-27*' /home/mobaxterm/MyDocuments/Logs/care3/app1
I'm sorry that this looks messy, but overall I'm trying to build the directory that its grabbing the log from, and if there are multiple days, just add onto the scp command. I'm trying to do this as opposed to running a whole separate command to save the user from entering their password 5 times if they need 5 files. Instead they would only have to enter it once.

Related

Is there a bash function for determining number of variables from a read provided from end user

I am currently working on a small command line interface tool that someone not very familiar with bash could run from their computer. I have changed content for confidentiality, however functionality has remained the same.
The user would be given a prompt
the user would then respond with their answer(s)
From this, I would be given two bits of information:
1.their responses now as individual variables
2.the number of variables that I have now been given: this value is now a variable as well
my current script is as follows
echo List your favorite car manufacturers
read $car1 $car2 $car3 #end user can list as many as they wish
for n in {1..$numberofmanufacturers} #finding the number of
variables/manufactures is my second question
do
echo car$n
done
I am wanting to allow for the user to enter as many car manufacturers as they please (1=<n), however I need each manufacturer to be a different variable. I also need to be able to automate the count of the number of manufacturers and have that value be its own new variable.
Would I be better suited for having the end user create a .txt file in which they list (vertically) their manufactures, thus forcing me to use wc -l to determine the number of manufacturers?
I appreciate any help in advance.
As I said in the comment, whenever you want to use multiple dynamically created variables, you should check if there isn't a better data structure for your use case; and in almost all cases there will be. Here is the implementation using bash arrays. It prints out the contents of the input array in three different ways.
echo List your favorite car manufacturers
# read in an array, split on spaces
read -a cars
echo Looping over array values
for car in "${cars[#]}"
do
echo $car
done
echo Looping over array indices
for i in ${!cars[#]}
do
echo ${cars[$i]}
done
echo Looping from 0 to length-1
let numcars=${#cars[#]}
for i in $(seq 0 $((numcars-1)))
do
echo ${cars[$i]}
done

Linux Read - Timeout after x seconds *idle*

I have a (bash) script on a server that I have inherited the administration aspect of, and have recently discovered a flaw in the script that nobody has brought to my attention.
After discovering the issue, others have told me that it has been irritating them, but never told me (great...)
So, the script follows this concept
#!/bin/bash
function refreshscreen(){
# This function refreshes a "statistics screen"
...
echo "Enter command to override update"
read -t 10 variable
}
This script refreshes a statistics screen, and allows the user to stall the update in lieu of commands built into a case statement. However, the read times-out (read -t 10) after 10 seconds, regardless of if the user is typing.
Long story short, is there a way to prevent read from timing out if the user is actively typing a command? Best case scenario would be a "Time out of SEC idle/inactive seconds" opposed to just timeout after x seconds.
I have thought about running a background script at the end of the cycle before the read command pauses the screen to check for inactivity, but have not found a way to make that command work.
You can use read in a loop, reading one character at a time, and adding it to a final read string. This would then give the user some timeout amount of time per character rather than per command. Here's a sample function you might be able to incorporate into your script that shows what I'm talking about:
read_with_idle_timeout() {
local input=""
read -t 10 -N 1 variable
while [ ! -z $variable ]
do
input+=$variable
read -t 10 -N 1 variable
done
echo "Read: $input"
}
This will give the user 10 seconds to type each character. If they stop typing, you'll get as much of the command as they had started typing before the timeout occurred, and then your case statement can handle it. Perhaps you can store the final string in a global variable, or just put this code directly into your other function.
If you need more than one word, since read breaks on $IFS, you could call this function multiple times until you get all the input you're expecting.
I have searched for a simple solution that will do the following:
timeout after 10 seconds, if there is no user input at all
the user has infinite time to finish his answer if the first character was typed within the first 10 sec.
This can be implemented in two lines as follows:
read -N 1 -t 10 -p "What is your name? > " a
[ "$a" != "" ] && read b && echo "Your name is $a$b" || echo "(timeout)"
In case the user waits 10 sec before he enters the first character, the output will be:
What is your name? > (timeout)
If the user types the first character within 10 sec, he has unlimited time to finish this task. The output will look like follows:
What is your name? > Oliver
Your name is Oliver
Caveat: the first character is not editable, once it was typed, while all other characters can be edited (backspace and re-type). Any ideas for a simple solution?

Using flock to lock file for X time

I am a Linux novice and currently developing a security system using Raspberry Pi 3 and MotionEye. To get notifications via e-mail, I am attempting to create a custom shell script that will send an e-mail if there is motion, lock for X minutes, then send another e-mail if there is still motion. However, I am having some difficulties.
I created a simple Python script named "send_email.py" using SMTP that works perfectly fine for sending e-mails when I execute it via command line.
The shell script (named "flock_email.sh") is where I run into troubles in a few regards:
Whenever I run flock_email.sh, it completely overwrites send_email.py. I have tried to change file permission so it is only executable by the user, but it still overwrites.
The flock command/function does not work as I intended or at all. I have looked all over the internet and tried multiple different codes, but none have worked. I have attached my various flock_email.sh scripts I have tried.
Not necessarily a problem, but I am a bit confused on what my "shebang" line should be. For flock_email.sh I have it as "!#/bin/bash", which I believe makes the script it executable, at least according to this. Do I still need to change the permissions via the command "chmod +x flock_email.sh"? The path is /home/pi, which is essentially the main directory of my Pi.
The different solutions I have tried:
In flock_email.sh, I have tried to directly change the file permissions to read-only instead of using flock, having it sleep, then changing the permissions back to allow execution of the file.
Multiple flock_email.sh implementations, as attached.
To summarize:
I need to execute send_email.py before locking the file flock_email.sh.
Once locked, it needs to stay locked for X time.
Does anyone have any pointers or suggestions? I have spent well over 15 hours tinkering with this and feel like I have gotten nowhere!
send_email.py:
#!/usr/bin/env
import smtplib
def send_email():
content = "Message I want to send to specified e-mail."
sender = "e-mail account that will send message"
pword = "password of sender"
receiver = "e-mail account that will receive message"
mail = smtplib.SMTP("smtp.gmail.com",587)
mail.ehlo
mail.starttls()
mail.login(sender,pword)
mail.sendmail(sender,receiver,content)
mail.close()
send_email()
flock_email.sh (1):
#!/bin/bash
(
python /home/pi/send_email.py
flock -e 200
sleep [time in seconds]
)
flock_email.sh (2):
#!/bin/bash
(
python /home/pi/send_email.py
exec 3>/home/pi/send_email.py
flock -x 3
sleep [time in seconds]
exec 3>&-
)
flock_email.sh (3):
#!/bin/bash
python /home/pi/send_email.py
chmod 444 /home/pi/send_email.py # modify to read only for all
sleep [time in seconds]
chmod 755 /home/pi/send_email.py # modify to rwx for owner, r-x for others
The reason why man flock and all posts say to use > is because you're supposed to use a dedicated lock file, typically in /var/lock:
#!/bin/bash
exec 3> /var/lock/motionmail
flock -ne 3 || exit
python /home/pi/send_email.py
sleep 3600
This additionally fixes you sending your email regardless, before you ever check the lock, and aborts new emails instead of queueing them all up.
You choose the lock file name based on the scope you want your lock to have:
If you only want one email per hour, you can use something like /var/lock/motionmail because there's just one per system.
If you want one email for each user per hour, you can use $HOME/.motionmail.lock because there's just one per user.
You can use /home/pi/send_email.py if you want with <, but this implies that you want one email per hour not only for each user, programming language and script copy, but also every time you hit save and replace the file with your editor*
* Editors differ in whether they replace or overwrite a file

Check latest file updates in directory on linux using bash shell scripting

I have basic knowledge of linux bash shell scripting, right now I am facing a problem that is like following:
Suppose I am working in an empty directory mydir
Then there is a process which is created by a C program to generate a file with one word. (Exp: file.txt would have one word, "hello")
Routinely, after a specific period of time, the file is updated by the C program with the same one word "hello".
I want check the file every time when it is updated.
But the issue is that I also want my script doing some other operation while checking the file updates and when it detects file updates that it returns something for which I can use to trigger something else.
So, can anyone help me.
Also, some proof of concept :
while true;
do
func1();
func2();
check file is updated or not
if updated ; then
break;
else
continue;
You probably want the stat command. Do man stat to see how yours works. You want to look for "modtime" or "time of last data modification" option. For mine that would be stat -c%Y file. Something like basemodtime=$(stat -c%Y file) before the loop, modtime=$(stat -c%Y file) after func2(), and then if [ $modtime != $basemodtime ]; then to detect "updated".

Condition to check a variable value present or not

there is a autosys job, whicn has 3 jobs init. all 3 jobs call a common script and there 3 diffrent profiles one for each of them.these 3 jobs called from 3 different machines
.Each profile has got a varialble and this varible contains a fixed value specfic to the machine.
In the comomna scirpt i wnt to execute code based on teh value in the variable passed.
If the variable matches the value in the profile of a machine the code for that machine will be executed.how can I check if the varialbe received in the common script is also present in the profile of the machine.
Thanks
Old fashioned way I fall back on (in sh) is :
if [ -z "$VarToCheck" ] ; then
echo "Variable not set"
fi
There is probably a more correct way though. Do you care if a variable exists but has 0 length?

Resources