Proper variable interpolation for $# - linux

#!/bin/bash
TARGET_ENV="$1"
shift
commandid=$(aws ssm send-command \
--document-name "AWS-RunShellScript" \
--targets Key=tag:Name,Values=$TARGET_ENV \
--parameters '{"commands":["su -c \"./'$#'\" - ec2-user"]}' \
--query 'Command.CommandId' \
--output text)
echo $commandid
(ssm_runner.sh)
My ec2 instance have a script called hello_world.sh that prints hello world and echo.sh which accepts parameters and echo it.
The following works
ssm_runner.sh dev hello_world.sh
but this one doesn't
ssm_runner.sh dev echo.sh hello

#!/bin/bash
TARGET_ENV="$1"
shift
# Compose a complete su command which can be safely interpreted with
# `eval` or `bash -c`.
printf -v cmd '%q ' "$#"
su="su -c ./${cmd% } - ec2-user"
# Create JSON using jq.
params=$(jq -c --arg su "$su" '.commands = [$su]' <<< '{}')
# Execute.
commandid=$(aws ssm send-command \
--document-name "AWS-RunShellScript" \
--targets Key=tag:Name,Values="$TARGET_ENV" \
--parameters "$params" \
--query 'Command.CommandId' \
--output text)
echo "$commandid"

Related

Python subprocess capture output of bash code in parentheses

I have a shell script that I run in a subprocess.popen() and would like to capture the output of that script and send it line by line into a logger object. The issue is that the output from the subprocess is captured by the parentheses (), and I never see the output. Has anyone here ever seen this happen, and would you be able to guide me on how to best do this?
The code takes a backup on a remote server and stores it locally. The parentheses are needed to encapsulate the ssh-agent in case more than one are open at the same time.
The bash code
( \
eval $(ssh-agent) > /dev/null; \
ssh-add -q /home/borgs/.ssh/borg-client_key; \
echo $borgPassword | \
ssh -p ${port} -A -o StrictHostKeyChecking=no ${user}#${ipAddress} \
"BORG_PASSPHRASE=\$(cat) \
borg \
--rsh 'ssh -p 9270 -o StrictHostKeyChecking=no' \
create \
--verbose \
--filter AME \
--list \
--stats \
--show-rc \
--compression lz4 \
--exclude-caches \
ssh://borgs#${ip_address}/~/${hostname}::$hostname-$backupTime $locations";
kill "${SSH_AGENT_PID}"; \
) >&1
The python code
def takeBackup(server, lg):
print(f"Taking backup of {server['hostname']}")
locationString = " ".join(server['locations'])
print(
"Running create-backup.sh command: create-backup.sh",
f"{server['hostname']} {server['ip-address']} '{locationString}'"
)
process = subprocess.Popen(
[
"./create-backup.sh",
server['hostname'],
server['ip-address'],
server['borgPassword'],
locationString
],
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE
)
for line in process.stdout:
line = line.decode().rstrip('\n')
lg.print_info(line)
process.wait()
print(process.returncode)

Enable or disable a task in shell script

I have a few executable files to execute one by one. e.g.
A_1.sh
B_0.sh
cs_2.sh
c_4.sh
3_d.sh
I can execute all files with the following script, but I can't able to disable one, if I don't need that.
#!/bin/sh
for type in \
A_1 \
B_0 \
cs_2 \
c_4 \
3_d
do
echo executing $type.sh
done
When I am disabling one e.g.
#!/bin/sh
for type in \
A_1 \
B_0 \
# cs_2 \
c_4 \
3_d
do
echo executing $type.sh
done
It is showing syntax error near c_4. How to handle it actually?
The simplest is to use Bash:
#!/bin/bash
arr=(
A_1
B_0
# This is a comment
# cs_2
c_4
3_d
)
for type in "${arr[#]}"; do
echo executing $type.sh
done
You can read from a here document and filter comments:
#!/bin/sh
while read -r -u 3 line; do
# Is a comment?
if grep '^ *#' <<<"$line"; then continue; fi
# Close fd3, on the safe side.
echo executing $type.sh <&3-
done 3<<EOF
A_1
B_0
# This is a comment
# cs_2
c_4
3_d
EOF
Or invert the logic, do a function:
run() {
echo executing $1.sh
}
run A_1
run B_0
# run cs_2
run c_4
run 3_d

Can I combine a long chain of sed commands into something shorter?

i successfully wrote a long line that works well for my usage.
it get a file, and format the text to be as i want.
is it possible to make it shorter ?
wget http://user:password#192.168.1.100/details.cgx \
&& sed "s/value/\n/g" details.cgx >> step1 \
&& sed "s/text/</g" step1 >> step2 \
&& sed "s/id/</g" step2 >> step3 \
&& tr -d '<>/' < step3 >> step4 \
&& sed "s/formFanLevel/FanLevel/g" step4 >> step5 \
&& sed '123,155d' step5 >> step6 \
&& sed '79,120d' step6 >> step7 \
&& sed '57,66d' step7 >> step8 \
&& sed '47,48d' step8 >> step9 \
&& sed '37,44d' step9 >> step10 \
&& sed '13,26d' step10 >> VMCDF.txt \
&& rm step* && rm details.cgx
I think you want this:
wget -O- http://user:password#192.168.1.100/details.cgx | sed -E '
s/value/\n/g
s/text|id/</g
s,<>/,,g
s/form(FanLevel)/\1/g
' | sed '
13,26d
37,44d
47,48d
57,66d
79,120d
123,155d
' > VMCDF.txt
A good start would be to learn how the pipe command (|) can be used to pass the output from one program as input to the next - this would remove the need to create, reference and clean up all the intermediate files (step1 .. step10) from your command line.
You can write a shell script that can apply the same command chain to any file, then use the script instead of typing all these lines.
If, though unlikely, you always use the same file, just make an alias for it by giving a name to the command chain, eg:
alias wgetsed='wget http://user:password#192.168.1.100/details.cgx \
&& sed "s/value/\n/g" details.cgx >> step1 \
&& sed "s/text/</g" step1 >> step2 \
...
&& rm step* && rm details.cgx'
You then add the code above to ~/.bash_rc to make it permanent for your user account.

Makefile not executing the commands as expected

I'm using the following Makefile, which should check whether the files are in some directory *.rpm, if the files hasn't been found i'm going to execute some command (run a script or make the rpm)
Here's a snippet from my Makefile
include /home/user/workspace/test/exec_recipe.mk
export BUILD_LOGS_DIR = $(CURDIR)/build_logs
.PHONY: my_rpm
libpdutil_rpm:
#echo "Making target 'libpdutil_rpm'. Time: $$(date +%T)"
cd /home/user/workspace/test/build/test && $(call exec_recipe,$(ls /home/user/workspace/test/build/test/rpmbuild/RPMS/x86_64d/*.rpm) || $(./test.sh),test.log,)
#echo "Finished making target 'my_rpm'. Time: $$(date +%T)"
And here's the exec_recipe.mk
SHELL:=/bin/bash
exec_recipe = \
echo "The logs dir is: $$BUILD_LOGS_DIR"; \
log_name="$(2)"; \
echo "The log name is $$log_name"; \
cmd="$(1)"; \
eval "$$cmd" 2>&1 | tee -a "$$BUILD_LOGS_DIR/$$log_name"; rc="$${PIPESTATUS[0]}"; \
if [ $$rc = 0 ]; then \
res="PASS"; \
else \
res="FAIL"; \
fi; \
flock $(SUMMARY) echo "Making target '$#': $$res" >> $(SUMMARY); \
exit $$rc
So the problem is when it's reaching the execution of the exec_recipe.mk it giving me some errors:
/bin/bash: eval: line 0: syntax error near unexpected token||'
/bin/bash: eval: line 0: || '
I'm sure that i'm doing something wrong..
I need the helper make file, because i'm using it for other purposes as well
You have to escape all the dollar signs that you don't want make to interpret. You've forgotten to escape the dollar signs in the arguments to $(call exec_recipe,...):
cd /home/user/workspace/test/build/test && $(call exec_recipe,$$(ls /home/user/workspace/test/build/test/rpmbuild/RPMS/x86_64d/*.rpm) || $$(./test.sh),test.log,)
I'm not sure what the final comma is for but whatever.

Bash script for fetching row from Postgresql

I have a Postgresql function which returns table of JSON strings.
I want to fetch those rows under the Linux platform and run CURL program for each string like this:
curl http://XXX.XX.XX.XX:XXXX/update/json -H 'Content-type:application/json' -d ['||solr_interface.dict_cursor_okpd_json()||']'
Do you have any example how to do this using say bash script?
Thanks in advise.
I've found solution. Maybe it will useful for somebody
#!/bin/bash
if [ $# != 2 ]; then
echo "Please select postgres procedure for data retrieving as first parameter"
echo "and Solr index name (dictionary or company) as second parameter"
exit 1;
fi
PSQL=/usr/bin/psql
DB_USER="xxxxxxxx"
DB_PASSWORD="xxxxxxxx"
DB_HOST="xxx.xxx.xxx.xxx"
DB_PORT="xxxx"
DB_NAME="portal_sources"
export PGPASSWORD=$DB_PASSWORD
$PSQL \
-X \
-h $DB_HOST \
-p $DB_PORT $DB_NAME \
-c "select 'curl http://xxx.xxx.xxx.xxx:xxxx/solr/$2/update/json -H ''Content-type:application/json'' -d ''['||$1||']'''" \
--single-transaction \
--set AUTOCOMMIT=off \
--set ON_ERROR_STOP=on \
--no-align \
-t \
--field-separator ' ' \
--quiet \
| while read command ; do
#echo "$command"
eval $command
done

Resources