Enable or disable a task in shell script - linux

I have a few executable files to execute one by one. e.g.
A_1.sh
B_0.sh
cs_2.sh
c_4.sh
3_d.sh
I can execute all files with the following script, but I can't able to disable one, if I don't need that.
#!/bin/sh
for type in \
A_1 \
B_0 \
cs_2 \
c_4 \
3_d
do
echo executing $type.sh
done
When I am disabling one e.g.
#!/bin/sh
for type in \
A_1 \
B_0 \
# cs_2 \
c_4 \
3_d
do
echo executing $type.sh
done
It is showing syntax error near c_4. How to handle it actually?

The simplest is to use Bash:
#!/bin/bash
arr=(
A_1
B_0
# This is a comment
# cs_2
c_4
3_d
)
for type in "${arr[#]}"; do
echo executing $type.sh
done
You can read from a here document and filter comments:
#!/bin/sh
while read -r -u 3 line; do
# Is a comment?
if grep '^ *#' <<<"$line"; then continue; fi
# Close fd3, on the safe side.
echo executing $type.sh <&3-
done 3<<EOF
A_1
B_0
# This is a comment
# cs_2
c_4
3_d
EOF
Or invert the logic, do a function:
run() {
echo executing $1.sh
}
run A_1
run B_0
# run cs_2
run c_4
run 3_d

Related

Python subprocess capture output of bash code in parentheses

I have a shell script that I run in a subprocess.popen() and would like to capture the output of that script and send it line by line into a logger object. The issue is that the output from the subprocess is captured by the parentheses (), and I never see the output. Has anyone here ever seen this happen, and would you be able to guide me on how to best do this?
The code takes a backup on a remote server and stores it locally. The parentheses are needed to encapsulate the ssh-agent in case more than one are open at the same time.
The bash code
( \
eval $(ssh-agent) > /dev/null; \
ssh-add -q /home/borgs/.ssh/borg-client_key; \
echo $borgPassword | \
ssh -p ${port} -A -o StrictHostKeyChecking=no ${user}#${ipAddress} \
"BORG_PASSPHRASE=\$(cat) \
borg \
--rsh 'ssh -p 9270 -o StrictHostKeyChecking=no' \
create \
--verbose \
--filter AME \
--list \
--stats \
--show-rc \
--compression lz4 \
--exclude-caches \
ssh://borgs#${ip_address}/~/${hostname}::$hostname-$backupTime $locations";
kill "${SSH_AGENT_PID}"; \
) >&1
The python code
def takeBackup(server, lg):
print(f"Taking backup of {server['hostname']}")
locationString = " ".join(server['locations'])
print(
"Running create-backup.sh command: create-backup.sh",
f"{server['hostname']} {server['ip-address']} '{locationString}'"
)
process = subprocess.Popen(
[
"./create-backup.sh",
server['hostname'],
server['ip-address'],
server['borgPassword'],
locationString
],
stdin=subprocess.PIPE,
stderr=subprocess.PIPE,
stdout=subprocess.PIPE
)
for line in process.stdout:
line = line.decode().rstrip('\n')
lg.print_info(line)
process.wait()
print(process.returncode)

Can I combine a long chain of sed commands into something shorter?

i successfully wrote a long line that works well for my usage.
it get a file, and format the text to be as i want.
is it possible to make it shorter ?
wget http://user:password#192.168.1.100/details.cgx \
&& sed "s/value/\n/g" details.cgx >> step1 \
&& sed "s/text/</g" step1 >> step2 \
&& sed "s/id/</g" step2 >> step3 \
&& tr -d '<>/' < step3 >> step4 \
&& sed "s/formFanLevel/FanLevel/g" step4 >> step5 \
&& sed '123,155d' step5 >> step6 \
&& sed '79,120d' step6 >> step7 \
&& sed '57,66d' step7 >> step8 \
&& sed '47,48d' step8 >> step9 \
&& sed '37,44d' step9 >> step10 \
&& sed '13,26d' step10 >> VMCDF.txt \
&& rm step* && rm details.cgx
I think you want this:
wget -O- http://user:password#192.168.1.100/details.cgx | sed -E '
s/value/\n/g
s/text|id/</g
s,<>/,,g
s/form(FanLevel)/\1/g
' | sed '
13,26d
37,44d
47,48d
57,66d
79,120d
123,155d
' > VMCDF.txt
A good start would be to learn how the pipe command (|) can be used to pass the output from one program as input to the next - this would remove the need to create, reference and clean up all the intermediate files (step1 .. step10) from your command line.
You can write a shell script that can apply the same command chain to any file, then use the script instead of typing all these lines.
If, though unlikely, you always use the same file, just make an alias for it by giving a name to the command chain, eg:
alias wgetsed='wget http://user:password#192.168.1.100/details.cgx \
&& sed "s/value/\n/g" details.cgx >> step1 \
&& sed "s/text/</g" step1 >> step2 \
...
&& rm step* && rm details.cgx'
You then add the code above to ~/.bash_rc to make it permanent for your user account.

Makefile not executing the commands as expected

I'm using the following Makefile, which should check whether the files are in some directory *.rpm, if the files hasn't been found i'm going to execute some command (run a script or make the rpm)
Here's a snippet from my Makefile
include /home/user/workspace/test/exec_recipe.mk
export BUILD_LOGS_DIR = $(CURDIR)/build_logs
.PHONY: my_rpm
libpdutil_rpm:
#echo "Making target 'libpdutil_rpm'. Time: $$(date +%T)"
cd /home/user/workspace/test/build/test && $(call exec_recipe,$(ls /home/user/workspace/test/build/test/rpmbuild/RPMS/x86_64d/*.rpm) || $(./test.sh),test.log,)
#echo "Finished making target 'my_rpm'. Time: $$(date +%T)"
And here's the exec_recipe.mk
SHELL:=/bin/bash
exec_recipe = \
echo "The logs dir is: $$BUILD_LOGS_DIR"; \
log_name="$(2)"; \
echo "The log name is $$log_name"; \
cmd="$(1)"; \
eval "$$cmd" 2>&1 | tee -a "$$BUILD_LOGS_DIR/$$log_name"; rc="$${PIPESTATUS[0]}"; \
if [ $$rc = 0 ]; then \
res="PASS"; \
else \
res="FAIL"; \
fi; \
flock $(SUMMARY) echo "Making target '$#': $$res" >> $(SUMMARY); \
exit $$rc
So the problem is when it's reaching the execution of the exec_recipe.mk it giving me some errors:
/bin/bash: eval: line 0: syntax error near unexpected token||'
/bin/bash: eval: line 0: || '
I'm sure that i'm doing something wrong..
I need the helper make file, because i'm using it for other purposes as well
You have to escape all the dollar signs that you don't want make to interpret. You've forgotten to escape the dollar signs in the arguments to $(call exec_recipe,...):
cd /home/user/workspace/test/build/test && $(call exec_recipe,$$(ls /home/user/workspace/test/build/test/rpmbuild/RPMS/x86_64d/*.rpm) || $$(./test.sh),test.log,)
I'm not sure what the final comma is for but whatever.

Bash script for fetching row from Postgresql

I have a Postgresql function which returns table of JSON strings.
I want to fetch those rows under the Linux platform and run CURL program for each string like this:
curl http://XXX.XX.XX.XX:XXXX/update/json -H 'Content-type:application/json' -d ['||solr_interface.dict_cursor_okpd_json()||']'
Do you have any example how to do this using say bash script?
Thanks in advise.
I've found solution. Maybe it will useful for somebody
#!/bin/bash
if [ $# != 2 ]; then
echo "Please select postgres procedure for data retrieving as first parameter"
echo "and Solr index name (dictionary or company) as second parameter"
exit 1;
fi
PSQL=/usr/bin/psql
DB_USER="xxxxxxxx"
DB_PASSWORD="xxxxxxxx"
DB_HOST="xxx.xxx.xxx.xxx"
DB_PORT="xxxx"
DB_NAME="portal_sources"
export PGPASSWORD=$DB_PASSWORD
$PSQL \
-X \
-h $DB_HOST \
-p $DB_PORT $DB_NAME \
-c "select 'curl http://xxx.xxx.xxx.xxx:xxxx/solr/$2/update/json -H ''Content-type:application/json'' -d ''['||$1||']'''" \
--single-transaction \
--set AUTOCOMMIT=off \
--set ON_ERROR_STOP=on \
--no-align \
-t \
--field-separator ' ' \
--quiet \
| while read command ; do
#echo "$command"
eval $command
done

Scale pdf to add border for printing full size pages

When printing a pdf with no border (or margins), the printer choppes off around 1mm of the image data at the edges of the paper. I am therefore looking for a solution to
scale/resize a pdf page slightly on the page to add a white border at the edges that will correspond with the white space at the edges produced by the printer.
I have tried using gs so far.. For instance, suppose i have an A4 size pdf 1.pdf, then I used:
gs -sDEVICE=pdfwrite \
-q -dBATCH -dNOPAUSE \
-dPDFFitPage \
-r300x300 \
-g2232x3157 \
-sOutputFile=1A.pdf \
1.pdf
Here, a full a4 paper is given by -g2480x3508 and I have tried to multiply by 0.9 to scale, but I do not see any effect of this..
Here's a Gist of a bash script that builds on the prev. Fixes a color compatibility problem (possibly specific to my pdf), and does some dependency checking:
#!/bin/bash
# pdfScale.sh
#
# Scale PDF to specified percentage of original size.
# Ref: http://ma.juii.net/blog/scale-page-content-of-pdf-files.
echo "This script doesn't handle files with spaces in them."
SCALE=0.95 # scaling factor (0.95 = 95%, e.g.)
# Validate args.
[ $# -eq 1 ] || { echo "***ERROR: Usage pdfScale.sh <inFile>.pdf"; exit 99; }
INFILEPDF="$1"
[[ "$INFILEPDF" =~ ^..*\.pdf$ ]] || { echo "***ERROR: Usage pdfScale.sh <inFile>.pdf"; exit 99; }
OUTFILEPDF=$(echo "$INFILEPDF" | sed -e s/\.pdf$// -).SCALED.pdf
# Dependencies
command -v identify >/dev/null 2>&1 || { echo >&2 "Please install 'imagemagick' (sudo apt-get install imagemagick). Aborting."; exit 1; }
command -v gs >/dev/null 2>&1 || { echo >&2 "Please install 'ghostscript' (sudo apt-get install ghostscript ?). Aborting."; exit 1; }
command -v bc >/dev/null 2>&1 || { echo >&2 "Please install 'bc' arbitrary precision calculator language. Aborting."; exit 1; }
# Get width/height in postscript points (1/72-inch), via ImageMagick identify command.
# (Alternatively, could use Poppler pdfinfo command; or grep/sed the PDF by hand.)
IDENTIFY=($(identify $INFILEPDF 2>/dev/null)) # bash array
[ $? -ne 0 ] &GEOMETRY=($(echo ${IDENTIFY[2]} | tr "x" " ")) # bash array — $IDENTIFY[2] is of the form PGWIDTHxPGHEIGHT
PGWIDTH=${GEOMETRY[0]}; PGHEIGHT=${GEOMETRY[1]}
# Compute translation factors (to center page.
XTRANS=$(echo "scale=6; 0.5*(1.0-$SCALE)/$SCALE*$PGWIDTH" | bc)
YTRANS=$(echo "scale=6; 0.5*(1.0-$SCALE)/$SCALE*$PGHEIGHT" | bc)
echo $PGWIDTH , $PGHEIGHT , $OUTFILEPDF , $SCALE , $XTRANS , $YTRANS , $INFILEPDF , $OUTFILEPDF
# Do it.
gs \
-q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dSAFER \
-dCompatibilityLevel="1.5" -dPDFSETTINGS="/printer" \
-dColorConversionStrategy=/LeaveColorUnchanged \
-dSubsetFonts=true -dEmbedAllFonts=true \
-dDEVICEWIDTH=$PGWIDTH -dDEVICEHEIGHT=$PGHEIGHT \
-sOutputFile="$OUTFILEPDF" \
-c "<</BeginPage{$SCALE $SCALE scale $XTRANS $YTRANS translate}>> setpagedevice" \
-f "$INFILEPDF"
https://gist.github.com/MichaelJCole/86e4968dbfc13256228a
More information about this method and a discussion of this gist is availanle on this blog post:
How to scale the page content of PDF files? (Aug 2008; by Matt)
See tavinus/pdfScale, it's a fork with some other features added to it.
It seems like the solution provided at
http://ma.juii.net/blog/scale-page-content-of-pdf-files
works well here..
Based on that solution, I wrote the following bash script (scaleA4Pdf) for scaling the page content of an A4 pdf file. You can now just write scaleA4Pdf 10 to scale the page 10%..
#! /bin/bash
if [ $# -ne 1 ] ; then
echo "Bad arguments!"
exit
fi
# assume 0<=$1<=100 (no error checks!)
xx="595" #width of A4 in post script points
yy="842" #height of A4 in pps
ss=$(echo "scale=4; $1 / 2" | bc)
sx=$(echo "scale=4; ${xx}"'*'"( ${ss}/ 100 )" | bc)
sy=$(echo "scale=4; ${yy}"'*'"( ${ss}/ 100 )" | bc)
s=$(echo "scale=4; 1 - $1 / 100" | bc)
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dSAFER \
-dCompatibilityLevel="1.3" -dPDFSETTINGS="/printer" \
-dSubsetFonts=true -dEmbedAllFonts=true \
-sPAPERSIZE=a4 -sOutputFile="1A.pdf" \
-c "<</BeginPage{${s} ${s} scale ${sx} ${sy} translate}>> setpagedevice" \
-f 1.pdf
Nice Håkon Hægland!
I make a little improvement to make easy select the input.
So if you run
$ scaleA4PDF 10 yourfile.pdf
you will receive a yourfile_scaled.pdf file.
#! /bin/bash
input=$2
output=$(echo $2 | sed s/.pdf/_scaled.pdf/)
if [ $# -ne 2 ] ; then
echo "Bad arguments!"
exit
fi
# assume 0<=$1<=100 (no error checks!)
xx="595" #width of A4 in post script points
yy="842" #height of A4 in pps
ss=$(echo "scale=4; $1 / 2" | bc)
sx=$(echo "scale=4; ${xx}"'*'"( ${ss}/ 100 )" | bc)
sy=$(echo "scale=4; ${yy}"'*'"( ${ss}/ 100 )" | bc)
s=$(echo "scale=4; 1 - $1 / 100" | bc)
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dSAFER \
-dCompatibilityLevel="1.3" -dPDFSETTINGS="/printer" \
-dSubsetFonts=true -dEmbedAllFonts=true \
-sPAPERSIZE=a4 -sOutputFile="${output}" \
-c "<</BeginPage{${s} ${s} scale ${sx} ${sy} translate}>> setpagedevice" \
-f ${input}
Since you did not specify a particular tool you are interested in, I would use iText to accomplish such a task. You could write simple code in Java or .NET (iTextSharp) to accomplish this task easily. Use this as inspiration (n-up tool). While it is actually putting multiple pages of a document into single pages, you could adopt this code to slightly scale individual pages in the same way.

Resources