I have a parameter file (param.env) having the following content.
MY_PARAM=com:27}WMV\)pviZN
also, a bash file where I am fetching the value of MY_PARAM and writing into a random file.
#!/bin/bash
value=$(grep "^MY_PARAM=" param.env | cut -d '=' -f2-)
value1=$(cat param.env | grep "^MY_PARAM" | sed 's/=/ /' | awk '{print $2}')
echo $value
echo $value1
printf '%s\n' "$value"
Output:
com:27}WMV\)pviZN
com:27}WMV\)pviZN
com:27}WMV\)pviZN
However, I am expecting \ to be escaped and should not be part of the output.
I am also not allowed to edit the param.env.
Expected output:
com:27}WMV)pviZN
You could source the file, then the string will behave as if you'd assigned it like that in an interactive shell:
$ (. param.env; echo "$MY_PARAM")
com:27}WMV)pviZN
I've put the commands in a subshell so they don't pollute the environment.
Related
When I run terraform plan it shows a changed resource, which happens to be JSON data in an aws_s3_bucket_object. But the JSON is long and it's difficult to see what changed. How can I display this as a diff?
https://github.com/coinbase/terraform-landscape can help with this.
gem install terraform_landscape (may need sudo on macOS)
terraform plan | landscape
This shows JSON changes as a diff. Here's an example from the github site:
I wrote a bash script to format terraforms ugly policy output:
#!/bin/bash
input=$( xclip -o )
old=$( echo "$input" | awk -F' => ' '{ print $1 }' | sed 's/\\n/\n\r/g' | sed 's/\\"/"/g' | sed 's/"{/{/' | sed 's/}"/}/' )
new=$( echo "$input" | awk -F' => ' '{ print $2 }' | sed 's/\\n/\n\r/g' | sed 's/\\"/"/g' | sed 's/"{/{/' | sed 's/}"/}/' )
echo "----------------------------------------------------------------------------------------------"
echo "old:"
echo "$old" | jq '.'
echo "----------------------------------------------------------------------------------------------"
echo "new:"
echo "$new" | jq '.'
echo "----------------------------------------------------------------------------------------------"
echo "diff:"
diff -u --color <( echo "$old" | jq '.' ) <( echo "$new" | jq '.' )
echo "----------------------------------------------------------------------------------------------"
It shows three blocks of output, the old, then the new and then the diff. It makes use of xclip, jq and diff. Usage is (on Linux) to highlight the terrafrom output and then invoke the script (I call it tf-diff and it lives in ~/bin).
How to print output of grep -o in a single line ? I am trying to print :
$ echo "Hello Guys!" |grep -E '[A-Z]'
Hello Guys!
$ echo "Hello Guys!" |grep -Eo '[A-Z]' <----Multiple lines
H
G
$ echo "Hello Guys!" |grep -Eo '[A-Z]'
Desired output:
HG
I am able to cheaply achieve it using following command ,but the issue is that number of letters(3 in this case) could be dynamic. So this approach cannot be used.
echo "HEllo Guys!" |grep -oE '[A-Z]' |xargs -L3 |sed 's/ //g'
HEG
You could do it all with this sed instruction
echo "Hello Guys!" |sed 's/[^A-Z]//g'
UPDATE
Breakdown of sed command:
The s/// is sed's substitute command. It simply replaces the first RegEx (the one between the first and the second slash) with the Expression between slash two and three. The trailing g stands for global, i.e, do this for every match of the RegEx in the current line. Without the g it would just stop processing after the first match. The RegEx itself is matching any non-capital letter and then those letters are replaced with nothing, i.e., effectively deleted.
You can use awk:
echo "Hello Guys!" | awk '{ gsub(/[^A-Z]/,"", $0); print;}'
HG
Also with tr:
echo "Hello Guys!" | tr -cd [:upper:]
HG
Also with sed :
echo "Hello Guys!" | sed 's/[^\[:upper:]]//g'
HG
You just need to remove the newline characters. You can use tr for that:
echo "HEllo Guys!" |grep -Eo '[A-Z]' |tr -d '\n'
HEG
Though, it cuts the last newline too.
You can use perl instead of grep
echo 'HEllo Guys!' | perl -lne 'print /([A-Z])/g'
HEG
When I try to run this script this error appears : operating extra /home/ubuntu/Desktop/Destino/, and I do not know why , someone help me please.
#!/bin/bash
input="/home/ubuntu/Desktop/Output/SAIDA.txt"
dt=`date +"%Y%m%d%H%M%S"`
layout='C'
if [ -e "$input" ] ; then
header=$(head -n 1 $input)
export header
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
do
NOME= ´cat $arquivo | cut -d "." -f1´
touch/home/ubuntu/Desktop/Destino/$NOME.cfg
echo $dt > $NOME.cfg
echo $layout > $NOME.cfg
done
else
echo "The input file does not exist."
fi
You have some strange quote characters in your script. To substitute the output of a command, wrap it with $() or backticks, not ´ characters.
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
I guess Is was meant to be ls, but you left out the space after it. But there's no need to parse the output of ls, just use the wildcard directly.
for arquivo in /home/ubuntu/Desktop/*.txt
On this line:
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
you need to put the output filename in quotes because of the spaces.
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >"/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_"
Also, the FILE variable is not set, you need to assign that earlier.
On this line:
NOME= ´cat $arquivo | cut -d "." -f1´
you're again using the wrong type of quotes to capture the output of the command. Also, you must not have a space between = and the value you want to assign. It should be:
NOME=$(cat $arquivo | cut -d "." -f1)
There's no need to do export header. The variable is only being used in this script, not in any child processes.
I want to check inside a binary file if it matches a binary pattern.
For that, I'm using clamAV signature database
Exploit.HTML.ObjectType:3:*:3c6f626a65637420747970653d222f2f2f2f2f2f2f2f2f2f2f2f
I code this to retrieve the hex signature string
signature=$(echo "$line" |awk -F':' '{ print $4 }')
Moreover I would like to change hex string to binary
tmp=$(echo -n $signature | sed 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI' | xargs print)
Finally I would like to check if my file ( *$raw_file_path* ) matches my binary pattern (now in $tmp)
test_var=$(cat $raw_file_path | grep -U -P "$tmp")
I don't why it doesn't work.
If you have any idea.
Thanks.
How about this?
line=Exploit.HTML.ObjectType:3:*:3c6f626a65637420747970653d222f2f2f2f2f2f2f2f2f2f2f2f
printf $(sed 's/.*://;s/\(..\)/\\x\1/g' <<< "$line")
Which yields:
<object type="////////////
You can put the bin output in a variable thus:
printf -v variable $(sed 's/.*://;s/\(..\)/\\x\1/g' <<< "$line")
Now, please avoid a useless use of cat!
grep -U "$variable" "$raw_file_path"
is enough. If you want to test the result of grep (and ask grep to be quiet):
if grep -qU "$variable" "$raw_file_path"; then
echo "Pattern found"
else
echo "Pattern not found"
fi
Can any one advise how to search on linux for some data between a tilde character. I need to get IP data however its been formed like the below.
Details:
20110906000418~118.221.246.17~DATA~DATA~DATA
One more:
echo '20110906000418~118.221.246.17~DATA~DATA~DATA' | sed -r 's/[^~]*~([^~]+)~.*/\1/'
echo "20110906000418~118.221.246.17~DATA~DATA~DATA" | cut -d'~' -f2
This uses the cut command with the delimiter set to ~. The -f2 switch then outputs just the 2nd field.
If the text you give is in a file (called filename), try:
grep "[0-9]*~" filename | cut -d'~' -f2
With cut:
echo "20110906000418~118.221.246.17~DATA~DATA~DATA" | cut -d~ -f2
With awk:
echo "20110906000418~118.221.246.17~DATA~DATA~DATA"
| awk -F~ '{ print $2 }'
In awk:
echo '20110906000418~118.221.246.17~DATA~DATA~DATA' | awk -F~ '{print $2}'
Just use bash
$ string="20110906000418~118.221.246.17~DATA~DATA~DATA"
$ echo ${string#*~}
118.221.246.17~DATA~DATA~DATA
$ string=${string#*~}
$ echo ${string%%~*}
118.221.246.17
one more, using perl:
$ perl -F~ -lane 'print $F[1]' <<< '20110906000418~118.221.246.17~DATA~DATA~DATA'
118.221.246.17
bash:
#!/bin/bash
IFS='~'
while read -a array;
do
echo ${array[1]}
done < ip
If string is constant, the following parameter expansion performs substring extraction:
$ a=20110906000418~118.221.246.17~DATA~DATA~DATA
$ echo ${a:15:14}
118.221.246.17
or using regular expressions in bash:
$ echo $(expr "$a" : '[^~]*~\([^~]*\)~.*')
118.221.246.17
last one, again using pure bash methods:
$ tmp=${a#*~}
$ echo $tmp
118.221.246.17~DATA~DATA~DATA
$ echo ${tmp%%~*}
118.221.246.17