My logic goes here,
vodCIntSvc_VideoSegments()
{
# logic to download video segments from mediaPlaylist-Video payload response based on the count
var_param1="#EXTINF:6.006,"
var_segment_cnt=$(echo $var_videoLayer | awk -F"$var_param1" '{print NF}')
echo $var_segment_cnt " count is "
if [ $input_segment_cnt -le $var_segment_cnt ]; then
var_segment_cnt=$(($input_segment_cnt+1))
fi
i=2
while [ $i -le $var_segment_cnt ]
do
segment_url=$(echo $var_videoLayer | awk -F"$var_param1" '{print $'$i'}')
#"#EXTINF:6.006,
echo $segment_url
i=$((i+1))
echo $segment_url >> /tmp/"$directory_name"/4_videoSegments_$(date +%Y%m%d%H%M).ts
done
echo "$(date +'%Y-%m-%d %H:%M:%S') :: Downloading video segments are successful with argument count- $input_segment_cnt & the output file is saved to :- /tmp/"$directory_name"/4_VideoSegments_$(date +%Y%m%d%H%M).ts"
}
and the output which I Am getting is :
https://manifest.vod.wb4.ott.eng.alticeusa.net/cdn/wb4vodmgmt7idpx01b1.wb4-7.eng.cv.net:5555/hls/NCPHAAAAAOBLNAOF.m3u8/Level(6)/Segment(0).ts **#EXT-X-KEY:METHOD=SAMPLE-AES,URI="skd://indemand.com~INMV1019201000659085",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1",IV=0x0C672EE95BE69AD3AEB30B44A04E966A**
https://manifest.vod.wb4.ott.eng.alticeusa.net/cdn/wb4vodmgmt7idpx01b1.wb4-7.eng.cv.net:5555/hls/NCPHAAAAAOBLNAOF.m3u8/Level(6)/Segment(1).ts **#EXT-X-KEY:METHOD=SAMPLE-AES,URI="skd://indemand.com~INMV1019201000659085",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1",IV=0x0C672EE95BE69AD3AEB30B44A04E966A**
the regular expressions #EXT-X-KEY for each url should be removed
and the expected output is :
https://manifest.vod.wb4.ott.eng.alticeusa.net/cdn/wb4vodmgmt7idpx01b1.wb4-7.eng.cv.net:5555/hls/NCPHAAAAAOBLNAOF.m3u8/Level(6)/Segment(0).ts
https://manifest.vod.wb4.ott.eng.alticeusa.net/cdn/wb4vodmgmt7idpx01b1.wb4-7.eng.cv.net:5555/hls/NCPHAAAAAOBLNAOF.m3u8/Level(6)/Segment(1).ts
could someone please provide any suggestions in shell script on above my code.
One way using sed:
echo "$segment_url" | sed 's/#EXT-X-KEY.*//'
This will remove text starting from #EXT-X-KEY till the end of the line.
Try following in awk:
echo "$segment_url" | awk '{gsub(/#EXT-X-KEY.*/,"")}1'
I used gsub here for global substitution you can remove it to sub in case you don't want global substitution here.
Correction 1: gsub(/#EXT-X-KEY.*/,"") added / after *.
Related
This question already has answers here:
Why does adding spaces around bash comparison operator change the result?
(2 answers)
How to pass the value of a variable to the standard input of a command?
(9 answers)
Closed 1 year ago.
I want to check which lines of the file /etc/passwd end with the "/bin/bash" string (field number 7, ":" as delimiter).
So far, I've written the following code:
while read line
do
if [ $("$line" | cut -d : -f 7)=="/bin/bash" ]
then
echo $line | cut -d : -f 1
echo "\n"
fi
done < /etc/passwd
Currently, executing the script throws errors that show a bad interpretation (most likely due to the syntax).
I'd appreciate if you could help me.
You MUST surround the == operator with spaces. [ and [[ do different things based on how many arguments are given:
if [ "$( echo "$line" | cut -d: -f7 )" == "/bin/bash" ]; ...
I would actually do this: parse the line into fields while you're reading it.
while IFS=: read -ra fields; do
[[ ${fields[-1]} == "/bin/bash" ]] && printf "%s\n\n" "${fields[0]}"
done < /etc/passwd
This line is wrong:
if [ $("$line" | cut -d : -f 7)=="/bin/bash" ]
Also, this is not going to do what you want:
echo "\n"
Bash echo doesn't understand backslash-escaped characters without
-e. If you want to print a new line use just echo but notice that
the previous echo:
echo $line | cut -d : -f 1
will add a newline already.
You should always check your scripts with
shellcheck. The correct script would be:
#!/usr/bin/env bash
while read -r line
do
if [ "$(echo "$line" | cut -d : -f 7)" == "/bin/bash" ]
then
echo "$line" | cut -d : -f 1
fi
done < /etc/passwd
But notice that you don't really need a loop which is very slow and
could use the following awk one-liner:
awk -v FS=: '$7 == "/bin/bash" {print $1}' /etc/passwd
Instead of looping through the rows, and then checking for the /bin/bash part, why not use something like grep to get all the desired rows, like so:
grep ':/bin/bash$' /etc/passwd
Optionality, you can loop over the rows by using a simple while;
grep ':/bin/bash$' /etc/passwd | while read -r line ; do
echo "Processing $line"
done
Don't do while read | cut. Use IFS as:
#!/bin/sh
while IFS=: read name passwd uid gid gecos home shell; do
if test "$shell" = /bin/bash; then
echo "$name"
fi
done < /etc/passwd
But for this particular use case, it's probably better to do:
awk '$7 == "/bin/bash"{print $1}' FS=: /etc/passwd
The issue your code has is a common error. Consider the line:
if [ $("$line" | cut -d : -f 7)=="/bin/bash" ]
Assume you have a value in $line in which the final field is /bin/dash. The process substitution will insert the string /bin/dash, and bash will attempt to execute:
if [ /bin/dash==/bin/bash ]
since /bin/bash==/bin/bash is a non-empty string, the command [ /bin/bash==/bin/bash ] returns succesfully. It does not perform any sort of string comparison. In order for [ to do a string comparison, you need to pass it 4 arguments. For example, [ /bin/dash = /bin/bash ] would fail. Note the 4 arguments to that call are /bin/dash, =, /bin/bash, and ]. [ is an incredibly bizarre command that requires its final argument to be ]. I strongly recommend never using it, and replacing it instead with its cousin test (very closely related, indeed both test and [ used to be linked to the same executable) which behaves exactly the same but does not require its final argument to be ].
i have a scenario where i want to check the below details in files match the syntax
Syntax : NAME/SURNAME/COUNTRY
Note : every record in file should match the syntax. if one records does not match then should display
failed or success
File data
Ajay/abc/india
Avik/bcs/delhi
Viraj/xyz/
As you can see the file data the last record does not match according to the syntax
then should display failed
My code
for i in `cat /demo/Student.txt`
do
check=`echo $i | cut -d '/' -f3`
if [[ -z $check ]];
then
echo failed syntax does not match NAME/SURNAME/COUNTRY
exit 1
fi
done
Syntax : NAME/SURNAME/COUNTRY
Is very much underspecified. First write a regex to match your line - then just use a utility to check if each line matches the regex.
With grep:
if grep -v -q '^[[:alpha:]]\+/[[:alpha:]]\+/[[:alpha:]]\+$' "$file"; then
echo "failed synax grep"
fi
grep matches a regex:
-v invert match
^ match beginning of the line
[[:alpha:]]\+ one or more consecutive characters sed character classes. You could use [^/]* or [^/]+ instead for the same behavior as your current code wanted to do.
/ a slash
etc.
$ match ending of the line
With gnu sed:
if ! sed -n '\#^[[:alpha:]]\+/[[:alpha:]]\+/[[:alpha:]]\+$#!q1' "$file"; then
echo "failed syntax"
fi
Sed does:
\#....# - match the regex inside
! - if the regex was not matched
q1 - quit with exit status 1
You could also remove all characters except newline and a slash and check if each line has 3 characters.
Notes:
for i in cat is a common antipattern in bash - you shouldn't use this form. How to read a file line by line in bash bashfaq
`...` Do not use backticks, they are discouraged. Use $(...) instead. bash hackers wiki obsolete and deprecated
Maybe use awk
#check value in third column, using / as a delimiter
check="$(awk -F"/" '{print $3}' /demo/Student.txt)"
#check if third column is empty
if [ "$check" == "" ]; then
echo "failed syntax..."
exit 1
fi
You can run this check on the three columns (change print $3 to $1 and $2)
Probably not the most efficient solution, since I am also fairly new to shell scripting, but I think it should work.
cut -d '/'-f3
This is invalid syntax, as cut freely tells me. Did you not get any console output indicating the problem?
cut: the delimiter must be a single character
Try 'cut --help' for more information.
Fix that by putting a space before the -f switch.
The script then works for me: it reports an error with your file as-is, then no error if I append a 'country' (hi) to the final line.
Demo:
input:
Ajay/abc/india
Avik/bcs/delhi
Viraj/xyz/
output:
$ sh -x test.sh
++ cat test.txt
+ for i in `cat test.txt`
++ echo Ajay/abc/india
++ cut -d / -f3
+ check=india
+ [[ -z india ]]
+ for i in `cat test.txt`
++ echo Avik/bcs/delhi
++ cut -d / -f3
+ check=delhi
+ [[ -z delhi ]]
+ for i in `cat test.txt`
++ echo Viraj/xyz/
++ cut -d / -f3
+ check=
+ [[ -z '' ]]
+ echo failed syntax does not match NAME/SURNAME/COUNTRY
failed syntax does not match NAME/SURNAME/COUNTRY
+ exit 1
I am trying to get this bit of code to work. and I am getting hung up on the second piped grep with the variable $pkgname. I am unable to find a way to get it to read the variable correctly either I get no output or as the code is currently written I get grep: illegal byte sequence. if I put either text with no space in the variable it works or I enter the text as part of the grep statement it works.
#!/bin/bash
counter=0
results2=Executing\ SSH\ MOTD\ banner
pkgname=SSH\ MOTD\ banner
until [ $counter = 1 ]
do
echo $counter
echo $pkgname
echo $results2
result=$(grep "$(date +"%b %d")" /var/log/test.log | grep “$pkgname” | cut -d':' -f 4 | sed 's/^ *//g')
echo $result
if [ “$result” == “$results2” ]; then
counter=1
fi
done
echo finished
so the log file line I am looking for looks like this.
Tue Jun 28 10:58:57 machinename process: Executing SSH MOTD banner
change to
pkgname="SSH MOTD banner" # Here use quotes to avoid using \
As [ #jack ] rightly pointed out in this [ comment ], you need the neutral quotation mark " for a variable to be expanded. That said you can simplify the regex to below
pkgname="SSH MOTD banner"
d=$(date +"%b %d")
result="$(awk -v FS=":" -v d="$d" -v pkg="$pkg_name" '{if($0 ~ date && $0 ~ pkg){sub(/^ */,"",$4);print $4}}' /var/log/test.log)"
I'm new to bash scripting, I'm learning how commands work, I stumble in this problem,
I have a file /home/fedora/file.txt
Inside of the file is like this:
[apple] This is a fruit.
[ball] This is a sport's equipment.
[cat] This is an animal.
What I wanted is to retrieve words between "[" and "]".
What I tried so far is :
while IFS='' read -r line || [[ -n "$line" ]];
do
echo $line | awk -F"[" '{print$2}' | awk -F"]" '{print$1}'
done < /home/fedora/file.txt
I can print the words between "[" and "]".
Then I wanted to put the echoed word into a variable but i don't know how to.
Any help I will appreciate.
Try this:
variable="$(echo $line | awk -F"[" '{print$2}' | awk -F"]" '{print$1}')"
or
variable="$(awk -F'[\[\]]' '{print $2}' <<< "$line")"
or complete
while IFS='[]' read -r foo fruit rest; do echo $fruit; done < file
or with an array:
while IFS='[]' read -ra var; do echo "${var[1]}"; done < file
In addition to using awk, you can use the native parameter expansion/substring extraction provided by bash. Below # indicates a trim from the left, while % is used to trim from the right. (note: a single # or % indicates removal up to the first occurrence, while ## or %% indicates removal of all occurrences):
#!/bin/bash
[ -r "$1" ] || { ## validate input is readable
printf "error: insufficient input. usage: %s filename\n" "${0##*/}"
exit 1
}
## read each line and separate label and value
while read -r line || [ -n "$line" ]; do
label=${line#[} # trim initial [ from left
label=${label%%]*} # trim through ] from right
value=${line##*] } # trim from left through '[ '
printf " %-8s -> '%s'\n" "$label" "$value"
done <"$1"
exit 0
Input
$ cat dat/labels.txt
[apple] This is a fruit.
[ball] This is a sport's equipment.
[cat] This is an animal.
Output
$ bash readlabel.sh dat/labels.txt
apple -> 'This is a fruit.'
ball -> 'This is a sport's equipment.'
cat -> 'This is an animal.'
I am not a Linux scripting expert and I have exhausted my knowledge on this matter. Here is my situation.
I have a list of states passed as a command line argument to a shell script ( e.g "AL,AK,AS,AZ,AR,CA..." ). The Shell script needs to extract each of the state code and write it to a file ( states.txt) , with each state in one line. See below
AL
AK
AS
AZ
AR
CA
..
..
How can this be achieved using a linux shell script.
Thanks in advance.
Use tr:
echo "AL,AK,AS,AZ,AR,CA" | tr ',' '\n' > states.txt
echo "AL,AK,AS,AZ,AR,CA" | awk -F, '{for (i = 1; i <= NF; i++) print $i}';
Naive solution:
echo "AL,AK,AS,AZ,AR,CA" | sed 's/,/\n/g'
I think awk is the simplest solution, but you could try using cut in a loop.
Sample script (outputs to stdout, but you can just redirect it):
#!/bin/bash
# Check for input
if (( ${#1} == 0 )); then
echo No input data supplied
exit
fi
# Initialise first input
i=$1
# While $i still contains commas
while { echo $i| grep , > /dev/null; }; do
# Get first item of $i
j=`echo $i | cut -d ',' -f '1'`
# Shift off the first item of $i
i=`echo $i | cut --complement -d ',' -f '1'`
echo $j
done
# Display the last item
echo $i
Then you can just run it as ./script.sh "AL,AK,AS,AZ,AR,CA" > states.txt (assuming you save it as script.sh in the local directory and give it execute permission)