bash: extract URLs from a string - string

So, I am trying to get information out of OS X's "favorite servers" .plist so that I can then decide whether or not I want to add certain servers to it. Some of the information for how this can be done can be found here:
http://jacobsalmela.com/bash-script-set-favorite-servers-in-connect-to-menu/
The problem with this is that you can't, for example, just do
/usr/libexec/Plistbuddy -c "Add favoriteservers:CustomListItems:0:Name string server1.fqdn.com" com.apple.sidebarlists.plist
over and over, because Plistbuddy is not smart enough to do an insert into the array. You have to know how long the array is, and then add things to the end of it, such that when you go to add things you have already determined whether you need to use 0 or 5 or 7 between "CustomListItems" and "Name" up there.
The obnoxiousness of that aside, I'm having trouble just parsing the output from the Plistbuddy print command, which looks like this:
Array { Dict { Name = afp://or-fs-001/vol1 URL = afp://or-fs-001/vol1 } Dict { Name = smb://or-fs-001/vol1 URL = smb://or-fs-001/vol1 } Dict { Name = vnc://or-fs-001/vol1 URL = vnc://or-fs-001/vol1 } Dict { Name = ftp://or-fs-001/vol1 URL = ftp://or-fs-001/vol1 } }
So you have the same URL twice for each entry (I have no idea why there is both a "Name" and "URL" when you can't actually make them different), and they may start with any protocol supported by Finder, which means afp, http, https, smb, or vnc. The first thing I'm trying to do is just split them up into pieces by the "Name" substring so that I know how many entries are in the list, but that results in weird behavior when I use tr for that; it starts cutting out way too many pieces.
Does anyone have ideas for better ways to do this? Can I count the number of times "Dict" shows up?

You can use grep -o to extract interesting parts of the input. An example:
#!/bin/bash
output='Array { Dict { Name = afp://or-fs-001/vol1 URL = afp://or-fs-001/vol1 } Dict { Name = smb://or-fs-001/vol1 URL = smb://or-fs-001/vol1 } Dict { Name = vnc://or-fs-001/vol1 URL = vnc://or-fs-001/vol1 } Dict { Name = ftp://or-fs-001/vol1 URL = ftp://or-fs-001/vol1 } }'
count=$(echo "$output" | grep -o 'Name =' | wc -l)
names=($(grep -o 'Name = [^ ]\+' <<< "$output" | cut -f3- -d' '))
echo $count = ${#names[#]}
for name in "${names[#]}" ; do
echo "$name"
done

Related

How do you validate file paths in TCL?

Given a string, I'd like to know if the string represents a valid file path. I've looked at [file isfile <string>] and [file isdirectory <string>] but it seems those two return false if the given string isn't pointing to an existing file/directory. I'm merely interested in the validity of the string.
Pretty much every string is a valid file path. Can you give some examples of what you consider invalid file paths?
You can use file pathtype to check if the path is absolute. Also file split and file normalize may be useful, depending on what you really need.
I don't know that there's a particular way to do that.
I suppose you could try:
set fn /home/bll/mytest.txt
set fail true
if { [file exists $fn] } {
set fail false
} else {
try {
set fh [open $fn w]
set fail false
file delete $fn
} on error {err res} {
}
}
Even if the above works, certain characters may cause issues if you use
the filename in an external command line.
I strip the following characters simply to avoid possible command line problems:
* : " ? < > | [:cntrl:]
I have this list as possibly problematical for unix:
* & [] ? ' " < > |
And this list for windows:
* : () & ^ | < > ' "
Also, a trailing dot is illegal for windows directory names.
And slashes or backslashes can cause issues as those are directory separators.

How to extract key value pairs from a file when values span multiple lines?

I'm a few weeks into bash scripting and I haven't advanced enough yet to get my head wrapped around this problem. Any help would be appreciated!
I have a "script.conf" file that contains the following:
key1=value1
key2=${HOME}/Folder
key3=( "k3v1" "k3 v2" "k3v3")
key4=( "k4v1"
"k4 v2"
"k4v3"
)
key5=value5
#key6="Do Not Include Me"
In a bash script, I want to read the contents of this script.conf file into an array. I've learned how to handle the scenarios for keys 1, 2, 3, and 5, but the key4 scenario throws a wrench into it with it spanning across multiple lines.
I've been exploring the use of sed -n '/=\s*[(]/,/[)]/{/' which does capture key4 and its value, but I can't figure out how to mix this so that the other keys are also captured in the matches. The range syntax is also new to me, so I haven't figured out how to separate the key/value. I feel like there is an easy regex that would accomplish what I want... in plain-text: "find and group the pattern ^(.*)= (for the key), then group everything after the '=' char until another ^(.*)= match is found, rinse and repeat". I guess if I do this, I need to change the while read line to not handle the key/value separation for me (I'll be looking into this while I'm waiting for a response). BTW, I think a solution where the value of key4 is flattened (new lines removed) would be acceptable; I know for key3 I have to store the value as a string and then convert it to an array later when I want to iterate over it since an array element apparently can't contain a list.
Am I on the right path with sed or is this a job for awk or some other tool? (I haven't ventured into awk yet). Is there an easier approach that I'm missing because I'm too deep into the forest (like changing the while read line in the LoadConfigFile function)?
Here is the code that I have so far in script.sh for processing and capturing the other pairs into the $config array:
__AppDir=$(dirname $0)
__AppName=${__ScriptName%.*}
typeset -A config #init config array
config=( #Setting Default Config values
[key1]="defaultValue1"
[key2]="${HOME}/defaultFolder"
[QuietMode]=0
[Verbose]=0 #Ex. Usage: [[ "${config[Verbose]}" -gt 0 ]] && echo ">>>Debug print"
)
function LoadConfigFile() {
local cfgFile="${1}"
shopt -s extglob #Needed to remove trailing spaces
if [ -f ${cfgFile} ]; then
while IFS='=' read -r key value; do
if [[ "${key:0:1}" == "#" ]]; then
#echo "Skipping Comment line: ${key}"
elif [ "${key:-EMPTY}" != "EMPTY" ]; then
value="${value%%\#*}" # Delete in-line, right comments
value="${value%%*( )}" # Delete trailing spaces
value="${value%%( )*}" # Delete leading spaces
#value="${value%\"*}" # Delete opening string quotes
#value="${value#\"*}" # Delete closing string quotes
#Manipulate any variables included in the value so that they can be expanded correctly
# - value must be stored in the format: "${var1}". `backticks`, "$var2", and "doubleQuotes" are left as is
value="${value//\"/\\\"}" # Escape double quotes for eval
value="${value//\`/\\\`}" # Escape backticks for eval
value="${value//\$/\\\$}" # Escape ALL '$' for eval
value="${value//\\\${/\${}" # Undo the protection of '$' if it was followed by a '{'
value=$(eval "printf '%s\n' \"${value}\"")
config[${key}]=${value} #Store the value into the config array at the specified key
echo " >>>DBG: Key = ${key}, Value = ${value}"
#else
# echo "Skipped Empty Key"
fi
done < "${cfgFile}"
fi
}
CONFIG_FILE=${__AppDir}/${__AppName}.conf
echo "Config File # ${CONFIG_FILE}"
LoadConfigFile ${CONFIG_FILE}
#Print elements of $config
echo "Script Config Values:"
echo "----------------------------"
for key in "${!config[#]}"; do #The '!' char gets an array of the keys, without it, we would get an array of the values
printf " %-20s = %s\n" "${key}" "${config[${key}]}"
done
echo "------ End Script Config ------"
#To convert to an array...
declare -a valAsArray=${config[RequiredAppPackages]} #Convert the value from a string to an array
echo "Count = ${#valAsArray[#]}"
for itemCfg in "${valAsArray[#]}"; do
echo " item = ${itemCfg}"
done
As I mentioned before, I'm just starting to learn bash and Linux scripting in general, so if you see that I'm doing some taboo things in other areas of my code too, please feel free to provide feedback in the comments... I don't want to start bad habits early on :-).
*If it matters, the OS is Ubuntu 14.04.
EDIT:
As requested, after reading the script.conf file, I would like for the elements in $config[#] to be equivalent to the following:
typeset -A config #init config array
config=(
[key1]="value1"
[key2]="${HOME}/Folder"
[key3]="( \"k3v1\" \"k3 v2\" \"k3v3\" )"
[key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )"
[key5]="value5"
)
I want to be able to convert the values of elements 'key4' and 'key3' into an array and iterated over them the same way in the following code:
declare -a keyValAsArray=${config[keyN]} #Convert the value from a string to an array
echo "Count = ${#keyValAsArray[#]}"
for item in "${keyValAsArray[#]}"; do
echo " item = ${item}"
done
I don't think it matters if \n is preserved for key4's value or not... that depends on if declare has a problem with it.
A shell is an environment from which to call tools with a language to sequence those calls. It is NOT a tool to manipulate text. The standard UNIX tool to manipulate text is awk. Trying to manipulate text in shell IS a bad habit, see why-is-using-a-shell-loop-to-process-text-considered-bad-pr‌​actice for SOME of the reasons why
You still didn't post the expected result of populating the config array so I'm not sure but I think this is what you wanted:
$ cat tst.sh
declare -A config="( $(awk '
{ gsub(/^[[:space:]]+|([[:space:]]+|#.*)$/,"") }
!NF { next }
/^[^="]+=/ {
name = gensub(/=.*/,"",1)
value = gensub(/^[^=]+=/,"",1)
n2v[name] = value
next
}
{ n2v[name] = n2v[name] OFS $0 }
END {
for (name in n2v) {
value = gensub(/"/,"\\\\&","g",n2v[name])
printf "[%s]=\"%s\"\n", name, value
}
}
' script.conf
) )"
declare -p config
$ ./tst.sh
declare -A config='([key5]="value5" [key4]="( \"k4v1\" \"k4 v2\" \"k4v3\" )" [key3]="( \"k3v1\" \"k3 v2\" \"k3v3\")" [key2]="/home/Ed/Folder" [key1]="value1" )'
The above uses GNU awk for gensub(), with other awks you'd use [g]sub() instead.

How to select a row using shell scripting?

Consider the below example:
Parameter1 = 5
Parameter2 = 10
Parameter3 = 15
Parameter4 = 20
I want to fetch the value depending on the parameter name by providing a user input as shown below:
echo ""
echo " Enter the parameter name"
read value
Parameter = "$value"
Check if the parameter is existing within the concern file
if grep -qs "$Parameter" "Filename"
then
echo " Parameter exist within the concern file"
val = #Here I want to fetch the value of the parameter that the user had input and I do not know how to do it?
Kindly let me know how I can fetch the value of the parameter which the user had given as input.
If I understand you correctly, you could use something like this:
read param_name # e.g. Parameter2
value=$(awk -v param="$param_name" '$1 == param { print $3 }' filename)
This reads the name of the parameter then prints the third field on the line when the first field matches the name. The result is assigned to the shell variable $value.

Shell Script to parse/retrieve a string found after another string/match

The shell script will be passed a string of arguments. The position of the key/value I am looking to parse out may change over time, i.e. it may come before or after another key at any time so parsing between two keys wouldn't be an option.
I am looking to parse the domain key out of a string like this:
maxpark 0 maxsub n domain sample.foo maxlst n max_defer_fail_percentage user oli force no_cache_update 0 maxpop n maxaddon 0 locale en contactemail
The key would be "domain" the value would be "sample.foo". The domain key could have more than one '.' in it so I would need to grab the entire domain key.
I am not the best with regular expressions but I imagine using 'sed' is what I'm going to need to do.
I am accessing this full string using $*, if I could simply reference the key by accessing $DOMAIN that would be great, but since my only option is to access based on position, $3, and the position could change, that isn't an option
Solved the problem using PERL.
#!/usr/bin/perl -w
use strict;
my %OPTS = #ARGV;
open(FILE, "</var/named/$OPTS{'domain'}.db") || die "File not found";
my #lines = <FILE>;
close(FILE);
my #newlines;
foreach(#lines) {
$_ =~ s/$LOCAL_IP/$PUBLIC_IP/g;
push(#newlines,$_);
}
open(FILE, ">/var/named/$OPTS{'domain'}.db") || die "File not found";
print FILE #newlines;
close(FILE);
If you do have perl, just use this one-liner from your shell script.
domain=$( echo $* | perl -ne '/domain\s([^\s]+)\s/ and print "$1"' )
Or if you'd rather just do it with sed:
domain=$( echo $* | sed 's/.*\<domain \([^ ]\+\).*/\1/' )

How can I check if an environmental variable is set?

I have the following code in perl
my %Opt =
(
boards_txt => "$ENV{'ARDUINO_DIR'}/hardware/arduino/boards.txt",
);
In this you can see that the env variable ARDUINO_DIR is append. Some users might not have this variable set. If that is the case, then I want to hardcode a path.
Question: How can I check if the env variable is set or not?
The correct answers have been given, but I wanted to add that you might make use of the rather handy defined-or assignment operator //=:
my $dir = $ENV{'ARDUINO_DIR'};
$dir //= "/other/path";
Or, as RobEarl points out in the comment:
my $dir = $ENV{'ARDUINO_DIR'} // "/other/path";
This is the logical equivalent of
my $dir;
if (defined $ENV{'ARDUINO_DIR'}) {
$dir = $ENV{'ARDUINO_DIR'};
} else {
$dir = "/other/path";
}
As mob points out, the defined-or operator requires perl v5.10. For those who still have not upgraded to that version it is also possible to use the || operator:
my $dir = $ENV{'ARDUINO_DIR'} || "/other/path";
The caveat being that this will overwrite values that are interpreted as false, which may in some context be considered proper values, such as the empty string or zero. In this case, however, it is unlikely that 0 or the empty string are valid paths.
You are already using the %ENV hash. It contains all environment variables, so you could do something like:
if (defined $ENV{'ARDUINO_DIR'}) { $prefix = $ENV{'ARDUINO_DIR'} }
else { $prefix = '/path/to/arduino/dir/' }
my $path_to_txt = $prefix . 'boards.txt';
I suggest you use File::Spec for working with paths.
You can check for existence of a hash key with exists:
perl -le 'print "fnord!" if exists $ENV{"ARDUINO_DIR"}'

Resources