I was just wondering how I would go about writing a shell script to separate proxy IPs from their ports.
The proxies are stored in this format
ip:port
ip:port
ip:port
How can I use a shell script to separate the IP on the left side of the colon from the Port on the right side, and put the IP and Port lists in separate .txt files with the same order? Is this even possible?
If the proxies are listed that way in a file, say proxy.txt, then all you need is cut:
cut -f1 -d: proxy.txt > proxy_ip.txt
cut -f2 -d: proxy.txt > proxy_port.txt
Try something like this:
#!/bin/bash
ips="1.2.3.4:123 2.3.4.5:356 4.5.6.7:576"
# or get IPs from stdin
# split them
ips_array=($ips)
for w in ${ips_array[#]}
do
echo $w | sed -e 's/:.*$//g' >> ips.txt
echo $w | sed -e 's/^.*://g' >> ports.txt
done
Key is using the ($ips) to split the list up.
EDIT:
I just realized that you didn't format your question correctly so it's not a single line with IP:PORTs separated by spaces, but one on a line by itself. You just need this then:
#!/bin/bash
while read w
do
echo $w | sed -e 's/:.*$//g' >> ips.txt
echo $w | sed -e 's/^.*://g' >> ports.txt
done
And you read from stdin.
Related
Basically, I have a file formatted like
ABC:123
And I would like to flip the strings around the delimiter, so it would look like this
123:ABC
I would prefer to do this with bash/linux tools.
Thanks for any help!
That's reasonably easy with internal bash commands, assuming two fields, as per the following transcript:
pax:~$ x='abc:123'
pax:~$ echo "${x#*:}:${x%:*}"
123:abc
The first substitution ${x#*:} removes everything from the start up to the colon. The second, ${x%:*}, removes everything from the colon to the end.
Then you just re-join them with the colon in-between.
It doesn't matter for your particular data but % and # use the shortest possible pattern. The %% and ## variants will give you the longest possible pattern (greedy).
As an aside, this is ideal if you doing it for one string at a time since you don't need to kick up an external process to do the work for you. But, if you're processing an entire file, there are better ways to do it, such as with awk:
pax:~$ printf "abc:123\ndef:456\nghi:789\n" | awk -F: '{print $2 FS $1}'
123:abc
456:def
789:ghi
#!/bin/sh -x
var1=$(echo -e 'ABC:123' | cut -d':' -f1)
var2=$(echo -e 'ABC:123' | cut -d':' -f2)
echo -e "${var2}":"${var1}"
I use cut to split the string into two parts, and store both of those parts as variables.
From there, it's possible to use echo to re-arrange the variables as you see fit.
Using sed.
sed -E 's/(.*):(.*)/\2:\1/' file.txt
Using paste and cut with process substitution.
paste -d: <(cut -d : -f2 file.txt) <(cut -d : -f1 file.txt)
A slower/slowest shell solution on large set of data/files.
while IFS=: read -r left rigth; do printf '%s:%s\n' "$rigth" "$left"; done < file.txt
I need to generate a script from within a script but am having problems because some of the commands going into the new script are being interpreted rather than written to the new file. For example i want to create a file called start.sh in it I want to set a variable to the current IP address:
echo "localip=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1 -d'/')" > /start.sh
what gets written to the file is:
localip=192.168.1.78
But what i wanted was the following text in the new file:
localip=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1 -d'/')"
so that the IP is determined when the generated script is run.
What am i doing wrong ?
You're making this unnecessary hard. Use a heredoc with a quoted sigil to pass literal contents through without any kind of expansion:
cat >/start.sh <<'EOF'
localip=$(ip addr | grep 'state UP' -A2 | tail -n1 | awk '{print $2}' | cut -f1 -d'/')
EOF
Using <<'EOF' or <<\EOF, as opposed to just <<EOF, is essential; the latter will perform expansion just as your original code does.
If anything you're writing to start.sh needs to be based on current variables, by the way, be sure to use printf %q to safely escape their contents. For instance, to set your current $1, $2, etc. to be active during start.sh execution:
# open start.sh for output on FD 3
exec 3>/start.sh
# build a shell-escaped version of your argument list
printf -v argv_str '%q ' "$#"
# add to the file we previously opened a command to set the current arguments to that list
printf 'set -- %s\n' "$argv_str" >&3
# pass another variable through safely, just to be sure we demonstrate how:
printf 'foo=%q\n' "$foo" >&3
# ...go ahead and add your other contents...
cat >&3 <<'EOF'
# ...put constant parts of start.sh here, which can use $1, $2, etc.
EOF
# close the file
exec 3>&-
This is far more efficient than using >>/start.sh on every line that needs to append: Using exec 3>file and then >&3 only opens the file once, rather than opening it once per command that generates output.
I have many configuration files ,
the line that start with LINE word have IP address
My target to read the line that start with LINE word from the file and print only the IP address
The problem is that IP address can be in any field in the line so I can’t capture the IP according to field number
example
grep LINE file1.txt
LINE /home/Ariate/run.pl "Voda STS 4 Test - " "102841" && ssh 17.77.170.130 -p 2022
grep LINE file2.txt
LINE /home/Ariate/run.pl 137.77.170.30 "Voda STS 4 Test - " "102841" && ssh ACTIVE
please advice how to capture the IP address from the line ( solution can be also with perl one liner )
expected results
echo $IP_FROM_LINE
17.77.170.130
echo $IP_FROM_LINE
137.77.170.30
perl -MRegexp::Common=net -lne 'print $1 if /^LINE.*\b($RE{net}{IPv4})/'
Using this grep -oE:
grep -oE '\d+\.\d+\.\d+\.\d+' file
17.77.170.130
137.77.170.30
OR else:
grep -oP '\d+\.\d+\.\d+\.\d+' file
The following will get you the desired IP addresses:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file
To place the result in a variable as request, you'll need to iterate of the results as follows:
grep -oP '^LINE.*\b\K\d+\.\d+\.\d+\.\d+' file |
while read IP_FROM_LINE ; do
echo $IP_FROM_LINE
done
grep -oE '[0-9]{2,3}(\.[0-9]{2,3}){3}'
matches
17.77.170.130
137.77.170.30
or
grep -oP '\d{2}(\.\d{2}){3}'
if your grep supports -P option.
both of them works with the data you have given.
But if you want really worried of what to be matched, use
grep -Eo '(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)'
which would match excat ip addresses.
I read some data out of a file, used grep for the only two columns needed, and redirected the output into a variable.
My script looks like this:
#!/bin/bash
cat hosts.cfg | grep 'address\|host_name' | sed -e 's/\<address\>//g' | while read line; do
echo $line | sed 's/host_name//g' | sed -r 's/\s+//g' ;
done
The output looks something like this now:
Host1
xx.xx.xx.xx
Host2
xx.xx.xx.xx
The problem is that hosts and ips must be saved into an array, not a file!
Output must look like this:
Host1(tab)xx.xx.xx.xx
Host2(tab)xx.xx.xx.xx
You are looking for process substitution. $(command), or old-style in `s.
(sorry, the description of how it should work is not clear enough for me to show modified version of your code)
You can use awk:
echo $output | awk 'NR%2{printf $0"\t";next;}1'
To save any command output, wrap it in backticks, or the newer (but less backward compatible) $(command) style substitution. E.g.:
result=`echo $output | awk 'NR%2{printf $0"\t";next;}1'`
using sed 'N;s/\n/\t/g'
change
Host1
xx.xx.xx.xx
Host2
xx.xx.xx.xx
to
Host1(tab)xx.xx.xx.xx
Host2(tab)xx.xx.xx.xx
You can use "set" to get faster output
exg:
I have file best2,
# cat best2
Host1 xx.xx.xx.xx Host2 xx.xx.xx.xx
make a script called: tabcheck.sh
# cat tabcheck.sh
#!/bin/bash
out=$(cat best2)
set $out
echo -e "$1\t$2\n$3\t$4"
# ./tabcheck.sh
Host1 xx.xx.xx.xx
Host2 xx.xx.xx.xx
If you use shift command(Usually only nine command line arguments can be accessed using positional parameters. The shift command gives access to command line arguments greater than nine by shifting each of the arguments.) as well.
Thanks.
I am trying to write a script that uses a user-created MAC address when using the wlan0 interface.
To test, I made a file called testFile.txt, which is a copy of /etc/network/interfaces. If there are interfaces after wlan0, I cannot use echo "$var" >> testFile.txt because that simply adds the text to the end.
I am able to find the end of the wlan0 interface text, but I am not sure how to insert there. Below is what I currently have:
#!/bin/bash
echo "Enter MAC Address"
read var
log=$(cat testFile.txt | grep -o "wlan0.*" | grep -o dhcp)
echo $log
echo $log prints dhcp.
I tried adding | echo "hwaddress ether $var" >> testFile.txt to $log but that still appends to the end of the file.
How do I insert directly after $log?
The Problem
Useless use of cat. Don't do that.
You're not using the right tool for the job. Use GNU sed to make life easier on yourself.
The Solution
# Insert hwaddress line into /etc/network/interfaces.
read -p "Enter MAC Address: "
sudo sed -i.bak "/iface wlan0/a\ hwaddress $REPLY" /etc/network/interfaces
# Replace existing hwaddress line in /etc/network/interfaces.
read -p "Enter MAC Address: "
sudo sed -ri.bak "s/(hwaddress).*/\1 $REPLY/"
You may want to use sed to insert the MAC address. If you want to insert it at the end of the line with wlan0, that would be
sed "s/\(wlan0.*\)/\1 $var/" testFile.txt > testFile.txt.tmp
If that works, then
mv testFile.txt.tmp testFile.txt
you need a different tool, SED for example
$ seq 5 | sed '/3/ s/.*/something else/'
1
2
something else
4
5
for example you can Substitute the line with '3' by 'something else' - and there is no limits