I'm having difficulty using the ipaddress.collapse_addresses() method.
# n is a list of 192.168.0.0/24 networks (1,2,3,4....etc)
def sumnet():
n = nlist()
for net in n:
snet = ipaddress.collapse_addresses(net)
return snet
I'm only getting back the original list:
Collapsed Networks
[IPv4Network('192.168.0.0/24'), IPv4Network('192.168.1.0/24'),
IPv4Network('192.168.2.0/24'), IPv4Network('192.168.3.0/24'),
IPv4Network('192.168.4.0/24'), IPv4Network('192.168.5.0/24'),
IPv4Network('192.168.6.0/24'), IPv4Network('192.168.7.0/24'),
IPv4Network('192.168.8.0/24')]
Assuming your input is a list of IPv4Networks from ipaddress like...
netlist = [ipaddress.IPv4Network('192.168.0.0/24'),
ipaddress.IPv4Network('192.168.1.0/24'),
ipaddress.IPv4Network('192.168.2.0/24'),
ipaddress.IPv4Network('192.168.3.0/24'),
ipaddress.IPv4Network('192.168.4.0/24'),
ipaddress.IPv4Network('192.168.5.0/24'),
ipaddress.IPv4Network('192.168.6.0/24'),
ipaddress.IPv4Network('192.168.7.0/24'),
ipaddress.IPv4Network('192.168.8.0/24')]
and your desired output is
[IPv4Network('192.168.0.0/21'), IPv4Network('192.168.8.0/24')]
All this can be done with...
import ipaddress
def sumnet(netlist):
return list(ipaddress.collapse_addresses(netlist))
netlist = [ipaddress.IPv4Network('192.168.0.0/24'),
ipaddress.IPv4Network('192.168.1.0/24'),
ipaddress.IPv4Network('192.168.2.0/24'),
ipaddress.IPv4Network('192.168.3.0/24'),
ipaddress.IPv4Network('192.168.4.0/24'),
ipaddress.IPv4Network('192.168.5.0/24'),
ipaddress.IPv4Network('192.168.6.0/24'),
ipaddress.IPv4Network('192.168.7.0/24'),
ipaddress.IPv4Network('192.168.8.0/24')]
print(sumnet(netlist))
The collapse_addresses method actually takes an entire list of addresses, you don't have to feed it ip_addresses one by one. It will return a generator for the collapsed network, but you can just convert that to a list to deal with it easier.
Let me know if this is not what you were trying to accomplish.
It is a little hard to understand exactly what your code is supposed to do as the following snippet starts a for loop where it grabs the first ip address collapses it into a generator and returns that generator with that single ip address, without looking at any of the other ip addresses. This however doesn't seem to be consistent with what your question claims the output to be.
for net in n:
snet = ipaddress.collapse_addresses(net)
return snet
Related
I'm using an API to find information on smart contracts. Each Contract has a unique address and the information can be pulled by plugging it into an API link. Example:
'https://contract-api.dev/contracts/xxxxxxxxxxxxxxxxxxxxxxxxx'
I have a CSV with 1000 contract addresses. Example:
Address
0
0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2
1
0xa5409ec958c83c3f309868babaca7c86dcb077c1
2
0xdac17f958d2ee523a2206206994597c13d831ec7
3
0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48
4
0xa2327a938febf5fec13bacfb16ae10ecbc4cbdcf
...
...
And this code allows me to get exactly what I want for one row,
ADDRESS = df['Address'][0]
total = requests.get(f"https://contract-api.dev/contracts/{ADDRESS}", headers={'CF-Access-Client-Id': 'xxxxxx.access', 'CF-Access-Client-Secret': 'xxxxx'}).json()
total['deployment']['created_by_user']
Where the output is:
'0x4f26ffbe5f04ed43630fdc30a87638d53d0b0876'
I just need to find a way to loop through every row and insert the contract Address into the API link, retrieve the "created_by_user" address, then move to the next row.
What do you think the best way to do this is?
def retrieve_created_by_user(address_):
return requests.get(f"https://contract-api.dev/contracts/{address_}", headers={'CF-Access-Client-Id': 'xxxxxx.access', 'CF-Access-Client-Secret': 'xxxxx'}).json()['deployment']['created_by_user']
for address in df['Address']:
created_by_user = retrieve_created_by_user(address)
...
Or since it can probably take a bit you can use a progress bar like tqdm:
from tqdm import tqdm
for address is tqdm(df['Address']):
created_by_user = retrieve_created_by_user(address)
...
Here do with that rows data whatever you plan to do.
If for some reason you want to be able to index the values (e.g. to restart from a manual index later) you can use list(df['Address']).
I'm using pysnmp to retrieve OID info from Cisco devices. The below code prints out all names of the 1k+ access points associated with the IP.
a = nextCmd(SnmpEngine(), CommunityData('myComm'), UdpTransportTarget((IP_Address, 161)), ContextData(), ObjectType(ObjectIdentity('1.3.6.1.4.1.14179.2.2.1.1.3')))
for errorIndication,errorStatus,errorIndex,varBinds in a:
for v in varBinds:
print(v)
The output looks like this
SNMPv2-SMI::enterprises.xxx.x.x.x.xx.x.160 = cftnnjguapnrs01v100c
SNMPv2-SMI::enterprises.xxx.x.x.x.xx.x.128 = nycmnykyapnrs01v100c
You can see the name payload at the end of the string. Next, I need to get the corresponding connection count for each. However, I don't know how to match the two afterwards. Is there a way to return a tuple for Name, Count, (Location?) in one request?
I would like to extract IP address and port number from the this link. Here is my Python code:
http://spys.one/free-proxy-list/FR/
import urllib.request
import re
url = 'http://spys.one/free-proxy-list/FR/'
req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
html = urllib.request.urlopen(req).read().decode('utf-8')
ip = re.findall( r'[0-9]+(?:\.[0-9]+){3}',html )
# ip = re.findall( r'[0-9]+(?:\.[0-9]+){3}:[0-9]+[0-9]',html) # This is also not working
print (ip)
Output -
['37.59.0.139', '212.47.239.185', '85.248.227.165', '167.114.250.199', '51.15.86.160', '212.83.164.85', '82.224.48.173']
I get only IP address but not the port numbers.
I'm expecting something like this - '37.59.0.139:17658'
Your code does not work because -- aside from several issues with your regex that have been pointed out in other answers -- the website you provided displays the port number of each IP by executing some javascript in the underlying HTML code.
In order to capture each IP and its associated port number, you first need to execute the javascript so that the port numbers are properly printed in the HTML response (you can follow the guidelines here: Web-scraping JavaScript page with Python). Then you need to extract this information from the javascript-computed HTML response.
By inspecting the HTML response, I found out that each port number is preceded by :</font> and followed by <.
A working code snippet can be found below. I took the liberty of slightly modifying your IP-regex as only certain IP addresses were associated with a port number (other IPs were related to the hostname column and should be discarded) - namely, the IPs of interest are those followed by the <script string.
import dryscrape
import re
url = 'http://spys.one/free-proxy-list/FR/'
#get html with javascript
session = dryscrape.Session()
session.visit(url)
response = session.body()
#capture ip:
IP = re.findall(r'[0-9]+(?:\.[0-9]+){3}(?=<script)',response)
#capture port:
port = re.findall(r'(?<=:</font>)(.*?)(?=\<)',response)
#join IP with ports
IP_with_ports = []
for i in range(len(IP)):
IP_with_ports.append(IP[i] + ":" + port[i])
print (IP_with_ports)
OUTPUT: ['178.32.213.128:80', '151.80.207.148:80', '134.119.223.242:80', '37.59.0.139:17459', ..., '37.59.0.139:17658']
Do note that the code above only works for the website you provided, as each website has its own logic for displaying data.
First, you've a bit of a wonky part of your regex: you have (?:, you probably mean (:?. Not sure what the former means, but the latter means zero or one :
Your regex is only looking for four groupings of numbers split by : or .. You need up to five groups of numbers: 0.0.0.0:0000 = five groups. Try this instead:
re.findall( r'([0-9]{1,3}\.){3}[0-9]{1,3}(:[0-9]{2,4})?'
[0-9]{1,3} = between one and 3 digits
\. = a period (escaped, because . means "any character")
{3} = the above needs to be repeated exactly three times
(:[0-9]{2,4}) a colon followed by a numeric sequence between two and four characters long. This is your port.
? the port is optional, it will either be there or it won't.
I have a List of Lists that looks like this (Python3):
myLOL = ["['1466279297', '703.0']", "['1466279287', '702.0']", "['1466279278', '702.0']", "['1466279268', '706.0']", "['1466279258', '713.0']"]
I'm trying to use a list comprehension to convert the first item of each inner list to an int and the second item to a float so that I end up with this:
newLOL = [[1466279297, 703.0], [1466279287, 702.0], [1466279278, 702.0], [1466279268, 706.0], [1466279258, 713.0]]
I'm learning list comprehensions, can somebody please help me with this syntax?
Thank you!
[edit - to explain why I asked this question]
This question is a means to an end - the syntax requested is needed for testing. I'm collecting sensor data on a ZigBee network, and I'm using an Arduino to format the sensor messages in JSON. These messages are published to an MQTT broker (Mosquitto) running on a Raspberry Pi. A Redis server (also running on the Pi) serves as an in-memory message store. I'm writing a service (python-MQTT client) to parse the JSON and send a LoL (a sample of the data you see in my question) to Redis. Finally, I have a dashboard running on Apache on the Pi. The dashboard utilizes Highcharts to plot the sensor data dynamically (via a web socket connection between the MQTT broker and the browser). Upon loading the page, I pull historical chart data from my Redis LoL to "very quickly" populate the charts on my dashboard (before any realtime data is added dynamically). I realize I can probably format the sensor data the way I want in the Redis store, but that is a problem I haven't worked out yet. Right now, I'm trying to get my historical data to plot correctly in Highcharts. With the data properly formatted, I can get this piece working.
Well, you could use ast.literal_eval:
from ast import literal_eval
myLOL = ["['1466279297', '703.0']", "['1466279287', '702.0']", "['1466279278', '702.0']", "['1466279268', '706.0']", "['1466279258', '713.0']"]
items = [[int(literal_eval(i)[0]), float(literal_eval(i)[1])] for i in myLOL]
Try:
import json
newLOL = [[int(a[0]), float(a[1])] for a in (json.loads(s.replace("'", '"')) for s in myLOL)]
Here I'm considering each element of the list as a JSON, but since it's using ' instead of " for the strings, I have to replace it first (it only works because you said there will be only numbers).
This may work? I wish I was more clever.
newLOL = []
for listObj in myLOL:
listObj = listObj.replace('[', '').replace(']', '').replace("'", '').split(',')
newListObj = [int(listObj[0]), float(listObj[1])]
newLOL.append(newListObj)
Iterates through your current list, peels the string apart into a list by replace un-wanted string chracters and utilizing a split on the comma. Then we take the modified list object and create another new list object with the values being the respective ints and floats. We then append the prepared newListObj to the newLOL list. Considering you want an actual set of lists within your list. Your previously documented input list actually contains strings, which look like lists.
This is a very strange format and the best solution is likely to change the code which generates that.
That being said, you can use ast.literal_eval to safely evaluate the elements of the list as Python tokens:
>>> lit = ast.literal_eval
>>> [[lit(str_val) for str_val in lit(str_list)] for str_list in myLOL]
[[1466279297, 703.0], [1466279287, 702.0], [1466279278, 702.0], [1466279268, 706.0], [1466279258, 713.0]]
We need to do it twice - once to turn the string into a list containing two strings, and then once per resulting string to convert it into a number.
Note that this will succeed even if the strings contain other valid tokens. If you want to validate the format too, you'd want to do something like:
>>> def process_str_list(str_list):
... l = ast.literal_eval(str_list)
... if not isinstance(l, list):
... raise TypeError("Expected list")
... str_int, str_float = l
... return [int(str_int), float(str_float)]
...
>>> [process_str_list(str_list) for str_list in myLOL]
[[1466279297, 703.0], [1466279287, 702.0], [1466279278, 702.0], [1466279268, 706.0], [1466279258, 713.0]]
Your input consists of a list of strings, where each string is the string representation of a list. The first task is to convert the strings back into lists:
import ast
lol2 = map(ast.literal_eval, mylol) # [['1466279297', '703.0'], ...]
Now, you can simply get int and float values from lol2:
newlol = [[int(a[0]), float(a[1])] for a in lol2]
Okay so I have a file called 'whois.txt' which contains
["96363612", "#a2743, coil, charge"]
["12101258", "#a0272, climate, vault"]
["83157521", "sith"]
["33907120", "#a1321, missile, wired"]
["55553768", "#a2722, legal, illegal"]
["22686400", "#a5619, mindless, #a5637, bank"]
["97436430", "jedi, #a5770, charge, lantern, #a9491, legal"]
["91645905", "sith"]
["89514799", "lantern, #a2563, #a2693"]
["19658307", "Umbrechu"]
["56112504", "#a0473, lantern, kryptonian"]
["12195491", "riyoken"]
["53281943", "#a5135, gateway, jedi"]
["76515035", "#a4023, gateway, wired"]
["79444876", "#a2716, loyalty"]
What I'm doing here is using json and using the first numbers as an ID and the accounts that are associated with the ID are linked by ', '. So using python I am using this code to try to get all the accounts that are associated
def getWhois(self):
x = []
f = open('whois.txt','r')
for line in f.readlines():
rid,names = json.loads(line.strip())
x.append([rid,names])
return x
def recvWhois(self,user):
returned = self.getWhois()
x = []
for data in returned:
rid,names = data[0],data[1]
if user in names:
x.append(names)
matches = list(set(', '.join(x).split(', ')))
return matches
So what that is doing is getting the matches of a user you are searching but I want to search the users in those matches also, I have done this but It feels Like I would have to do this an infinite amount of times of researching matches that are pulled so if I were to do self.recvWhois('missile') It would pull "['missile', 'wired', '#a1321']" I would then try to search all of those accounts to link more, and by now you probably see my problem because I would have to do that x amount of times depending on how many matches there are linked to the previous matched accounts If any of you have a solution to my problem it would be very appreciated.
First i would suggest to maintain an index for searching. You could use a search engine but a python map can also serve as a poor man's search engine. So idea is to have an inverted index where the usernames points to records to which they belong. For searching all linked accounts you can write a memoized recursive function which will cut down the infinite recursive paths. Also in case you have large no. of records you can limit recursion to a predefined maximum level.
It is really hard to tell what you are trying to do, but I think you are making it too complicated. Your data structure lends itself to a dictionary. Why not load it using rid as the key and names as the values?