i like the function pkt.sessioins() in scapy, how i can add more columns to this table:
x.sessions()
{'TCP 10.0.2.15:32896 > 173.194.34.159:80': <PacketList: TCP:11 UDP:0 ICMP:0 Other:0>,
Thanks
To add more information you would need to define your own "session extractor" function and pass it to the .sessions() method.
But you cannot add information about the whole session (like total data size) since this is known only when the whole session has been read.
A quick way to add total data size, for example, would be to create your own sessions() function (adapted from the original PacketList.sessions() method:
def sessions(plist, session_extractor=None):
if session_extractor is None:
def session_extractor(p):
sess = "Other"
if 'Ether' in p:
if 'IP' in p:
if 'TCP' in p:
sess = p.sprintf("TCP %IP.src%:%r,TCP.sport% > %IP.dst%:%r,TCP.dport%")
elif 'UDP' in p:
sess = p.sprintf("UDP %IP.src%:%r,UDP.sport% > %IP.dst%:%r,UDP.dport%")
elif 'ICMP' in p:
sess = p.sprintf("ICMP %IP.src% > %IP.dst% type=%r,ICMP.type% code=%r,ICMP.code% id=%ICMP.id%")
else:
sess = p.sprintf("IP %IP.src% > %IP.dst% proto=%IP.proto%")
elif 'ARP' in p:
sess = p.sprintf("ARP %ARP.psrc% > %ARP.pdst%")
else:
sess = p.sprintf("Ethernet type=%04xr,Ether.type%")
return sess
sessions = defaultdict(lambda: [plist.__class__(), 0])
for p in plist.res:
sess = session_extractor(plist._elt2pkt(p))
sessions[sess][0].append(p)
sessions[sess][1] += len(p)
return dict(sessions)
Each value (the keys do not change) in the resulting dict is a two elements list, the first one being the PacketList object, just as with the .sessions() method, and the second one the total data size for the session.
Related
I am working on a computer vision project where the model is predicting the objects in the frame. I am appending all the objects in a list detectedObjs. I have to create a list of dicts for these detected objects which will contain the name, start time and end time of the object. Start time basically means when the object was first detected and end time means when the object was last detected. So for this I have below code:
for obj in detectedObjs:
if not objList:
# First object is detected, save its information
tmp = dict()
tmp['Name'] = obj
tmp['StartTime'] = datetime.datetime.utcnow().isoformat()
tmp['EndTime'] = datetime.datetime.utcnow().isoformat()
objList.append(tmp)
else:
# Here check if the object is alreay present in objList
# If yes, then keep updating end time
# If no, then add the object information in objList
for objDict in objList:
if objDict['Name'] == obj:
objDict["EndTime"] = datetime.datetime.utcnow().isoformat()
break
else:
tmp = dict()
tmp['Name'] = obj
tmp['StartTime'] = datetime.datetime.utcnow().isoformat()
tmp['EndTime'] = datetime.datetime.utcnow().isoformat()
objList.append(tmp)
So first in for loop I am saving the information of the first detected object. After that in else, I am checking if the current object is already added in objList, if yes then keep updating the end time otherwise, add it in objList.
The detectedObjs list have item1 and then after few secs item2 is also added. But in the output of objList I can see item1 properly added but item2 is added lot many times. Is there any way to optimize this code so that I can have proper start and end times. Thanks
Below is the full reproducible code. I cannot put the code of prediction from the model here so I have added a thread which will keep on adding items to detectedObj list
from threading import Thread
import datetime
import time
detectedObjs = []
def doJob():
global detectedObjs
for i in range(2):
if i == 0:
detectedObjs.append("item1")
elif i == 1:
detectedObjs.append("item2")
elif i == 2:
detectedObjs.append("item3")
elif i == 3:
detectedObjs.remove("item1")
elif i == 4:
detectedObjs.remove("item2")
elif i == 5:
detectedObjs.remove("item3")
time.sleep(3)
Thread(target=doJob).start()
while True:
objList = []
for obj in detectedObjs:
if not objList:
# First object is detected, save its information
tmp = dict()
tmp['Name'] = obj
tmp['StartTime'] = datetime.datetime.utcnow().isoformat()
tmp['EndTime'] = datetime.datetime.utcnow().isoformat()
objList.append(tmp)
else:
# Here check if the object is alreay present in objList
# If yes, then keep updating end time
# If no, then add the object information in objList
for objDict in objList:
if objDict['Name'] == obj:
objDict["EndTime"] = datetime.datetime.utcnow().isoformat()
break
else:
tmp = dict()
tmp['Name'] = obj
tmp['StartTime'] = datetime.datetime.utcnow().isoformat()
tmp['EndTime'] = datetime.datetime.utcnow().isoformat()
objList.append(tmp)
print(objList)
I would recommend you use a dict containing dicts… here is an untested version of your code…
obj_dict = {}
for obj in detectedObjs:
if obj not in obj_dict: # checks the keys for membership
# first entry
time_seen = datetime.datetime.utcnow().isoformat()
obj_dict[obj] = {
“name”: obj,
“start”: time_seen,
“end”: time_seen,
}
else: # additional time(s) seen
time_seen = datetime.datetime.utcnow().isoformat()
obj_dict[obj][“end”] = time_seen
Additionally this will save on processing as your list grows larger, it won’t have to search the whole list for an entry each time to update it.
I know that it is possible to backup your data by simply copying them
to some storage (on Linux they are usually installed under
/var/lib/couchdb/*.couch). Also in that same directories are data
indexed by views. I also know that you can make a copy of views by
sending HTTP requests to another database. But what I don't know is, is it
possible to save your views by simply copying them to a storage like
with data? To make my question a little bit clearer: Can you make
a backup of views without another database instance? Also, I am using
CouchDB 1.6 which has different file structure than versions greater
than 2.
Let me post an answer, this is only solution that i could think of...Of course it can be improved or changed but it works for me.
Script needs to be called from terminal (unless you edit it in some IDE) with
sys.argv[1] like python3 viewBackup.py 1 or python3 viewBackup.py 2
(1 is to make backup file, 2 is to upload that backup).Since this script is so simple there's no need for checking if backup file already exists (if there is backup made)
import requests
import json
import sys
# these are credentials of DB which views we want to backup
from_db = 'http://localhost:5984/'
from_user = 'root'
from_pswd = '1234'
from_auths = (from_user, from_pswd)
#`and these are credentials of DB where we want to upload our backuped views`
to_db = 'http://localhost:5984/'
to_user = 'root'
to_pswd = '1234'
to_auths = (from_user, from_pswd)
tf = 'bucket2views.json'
def getViews(source, credS):
global tf
data = _getDBList(source, credS)
temp = []
f = open(tf, 'w')
for d in data:
designs = requests.get(source + d + '/_all_docs?startkey="_design/"&endkey="_desing0"&include_docs=true',auth=credS).json()
designs = designs['rows']
for z in designs:
z = z['doc']['_id'].split('/')
z = z[1]
try:
print("getting views, it might take some time, please be patient")
views = requests.get(source + d + '/_design/' + z, auth=credS).json()['views']
print(" views gathered, start processing them")
views2 = {'views': views}
payload = json.dumps(views2)
temp.append((d, z, payload))
except Exception as e:
pass
f.write(json.dumps(temp))
f.close()
print(' views has been saved')
def _getDBList(server, auth):
response = requests.get(server + '_all_dbs', auth=auth).json()
return response
def uploadViews(srv, cred):
f = open(tf, 'r')
txt = f.read()
f.close()
jt = json.loads(txt)
for item in jt:
d = item[0]
z = item[1]
full_name = item[2]
resp = requests.put(srv+ d +'/_design/'+ z, data=json.dumps(payload), auth=cred).json()
print (json.dumps(resp))
def main():
ans = sys.argv[1]
# ans = 2
if ans in [1, '1', 'getViews']:
getViews(from_db, from_auths)
elif ans in [2, '2', "uploadViews"]:
uploadViews(to_db, to_auths)
else:
print('Possible inputs: 1, getViews, 2, uploadViews')
main()
I am trying to create a calorie counter the standard input goes like this:
python3 calories.txt < test.txt
Inside calories the food is the following format: apples 500
The problem I am having is that whenever I calculate the values for the person it seems to never return to an empty list..
import sys
food = {}
eaten = {}
finished = {}
total = 0
#mappings
def calories(x):
with open(x,"r") as file:
for line in file:
lines = line.strip().split()
key = " ".join(lines[0:-1])
value = lines[-1]
food[key] = value
def calculate(x):
a = []
for keys,values in x.items():
for c in values:
try:
a.append(int(food[c]))
except:
a.append(100)
print("before",a)
a = []
total = sum(a) # Problem here
print("after",a)
print(total)
def main():
calories(sys.argv[1])
for line in sys.stdin:
lines = line.strip().split(',')
for c in lines:
values = lines[0]
keys = lines[1:]
eaten[values] = keys
calculate(eaten)
if __name__ == '__main__':
main()
Edit - forgot to include what test.txt would look like:
joe,almonds,almonds,blue cheese,cabbage,mayonnaise,cherry pie,cola
mary,apple pie,avocado,broccoli,butter,danish pastry,lettuce,apple
sandy,zuchini,yogurt,veal,tuna,taco,pumpkin pie,macadamia nuts,brazil nuts
trudy,waffles,waffles,waffles,chicken noodle soup,chocolate chip cookie
How to make it easier on yourself:
When reading the calories-data, convert the calories to int() asap, no need to do it every time you want to sum up somthing that way.
Dictionary has a .get(key, defaultvalue) accessor, so if food not found, use 100 as default is a 1-liner w/o try: ... except:
This works for me, not using sys.stdin but supplying the second file as file as well instead of piping it into the program using <.
I modified some parsings to remove whitespaces and return a [(name,cal),...] tuplelist from calc.
May it help you to fix it to your liking:
def calories(x):
with open(x,"r") as file:
for line in file:
lines = line.strip().split()
key = " ".join(lines[0:-1])
value = lines[-1].strip() # ensure no whitespaces in
food[key] = int(value)
def getCal(foodlist, defValueUnknown = 100):
"""Get sum / total calories of a list of ingredients, unknown cost 100."""
return sum( food.get(x,defValueUnknown ) for x in foodlist) # calculate it, if unknown assume 100
def calculate(x):
a = []
for name,foods in x.items():
a.append((name, getCal(foods))) # append as tuple to list for all names/foods eaten
return a
def main():
calories(sys.argv[1])
with open(sys.argv[2]) as f: # parse as file, not piped in via sys.stdin
for line in f:
lines = line.strip().split(',')
for c in lines:
values = lines[0].strip()
keys = [x.strip() for x in lines[1:]] # ensure no whitespaces in
eaten[values] = keys
calced = calculate(eaten) # calculate after all are read into the dict
print (calced)
Output:
[('joe', 1400), ('mary', 1400), ('sandy', 1600), ('trudy', 1000)]
Using sys.stdin and piping just lead to my console blinking and waiting for manual input - maybe VS related...
I'm creating a game where i have the data imported from a database, but i have a little problem...
Currently i get a copy of the data as a dictionary, which i need to pass as argument to my GUI, however i also need to process some data, like in this example:
I get the data as a dict (I've created the UseDatabase context manager and is working):
def get_user(name: str, passwd: str):
user = {}
user['name'] = name
user['passwd'] = passwd
with UseDatabase() as cursor:
_SQL = "SELECT id, cash, ruby FROM user WHERE name='Admin' AND password='adminpass'"
cursor.execute(_SQL)
res = cursor.fetchall()
if res:
user['id'] = res[0][0]
user['cash'] = res[0][1]
user['ruby'] = res[0][2]
return user
return res
.
.
.
def get_activities():
with UseDatabase() as cursor:
_SQL = "SELECT * FROM activities WHERE user_id='2'"
cursor.execute(_SQL)
res = cursor.fetchall()
if res:
ids = [i[0] for i in res]
activities = {}
for i in res:
activities[i[0]] = {'title':i[1],'unlock':i[2],'usr_progress':i[3]}
return (ids, activities)
return res
Need it as a dict in my GUI ("content" argument):
class SideBar:
def __init__(self, screen: 'pygame.display.set_mode()', box_width: int, box_height: int, content: dict, font: 'font = pygame.font.Font()'):
#content dict: {id: {'title':'','unlock':'','usr_progress':''},...}
self.box_width = box_width
self.box_height = box_height
self.box_per_screen = screen.get_height() // box_height
self.content = content
self.current_box = 1
self.screen = screen
self.font = font
self.generate_bar()
def generate_bar (self):
active = [i for i in self.content.keys() if i in range(self.current_box, self.current_box+self.box_per_screen)]
for i in range(self.box_per_screen):
gfxdraw.box(self.screen,pygame.Rect((0,i*self.box_height),(self.screen.get_width()/3,self.screen.get_height()/3)),(249,0,0,170))
self.screen.blit(self.font.render(str(active[i]) + ' - ' + self.content[active[i]]['title'], True, (255,255,255)),(10,i*self.box_height+4))
for i in range(self.box_per_screen):
pygame.draw.rect(self.screen,(50,0,0),pygame.Rect((0,i*self.box_height),(self.screen.get_width()/3,self.screen.get_height()/3)),2)
But still need to make some changes in the data:
def unlock_act(act_id):
if user['cash'] >= activities[act_id]['unlock'] and activities[act_id]['usr_progress'] == 0:
user['cash'] -= activities[act_id]['unlock']
activities[act_id]['usr_progress'] = 1
So the question is: in this situation should i keep a copy of the data as dict, and create a class with it plus the methods i need or use functions to edit the data inside the dict?
I am trying to learn how to use multiprocessing and have managed to get the code below to work. The goal is to work through every combination of the variables within the CostlyFunction by setting n equal to some number (right now it is 100 so the first 100 combinations are tested). I was hoping I could manipulate w as each process returned its list (CostlyFunction returns a list of 7 values) and only keep the results in a given range. Right now, w holds all 100 lists and then lets me manipulate those lists but, when I use n=10MM, w becomes huge and costly to hold. Is there a way to evaluate CostlyFunction's output as the workers return values and then 'throw out' values I don't need?
if __name__ == "__main__":
import csv
csvFile = open('C:\\Users\\bryan.j.weiner\\Desktop\\test.csv', 'w', newline='')
#width = -36000000/1000
#fronteir = [None]*1000
currtime = time()
n=100
po = Pool()
res = po.map_async(CostlyFunction,((i,) for i in range(n)))
w = res.get()
spamwriter = csv.writer(csvFile, delimiter=',')
spamwriter.writerows(w)
print(('2: parallel: time elapsed:', time() - currtime))
csvFile.close()
Unfortunately, Pool doesn't have a 'filter' method; otherwise, you might've been able to prune your results before they're returned. Pool.imap is probably the best solution you'll find for dealing with your memory issue: it returns an iterator over the results from CostlyFunction.
For sorting through the results, I made a simple list-based class called TopList that stores a fixed number of items. All of its items are the highest-ranked according to a key function.
from collections import Userlist
def keyfunc(a):
return a[5] # This would be the sixth item in a result from CostlyFunction
class TopList(UserList):
def __init__(self, key, *args, cap=10): # cap is the largest number of results
super().__init__(*args) # you want to store
self.cap = cap
self.key = key
def add(self, item):
self.data.append(item)
self.data.sort(key=self.key, reverse=True)
self.data.pop()
Here's how your code might look:
if __name__ == "__main__":
import csv
csvFile = open('C:\\Users\\bryan.j.weiner\\Desktop\\test.csv', 'w', newline='')
n = 100
currtime = time()
po = Pool()
best = TopList(keyfunc)
result_iter = po.imap(CostlyFunction, ((i,) for i in range(n)))
for result in result_iter:
best.add(result)
spamwriter = csv.writer(csvFile, delimiter=',')
spamwriter.writerows(w)
print(('2: parallel: time elapsed:', time() - currtime))
csvFile.close()