Why conversion works on the idle but not executing the program? - scapy

I'm playing with scapy library in python3 (https://github.com/phaethon/scapy) and sniffing the packets on the net, I get these format of packets, here are some examples:
b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x00b\xe7\xb1#\x00#\x06\xf2j\xc0\xa8\x01h\xd8:\xc6.\xd2B\x01\xbb\xa1\x9a\xbf\x1e\x0b"q\xc1\x80\x18\x00\xfe^\x95\x00\x00\x01\x01\x08\n\x00CX6)\x818P\x17\x03\x03\x00)\x00\x00\x00\x00\x00\x00\x00\x0f\x8d\xdc\x86`:\xe9\x8f=\xec\x89u~\xdf\x7f\x1aN\x85\xf4ESt\xa0oT\xdba\xfan\xc7\xb8\x01\xf8\t'
or
b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x01\x03\xe3\x9e#\x00#\x06\xda\xbe\xc0\xa8\x01h\xc0|\xf9\n\xcc6\x01\xbbDo\x9a\xe4N\xaf\x11~\x80\x18\x05\xa4\xc6x\x00\x00\x01\x01\x08\n\x00CY\xed`\xa8Vb\x17\x03\x03\x00\xca\x00\x00\x00\x00\x00\x00\x004;wn,\x13\x8d9\x17G\xee/\x1c\x15\x12,\xc0\xde\xf4\xab\xa5emvZ\xf5\x1f\x94\x8a\xcc4g>#\xb9\xf2dYs\xeb<l;\x8b\x8a\xccT\n\xacUT\x049\xcbp\x1c#\x87z\\[8\xde\xd7I\x14\xfd\x94\x81\x13x`\x9eH\xd8\xbc)lc\x0fE\xfc\x99F\xb0c\xb9v\x86\x1c\xc4\xec\x14\xb6\x00\x1e\x8f\xd5\x08H/\xa4\xea\xf4~2\x90\xc7\xabJ\xb9\x9f\xf9\x06\xc6\xe5$\xe4(\xab#\xd8\xf4<\x7f\xfa\xe6Y_\x80\x14\xb2H\x10\x1cT\xdf\x1f\x0f\xbb\xce\xf8\xbe q\xf3=qQ\xd2a!e\x8a\xbf9Y\x7f\xb8\xe5Bo\xd8\r\xcb\xcf\x01H\xf8x\x90\x98\xfc>$\xef>\x80;\xa9\xf9\x7f\x9f\xc5?V\xecH:\x7f\xfeb\xa5E\xab'
The weird thing, is that if I try to convert them to idle3, typing:
> from scapy.all import *
> text = b'blablablabytesstring'
> text2 = Ether(text)
> text2
And it works, it correctly converts.
But if I do the same thing, but writing a simple script, like:
from scapy.all import *
text = b'sameblablablabytesstringasidle3'
text2 = Ether(text)
print (text2)
It doesn't converts, and it prints the same bytes string.
Why? How can I solve this?
--UPDATE--
This is the output of text2.show()
###[ Ethernet ]###
dst = e8:74:e6:2c:f1:3a
src = 30:10:b3:18:8b:fd
type = 0x800
###[ IP ]###
version = 4
ihl = 5
tos = 0x0
len = 259
id = 24485
flags = DF
frag = 0
ttl = 64
proto = tcp
chksum = 0x5eb8
src = 192.168.1.104
dst = 192.124.249.10
\options \
###[ TCP ]###
sport = 51172
dport = https
seq = 3204149667
ack = 3154139287
dataofs = 8
reserved = 0
flags = PA
window = 1444
chksum = 0xb3ea
urgptr = 0
options = [('NOP', None), ('NOP', None), ('Timestamp', (4182786, 1620720294))]
###[ Raw ]###
load = b'\x17\x03\x03\x00\xca\x00\x00\x00\x00\x00\x00\x01\xba\xc1\xa0H\xa2\xb3\t\x02:P>\x8b\xeb\xaa\xbd\x83H\x05\x1f(\x02\x80\x9e\x96\xa5\xd9\xf7\xf4\x07&s\xfd6\xb9 \x00\x8c\xcf\xd7\xe4\x04pQ\x992"$\x07R\x02\xb4\x97\xad!\xafB\xc8I\xa6\xe6\x18\xc7x\x16\x9a$c\xc4\x05}\xffl\xe7\x89\x93\xa4\x9a\xf1\x90\x8d\xde"\xe0\xb2\x06\x14(Yk\xf441\xbf\xfa?\xd4puE\x97\xcd\xb1\xc2\xfaaa\x14{\xd4f\x80\xb9\x81keO\x90\x13\x9c\x01\xaa\xe4}\xab{\xd8\x9a\xb3\xe3\x88\xcd:5\xe1\xa9\xd6:\x06l_\xed\xb3\x96=\xcc\x0f?c\xf7\xad\xed\xde\xca||\xd6\xbc\xa4\x99u\xff\xbcI\xe2\x8cQ\xe3\x87\x9f\x18(\rp\x04u\xad\r\xa55\xbe`q\x1b=`5\'\xbc\xf2\x0b\xb9V\x1ecp'
--UPDATE2--
(from idle3)
>>> from scapy.all import *
WARNING: No route found for IPv6 destination :: (no default route?). This affects only IPv6
>>> text = b"0\x10\xb3\x18\x8b\xfd\xe8t\xe6,\xf1:\x08\x00E\xa0\x004a'#\x004\x06\x90\xa2\x173{\x1b\xc0\xa8\x01d\x00P\xec\xe6\x01\x81%|\xfcY#8\x80\x10\x03\xd4(\xbe\x00\x00\x01\x01\x08\nx\x93UR\x00\x00\xd7$"
>>> text2 = Ether(text)
>>> text2
<Ether dst=e8:74:e6:2c:f1:3a src=30:10:b3:18:8b:fd type=0x800 |<IP version=4 ihl=5 tos=0x0 len=98 id=59313 flags=DF frag=0 ttl=64 proto=tcp chksum=0xf26a src=192.168.1.104 dst=216.58.198.46 options=[] |<TCP sport=53826 dport=https seq=2711273246 ack=186806721 dataofs=8 reserved=0 flags=PA window=254 chksum=0x5e95 urgptr=0 options=[('NOP', None), ('NOP', None), ('Timestamp', (4413494, 696334416))] |<Raw load=b'\x17\x03\x03\x00)\x00\x00\x00\x00\x00\x00\x00\x0f\x8d\xdc\x86`:\xe9\x8f=\xec\x89u~\xdf\x7f\x1aN\x85\xf4ESt\xa0oT\xdba\xfan\xc7\xb8\x01\xf8\t' |>>>>
--UPDATE3-- (print from program.py)
Ether / IP / TCP 192.168.1.104:53826 > 216.58.198.46:https PA / Raw

Can you try text2.show() instead of print(text2) and see how that goes?
from scapy.all import *
text = b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x00b\xe7\xb1#\x00#\x06\xf2j\xc0\xa8\x01h\xd8:\xc6.\xd2B\x01\xbb\xa1\x9a\xbf\x1e\x0b"q\xc1\x80\x18\x00\xfe^\x95\x00\x00\x01\x01\x08\n\x00CX6)\x818P\x17\x03\x03\x00)\x00\x00\x00\x00\x00\x00\x00\x0f\x8d\xdc\x86`:\xe9\x8f=\xec\x89u~\xdf\x7f\x1aN\x85\xf4ESt\xa0oT\xdba\xfan\xc7\xb8\x01\xf8\t'
text2 = Ether(text)
text2.show()

Related

How do I convert a CAN message to the bytearray format needed by UDP sendto()? [duplicate]

I made UDP socket connection between two Linux machines and can send for example b"Hello, World!" easily. But now I need to send the below CAN frame
from can import Message
send_msg = Message(data=[1, 2, 3, 4, 5])
So if I print send_msg it shows:
Timestamp: 0.000000 ID: 00000000 X DLC: 5 01 02 03 04 05
I want to get this printed on the receiving end. The sending and receiving end codes I am using are below:
Sending:
import socket
UDP_IP = "10.140.189.249"
UDP_PORT = 5005
from can import Message
send_msg = Message(data=[1, 2, 3, 4, 5])
print(send_msg)
MESSAGE = send_msg
print("UDP target IP: %s" % UDP_IP)
print("UDP target port: %s" % UDP_PORT)
print("message: %s" % MESSAGE)
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))
Here I know MESSAGE = send_msg is wrong.
Receiving
import socket
UDP_IP = "0.0.0.0"
UDP_PORT = 5005
sock = socket.socket(socket.AF_INET, # Internet
socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))
while True:
rec_msg, addr = sock.recvfrom(1024) # buffer size is 1024 bytes
print("received message: %s" % rec_msg)
Please advise
As the physical layer of an UDP connection and a CAN connection are substantially different, you cannot send the CAN frames over UDP. What is of course possible is to send the payload of the CAN frame and assemble the CAN message on the receiving side: I.e. on the sending side:
sock.sendto(b“12345“, (UDP_IP, UDP_PORT))
And on the receiving side:
msg = Message(data=bytearray(recv_msg))
Most likely you do not only want to transfer the data of the CAN frame, but also the ids and other fields.
Another possibility would be to pickle the Message object on the sending side and unpickle it on the receiving side using pickle.dumps and pickle.loads
All features of the CAN bus like arbitration, error-frames etc. cannot be mimicked on a UDP connection.

Scapy extract TCP window size

I wanted to know how to extract the TCP window size using Scapy
For example to get the IP packet payload I use the following code
def packethandler(pkts):
with codecs.open("Dropcam.csv", mode='w', encoding='utf-8') as csvfile:
writer = csv.writer(csvfile)
for pkts in pkt:
data = (len(pkts))
data2 = (len(pkts.payload))
ether_len.append(data)
IP_len.append(data2)
ethernet_header_min = min(ether_len) # min ethernet frame length
ethernet_header_max = max(ether_len) # max ethernet frame length
````
You can use this syntax:
pkt[TCP].window

removing a particular line from a file in python.

I'm trying to remove the First line from the output file (arista_dataset2), i serached and have seen lot of suggestion but most of the solution didn'y worked for me and i came around with the solution res_data = res_Out.split("\n",1)[1];
which works. Though, i'm looking for an elegant solution if here can be one.
from __future__ import absolute_import, division, print_function
import netmiko
import json
def arista_Connect():
filename = "arista_dataset2"
connection = netmiko.ConnectHandler(ip='192.1.1.1', device_type='arista_eos', username='netuser', password='secrete')
#print(connection.send_command('show ip arp', max_loops=1000, delay_factor=20))
res_Out = connection.send_command('show ip arp',delay_factor=20)
#res_data = res_Out[1:]
res_data = res_Out.split("\n",1)[1];
#print(res_Out)
#return res_Out
filedata = open(filename, mode='w+', encoding='utf-8')
# while writing the entire dataset to a outfile Just eleminate the first line from it
filedata.write(res_data)
filedata.close
connection.disconnect()
#arista_Connect()
if __name__=='__main__':
arista_Connect()
The data into arista_dataset2 File is as follows:
I'm Just removing the line starting with ^Address.
Address Age (min) Hardware Addr Interface
127.1.0.2 0 0010.gh587.b001 Internal1/1
192.102.1.20 0 2c21.099a.9c01 Management1/1
192.102.1.23 0 c4f5.6061.b8dd Management1/1
192.102.1.25 0 c4f5.1b49.66a9 Management1/1
192.102.1.46 0 001c.7c03.1470 Management1/1

SNMP Agent using pysnmp in a multithreaded environment Timeout

I am trying to write an SNMP agent that I can use to monitor my python processes. To do so I have written a class that implements the SNMP agent using pysnmp.
The core part of this agent works (i.e. i can use snmpwalk to interrogate the agent and the returned data is correct). To allow me to update the agent MIB values I have run the dispatcher() within it's own thread. My problem is that I then get timeouts when talking to the agent using snmpwalk (snmpwalk correctly walks the MIB but then times out).
Does anyone have insight into what I'm doing wrong?
The agent code is as follows:
import logging
from pysnmp import debug
from pysnmp.carrier.asyncore.dgram import udp
from pysnmp.entity import engine, config
from pysnmp.entity.rfc3413 import cmdrsp, context
from pysnmp.smi import exval
import threading
formatting = '[%(asctime)s-%(levelname)s]-(%(module)s) %(message)s'
logging.basicConfig(level=logging.DEBUG, format=formatting, )
class SNMPAgent(object):
def _main(self):
logging.debug("Creating SNMP Agent....")
self._snmpEngine = engine.SnmpEngine()
config.addTransport(
self._snmpEngine,
udp.domainName,
udp.UdpTransport().openServerMode((self._agentHost, self._agentPort))
)
config.addV1System(self._snmpEngine, 'my-area', self._communityName)
config.addVacmUser(self._snmpEngine,
2,
'my-area',
'noAuthNoPriv',
(1, 3, 6),
(1, 3, 6))
snmpContext = context.SnmpContext(self._snmpEngine)
mibBuilder = snmpContext.getMibInstrum().getMibBuilder()
mibBuilder.loadModules('HOST-RESOURCES-MIB')
self._mibInstrum = snmpContext.getMibInstrum()
self._hostRunTable, = mibBuilder.importSymbols('HOST-RESOURCES-MIB', 'hrSWRunEntry')
self._instanceId = self._hostRunTable.getInstIdFromIndices(1)
# The following shows the OID name mapping
#
# hrSWRunTable 1.3.6.1.2.1.25.4.2 <TABLE>
# hrSWRunEntry 1.3.6.1.2.1.25.4.2.1 <SEQUENCE>
# hrSWRunIndex 1.3.6.1.2.1.25.4.2.1.1 <Integer32>
# hrSWRunName 1.3.6.1.2.1.25.4.2.1.2 <InternationalDisplayString> 64 Char
# hrSWRunID 1.3.6.1.2.1.25.4.2.1.3 <ProductID>
# hrSWRunPath 1.3.6.1.2.1.25.4.2.1.4 <InternationalDisplayString> 128 octets
# hrSWRunParameters 1.3.6.1.2.1.25.4.2.1.5 <InternationalDisplayString> 128 octets
# hrSWRunType 1.3.6.1.2.1.25.4.2.1.6 <INTEGER>
# hrSWRunStatus 1.3.6.1.2.1.25.4.2.1.7 <INTEGER> <<===== This is the key variable used by Opennms
self._setVars()
cmdrsp.GetCommandResponder(self._snmpEngine, snmpContext)
cmdrsp.SetCommandResponder(self._snmpEngine, snmpContext)
cmdrsp.NextCommandResponder(self._snmpEngine, snmpContext)
cmdrsp.BulkCommandResponder(self._snmpEngine, snmpContext)
def runAgent(self):
'''
Run the Agent
'''
t = threading.Thread(target=self._runAgentFunc, args = ())
t.daemon = True
t.start()
def _runAgentFunc(self):
try:
self._snmpEngine.transportDispatcher.jobStarted(1)
self._snmpEngine.transportDispatcher.runDispatcher()
except:
self._snmpEngine.transportDispatcher.closeDispatcher()
raise
def updateAgentStatus(self, runStatus, text1, text2):
self._mibDict = {"hrSWRunIndex" : 1,
"hrSWRunName" : self._name,
"hrSWRunID" : self._enterpriseMIB,
"hrSWRunPath" : text1[:128] if text1 is not None else '',
"hrSWRunParameters" : text2[:128] if text1 is not None else '',
"hrSWRunType" : 4,
"hrSWRunStatus" : 1
}
self._setVars()
def _setVars(self):
varBinds = self._mibInstrum.writeVars((
(self._hostRunTable.name + (1,) + self._instanceId, self._mibDict["hrSWRunIndex"]),
(self._hostRunTable.name + (2,) + self._instanceId, self._mibDict["hrSWRunName"]), # <=== Must match OpenNMS service-name variable
(self._hostRunTable.name + (3,) + self._instanceId, self._mibDict["hrSWRunID" ]), #
(self._hostRunTable.name + (4,) + self._instanceId, self._mibDict["hrSWRunPath"]),
(self._hostRunTable.name + (5,) + self._instanceId, self._mibDict["hrSWRunParameters"]),
(self._hostRunTable.name + (6,) + self._instanceId, self._mibDict["hrSWRunType"]), # Values are ==> unknown(1), operatingSystem(2), deviceDriver(3), application(4)
(self._hostRunTable.name + (7,) + self._instanceId, self._mibDict["hrSWRunStatus"]) #<<=== This is the status number OpenNMS looks at Values are ==> running(1), runnable(2), notRunnable(3), invalid(4)
))
def __init__(self, name, host, port, community, text1='Service up', text2=''):
'''
#=======================================================================
# Constructor
# name -- the (process) name the agent should publish (must match
# the openNMS name
# host -- the host name or ip the agent will run on
# port -- the port the snmp agent will listen on
# community -- the community name the agent will use (usually 'public')
# text1 -- the first status text string published (128 char max)
# text2 -- the second status text string published (128 char max)
#=======================================================================
'''
self._name = name
self._agentHost = host
self._agentPort = port
self._communityName = community
self._enterpriseMIB = (1, 3, 6, 1, 4, 1, 50000, 0) # Made up for now
self._mibInstrum = None
self._snmpEngine = None
self._dataChanged = False
self._mibDict = {"hrSWRunIndex" : 1,
"hrSWRunName" : self._name,
"hrSWRunID" : self._enterpriseMIB,
"hrSWRunPath" : text1[:128] if text1 is not None else '',
"hrSWRunParameters" : text2[:128] if text1 is not None else '',
"hrSWRunType" : 4,
"hrSWRunStatus" : 1
}
self._main()
and I call this code as follows (this is just testing that I can change the status):
from SNMPAgent import SNMPAgent
from time import sleep
agent = SNMPAgent("test", "127.0.0.1", 12345, "public", "This is my test message 1", "This is my test message 2")
agent.runAgent()
sleep(10) # Wait for it to start
while True:
agent.updateAgentStatus(3, "Oh no", "Something is wrong!")
sleep(30)
agent.updateAgentStatus(2, "Whew", "Everything is fixed")
sleep(30)
To walk the agent MIB i use:
snmpwalk -v 2c -c public -n my-context 127.0.0.1:12345 1.3.6.1.2.1.25.4.2
This shows the data updating but at the end of walking the MIB the agent times out:
HOST-RESOURCES-MIB::hrSWRunIndex.1 = INTEGER: 1
HOST-RESOURCES-MIB::hrSWRunName.1 = STRING: "test"
HOST-RESOURCES-MIB::hrSWRunID.1 = OID: SNMPv2-SMI::enterprises.50000.0
HOST-RESOURCES-MIB::hrSWRunPath.1 = STRING: "Whew"
HOST-RESOURCES-MIB::hrSWRunParameters.1 = STRING: "Everything is fixed"
HOST-RESOURCES-MIB::hrSWRunType.1 = INTEGER: application(4)
HOST-RESOURCES-MIB::hrSWRunStatus.1 = INTEGER: running(1)
Timeout: No Response from 127.0.0.1:12345
Enabling pysnmp debugging reveals serialization error caused by unset integer value for specific OID:
[2017-01-13 02:05:18,387-DEBUG]-(debug) generateResponseMsg: Message:
version='version-2c'
community=public
data=PDUs:
response=ResponsePDU:
request-id=1950819527
error-status='noError'
error-index=0
variable-bindings=VarBindList:
VarBind:
name=1.3.6.1.2.1.25.5.1.1.1.1
=_BindValue:
value=ObjectSyntax:
simple=SimpleSyntax:
integer-value=<no value>
2017-01-13 02:05:18,387 pysnmp: generateResponseMsg: serialization failure: Uninitialized ASN.1 value ("__eq__" attribute looked up)
[2017-01-13 02:05:18,387-DEBUG]-(debug) generateResponseMsg: serialization failure: Uninitialized ASN.1 value ("__eq__" attribute looked up)
2017-01-13 02:05:18,388 pysnmp: StatusInformation: {'errorIndication': <pysnmp.proto.errind.SerializationError object at 0x10162f828>}
[2017-01-13 02:05:18,388-DEBUG]-(debug) StatusInformation: {'errorIndication': <pysnmp.proto.errind.SerializationError object at 0x10162f828>}
You should probably make sure that you set values for all non-defaulted SNMP table columns.
As a side note, you seem to manage your MIB from main and SNMP agent threads without explicit synchronization. That may cause race conditions...

How to verify if a packet in Scapy has a TCP layer

I want to know how can I verify that a packet I received from the sr1() function in Scapy contains a TCP layer, in order to do some treatment on the TCP flags.
You have two options, the in operator is one.
>>> TCP in pkt
True
>>> if TCP in pkt:
... # Handle TCP Flags
Packet objects in Scapy also have a function called haslayer().
>>> pkt = IP()/TCP()
>>> pkt.haslayer(TCP)
1
>>> pkt2 = IP()/UDP()
>>> pkt2.haslayer(TCP)
0
>>> Packet.haslayer.__doc__
'true if self has a layer that is an instance of cls. Superseded by "cls in self" syntax.'
The accepted answer may not give the wanted result in cases of encapsulation. Normally you should check the protocol at particular place using the method layers():
>>> pkt1 = IP()/TCP()
>>> pkt1.layers()
[scapy.layers.inet.IP, scapy.layers.inet.TCP]
>>> pkt1.layers()[1] == TCP
True
>>> pkt2 = IP()/UDP()
>>> pkt2.layers()[1] == TCP
False
Demonstration how the answer by RyPeck may give unwanted result:
>>> pkt3 = IP()/IP()/TCP()
>>> pkt3.layers()[1] == TCP
False
>>> pkt3.haslayer(TCP)
True

Resources