I want to create a sniffer script for monitoring windows and linux. For linux, it is easy, but for windows it is not. I met an error as follows:
Traceback (most recent call last):
File "test_scapy.py", line 45, in <module>
main()
File "test_scapy.py", line 38, in main
sniff(filter="tcp port 80", prn=packet_callback, count=10)
File "C:\Python27\lib\site-packages\scapy\sendrecv.py", line 575, in sniff
sel = select([s],[],[],remain)
select.error: (10038, '')
My code is
from scapy.all import *
def packet_callback(packet):
print packet.show()
sniff(filter="tcp port 80", prn=packet_callback, count=10)
I have read the post Scapy sniff() "an operation was performed on something that is not a socket"
and read the select document https://docs.python.org/2/library/select.html
"File objects on Windows are not acceptable, but sockets are. On Windows, the underlying select() function is provided by the WinSock library, and does not handle file descriptors that don’t originate from WinSock."
I think that means I have to re-define the socket for sniffing. So I upgrade the code
# create a raw socket and bind it to the public interface
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
s.bind((HOST, 0))
# Include IP headers
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
# receive all packages
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_ON)
sniff(filter="tcp port 80", prn=packet_callback, count=10, opened_socket=s)
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
That is ok now and the error disappear. But I have to parse the TCP RAW data manually.
Now the question is: is there any easy way for me to sniffer for
windows platform like using linux scapy layer wrapper, like call package[TCP].dst
Any suggestion is appreciated.
Which version of Scapy are you using? You should probably update to the latest (2.3.3) version or event better, to the current development version from GitHub.
Related
I am trying to determine vendor + version (using python NAPALM and parallel-ssh) of network switches (Huawei VRP5/8, Cisco Catalyst and Cisco SMB (SF/SG):
admin#server:~$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from napalm import get_network_driver
>>> driver = get_network_driver('ios')
>>> device = driver('ip', 'username', 'password')
>>> device.open()
>>> print(device.get_facts())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/altepro/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 811, in get_facts
show_ver = self._send_command('show version')
File "/home/altepro/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 165, in _send_command
output = self.device.send_command(command)
File "/home/altepro/.local/lib/python3.8/site-packages/netmiko/utilities.py", line 600, in wrapper_decorator
return func(self, *args, **kwargs)
File "/home/altepro/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1694, in send_command
raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:
Pattern not detected: '\x1b\\[Ksg300\\-ab\\-1\\#' in output.
Things you might try to fix this:
1. Explicitly set your pattern using the expect_string argument.
2. Increase the read_timeout to a larger value.
Where sg300-ab-1 is sysname of the switch (Cisco SMB - sg300 in this case, but i have tested this on several versions and types of the SMB lineup)
Things that i have tried:
Tried several version of netmiko, napalm (And its drivers including ios-350) and parallel-ssh. Tried several fresh linux servers with fresh install of napalm and parallel-ssh.
SSH is tested using the same server and credentials and it works without any problems.
When i use parallel-ssh the device doesnt even raise exception or timeout - it just goes stuck in the command:
output = client.run_command(cmd)
hosts = ['192.168.1.50']
client = ParallelSSHClient(hosts, user='my_user', password='my_pass')
cmd = 'show version'
output = client.run_command(cmd)
for host_out in output:
for line in host_out.stdout:
print(line)
Thanks for any kind of help !
It looks like the prompt isn't getting recognized properly. I'm not very familiar with either ParallelSSHClient or napalm, but I have worked with netmiko and that looks like where the error is. Here's some steps that can possible get you closer to figuring out what's happening. I suspect it's the prompt not being read correctly from the device.
Set up debugging and a netmiko session and run a simple command
import logging
import netmiko
logging.basicConfig(level=logging.DEBUG)
session = netmiko.ConnectHandler(
host='192.168.1.50',
username='my_user',
password='my_pass',
device_type='cisco_ios')
results = session.send_command('show version')
If this fails with the same error, then it's the prompt (possibly the \x1b escape character). Try again but with a simpler expect_string, like what's expected at the end of the prompt:
session.send_command('show version', expect_string="#")
If this gets you a result, then it's something about the how the prompt is being set for this device.
To see what's being found for the prompt:
session.find_prompt()
Edit:
Based on what you're reporting, the issue seems to be with the control code \x1b\[ being included in the prompt. It's possible this can be disabled on the device itself, but I'm unfamiliar with that platform. The napalm API doesn't expose netmiko's send_command method. It should still be fixable. This solution would be a hack to make things work, nothing that I'd recommend relying on.
Establish a class that will act as your fix. This will be instantiated with the netmiko session (device.device) and will be used to replace the send_command method.
class HackyFix:
def __init__(self, session):
self.session = session
self.original_send_command = session.send_command
def send_command(self, command):
original_prompt = self.session.find_prompt()
fixed_prompt = original_prompt.replace(r"\x1b[", "")
print(
f"send_command intercepted. {original_prompt} replaced with {fixed_prompt}"
)
return self.original_send_command(command, expect_string=fixed_prompt)
Then in your existing napalm code, add this right after device.open():
hackyfix = HackyFix(device.device)
device.device.send_command = hackyfix.send_command
Now all of napalm's calls to send_command will go through your custom fix that will find the prompt and modify it before passing it to expect_string.
Last edit.
It's an ANSI Escape Code that's being thrown in by the SG300. Specifically it's the one that clears from cursor to end of line. It's also a known issue with the SG300. The good news is that someone made a napalm driver to support it. One big difference between the SG300 driver and the IOS driver is the netmiko device_type is cisco_s300. When this device_type is used, strip_ansi_escape_codes is ran against the output.
Behavior of that escape code tested in bash:
$ printf "This gets cleared\r"; code="\x1b[K"; printf "${code}This is what you see\n"
This is what you see
You can validate that setting cisco_s300 as the device_type fixes the issue:
session = netmiko.ConnectHandler(
host='192.168.1.50',
username='my_user',
password='my_pass',
device_type='cisco_s300')
results = session.send_command('show version')
This should give a result with no modification to the expect_string value. If that works and you're looking to get results sooner or later, the following is a better fix than the hacky fix above.
from napalm.ios import IOSDriver
class QuickCiscoSG300Driver(IOSDriver):
def __init__(self, hostname, username, password, timeout=60, optional_args=None):
super().__init__(hostname, username, password, timeout, optional_args)
def open(self):
device_type = "cisco_s300"
self.device = self._netmiko_open(
device_type, netmiko_optional_args=self.netmiko_optional_args
)
device = QuickCiscoSG300Driver("192.168.1.50", "my_user", "my_pass")
device.open()
device.get_facts()
Or you can get the driver (better option, unless this happens to be the driver you already tried)
The program should allow to run several https get requests with one aiohttp.ClientSession as the documentation suggests. It is intended to run a telegram bot.
I was not able to catch the exception with try ... except. Therefore the program hangs when exiting. During extended sessions the error is printed in the command windows (but not in the error log).
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000016A581E4400>
transport: <_SelectorSocketTransport fd=644 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 207, in feed_ssldata
self._sslobj.unwrap()
File "C:\Users\annet\Anaconda3\lib\ssl.py", line 767, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2592)
^C
As the error information is very unspecific I could not really isolate the source and have a short code to reproduce the error.
A sample code is on github under https://github.com/fhag/telegram2.git
In order to run the code you will need an API token from telegram of your own bot.
This error showed up the first time when I upgraded to python 3.7.1.
Python is running on Windows 10.
Alright I'm at the end of my tether trying to get phantomJS to work with selenium in an openshift environment. I've downloaded the phantomjs binary using ssh and can even run it in the shell. But when it comes to starting a webdriver service using selenium I keep getting this traceback error no matter the args I put in.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/phantomjs/webdriver.py", line 50, in __init__
service_args=service_args, log_path=service_log_path)
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/phantomjs/service.py", line 50, in __init__
service.Service.__init__(self, executable_path, port=port, log_file=open(log_path, 'w'))
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/common/service.py", line 33, in __init__
self.port = utils.free_port()
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/common/utils.py", line 36, in free_port
free_socket.bind(('0.0.0.0', 0))
PermissionError: [Errno 13] Permission denied
Not sure what's going on, am I supposed to bind to an IP address? If so I tried using service args but that hasn't helped.
I came across the same issues trying to run phantomJS on my Openshift-hosted Django application, running on a Python 3 gear. Finally I managed to make it work, this is how:
The main issue to overcome is that Openshift does not allow applications to bind on localhost (nor '0.0.0.0' nor '127.0.0.1). So the point is to bind to the actual IP address of your Openshift Gear instead
You have to deal with this issue at the ghostdriver level as well as within the Python-selenium binding.
ghostdriver (phantomJS binary)
Unfortunately, as explained brilliantly by Paolo Bernardi in this post : http://www.bernardi.cloud/2015/02/25/phantomjs-with-ghostdriver-on-openshift/ you have to use a patched version of phantomjs for this, as the released version doesn't allow to bind to a specified IP. The binary linked by Paolo did not work on my Python3 cardridge, yet this one worked perfectly: https://github.com/jrestful/server/blob/master/seo/phantomjs-1.9.8-patched.tar.gz?raw=true (see question Trying to run PhantomJS on OpenShift: cannot patch GhostDriver so that it can bind on the server IP address for details)
Upload this phantomjs binary to app-root/data/phantomjs/bin (for example) and make sure it is runnable :
> chmod 711 app-root/data/phantomjs/bin/phantomjs
You can now check that you can bin to your IP like this (I chose port number 15002 for my app, I reckon you can pick any value you want above 15000):
> echo $OPENSHIFT_PYTHON_IP
127.13.XXX.XXX
> app-root/data/phantomjs/bin/phantomjs --webdriver=127.13.XXX.XXX:15002
PhantomJS is launching GhostDriver...
[INFO - 2017-03-24T13:16:36.031Z] GhostDriver - Main - running on port 127.13.XXX.XXX:15002
Ok, now kill this process and proceed to step 2 : python webdriver
Custom python-selenium webdriver for PhantomJS
The point is to add the IP address to bind to as a parameter of the PhantomJS Webdriver.
First, I defined new settings to adapt to Openshift's constraint in my settings.py
PHANTOMJS_BIN_PATH = os.path.join(os.getenv('OPENSHIFT_DATA_DIR'), 'phantomjs', 'bin', 'phantomjs')
PHANTOMJS_LOG_PATH = os.path.join(os.getenv('OPENSHIFT_LOG_DIR'), 'ghostdriver.log')
(make sure app-root/logs/ is writable, maybe you'll have to chmod it)
Then, I had to override the PhantomJS Webdriver class, to provide the ip address as an argument. Here is my own implementation:
from selenium.webdriver import phantomjs
from selenium.webdriver.common import utils
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
class MyPhantomJSService(phantomjs.service.Service):
def __init__(self, executable_path, port=0, service_args=None, log_path=None, ip=None):
if ip is None:
self.ip = '0.0.0.0'
else:
self.ip = ip
phantomjs.service.Service.__init__(self, executable_path, port, service_args, log_path)
def command_line_args(self):
return self.service_args + ["--webdriver=%s:%d" % (self.ip, self.port)]
def is_connectable(self):
return utils.is_connectable(self.port, host=self.ip)
#property
def service_url(self):
"""
Gets the url of the GhostDriver Service
"""
return "http://%s:%d/wd/hub" % (self.ip, self.port)
class MyPhantomWebDriver(RemoteWebDriver):
"""
Wrapper to communicate with PhantomJS through Ghostdriver.
You will need to follow all the directions here:
https://github.com/detro/ghostdriver
"""
def __init__(self, executable_path="phantomjs",
ip=None, port=0, desired_capabilities=DesiredCapabilities.PHANTOMJS,
service_args=None, service_log_path=None):
"""
Creates a new instance of the PhantomJS / Ghostdriver.
Starts the service and then creates new instance of the driver.
:Args:
- executable_path - path to the executable. If the default is used it assumes the executable is in the $PATH
- ip - IP sur lequel on veut se binder : c'est la spécificité de ce monkeypatch
- port - port you would like the service to run, if left as 0, a free port will be found.
- desired_capabilities: Dictionary object with non-browser specific
capabilities only, such as "proxy" or "loggingPref".
- service_args : A List of command line arguments to pass to PhantomJS
- service_log_path: Path for phantomjs service to log to.
"""
self.service = MyPhantomJSService(
executable_path,
port=port,
service_args=service_args,
log_path=service_log_path,
ip=ip)
self.service.start()
try:
RemoteWebDriver.__init__(
self,
command_executor=self.service.service_url,
desired_capabilities=desired_capabilities)
except Exception:
self.quit()
raise
self._is_remote = False
def quit(self):
"""
Closes the browser and shuts down the PhantomJS executable
that is started when starting the PhantomJS
"""
try:
RemoteWebDriver.quit(self)
except Exception:
# We don't care about the message because something probably has gone wrong
pass
finally:
self.service.stop()
Finally, invoke this custom webdriver instead of webdriver.PhantomJS(..., like this:
from .myphantomjs import MyPhantomWebDriver
browser = MyPhantomWebDriver(executable_path=settings.PHANTOMJS_BIN_PATH, service_log_path=settings.PHANTOMJS_LOG_PATH, ip=os.getenv('OPENSHIFT_PYTHON_IP'), port=15002)
From then on, you can use the browser object normally
I am trying to implement websockets on a openshift.com server (which should support them).
openshift.com provides me a WSGI, so I embed my cherrypy to it, so that my wsgi.pyscript define an application object.
Also, cherrypy has a websocket tool, as defined by ws4py.
This is a minimal cherrypy application that works under WSGI in OpenShift, and that should use websockets too!
import cherrypy
from ws4py.server.cherrypyserver import WebSocketPlugin, WebSocketTool
from ws4py.websocket import EchoWebSocket
import atexit
import logging
# see http://tools.cherrypy.org/wiki/ModWSGI
cherrypy.config.update({'environment': 'embedded'})
if cherrypy.__version__.startswith('3.0') and cherrypy.engine.state == 0:
cherrypy.engine.start(blocking=False)
atexit.register(cherrypy.engine.stop)
class Root(object):
def index(self): return 'I work!'
def ws(self): print('THIS IS NEVER PRINTED :(')
index.exposed=True
ws.exposed=True
# registering the websocket
conf={'/ws':{'tools.websocket.on': True,'tools.websocket.handler_cls': EchoWebSocket}}
WebSocketPlugin(cherrypy.engine).subscribe()
cherrypy.tools.websocket = WebSocketTool()
#show stacktraces in console (for some reason this is not default in cherrypy+WSGI)
logger = logging.getLogger()
logger.setLevel(logging.INFO)
stream = logging.StreamHandler()
stream.setLevel(logging.INFO)
logger.addHandler(stream)
application = cherrypy.Application(Root(), script_name='', config=conf)
Everything work wonderfully, except when I create a websocket ( connecting to ws://myserver:8000/ws ), this is the stacktrace I get:
cherrypy/_cplogging.py, 214, HTTP Traceback (most recent call last):
File "cherrypy/_cprequest.py", line 661, in respond
self.hooks.run('before_request_body')
File "cherrypy/_cprequest.py", line 114, in run
raise exc
File "cherrypy/_cprequest.py", line 104, in run
hook()
File "cherrypy/_cprequest.py", line 63, in __call__
return self.callback(**self.kwargs)
File "ws4py/server/cherrypyserver.py", line 200, in upgrade
ws_conn = get_connection(request.rfile.rfile)
AttributeError: 'mod_wsgi.Input' object has no attribute 'rfile'
(I manually deleted the absolute path from the filenames)
PS: I use python3.3, cherrypy==3.5.0, ws4py==0.3.4.
It is not clear to me:
if this is a lack of compatibility between cherrypy and ws4py when in an WSGI environment.
if it is a problem of ws4py when in a WSGI environment
if it is because Openshift websockets have a different port than the http one
PPS: this is a complete OpenShift project, that you can run and try this yourself: https://github.com/spocchio/wsgi-cherrypy-ws4py
I don't think it is possible at all. WSGI is a synchronous protocol (1, 2), WebSocket protocol is asynchronous. Wiki states that for a Python application interface OpenShift uses WSGI (3). Alas.
However I've recently played with ws4py in pub/sub scenario and it works really well on top of CherryPy standard HTTP-server deployment. So it shouldn't be a problem on a generic virtual server with no application interface constraints.
I'm making a secure SSL connection to a server using python and M2Crypto. See code below.
from M2Crypto import SSL, m2,x509
from M2Crypto.m2xmlrpclib import Server, SSL_Tranport
ctx = SSL.Context()
m2.ssl_ctx_use_pkey_privkey(ctx.ctx,myKey.pkey)
m2.ssl_ctx_use_x509(ctx.ctx,myCert.x509)
server = Server(serverUrl, SSL_Transport(ctx))
server.ping()
The above works fine. If I try to change the default socket timeout by adding the following two lines at the beginning of the code, I get a protocol error.
import socket
socket.setdefaulttimeout(40)
This is the error I receive:
File "/usr/local/lib/python2.4/xmlrpclib.py", line 1096, in call
return self.__send(self.__name, args)
File "/usr/local/lib/python2.4/xmlrpclib.py", line 1383, in __request
verbose=self.__verbose
File "/usr/local/lib/python2.4/site-packages/M2Crypto/m2xmlrpclib.py", line 68, in request
headers
xmlrpclib.ProtocolError:
Why is the default socket timeout causing problems?
There is a patch that can fix this. It is for Linux only. See Bug 2341 --> https://bugzilla.osafoundation.org/show_bug.cgi?id=2341
I have not tried the patch. I will use a different work around. I set the socket timeout to None then run my M2Crypto code then set the socket timeout back to the value I need for the rest of my code.
origTimeout = socket.getdefaulttimeout()
socket.setdefaulttimeout(None)
# run M2Crypto code
socket.setdefaulttimeout(origTimeout)