I'm trying to write a code that basically grabs the network traffic sniffed by wireshark and opens the ips in tabs in selenium.
At first I tried using whois and socket.gethostbyaddr() as All I needed was to translate the ips to domains. But it didn't workout well.
I ran into a problem with permissions because pyshark needs root privilege and selenium doesn't work as root.
here is the code:
import pyshark
from selenium import webdriver
global ips
ips = ''
options = webdriver.ChromeOptions()
options.headless = False
options.add_argument("user-data-dir=/home/afterlife/Documents/chrome_profiles/cyber")
browser = webdriver.Chrome(executable_path='/home/afterlife/Downloads/chromedriver', options=options)
def get_domain_name(ip_address):
for line in ips:
if ip_address not in line:
ips += ip_address+'\n'
browser.execute_script('''window.open(''' + ip_address + ''',"_blank");''')
capture = pyshark.LiveCapture(interface="wlp1s0")
capture.set_debug()
for packet in capture.sniff_continuously():
try:
if hasattr(packet, 'http'):
protocol = packet.transport_layer
source_address = packet.ip.src
source_port = packet[packet.transport_layer].srcport
destination_address = packet.ip.dst
destination_port = packet[packet.transport_layer].dstport
print(destination_address)
get_domain_name(destination_address)
except:
print("exception")
I tried to make pyshark work as non root but failed, I used sudo dpkg-reconfigure wireshark-common but it haven't effected tshark of pyshark.
I also tried using os.setuid() which I successfully used once but forgot/now it doesn't work. I used 0 for root and 1000 for my non-root user.
How would you accomplish what I want?
Thank you!
Related
I am trying to run an XMLRPC server and an XMLRPC client on Mininet hosts, using the script below.
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.node import OVSController
class MyTopo(Topo):
def __init__(self):
# Initialize topology
Topo.__init__(self)
# Add hosts
server1 = self.addHost('server1')
server2 = self.addHost('server2')
# Add switch
s1 = self.addSwitch('s1')
# Add links
self.addLink(server1, s1)
self.addLink(server2, s1)
if __name__ == '__main__':
net = Mininet(topo=MyTopo(), controller=OVSController)
net.start()
print(net.hosts[0].cmd('python3 xmlrpc_server.py'))
print(net.hosts[1].cmd('python3 xmlrpc_client.py'))
The file xmlrpc_server.py is:
from xmlrpc.server import SimpleXMLRPCServer
import threading
def is_even(n):
return n%2 == 0
server = SimpleXMLRPCServer(("0.0.0.0", 8000), logRequests=True, allow_none = True)
server.register_function(is_even, "is_even")
print("Listening on port 8000...")
server_thread = threading.Thread(target=server.serve_forever)
server_thread.start()
The file xmlrpc_client.py is:
import xmlrpc.client
proxy = xmlrpc.client.ServerProxy("http://10.0.0.1:8000/")
print("3 is even: %s" % str(proxy.is_even(3)))
print("100 is even: %s" % str(proxy.is_even(100)))
The problem is that although I have used a thread, when I run the xmlrpc_server.py script on server1, the execution pauses at line server_thread.start() waiting for the script execution to be completed before moving on and thus never goes on to the next line, which means that the XMLRPC client script never runs. How do I overcome this problem?
P.S.: xmlrpc_server.py and xmlrpc_client.py can be executed through the server terminals (by using the commands xterm server1 and xterm server2 on Mininet CLI and then using the commands python3 xmlrpc_server.py and python3 xmlrpc_client.py on the xterm terminals that open), but I need to start the server and client through a python script so as to perform some further calculations after the communication between the two servers.
Replace print(net.hosts[0].cmd('python3 xmlrpc_server.py')) with print(net.hosts[0].sendCmd('python3 xmlrpc_server.py')). Connection is sometimes refused, but that issue can be resolved with exception handling on the client script.
I am making python selenium script to automate some google search with firefox.
I am using python 3.7 on Windows 10 64b.
Something weird happened. When I run my python script, it’is fine.
When I compile it with Nuitka and I run the exe, Firefox is opening with some proxy added (127.0.01:53445).
So I added this line:
profile.set_preference("network.proxy.type", 0)
And again, the script run fine but when I compile it, the exe opens Firefox with a proxy.
It is a pain as this 127.0.01 proxy creates an issue to open google and my program is broken.
Does anyone already faced this weird behaviour of selenium?
Without seeing what code you are using relevant to webdriver I am mostly guessing here. I would suggest rotating the proxy as well.
import requests
from bs4 import BeautifulSoup
from random import choice
firefox_capabilities = webdriver.DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
def proxy_generator():
# REQUIRED - pip install html5lib
response = requests.get("https://sslproxies.org/")
soup = BeautifulSoup(response.content, 'html5lib')
proxy = {'https': choice(list(map(lambda x: x[0] + ':' + x[1], list(zip(map(lambda x: x.text, soup.findAll('td')[::8]), map(lambda x: x.text, soup.findAll('td')[1::8]))))))}
return proxy
PROXY = proxy_generator()# Commented out proxy option "58.216.202.149:8118"
firefox_capabilities['proxy'] = {
"proxyType": "MANUAL",
"httpProxy": PROXY,
"ftpProxy": PROXY,
"sslProxy": PROXY
}
driver = webdriver.Firefox(capabilities=firefox_capabilities)
I am able to execute a command inside the server but I want to get that output in my local pc. I don't want want to use ssh keys. I want to use just the host's username and password as well as vm's(server's) username and password. I couldn't find a direct method to get the shell output of server in my pc, but this method seems half done like generate and save the output in server and then get the file from server. I am finding it difficult to get file(sample.txt) from server to local host(pc).
[EDIT: I am able to do this using via via method(Store the output to server and then get it back into local pc, is there a direct method ?) ]
from pyVim import connect
from config import *
from pyVmomi import vim, vmodl
import ssl
service_instance = connect.SmartConnect(host="yyyyyyy", port=some_number,user="xxx" , pwd=pwd,sslContext=ssl._create_unverified_context())
content = service_instance.RetrieveContent()
vm = searcher.FindByIp(ip="zzzzzz", vmSearch=True)
creds = vim.vm.guest.NamePasswordAuthentication(username='root', password=vmpwd)
pm = service_instance.content.guestOperationsManager.processManager
#checks python version and stores in sample.txt in server
ps = vim.vm.guest.ProcessManager.ProgramSpec(programPath='/usr/bin/python', arguments='--version &> sample.txt')
res = pm.StartProgramInGuest(vm, creds, ps)
print(res) #Prints pid
This does the job but I would appreciate if someone knows how to directly get the output of shell command from server to my local pc. This code makes a file with the output of cmd inside the server and it gets copied into my local pc
from pyVim import connect
from config import *
from pyVmomi import vim, vmodl
import ssl
import os
import requests
service_instance = connect.SmartConnect(host="xxxx", port=aaa,user="yyy" , pwd=pwd,sslContext=ssl._create_unverified_context())
content = service_instance.RetrieveContent()
# # Find a VM
vm = searcher.FindByIp(ip="aaaa", vmSearch=True)
creds = vim.vm.guest.NamePasswordAuthentication(username='root', password=vmpwd)
pm = service_instance.content.guestOperationsManager.processManager
#executes and saves sample.txt into server
ps = vim.vm.guest.ProcessManager.ProgramSpec(programPath='/usr/bin/python', arguments='--version &> sample.txt')
res = pm.StartProgramInGuest(vm, creds, ps)
dest="/Users/username/Desktop/vcenter/sample.txt" #My local pc
src="/root/sample.txt" #Server's directory
fti = content.guestOperationsManager.fileManager.InitiateFileTransferFromGuest(vm, creds, src)
resp=requests.get(fti.url, verify=False)
#Writes into file
with open(dest, 'wb') as f:
f.write(resp.content)
I am trying to connect to a Tor browser but get an error stating "proxyConnectFailure" any ideas I have tried multiple attempts to get into the basics of Tor browser to get it connected but all in vain if any could help life could be saved big time:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Users\Admin\Desktop\Tor Browser\Browser\firefox.exe")
profile = FirefoxProfile(r"C:\Users\Admin\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default")
# Configured profile settings.
proxyIP = "127.0.0.1"
proxyPort = 9150
proxy_settings = {"network.proxy.type":1,
"network.proxy.socks": proxyIP,
"network.proxy.socks_port": proxyPort,
"network.proxy.socks_remote_dns": True,
}
driver = webdriver.Firefox(firefox_binary=binary,proxy=proxy_settings)
def interactWithSite(driver):
driver.get("https://www.google.com")
driver.save_screenshot("screenshot.png")
interactWithSite(driver)
To connect to a Tor Browser through a FirefoxProfile you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os
torexe = os.popen(r'C:\Users\AtechM_03\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile = FirefoxProfile(r'C:\Users\AtechM_03\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile= profile, executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("http://check.torproject.org")
Browser Snapshot:
You can find a relevant discussion in How to use Tor with Chrome browser through Selenium
I would like to expand on #DebanjanB answer by adding the Linux counterpart:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os
torexe = os.popen('some/path/tor-browser_en-US/Browser/start-tor-browser')
# in my case, I installed it under a folder tor-browser_en-US after
# downloading and extracting it from
# https://www.torproject.org/download/ for linux
profile = FirefoxProfile(
'some/path/tor-browser_en-US/Browser/TorBrowser/Data/Browser/profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
firefox_options = webdriver.FirefoxOptions()
firefox_options.binary_location = '/usr/bin/firefox'
# /usr/bin/firefox is default location of firefox - for me anyway
driver = webdriver.Firefox(
firefox_profile=profile, options=firefox_options,
executable_path='wherever/you/installed/geckodriver')
# I keep my geckodriver(s) in a special folder sorted by versions.
# Geckodriver downloadable here:
# https://github.com/mozilla/geckodriver/releases/
driver.get("http://check.torproject.org")
The verified answer does not work in case of opening dot onion sites(I believe that's something to do with tor network which is not allowing access to normal firefox).
As for the latest tor browser (from the tor browser bundle), starting it using selenium causes some error due to which the browser cannot start tor proxy itself causing proxy and timeout errors(doesn't matter if tor proxy is started by python or manually or not started at all). This could also be due to port 9050 or 9150 being used by tor proxy and not being available to browser's tor instance but this does not explain the error caused when no instance of tor proxy is running.
The solution i have found is to start the tor proxy as normal, manually or using os.popen("tor.exe") and configure tor browser to not start tor proxy.
here's the code:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
os.popen(r'e:\\bla\\bla\\bla\\tor\\Tor\\tor.exe')
binary=FirefoxBinary(r'e:\\bla\\bla\\bla\\Tor Browser\\Browser\\firefox.exe')
fp=FirefoxProfile(r'e:\\foo\\bar\\bla\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default')
fp.set_preference('extensions.torlauncher.start_tor',False)#note this
fp.set_preference('network.proxy.type',1)
fp.set_preference('network.proxy.socks', '127.0.0.1')
fp.set_preference('network.proxy.socks_port', 9050)
fp.set_preference("network.proxy.socks_remote_dns", True)
fp.update_preferences()
driver = webdriver.Firefox(firefox_profile=fp,firefox_binary=binary)
driver.get("http://check.torproject.org")
driver.get('https://www.bbcnewsv2vjtpsuy.onion/')
*note fp.set_preference('extensions.torlauncher.start_tor',False) on line 10 is being used to configure tor to not start its own tor instance so that it uses the proxy config and tor instance started above.
lo and behold as the tbb starts working like normal firefox bot browser
Assume I have this script:
def connect():
print('[*] Checking for updates...')
subprocess.call(['sudo', 'apt-get', 'update'])
subprocess.call(['sudo', 'apt-get', 'upgrade'])
print('[+] Starting Tor service...')
subprocess.call(['sudo', 'service', 'tor', 'start'])
vpn_path = str(input('[*] Type the path where your .ovpn files are: '))
os.chdir(vpn_path)
path = os.listdir(vpn_path)
for file in path:
if file.lower().endswith('ovpn'):
print(file)
vpn = str(input("Enter a .ovpn file name to connect to: "))
if vpn in path:
subprocess.call(['sudo', 'openvpn', '--config', vpn])
The first two subprocess.calls run in succession - first the system is updated and then Tor is started. After this, the script needs to initiate a VPN connection. The only problem is that the script won't progress until the OpenVPN process is stopped. As soon as the openvpn subprocess is called, it should be opened and run in a new terminal while the script continues. I've seen a few questions similar to this, but they use popen and I'm not sure if they will be able to efficiently solve my problem. Is there a simple and efficient way of doing this in python3?
EDIT (new code):
vpn = str(input("Enter a .ovpn file name to connect to: "))
vpncommand = 'sudo openvpn --config ' + "'" + vpn + "'"
vpnprocess = subprocess.Popen(vpncommand, shell=True)
vpnprocess.wait()