Most efficient way of multithreading processes in python3 - python-3.x

Assume I have this script:
def connect():
print('[*] Checking for updates...')
subprocess.call(['sudo', 'apt-get', 'update'])
subprocess.call(['sudo', 'apt-get', 'upgrade'])
print('[+] Starting Tor service...')
subprocess.call(['sudo', 'service', 'tor', 'start'])
vpn_path = str(input('[*] Type the path where your .ovpn files are: '))
os.chdir(vpn_path)
path = os.listdir(vpn_path)
for file in path:
if file.lower().endswith('ovpn'):
print(file)
vpn = str(input("Enter a .ovpn file name to connect to: "))
if vpn in path:
subprocess.call(['sudo', 'openvpn', '--config', vpn])
The first two subprocess.calls run in succession - first the system is updated and then Tor is started. After this, the script needs to initiate a VPN connection. The only problem is that the script won't progress until the OpenVPN process is stopped. As soon as the openvpn subprocess is called, it should be opened and run in a new terminal while the script continues. I've seen a few questions similar to this, but they use popen and I'm not sure if they will be able to efficiently solve my problem. Is there a simple and efficient way of doing this in python3?
EDIT (new code):
vpn = str(input("Enter a .ovpn file name to connect to: "))
vpncommand = 'sudo openvpn --config ' + "'" + vpn + "'"
vpnprocess = subprocess.Popen(vpncommand, shell=True)
vpnprocess.wait()

Related

Network sniffer that opens the ips as tabs in browser?

I'm trying to write a code that basically grabs the network traffic sniffed by wireshark and opens the ips in tabs in selenium.
At first I tried using whois and socket.gethostbyaddr() as All I needed was to translate the ips to domains. But it didn't workout well.
I ran into a problem with permissions because pyshark needs root privilege and selenium doesn't work as root.
here is the code:
import pyshark
from selenium import webdriver
global ips
ips = ''
options = webdriver.ChromeOptions()
options.headless = False
options.add_argument("user-data-dir=/home/afterlife/Documents/chrome_profiles/cyber")
browser = webdriver.Chrome(executable_path='/home/afterlife/Downloads/chromedriver', options=options)
def get_domain_name(ip_address):
for line in ips:
if ip_address not in line:
ips += ip_address+'\n'
browser.execute_script('''window.open(''' + ip_address + ''',"_blank");''')
capture = pyshark.LiveCapture(interface="wlp1s0")
capture.set_debug()
for packet in capture.sniff_continuously():
try:
if hasattr(packet, 'http'):
protocol = packet.transport_layer
source_address = packet.ip.src
source_port = packet[packet.transport_layer].srcport
destination_address = packet.ip.dst
destination_port = packet[packet.transport_layer].dstport
print(destination_address)
get_domain_name(destination_address)
except:
print("exception")
I tried to make pyshark work as non root but failed, I used sudo dpkg-reconfigure wireshark-common but it haven't effected tshark of pyshark.
I also tried using os.setuid() which I successfully used once but forgot/now it doesn't work. I used 0 for root and 1000 for my non-root user.
How would you accomplish what I want?
Thank you!

How to run an XMLRPC server and an XMLRPC client on Mininet hosts through a python script?

I am trying to run an XMLRPC server and an XMLRPC client on Mininet hosts, using the script below.
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.node import OVSController
class MyTopo(Topo):
def __init__(self):
# Initialize topology
Topo.__init__(self)
# Add hosts
server1 = self.addHost('server1')
server2 = self.addHost('server2')
# Add switch
s1 = self.addSwitch('s1')
# Add links
self.addLink(server1, s1)
self.addLink(server2, s1)
if __name__ == '__main__':
net = Mininet(topo=MyTopo(), controller=OVSController)
net.start()
print(net.hosts[0].cmd('python3 xmlrpc_server.py'))
print(net.hosts[1].cmd('python3 xmlrpc_client.py'))
The file xmlrpc_server.py is:
from xmlrpc.server import SimpleXMLRPCServer
import threading
def is_even(n):
return n%2 == 0
server = SimpleXMLRPCServer(("0.0.0.0", 8000), logRequests=True, allow_none = True)
server.register_function(is_even, "is_even")
print("Listening on port 8000...")
server_thread = threading.Thread(target=server.serve_forever)
server_thread.start()
The file xmlrpc_client.py is:
import xmlrpc.client
proxy = xmlrpc.client.ServerProxy("http://10.0.0.1:8000/")
print("3 is even: %s" % str(proxy.is_even(3)))
print("100 is even: %s" % str(proxy.is_even(100)))
The problem is that although I have used a thread, when I run the xmlrpc_server.py script on server1, the execution pauses at line server_thread.start() waiting for the script execution to be completed before moving on and thus never goes on to the next line, which means that the XMLRPC client script never runs. How do I overcome this problem?
P.S.: xmlrpc_server.py and xmlrpc_client.py can be executed through the server terminals (by using the commands xterm server1 and xterm server2 on Mininet CLI and then using the commands python3 xmlrpc_server.py and python3 xmlrpc_client.py on the xterm terminals that open), but I need to start the server and client through a python script so as to perform some further calculations after the communication between the two servers.
Replace print(net.hosts[0].cmd('python3 xmlrpc_server.py')) with print(net.hosts[0].sendCmd('python3 xmlrpc_server.py')). Connection is sometimes refused, but that issue can be resolved with exception handling on the client script.

Controlling a minecraft server with python

I've searched a lot for this and have not yet found a definitive solution. The closest thing I've found is this:
import shutil
from os.path import join
import os
import time
import sys
minecraft_dir = ('server diectory')
world_dir = ('server world driectory')
def server_command(cmd):
os.system('screen -S -X stuff "{}\015"'.format(cmd))
on = "1"
while True:
command=input()
command=command.lower()
if on == "1":
if command==("start"):
os.chdir(minecraft_dir)
os.system('"C:\Program Files\Java\jre1.8.0_111\bin\java.exe" -Xms4G -Xmx4G -jar craftbukkit-1.10.2.jar nogui java')
print("Server started.")
on = "0"
else:
server_command(command)
When I launch this program and type 'start' the CMD flashes up and closes instantly. Instead I want the CMD to stay open with the minecraft sever running from it. I'm not sure why this happens or what the problem is, any help would be greatly appreciated.
p.s. I have edited this to my needs (such as removing a backup script that was unnecessary) but it didn't work before. The original link is: https://github.com/tschuy/minecraft-server-control
os.system will simply run the command then return to your python script with no way to further communicate with it.
On the other hand using subprocess.Popen gives you access to the process while it runs, including writing to it's .stdin which is how you send data to the server:
def server_command(cmd):
process.stdin.write(cmd+"\n") #just write the command to the input stream
process = None
executable = '"C:\Program Files\Java\jre1.8.0_111\bin\java.exe" -Xms4G -Xmx4G -jar craftbukkit-1.10.2.jar nogui java'
while True:
command=input()
command=command.lower()
if process is not None:
if command==("start"):
os.chdir(minecraft_dir)
process = subprocess.Popen(executable, stdin=subprocess.PIPE)
print("Server started.")
else:
server_command(command)
you can also pass stdout=subprocess.PIPE so you can also read it's output and stderr=subprocess.PIPE to read from it's error stream (if any)
As well instead of process.stdin.write(cmd+"\n") you could also use the file optional parameter of the print function, so this:
print(cmd, file=process.stdin)
Will write the data to process.stdin formatted in the same way that print normally does, like ending with newline for you unless passing end= to override it etc.
Both of the above answers do not work in the environment I tried them in.
I think the best way is to use RCON, not sending keys to a window.
RCON is the protocol used by games to run commands.
Many python libraries support Minecraft RCON, and the default server.properties file has an option for RCON.
We will use the python module: MCRON.
Install it. It works for windows, mac, linux.
Type:
pip install mcrcon
Lets configure your server to allow RCON.
In server.properties, find the line 'enable-rcon' and make it look like this:
enable-rcon=true
Restart and stop your server.
Find the line 'rcon.password' and set it to any password you will remember.
You can leave the port default at 25575.
Now, open your terminal and type:
mcron localhost
Or your server ip.
You will be prompted to enter the password you set.
Then you can run commands and will get the result.
But we are doing this with python, not the PYPI MCRON scripts - so do this.
from mcrcon import MCRcon as r
with r('localhost', 'insertyourpasswordhere') as mcr:
resp = mcr.command('/list')
print(resp) #there are 0/20 players online: - This will be different for you.

using python, selenium and phantomjs on openshift (socket binding permission denied?)

Alright I'm at the end of my tether trying to get phantomJS to work with selenium in an openshift environment. I've downloaded the phantomjs binary using ssh and can even run it in the shell. But when it comes to starting a webdriver service using selenium I keep getting this traceback error no matter the args I put in.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/phantomjs/webdriver.py", line 50, in __init__
service_args=service_args, log_path=service_log_path)
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/phantomjs/service.py", line 50, in __init__
service.Service.__init__(self, executable_path, port=port, log_file=open(log_path, 'w'))
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/common/service.py", line 33, in __init__
self.port = utils.free_port()
File "/var/lib/openshift/576e22027628e1fb13000211/python/virtenv/venv/lib/python3.3/site-packages/selenium/webdriver/common/utils.py", line 36, in free_port
free_socket.bind(('0.0.0.0', 0))
PermissionError: [Errno 13] Permission denied
Not sure what's going on, am I supposed to bind to an IP address? If so I tried using service args but that hasn't helped.
I came across the same issues trying to run phantomJS on my Openshift-hosted Django application, running on a Python 3 gear. Finally I managed to make it work, this is how:
The main issue to overcome is that Openshift does not allow applications to bind on localhost (nor '0.0.0.0' nor '127.0.0.1). So the point is to bind to the actual IP address of your Openshift Gear instead
You have to deal with this issue at the ghostdriver level as well as within the Python-selenium binding.
ghostdriver (phantomJS binary)
Unfortunately, as explained brilliantly by Paolo Bernardi in this post : http://www.bernardi.cloud/2015/02/25/phantomjs-with-ghostdriver-on-openshift/ you have to use a patched version of phantomjs for this, as the released version doesn't allow to bind to a specified IP. The binary linked by Paolo did not work on my Python3 cardridge, yet this one worked perfectly: https://github.com/jrestful/server/blob/master/seo/phantomjs-1.9.8-patched.tar.gz?raw=true (see question Trying to run PhantomJS on OpenShift: cannot patch GhostDriver so that it can bind on the server IP address for details)
Upload this phantomjs binary to app-root/data/phantomjs/bin (for example) and make sure it is runnable :
> chmod 711 app-root/data/phantomjs/bin/phantomjs
You can now check that you can bin to your IP like this (I chose port number 15002 for my app, I reckon you can pick any value you want above 15000):
> echo $OPENSHIFT_PYTHON_IP
127.13.XXX.XXX
> app-root/data/phantomjs/bin/phantomjs --webdriver=127.13.XXX.XXX:15002
PhantomJS is launching GhostDriver...
[INFO - 2017-03-24T13:16:36.031Z] GhostDriver - Main - running on port 127.13.XXX.XXX:15002
Ok, now kill this process and proceed to step 2 : python webdriver
Custom python-selenium webdriver for PhantomJS
The point is to add the IP address to bind to as a parameter of the PhantomJS Webdriver.
First, I defined new settings to adapt to Openshift's constraint in my settings.py
PHANTOMJS_BIN_PATH = os.path.join(os.getenv('OPENSHIFT_DATA_DIR'), 'phantomjs', 'bin', 'phantomjs')
PHANTOMJS_LOG_PATH = os.path.join(os.getenv('OPENSHIFT_LOG_DIR'), 'ghostdriver.log')
(make sure app-root/logs/ is writable, maybe you'll have to chmod it)
Then, I had to override the PhantomJS Webdriver class, to provide the ip address as an argument. Here is my own implementation:
from selenium.webdriver import phantomjs
from selenium.webdriver.common import utils
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
class MyPhantomJSService(phantomjs.service.Service):
def __init__(self, executable_path, port=0, service_args=None, log_path=None, ip=None):
if ip is None:
self.ip = '0.0.0.0'
else:
self.ip = ip
phantomjs.service.Service.__init__(self, executable_path, port, service_args, log_path)
def command_line_args(self):
return self.service_args + ["--webdriver=%s:%d" % (self.ip, self.port)]
def is_connectable(self):
return utils.is_connectable(self.port, host=self.ip)
#property
def service_url(self):
"""
Gets the url of the GhostDriver Service
"""
return "http://%s:%d/wd/hub" % (self.ip, self.port)
class MyPhantomWebDriver(RemoteWebDriver):
"""
Wrapper to communicate with PhantomJS through Ghostdriver.
You will need to follow all the directions here:
https://github.com/detro/ghostdriver
"""
def __init__(self, executable_path="phantomjs",
ip=None, port=0, desired_capabilities=DesiredCapabilities.PHANTOMJS,
service_args=None, service_log_path=None):
"""
Creates a new instance of the PhantomJS / Ghostdriver.
Starts the service and then creates new instance of the driver.
:Args:
- executable_path - path to the executable. If the default is used it assumes the executable is in the $PATH
- ip - IP sur lequel on veut se binder : c'est la spécificité de ce monkeypatch
- port - port you would like the service to run, if left as 0, a free port will be found.
- desired_capabilities: Dictionary object with non-browser specific
capabilities only, such as "proxy" or "loggingPref".
- service_args : A List of command line arguments to pass to PhantomJS
- service_log_path: Path for phantomjs service to log to.
"""
self.service = MyPhantomJSService(
executable_path,
port=port,
service_args=service_args,
log_path=service_log_path,
ip=ip)
self.service.start()
try:
RemoteWebDriver.__init__(
self,
command_executor=self.service.service_url,
desired_capabilities=desired_capabilities)
except Exception:
self.quit()
raise
self._is_remote = False
def quit(self):
"""
Closes the browser and shuts down the PhantomJS executable
that is started when starting the PhantomJS
"""
try:
RemoteWebDriver.quit(self)
except Exception:
# We don't care about the message because something probably has gone wrong
pass
finally:
self.service.stop()
Finally, invoke this custom webdriver instead of webdriver.PhantomJS(..., like this:
from .myphantomjs import MyPhantomWebDriver
browser = MyPhantomWebDriver(executable_path=settings.PHANTOMJS_BIN_PATH, service_log_path=settings.PHANTOMJS_LOG_PATH, ip=os.getenv('OPENSHIFT_PYTHON_IP'), port=15002)
From then on, you can use the browser object normally

pxssh error 'could not synchronize with original prompt'

I have been getting the below error while using pxssh to get into remote servers to run unix commands ( like uptime )
Traceback (most recent call last):
File "./ssh_pxssh.py", line 33, in
login_remote(hostname, username, password)
File "./ssh_pxssh.py", line 12, in login_remote
if not s.login(hostname, username, password):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect/pxssh.py", line 278, in login
**raise ExceptionPxssh ('could not synchronize with original prompt')
pexpect.pxssh.ExceptionPxssh: could not synchronize with original prompt**
Line 33 is where I call this function in main.
The function I am using is here :
def login_remote(hostname, username, password):
s = pxssh.pxssh()
s.force_password = True
if not s.login(hostname, username, password, auto_prompt_reset=False):
print("ssh to host :"+ host + " failed")
print(str(s))
else:
print("SSH to remote host " + hostname + " successfull")
s.sendline('uptime')
s.prompt()
print(s.before)
s.logout()
The error does not come each time I run the script. Rather it is intermittent. It comes 7 out of 10 times I run my script.
I have solved it by adding sync_multiplier argument to the login function.
s.login(hostname, username, password, sync_multiplier=5 auto_prompt_reset=False)
note that sync_multiplier is a communication timeout argument to perform successful synchronization. it tries to read prompt for at least sync_multiplier seconds.
Worst case performance for this method is sync_multiplier * 3 seconds.
I personally set sync_multiplier=2 but it depends on the communication speed on the system I work on.
I had the same problem when pxssh tried to login on a very slow connection.
The pexpect lib apparently was fooled by the remote motd prompt.
This remote motd prompt contained a uname -svr prompt, which itself contained a # character inside.
Apparently, pexpect saw it like a prompt. From that point, the lib was not in line anymore with the ssh session.
The following workaround was working for me:
just remove the # char inside /var/run/motd.dynamic (debian), or in /var/run/motd (ubuntu).
Another solution is to ask ssh to no prompt the motd while logging in . But this is not working for me:
i added the following:
PrintMotd no
in /etc/ssh/sshd_config
=> not working
Another workaround:
create a file in the home directory:
.hushlogin in the ~ directory

Resources