I have been getting the below error while using pxssh to get into remote servers to run unix commands ( like uptime )
Traceback (most recent call last):
File "./ssh_pxssh.py", line 33, in
login_remote(hostname, username, password)
File "./ssh_pxssh.py", line 12, in login_remote
if not s.login(hostname, username, password):
File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pexpect/pxssh.py", line 278, in login
**raise ExceptionPxssh ('could not synchronize with original prompt')
pexpect.pxssh.ExceptionPxssh: could not synchronize with original prompt**
Line 33 is where I call this function in main.
The function I am using is here :
def login_remote(hostname, username, password):
s = pxssh.pxssh()
s.force_password = True
if not s.login(hostname, username, password, auto_prompt_reset=False):
print("ssh to host :"+ host + " failed")
print(str(s))
else:
print("SSH to remote host " + hostname + " successfull")
s.sendline('uptime')
s.prompt()
print(s.before)
s.logout()
The error does not come each time I run the script. Rather it is intermittent. It comes 7 out of 10 times I run my script.
I have solved it by adding sync_multiplier argument to the login function.
s.login(hostname, username, password, sync_multiplier=5 auto_prompt_reset=False)
note that sync_multiplier is a communication timeout argument to perform successful synchronization. it tries to read prompt for at least sync_multiplier seconds.
Worst case performance for this method is sync_multiplier * 3 seconds.
I personally set sync_multiplier=2 but it depends on the communication speed on the system I work on.
I had the same problem when pxssh tried to login on a very slow connection.
The pexpect lib apparently was fooled by the remote motd prompt.
This remote motd prompt contained a uname -svr prompt, which itself contained a # character inside.
Apparently, pexpect saw it like a prompt. From that point, the lib was not in line anymore with the ssh session.
The following workaround was working for me:
just remove the # char inside /var/run/motd.dynamic (debian), or in /var/run/motd (ubuntu).
Another solution is to ask ssh to no prompt the motd while logging in . But this is not working for me:
i added the following:
PrintMotd no
in /etc/ssh/sshd_config
=> not working
Another workaround:
create a file in the home directory:
.hushlogin in the ~ directory
Related
Been working through this for a minute now, and figured I would ask here as I am sure it is something simple that I am missing.
Trying to silently run an installer using subprocess.run() (Have used subprocess.Popen, and os.system as well). I pass the full installer path, along with the arguments (As a list). Using all of the above, the installer runs successfully, and python waits for the installer to finish. I added a check to look at the PID of the installer process as just a backup check. However, running the script in full, I install some prerequisites (mainly dotnet 4.8) and then I restart. For cleanliness, I add the python script + arguments to the windows RunOnce registry key so it runs on next login. That works fine, but the installer never actually runs. It returns a 0 exit code, but never actually runs. Digging into it with event viewer, child processes (and maybe the parent process) is run by the NT System account. This causes the installer to silently fails since the System account doesn't have the permissions to run it. Looking for a way to run everything as a user to avoid the System account being used. Hence the question about running as an alternate user or the logged in user. (I don't know if this works on windows in all honesty)
Thoughts? Sample code snippets are below as well. Targeting Windows Server 2016+. Using python 3.11.2
Typical install code looks like the below. Installing it with subprocess works fine when run from command prompt manually. Its only when the RunOnce registry key triggers it does it silently fail. This leads me to believe its just the permissions/user error.
I also took a look at Python subprocess.Popen as different user on Windows but didn't have much luck implementing this. (Given it is 13 years old at this point)
# Installer path
installer = os.path.dirname(os.path.realpath(__file__)) + "\\setup.exe"
# Command for installing silently
# variables referenced here are all strings
ss_command = [
installer,
"-q",
"-s",
"InstallSecretServer=1",
"InstallPrivilegeManager=1",
'SecretServerUserDisplayName="Administrator"',
'SecretServerUserName="Administrator"',
"SecretServerUserPassword=" + '"' + administrator_password + '"',
"SecretServerAppUserName=" + '"' + service_account + '"',
"SecretServerAppPassword=" + '"' + service_account_password + '"',
"DatabaseIsUsingWindowsAuthentication=True",
"DatabaseServer=" + '"' + sql_hostname + '"',
"DatabaseName=" + '"' + database_name + '"',
"/l",
log_file
]
# Install attempt using Popen
installer_process = subprocess.Popen(ss_command)
# Install attempt using run
installer_process = subprocess.run(ss_command, capture_output=True, shell=False)
# Below are the functions for running executables and code from the registry.
# These work just fine, and arguments fed to the original script get passed to the registry key as well.
# Define a function to run an executable at boot
def run_at_startup_set(appname, arguments, path=None, user=False):
# Store the entry in the registry for running the application at startup
# Open the registry key path for applications that are run at login
key = win32api.RegOpenKeyEx(
win32con.HKEY_CURRENT_USER if user else win32con.HKEY_LOCAL_MACHINE,
Startup_Key_Path,
0,
win32con.KEY_WRITE | win32con.KEY_QUERY_VALUE
)
# Make sure the application is not already in the registry
i = 0
while True:
try:
name, _, _ = win32api.RegEnumValue(key, i)
except pywintypes.error as e:
if e.winerror == winerror.ERROR_NO_MORE_ITEMS:
break
else:
raise
if name == appname:
win32api.RegCloseKey(key)
return
i += 1
# Add arguments to the key
run_parameters = path or win32api.GetModuleFileName(0)
for arg in arguments:
run_parameters += ' "' + str(arg) + '"'
# Create a new key
win32api.RegSetValueEx(key, appname, 0, win32con.REG_SZ, run_parameters)
# Close the key
win32api.RegCloseKey(key)
return
# Define a function to run a script at boot
def run_script_at_startup_set(appname, arguments, user=False):
# Like run_at_startup_set(), but for source code files
run_at_startup_set(
appname,
arguments,
# Set the interpreter path (returned by GetModuleFileName())
# followed by the path of the current Python file (__file__).
'{} "{}"'.format(win32api.GetModuleFileName(0), __file__),
user
)
return
I am trying to determine vendor + version (using python NAPALM and parallel-ssh) of network switches (Huawei VRP5/8, Cisco Catalyst and Cisco SMB (SF/SG):
admin#server:~$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from napalm import get_network_driver
>>> driver = get_network_driver('ios')
>>> device = driver('ip', 'username', 'password')
>>> device.open()
>>> print(device.get_facts())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/altepro/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 811, in get_facts
show_ver = self._send_command('show version')
File "/home/altepro/.local/lib/python3.8/site-packages/napalm/ios/ios.py", line 165, in _send_command
output = self.device.send_command(command)
File "/home/altepro/.local/lib/python3.8/site-packages/netmiko/utilities.py", line 600, in wrapper_decorator
return func(self, *args, **kwargs)
File "/home/altepro/.local/lib/python3.8/site-packages/netmiko/base_connection.py", line 1694, in send_command
raise ReadTimeout(msg)
netmiko.exceptions.ReadTimeout:
Pattern not detected: '\x1b\\[Ksg300\\-ab\\-1\\#' in output.
Things you might try to fix this:
1. Explicitly set your pattern using the expect_string argument.
2. Increase the read_timeout to a larger value.
Where sg300-ab-1 is sysname of the switch (Cisco SMB - sg300 in this case, but i have tested this on several versions and types of the SMB lineup)
Things that i have tried:
Tried several version of netmiko, napalm (And its drivers including ios-350) and parallel-ssh. Tried several fresh linux servers with fresh install of napalm and parallel-ssh.
SSH is tested using the same server and credentials and it works without any problems.
When i use parallel-ssh the device doesnt even raise exception or timeout - it just goes stuck in the command:
output = client.run_command(cmd)
hosts = ['192.168.1.50']
client = ParallelSSHClient(hosts, user='my_user', password='my_pass')
cmd = 'show version'
output = client.run_command(cmd)
for host_out in output:
for line in host_out.stdout:
print(line)
Thanks for any kind of help !
It looks like the prompt isn't getting recognized properly. I'm not very familiar with either ParallelSSHClient or napalm, but I have worked with netmiko and that looks like where the error is. Here's some steps that can possible get you closer to figuring out what's happening. I suspect it's the prompt not being read correctly from the device.
Set up debugging and a netmiko session and run a simple command
import logging
import netmiko
logging.basicConfig(level=logging.DEBUG)
session = netmiko.ConnectHandler(
host='192.168.1.50',
username='my_user',
password='my_pass',
device_type='cisco_ios')
results = session.send_command('show version')
If this fails with the same error, then it's the prompt (possibly the \x1b escape character). Try again but with a simpler expect_string, like what's expected at the end of the prompt:
session.send_command('show version', expect_string="#")
If this gets you a result, then it's something about the how the prompt is being set for this device.
To see what's being found for the prompt:
session.find_prompt()
Edit:
Based on what you're reporting, the issue seems to be with the control code \x1b\[ being included in the prompt. It's possible this can be disabled on the device itself, but I'm unfamiliar with that platform. The napalm API doesn't expose netmiko's send_command method. It should still be fixable. This solution would be a hack to make things work, nothing that I'd recommend relying on.
Establish a class that will act as your fix. This will be instantiated with the netmiko session (device.device) and will be used to replace the send_command method.
class HackyFix:
def __init__(self, session):
self.session = session
self.original_send_command = session.send_command
def send_command(self, command):
original_prompt = self.session.find_prompt()
fixed_prompt = original_prompt.replace(r"\x1b[", "")
print(
f"send_command intercepted. {original_prompt} replaced with {fixed_prompt}"
)
return self.original_send_command(command, expect_string=fixed_prompt)
Then in your existing napalm code, add this right after device.open():
hackyfix = HackyFix(device.device)
device.device.send_command = hackyfix.send_command
Now all of napalm's calls to send_command will go through your custom fix that will find the prompt and modify it before passing it to expect_string.
Last edit.
It's an ANSI Escape Code that's being thrown in by the SG300. Specifically it's the one that clears from cursor to end of line. It's also a known issue with the SG300. The good news is that someone made a napalm driver to support it. One big difference between the SG300 driver and the IOS driver is the netmiko device_type is cisco_s300. When this device_type is used, strip_ansi_escape_codes is ran against the output.
Behavior of that escape code tested in bash:
$ printf "This gets cleared\r"; code="\x1b[K"; printf "${code}This is what you see\n"
This is what you see
You can validate that setting cisco_s300 as the device_type fixes the issue:
session = netmiko.ConnectHandler(
host='192.168.1.50',
username='my_user',
password='my_pass',
device_type='cisco_s300')
results = session.send_command('show version')
This should give a result with no modification to the expect_string value. If that works and you're looking to get results sooner or later, the following is a better fix than the hacky fix above.
from napalm.ios import IOSDriver
class QuickCiscoSG300Driver(IOSDriver):
def __init__(self, hostname, username, password, timeout=60, optional_args=None):
super().__init__(hostname, username, password, timeout, optional_args)
def open(self):
device_type = "cisco_s300"
self.device = self._netmiko_open(
device_type, netmiko_optional_args=self.netmiko_optional_args
)
device = QuickCiscoSG300Driver("192.168.1.50", "my_user", "my_pass")
device.open()
device.get_facts()
Or you can get the driver (better option, unless this happens to be the driver you already tried)
I am writing a program using pysftp, and it wants to verify the SSH host Key against C:\Users\JohnCalvin\.ssh\known_hosts.
Using PuTTY, the terminal program is saving it to the Registry [HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\SshHostKeys].
How do I reconcile the difference between pysftp and PuTTY?
My code is:
import pysftp as sftp
def push_file_to_server():
s = sftp.Connection(host='138.99.99.129', username='root', password='*********')
local_path = "testme.txt"
remote_path = "/home/testme.txt"
s.put(local_path, remote_path)
s.close()
push_file_to_server()
The error response I am receiving is:
E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp\__init__.py:61:
UserWarning: Failed to load HostKeys from C:\Users\JohnCalvin\.ssh\known_hosts. You will need to explicitly load HostKeys (cnopts.hostkeys.load(filename)) or disableHostKey checking (cnopts.hostkeys = None).
warnings.warn(wmsg, UserWarning)
Traceback (most recent call last):
File "E:\OneDrive\Python\GIT\DigitalCloud\pysftp_tutorial.py", line 14, in <module>
push_file_to_server()
File "E:\OneDrive\Python\GIT\DigitalCloud\pysftp_tutorial.py", line 7, in push_file_to_server
s = sftp.Connection(host='138.99.99.129', username='root', password='********')
File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp\__init__.py", line 132, in __init__
self._tconnect['hostkey'] = self._cnopts.get_hostkey(host)
File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp\__init__.py", line 71, in get_hostkey
raise SSHException("No hostkey for host %s found." % host) paramiko.ssh_exception.SSHException: No hostkey for host 138.99.99.129 found.
Exception ignored in: <bound method Connection.__del__ of <pysftp.Connection object at 0x00000222FF3A6BE0>>
Traceback (most recent call last):
File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp\__init__.py", line 1013, in __del__
self.close()
File "E:\Program Files (x86)\Anaconda3\lib\site-packages\pysftp\__init__.py", line 784, in close
if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'
The pysftp has some bugs regarding host key handling, as described below. It also seems that the pysftp project was abandoned. Consider using Paramiko directly instead. The pysftp is just a wrapper on top of Paramiko and it does not add anything really significant. See pysftp vs. Paramiko.
For handling of host keys in Paramiko, see:
Paramiko "Unknown Server"
If you want to keep using pysftp, do not set cnopts.hostkeys = None (as the second most upvoted answer shows), unless you do not care about security. You lose a protection against Man-in-the-middle attacks by doing so.
Use CnOpts.hostkeys (returns HostKeys) to manage trusted host keys.
cnopts = pysftp.CnOpts(knownhosts='known_hosts')
with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:
where the known_hosts contains a server public key(s)] in a format like:
example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...
If you do not want to use an external file, you can also use
from base64 import decodebytes
# ...
keydata = b"""AAAAB3NzaC1yc2EAAAADAQAB..."""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('example.com', 'ssh-rsa', key)
with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:
Though as of pysftp 0.2.9, this approach will issue a warning, what seems like a bug:
"Failed to load HostKeys" warning while connecting to SFTP server with pysftp
An easy way to retrieve the host key in the needed format is using OpenSSH ssh-keyscan:
$ ssh-keyscan example.com
# example.com SSH-2.0-OpenSSH_5.3
example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...
(due to a bug in pysftp, this does not work, if the server uses non-standard port – the entry starts with [example.com]:port + beware of redirecting ssh-keyscan to a file in PowerShell)
You can also make the application do the same automatically:
Use Paramiko AutoAddPolicy with pysftp
(It will automatically add host keys of new hosts to known_hosts, but for known host keys, it will not accept a changed key)
Though for an absolute security, you should not retrieve the host key remotely, as you cannot be sure, if you are not being attacked already.
See my article Where do I get SSH host key fingerprint to authorize the server?
It's for my WinSCP SFTP client, but most information there is valid in general.
If you need to verify the host key using its fingerprint only, see Python - pysftp / paramiko - Verify host key using its fingerprint.
One option is to disable the host key requirement:
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:
sftp.put(local_path, remote_path)
You can find more info about that here:
https://stackoverflow.com/a/38355117/1060738
Important note:
By setting cnopts.hostkeys=None you'll lose the protection against Man-in-the-middle attacks by doing so. Use #martin-prikryl answer to avoid that.
Try to use the 0.2.8 version of pysftp library.
$ pip uninstall pysftp && pip install pysftp==0.2.8
And try with this:
try:
ftp = pysftp.Connection(host, username=user, password=password)
except:
print("Couldn't connect to ftp")
return False
Why this?
Basically is a bug with the 0.2.9 of pysftp
here all details
https://github.com/Yenthe666/auto_backup/issues/47
Cook book to use different ways of pysftp.CnOpts() and hostkeys options.
Source : https://pysftp.readthedocs.io/en/release_0.2.9/cookbook.html
Host Key checking is enabled by default. It will use ~/.ssh/known_hosts by default. If you wish to disable host key checking (NOT ADVISED) you will need to modify the default CnOpts and set the .hostkeys to None.
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
# do stuff here
To use a completely different known_hosts file, you can override CnOpts looking for ~/.ssh/known_hosts by specifying the file when instantiating.
import pysftp
cnopts = pysftp.CnOpts(knownhosts='path/to/your/knownhostsfile')
with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
# do stuff here
If you wish to use ~/.ssh/known_hosts but add additional known host keys you can merge with update additional known_host format files by using .load method.
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys.load('path/to/your/extra_knownhosts')
with pysftp.Connection('host', username='me', password='pass', cnopts=cnopts):
# do stuff here
If You try to connect by pysftp to "normal" FTP You have to set hostkey to None.
import pysftp
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
with pysftp.Connection(host='****',username='****',password='***',port=22,cnopts=cnopts) as sftp:
print('DO SOMETHING')
Connect to the server first with a Windows ssh client that uses the known_hosts file.
PuTTy stores the data in the windows registry,however OpenSSH uses the known_hosts file, and will add entries in there after you connect.
Default location for the file is %USERPROFILE%.ssh. I hope this helps
I've implemented auto_add_key in my pysftp github fork.
auto_add_key will add the key to known_hosts if auto_add_key=True
Once a key is present for a host in known_hosts this key will be checked.
Please reffer Martin Prikryl -> answer about security concerns.
Though for an absolute security, you should not retrieve the host key remotely, as you cannot be sure, if you are not being attacked already.
import pysftp as sftp
def push_file_to_server():
s = sftp.Connection(host='138.99.99.129', username='root', password='pass', auto_add_key=True)
local_path = "testme.txt"
remote_path = "/home/testme.txt"
s.put(local_path, remote_path)
s.close()
push_file_to_server()
Note: Why using context manager
import pysftp
with pysftp.Connection(host, username="whatever", password="whatever", auto_add_key=True) as sftp:
#do your stuff here
#connection closed
Hi We sort of had the same problem if I understand you well. So check what pysftp version you're using. If it's the latest one which is 0.2.9 downgrade to 0.2.8.
Check this out. https://github.com/Yenthe666/auto_backup/issues/47
FWIR, if authentication is only username & pw, add remote server ip address to known_hosts like ssh-keyscan -H 192.168.1.162 >> ~/.ssh/known_hosts for ref https://www.techrepublic.com/article/how-to-easily-add-an-ssh-fingerprint-to-your-knownhosts-file-in-linux/
Assume I have this script:
def connect():
print('[*] Checking for updates...')
subprocess.call(['sudo', 'apt-get', 'update'])
subprocess.call(['sudo', 'apt-get', 'upgrade'])
print('[+] Starting Tor service...')
subprocess.call(['sudo', 'service', 'tor', 'start'])
vpn_path = str(input('[*] Type the path where your .ovpn files are: '))
os.chdir(vpn_path)
path = os.listdir(vpn_path)
for file in path:
if file.lower().endswith('ovpn'):
print(file)
vpn = str(input("Enter a .ovpn file name to connect to: "))
if vpn in path:
subprocess.call(['sudo', 'openvpn', '--config', vpn])
The first two subprocess.calls run in succession - first the system is updated and then Tor is started. After this, the script needs to initiate a VPN connection. The only problem is that the script won't progress until the OpenVPN process is stopped. As soon as the openvpn subprocess is called, it should be opened and run in a new terminal while the script continues. I've seen a few questions similar to this, but they use popen and I'm not sure if they will be able to efficiently solve my problem. Is there a simple and efficient way of doing this in python3?
EDIT (new code):
vpn = str(input("Enter a .ovpn file name to connect to: "))
vpncommand = 'sudo openvpn --config ' + "'" + vpn + "'"
vpnprocess = subprocess.Popen(vpncommand, shell=True)
vpnprocess.wait()
I've searched a lot for this and have not yet found a definitive solution. The closest thing I've found is this:
import shutil
from os.path import join
import os
import time
import sys
minecraft_dir = ('server diectory')
world_dir = ('server world driectory')
def server_command(cmd):
os.system('screen -S -X stuff "{}\015"'.format(cmd))
on = "1"
while True:
command=input()
command=command.lower()
if on == "1":
if command==("start"):
os.chdir(minecraft_dir)
os.system('"C:\Program Files\Java\jre1.8.0_111\bin\java.exe" -Xms4G -Xmx4G -jar craftbukkit-1.10.2.jar nogui java')
print("Server started.")
on = "0"
else:
server_command(command)
When I launch this program and type 'start' the CMD flashes up and closes instantly. Instead I want the CMD to stay open with the minecraft sever running from it. I'm not sure why this happens or what the problem is, any help would be greatly appreciated.
p.s. I have edited this to my needs (such as removing a backup script that was unnecessary) but it didn't work before. The original link is: https://github.com/tschuy/minecraft-server-control
os.system will simply run the command then return to your python script with no way to further communicate with it.
On the other hand using subprocess.Popen gives you access to the process while it runs, including writing to it's .stdin which is how you send data to the server:
def server_command(cmd):
process.stdin.write(cmd+"\n") #just write the command to the input stream
process = None
executable = '"C:\Program Files\Java\jre1.8.0_111\bin\java.exe" -Xms4G -Xmx4G -jar craftbukkit-1.10.2.jar nogui java'
while True:
command=input()
command=command.lower()
if process is not None:
if command==("start"):
os.chdir(minecraft_dir)
process = subprocess.Popen(executable, stdin=subprocess.PIPE)
print("Server started.")
else:
server_command(command)
you can also pass stdout=subprocess.PIPE so you can also read it's output and stderr=subprocess.PIPE to read from it's error stream (if any)
As well instead of process.stdin.write(cmd+"\n") you could also use the file optional parameter of the print function, so this:
print(cmd, file=process.stdin)
Will write the data to process.stdin formatted in the same way that print normally does, like ending with newline for you unless passing end= to override it etc.
Both of the above answers do not work in the environment I tried them in.
I think the best way is to use RCON, not sending keys to a window.
RCON is the protocol used by games to run commands.
Many python libraries support Minecraft RCON, and the default server.properties file has an option for RCON.
We will use the python module: MCRON.
Install it. It works for windows, mac, linux.
Type:
pip install mcrcon
Lets configure your server to allow RCON.
In server.properties, find the line 'enable-rcon' and make it look like this:
enable-rcon=true
Restart and stop your server.
Find the line 'rcon.password' and set it to any password you will remember.
You can leave the port default at 25575.
Now, open your terminal and type:
mcron localhost
Or your server ip.
You will be prompted to enter the password you set.
Then you can run commands and will get the result.
But we are doing this with python, not the PYPI MCRON scripts - so do this.
from mcrcon import MCRcon as r
with r('localhost', 'insertyourpasswordhere') as mcr:
resp = mcr.command('/list')
print(resp) #there are 0/20 players online: - This will be different for you.