This question is related to this other one: How to use sockets to send user and password to a devboard using ssh
I want to connect to the devboard in order to execute a script. All the outputs of that script I want to send to a Elasticsearch machine.
I can connect to the devboard (see IMAGE below) using my laptop which happens to have Elasticsearch installed. But, when I want to send data to the devboard, the script shows nothing.
What I am doing is:
As soon as you find mendel#undefined-eft:~$ , send the command: cd coral/tflite/python/examples/classification/Auto_benchmark\n
What am I doing wrong?
import paramiko
import os
#Server's data
IP = '172.16.2.47'
PORT = 22
USER = 'mendel'
PASSWORD = 'mendel'
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname = IP, port=PORT, username = USER, password = PASSWORD)
channel = ssh.invoke_shell() #to get a dedicated channel
channel_data = str()
host = str()
while True:
if channel.recv_ready(): #is there data to be read?
channel_data += channel.recv(9999).decode("utf-8")
os.system('clear')
print(channel_data)
#ONLY WORKS UNTIL HERE!!!
else:
continue
if channel_data.endswith('mendel#undefined-eft:~$'):
channel.send('cd coral/tflite/python/examples/classification/Auto_benchmark\n')
channel_data += channel.recv(9999).decode("utf-8")
print(channel_data)
IMAGE
EDIT
channel = ssh.invoke_shell() #to get a dedicated channel
channel_data = str()
host = str()
while True:
if channel.recv_ready(): #is there data to be read?
channel_data += channel.recv(9999).decode("utf-8")
os.system('clear')
print(channel_data)
else:
continue
if channel_data.endswith('mendel#undefined-eft:~$ '):#it is good to send commands
channel.send('cd coral/tflite/python/examples/classification/Auto_benchmark\n')
#channel_data += channel.recv(9999).decode("utf-8")
#print(channel_data)
elif channel_data.endswith('mendel#undefined-eft:~/coral/tflite/python/examples/classification/Auto_benchmark$ '):
channel.send('ls -l\n') #python3 auto_benchmark.py')
channel_data += channel.recv(9999).decode("utf-8")
print(channel_data)
I guess you have to change the
if channel_data.endswith('mendel#undefined-eft:~$'):
to
if channel_data.endswith('mendel#undefined-eft:~$ '):
according to your prompt. Please note the space after :~$
Related
I develop a Python two scripts to transfer lot of data (~120Go) on my vm, with Paramiko.
My vm is on OVH server.
First script transfert ~ 40Go, and the second script ~ 80Go.
Stack :
Python 3.9.1
Paramiko 2.7.2
SCP 0.13.3
On my both scripts, I use this function to setup SSH connection.
def connect():
transport = paramiko.Transport((target_host, target_port))
transport.connect(None, target_username, target_pwd)
sftp_client = paramiko.SFTPClient.from_transport(transport)
green_print("SSH connected")
return sftp_client, transport
If I create one script which do the two transfer, I'm timeout after 3 hours.
With two distincts script which run in the same time, I'm timeout after 2h30 of transfer.
I already read many many many post on Paramiko, SSH connection, timeout parameter, ClientAliveInterval, etc... But nothing works.
After this time, I have this error
Connexion fermée par l'hôte distant / Connection closed by remote host
Three functions of my script :
def connect():
transport = paramiko.Transport((target_host, target_port))
transport.connect(None, target_username, target_pwd)
sftp_client = paramiko.SFTPClient.from_transport(transport)
green_print("SSH connected")
return sftp_client, transport
def transfert(sftp, vm, object_path):
os.chdir(os.path.split(object_path)[0])
parent = os.path.split(object_path)[1]
try:
sftp.mkdir(vm)
except:
pass
for path, _, files in os.walk(parent):
try:
sftp.mkdir(os.path.join(vm, path))
except:
pass
for filename in files:
sftp.put(os.path.join(object_path, filename),
os.path.join(vm, path, filename))
def job():
green_print("\nProcess start...")
check_folder()
folder = forfiles_method()
vm, lidar, pos = name_path(folder)
sftp, transport = connect()
transfert(sftp, vm, pos)
sftp.close()
transport.close()
minimal reproducible example :
from paramiko.sftp_client import SFTPClient
import paramiko
import os
target_host = 'xx.xx.x.xxx'
target_port = 22
target_username = "xxxxxxx"
target_pwd = 'xxxxxx'
remote_path = "e:/x/" # => on your vm
target_folder = '/folder1' # => on your computer
def connect():
transport = paramiko.Transport((target_host, target_port))
transport.connect(None, target_username, target_pwd)
sftp_client = paramiko.SFTPClient.from_transport(transport)
return sftp_client, transport
def transfert(sftp, remote_path, object_path):
os.chdir(os.path.split(object_path)[0])
parent = os.path.split(object_path)[1]
try:
sftp.mkdir(remote_path)
except:
pass
for path, _, files in os.walk(parent):
try:
sftp.mkdir(os.path.join(remote_path, path))
except:
pass
for filename in files:
sftp.put(os.path.join(object_path, filename),
os.path.join(remote_path, path, filename))
def job():
sftp, transport = connect()
transfert(sftp, remote_path, target_folder)
sftp.close()
transport.close()
The tree structure of my files, and I want to transfer only the "test" folder which contains more than 120GB.
folder / test
I'm new in Python dev.
If someone have a solution, I take it !
So the solution :
subprocess.run(["winscp.com", "/script=" + cmdFile], shell=True)
If winscp.com is not found like command, insert the path like : C:/Program Files (x86)/WinSCP/winscp.com
Write your commandes line in a txt file, here cmdFile.
Links, which can help you :
Running WinSCP command from Python
From Python run WinSCP commands in console
https://winscp.net/eng/docs/commandline
I need to further optimize my code in Python.
I was earlier executing commands on the Device Under Test step by step which was a lot as I also required sleep timers. However I was able to minimize it through a list and calling elements of the list in a for loop:
I need your inputs to further optimize this code:
ConfigListBFD = ['conf t' , 'int Fa1/0' , 'ip address 10.10.10.1 255.255.255.0', 'no shut']
for i in ConfigListBFD:
tn.write(i.encode('ascii') + b"\n")
print (i, "command entered successfully")
time.sleep(2)
Please note: I am telnetting the DUT as ssh is not supported.
i am using this optimized common code for telnet. we can create a common file where you can add this method
import telnetlib
import time
def telnet(host):
user = <username>
password = <password>
try :
tn = telnetlib.Telnet(host)
except :
print("Unable to connect")
sys.exit()
tn.read_until(b"Username:") # read until username prompt
tn.write(user.encode('ascii') + b"\n")
if password:
tn.read_until(b"password:") #read until password prompt
tn.write(password.encode('ascii') + b"\n")
tn.read_until(b"#")
return tn #return telnetlib handle
than import this method to another file, where we write our script
I am running a script that acts as a server, allows two clients to connect to it, and for one specific client to send a message to the server, the server modifies it, then sends it to the other client.
This appears to work, as the receiving client acknowledges that the input was received and is valid. This is a script that I intend to run continuously.
However, a big issue is that my /tmp/ directory is filling up with directories named _M... (The ellipses representing a random string), that contains python modules (such as cryptography, which, as far as I'm aware, I'm not using), and timezone information (quite literally every timezone that python supports). It seems to be creating them very frequently, but I can't identify what in the process exactly is doing this.
I have created a working cleanup bash script that removes files older than 5 minutes from the directory every 5 minutes, however, I cannot guarantee that when I am duplicating this process for other devices, that the directories will have the same name formatting. Rather than create a unique bash script for each process that I create, I'd rather be able to clean up the directories from within the python script, or even better, to prevent the directories from being created at all.
The problem is, I'm not certain of how this is accomplished, and I do not see anything on SO regarding what is creating these directories, nor how to delete them.
The following is my script
import time, socket, os, sys, re, select
IP = '192.168.109.8'
PORT = [3000, 3001]
PID = str(os.getpid())
PIDFILE = "/path/to/pidfile.pid"
client_counter = 0
sockets_list = []
def runCheck():
if os.path.isfile(PIDFILE):
return False
else:
with open(PIDFILE, 'w') as pidfile:
pidfile.write(PID)
return True
def openSockets():
for i in PORT:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((IP, i))
s.listen(1)
sockets_list.append(s)
def receiveMessage(client_socket):
try:
message = client_socket.recv(2048).decode('utf-8')
if not message:
return False
message = str(message)
return message
except:
return False
def fixString(local_string):
#processes
return local_string
def main():
try:
openSockets()
clients = {}
print(f'Listening for connections on {IP}:{PORT[0]} and {PORT[1]}...')
client_count = 0
while True:
read_sockets, _, exception_sockets = select.select(sockets_list, [], sockets_list)
for notified_socket in read_sockets:
if notified_socket == sockets_list[0] or notified_socket == sockets_list[1]:
client_socket, client_address = sockets_list[client_count].accept()
client_count = (client_count + 1) % 2
sockets_list.append(client_socket)
clients[client_socket] = client_socket
print('Accepted new connection from: {}'.format(*client_address))
else:
message = receiveMessage(notified_socket)
if message is False:
continue
message = fixString(message)
for client_socket in clients:
if client_socket != notified_socket:
if message != "N/A":
client_socket.send(bytes(message, "utf-8"))
for notified_socket in exception_sockets:
sockets_list.remove(notified_socket)
del clients[notified_socket]
time.sleep(1)
except socket.timeout:
for i in sockets_list:
i.close()
os.remove(PIDFILE)
sys.exit()
except Exception as e:
for i in sockets_list:
i.close()
err_details = str('Error in line {}'.format(sys.exc_info()[-1].tb_lineno), type(e).__name__, e)
os.remove(PIDFILE)
print("Exception: {}".format(err_details))
sys.exit()
if __name__ == "__main__":
if runCheck():
main()
else:
pass
How might I set it up so that the python script will delete the directories it creates in the /tmp/ directory, or better, to not create them in the first place? Any help would be greatly appreciated.
As it would turn out, it is PyInstaller that was generating these files. In the documentation, it states that pyinstaller generates this _MEI directory when creating the executable in single-file mode, and it is supposed to delete it as well, but for some reason it didn't.
So I'm building this socket application, and it works just fine on my computer. But when i start server socket on another laptop, it just crashes with a invalid start byte error:
How do i proper encode the program to work with all laptops
This is the error i get on :
Other laptops.
My laptop.
I have tried to change the encoding, but I'm just not quite sure where i have to change it.
Class Listener:
def __init__(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server_address = (socket.gethostbyname(socket.gethostname()), 10000)
self.sock.bind(self.server_address)
print(f"LISTENER : {str(self.server_address[0])} port {str(self.server_address[1])}")
def listen(self):
self.sock.listen(1)
while True:
print("Connection Open")
print(" Waiting for connections")
self.connection, self.client_address = self.sock.accept()
try:
print(f"Connection from {str(self.client_address)}")
while True:
data = self.connection.recv(1024)
if data:
message = str(data)
if not "print" in message.lower(): # This just checks if the client wants to print system information from the server
Validate(message) # this checks for a command the server have to do
else:
self.connection.sendall(pickle.dumps(self.computerinfomation))
else:
self.listen()
except Exception as e:
print(e)
I want it to work on other laptops as well, and i just cant see why it wont.
Furas came with a solution.
I changed the
message = str(data)
to
message = str(data, encoding="utf-8")
I did the same on the client side
Not going to lie. I just changed the encoding = utf-16.
Example:
df = pd.read_csv(C:/folders path/untitled.csv, encoding = "utf-16")
In python, I am creating a message system where a client and server can send messages back and forth simeltaneously. Here is my code for the client:
import threading
import socket
# Global variables
host = input("Server: ")
port = 9000
buff = 1024
# Create socket instance
s = socket.socket()
# Connect to server
s.connect( (host, port) )
print("Connected to server\n")
class Recieve(threading.Thread):
def run(self):
while True: # Recieve loop
r_msg = s.recv(buff).decode()
print("\nServer: " + r_msg)
recieve_thread = Recieve()
recieve_thread.start()
while True: # Send loop
s_msg = input("Send message: ")
if s_msg.lower() == 'q': # Quit option
break
s.send( s_msg.encode() )
s.close()
I have a thread in the background to check for server messages and a looping input to send messages to the server. The problem arises when the server sends a message and the user input is immediately bounced up to make room for the servers message. I want it so that the input stays pinned to the bottom of the shell window, while the output is printed from the 2nd line up, leaving the first line alone. I have been told that you can use curses or Queues to do this, but I am not sure which one would be best in my situation nor how to implement these modules into my project.
Any help would be appreciated. Thank you.
I want it so that the input stays pinned to the bottom of the shell
window, while the output is printed from the 2nd line up, leaving the
first line alone. I have been told that you can use curses
Here's a supplemented version of your client code using curses.
import threading
import socket
# Global variables
host = input("Server: ")
port = 9000
buff = 1024
# Create socket instance
s = socket.socket()
# Connect to server
s.connect( (host, port) )
print("Connected to server\n")
import sys
write = sys.stdout.buffer.raw.write
from curses import *
setupterm()
lines = tigetnum('lines')
change_scroll_region = tigetstr('csr')
cursor_up = tigetstr('cuu1')
restore_cursor = tigetstr('rc')
save_cursor = tigetstr('sc')
def pin(input_lines): # protect input_lines at the bottom from scrolling
write(save_cursor + \
tparm(change_scroll_region, 0, lines-1-input_lines) + \
restore_cursor)
pin(1)
class Recieve(threading.Thread):
def run(self):
while True: # Recieve loop
r_msg = s.recv(buff).decode()
write(save_cursor+cursor_up)
print("\nServer: " + r_msg)
write(restore_cursor)
recieve_thread = Recieve()
recieve_thread.daemon = True
recieve_thread.start()
while True: # Send loop
s_msg = input("Send message: ")
if s_msg.lower() == 'q': # Quit option
break
s.send( s_msg.encode() )
pin(0)
s.close()
It changes the scrolling region to leave out the screen's bottom line, enters the scrolling region temporarily to output the server messages, and changes it back at the end.