Multi-client TCP socket server (server sends query upon connection then waits for client response + polls client) - multithreading

I modified the code above from what Torxed posted in the URL: Sending string via socket (python)
I used this as a base as want a multi-threading python3 server. I'm still learning Python so am getting a little stuck.
I've got 2 threads working where the 1st (a background) thread sends a UDP broadcast to inform clients that the source host (the python server) is waiting for client connections.
The client detects the UDP broadcast (on port 1300) & initiates a new TCP session to port 1200. I have got the python server to immediately send some bytes (bytes as not ascii) to the client which 'asks' the client to identity itself upon connection (the client does not initiate the conversation other than establishing the TCP session). In this case the client host is a solar inverter.
The client responds with some basics such as it's serial number, software version, etc which I want to store. Currently I'm not able to store the reply in a variable as it's byte sequence, not ascii so I'm currently writing the datagram to a local file until I figure out how to unpack the byte sequence & save in variables associated to the client session. This will be used to aid identifying which data-set is associated to which client.
Every 15 seconds after that, I want the python server to send some more bytes (again bytes not ascii) which will inform the client to reply with some PV (solar) data.
The client will reply with the PV data which I want to store associated to the client session.
Lastly if an event occurs on the client (inverter such as AC loss power) it will send to the server another datagram in the established TCP session a specifically formatted byte sequence so the server needs to handle an independent response from the client which will be to initiate sending an email (to me) containing the event type. Is it better to use a single recv datagram handler with something like a case statement to identify the response then process it accordingly (but the server must first receive the client's id).
I want the TCP session to remain open indefinitely but also support a 2nd inverter to connect too & allow them should connectivity is temporally interpreted to re-connection without issue.
Any help appreciated as I'm stuck with being unable to read the response from the client.
Here is what I currently have:
#!/usr/bin/env python3
# need threading support to run broadcast UDP 1300 thread
from threading import *
from socket import *
# import time for sleep/delay timers
import time
# Now we can create socket object
serversocket = socket(AF_INET, SOCK_STREAM)
#serversocket.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
#serversocket.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
# Set the IP address and port to listen on
HOST = "192.168.1.11"
PORT = 1200
# Bind the socket on the interface/port
serversocket.bind((HOST, PORT))
# This is the client thread allowing multiple clients to connect
class client(Thread):
# This section is what will run in the background
def background():
while True:
cs.sendto(b'\x55\xaa\x00\x40\x02\x00\x0b\x49\x20\x41\x4d\x20\x53\x45\x52\x56\x45\x52\x04\x3a', ("192.168.1.255", 1301))
time.sleep(5)
# This section is what will run in the foreground
def __init__(self, socket, address):
Thread.__init__(self)
self.sock = socket
self.addr = address
self.start()
def run(self):
while True:
msg = (b'\x55\xaa\x01\x03\x02\x00\x00\x01\x05')
self.sock.send(msg)
#Open recv.txt file in write mode
recvdatafile1 = open("recv1.txt", "wb")
print("Destination file name will be recv1.txt on the server\n")
# Receive intiate data from client side
# RecvData = self.sock.recv(1024)
while True:
recvdatafile1.write(RecvData)
# recvdatafile1.write(self.sock.recv(1024))
RecvData = self.sock.recv(1024)
# Request initial configuration information (replies with S/N, model number, etc
#msg = (b'\x55\xaa\x01\x03\x02\x00\x00\x01\x05')
#conn.send(msg)
# Close the file opened at server side once copy is completed
recvdatafile1.close()
print("\n File has been copied successfully \n")
#Open recv.txt file in write mode
recvdatafile2 = open("recv2.txt", "wb")
print("Destination file name will be recv2.txt on the server\n")
# Request PV data
msg = (b'\x55\xaa\x01\x02\x02\x00\x00\x01\x04')
conn.send(msg)
# Receive any data from client side
RecvData = conn.recv(1024)
while RecvData:
recvdatafile2.write(RecvData)
RecvData = conn.recv(1024)
# Close the file opened at server side once copy is completed
recvdatafile2.close()
print("\n File has been copied successfully \n")
# Close connection with client
conn.close()
print("\n Server closed the connection \n")
time.sleep(3)
# Come out from the infinite while loop as the file has been copied from client.
#break
serversocket.listen(5)
print("Server is listing on port:", PORT, "\n")
while 1:
clientsocket, address = serversocket.accept()
client(clientsocket, address)
print ("connection found!")
data = clientsocket.recv(1024).decode()
print (data)
b = threading.Thread(name='background', target=background)
f = threading.Thread(name='foreground', target=foreground)
b.start()
f.start()
Server Output:
user#server:~# python3 ./Samil_Solar_River_Monitor_Server.py
Server is listing on port: 1200
connection found!
Destination file name will be recv1.txt on the server
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "./Samil_Solar_River_Monitor_Server.py", line 62, in run
recvdatafile1.write(RecvData)
UnboundLocalError: local variable 'RecvData' referenced before assignment
Traceback (most recent call last):
File "./Samil_Solar_River_Monitor_Server.py", line 109, in <module>
data = clientsocket.recv(1024).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaa in position 1: invalid start byte
user#server:~#
Client Output:
user#client:~$ python3 test-solar-client.py 192.168.1.11
################## Below rcved from srv ##################
b'U\xaa\x01\x03\x02\x00\x00\x01\x05' <-- This is good/correct
################## Below rcved from srv ##################
b'' <-- this & below appears when traceback on server occurs
Traceback (most recent call last):
File "test-solar-client.py", line 47, in <module>
s.send(SendData)
BrokenPipeError: [Errno 32] Broken pipe
user#client:~$

Related

Python, TCP, Server and client

I have problem with TCP (Server and Client.) over python3.
What does i do wrong?
my code is in TCP Server:
import socket
server = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ip = socket.gethostbyname("127.0.0.1")
port = 8892
address=(ip,port)
server.bind(address)
server.listen(1)
print("[*] Started listening on" , ip, ":", port)
client,addr = server.accept()
print("[*] Got Connection from", addr[0], ":", addr[1])
while True:
data = client.recv(1024)
data = data.decode('utf-8')
print("[*] Received ", data, " from the client")
print("[*] Processing data")
if data == "hello":
client.send("Hello client")
print("[*] Processing done\n[*] Reply sent")
elif data == "disconnect":
client.send("Goodbye")
client.close()
break
else:
client.send("Invalid type")
print("Processing done Invalid data \n[*] Reply sent")
my code is in TCP Client:
import socket
client=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ip = socket.gethostbyname("127.0.0.1")
print(ip)
port=8892
address=(ip,port)
client.connect(address)
while True:
message = input()
client.send(message.encode('utf-8'))
print(client.recv(1024))
Erorr message:
Traceback (most recent call last): File "tcpServer.py", line 25, in
client.send("Invalid type") TypeError: a bytes-like object is required, not 'str'
Can some one help me please, what can i do?
Thanks for helping me. :)
in python 3 you are no more sending/reading characters or string, but byte-array (or bytes).
You might also need to manage encoding (utf-8 <-> bytes).
ex:
>>> s = "énorme"
>>> b = s.encode("utf-8")
>>> b
b'\xc3\xa9norme'
and use the bytes.decode() to reverse it.
For your very trivial case, you can use direcly bytes string as b"hello", as plain ASCII string are converted "as-is" to bytes.
client.send("Hello client") will become client.send(b"Hello client")
But don't go too deeper with this solution.
I personally allow this usage when using commands to talk to some hardware, but absolutely not for a chat or text exchange.

Sending Bytes properly to client

In Python, I am trying to build a simple server/client application that sends video bytes to the client.
I have been trying to figure this out for months.
The server is listening to a connection, and is receiving this much data every second, length of 1316
for example:
data = sock_reciever.recv(16000000)
If I print data, I get this:
b'G\x05\x14\x1e\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x14\x1f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x14\x10\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x14\x11\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x14\x12\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x14\x13\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xffG\x05\x17\x12\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\r\xff\xe0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
As this comes in, I can send it back out via sock_sender.sendto(data, (SENDMCAST_GRP, SENDMCAST_PORT)), and watch the video in VLC player.
Ultimately, I want to take this data and send it to the client through the socket via conn.send(data).
However, on the client side, I believe data is coming in in the wrong order.
I've compared the data between server and client and it appears to be the same, but it builds up on the client side, which might be causing some sort of overflow.
On the client side, I take the data coming in with data = sock_reciever.recv(16000000) and send it back out via sock_sender.sendto(data, (SENDMCAST_GRP, SENDMCAST_PORT)).
But I do not get the same results on the client side, as on the server side.
How do I add headers to this data to ensure it arrives to the client in sequential order?
If possible, please explain this to me like you would explain what oxygen is to a fish.
Which Sockets I Am Using
I am trying to use TCP sockets between server and client. I thought using a stream socket would be more reliable, but I might be wrong, the only reason being that I would like to do a reverse connection, so that the client connects to the server and then receives data.
I am also using sock_sender.sendto(data, (SENDMCAST_GRP, SENDMCAST_PORT)) just to relay the incoming data on the client side, it is not the actual connection between server and client.
The socket used for the communication between server/client
is called sock_client.
This is the server-side code:
sock_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# recieves connection from client
conn, address = sock_client.accept()
# sends data to client
conn.send(data)
Client Side is set up like so:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
data = s.recv(2400000)
Then the data is just relayed back out for use with other applications via sock_sender.sendto(data, (SENDMCAST_GRP, SENDMCAST_PORT)).
I need to ensure the data between the client/server socket connection is accurate and complete.
Let me explain it a little bit. The video can be sent in the form of bytes with the aid of struct. It is part of the python standard library string services.
It interprets strings as packed binary data and performs conversions between Python values and C structs represented as Python strings. This is helpful handling binary data in files, socket network connections and other sources. In server.py, the frame matrix is pickled dump as a string in variable a and then used struct.pack to add a reference payloadsize (for the string in a). The payload_size is attached using + to the a and then .sendall sends the whole frame. To simply explain it, lets suppose our frame is just 3x3 matrix:
frame=np.array([[1,2,3],[4,5,6],[7,8,9]])
a = pickle.dumps(frame)
The pickled form of frame is shown below as a string:
b'\x80\x03cnumpy.core.multiarray\n_reconstruct\nq\x00cnumpy\nndarray\nq\
x01K\x00\x85q\x02C\x01bq\x03\x87q\x04Rq\x05(K\x01K\x03K\x03\x86q\x06cnumpy
\ndtype\nq\x07X\x02\x00\x00\x00i4q\x08K\x00K\x01\x87q\tRq\n(K\x03X\x01\x00
\x00\x00
Here payload_size: len(a) is 193 and its string representation in a 4 byte 'L' format is b'\xc1\x00\x00\x00'.
Then, message = struct.pack("L", len(a))+a
b'\xc1\x00\x00\x00\x80\x03cnumpy.core.multiarray\n_reconstruct\nq\x00cnumpy\nndarray\nq\
x01K\x00\x85q\x02C\x01bq\x03\x87q\x04Rq\x05(K\x01K\x03K\x03\x86q\x06cnumpy
\ndtype\nq\x07X\x02\x00\x00\x00i4q\x08K\x00K\x01\x87q\tRq\n(K\x03X\x01\x00
\x00\x00
And now the message also contains the payload_size infront of the data. The len(message) = 197.
In client.py the s.recv(4096) means at the most 4096 bytes will be received through a blocking call.
The data is appended to the string in 4096 bytes per packet. The rest of the part is quite obvious.
For reference working versions of server.py and client.py are below. First run the server.py and then in another command window run client.py. The transmitted and received frames will be displayed. You can increase the number of maximum bytes. I hope this solves your problem!
server.py
import socket,cv2,pickle
import struct
serversocket = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 9999
serversocket.bind((host, port))
serversocket.listen(5)
while True:
clientsocket,addr = serversocket.accept()
print("Got a connection from %s" % str(addr))
if clientsocket:
vid = cv2.VideoCapture(0)
while(vid.isOpened()):
img, frame = vid.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
print(frame)
a = pickle.dumps(frame)
message =struct.pack("L", len(a))+a
clientsocket.sendall(message)
cv2.imshow('Transmitted',frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
clientsocket.close()
client.py
import socket,cv2,pickle
import struct
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = socket.gethostname()
port = 9999
s.connect((host, port))
data = b""
payload_size = struct.calcsize("L")
while True:
while len(data) < payload_size:
packet= s.recv(4096)
if not packet: break
data += packet
packed_msg_size = data[:payload_size]
data = data[payload_size:]
msg_size = struct.unpack("L", packed_msg_size)[0]
while len(data) < msg_size:
data += s.recv(4096)
frame_data = data[:msg_size]
data = data[msg_size:]
frame=pickle.loads(frame_data)
cv2.imshow('Received',frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
s.close()

Close a HTTP Proxy Server

I have a little problem with my code, I'm doing a HTTP Proxy Server and I send it a random number of HTTP Request and I want that my program close when I stop of send.
I think the problem is in the accept because the program still working always
I tried to put a recv after the accept for checking if there if empty but the program does't arrive there
My code is the following
from socket import *
from _thread import *
MAX_DATA_RECV = 4096 # max number of bytes we receive at once
def start(port_5, my_port):
s=socket(AF_INET, SOCK_STREAM)
s.bind(('', my_port))
s.listen(1)
while 1:
try:
conn, client_addr = s.accept()
except KeyboardInterrupt:
print('\nProgram closed. Interrupted by the user')
exit()
proxy_thread(conn, client_addr)
s.close()
def proxy_thread(conn, client_addr):
# get the request from browser
request = conn.recv(MAX_DATA_RECV).decode('utf-8')
# parse the first line
first_line = request.split('n')[0]
# get url
url = first_line.split(' ')[1]
# find the webserver and port
http_pos = url.find("://") # find pos of ://
if (http_pos==-1):
temp = url
else:
temp = url[(http_pos+3):] # get the rest of url
port_pos = temp.find(":") # find the port pos (if any)
# find end of web server
webserver_pos = temp.find("/")
if webserver_pos == -1:
webserver_pos = len(temp)
webserver = ""
port = -1
if (port_pos==-1 or webserver_pos < port_pos): # default port
port = 80
webserver = temp[:webserver_pos]
else: # specific port
port = int((temp[(port_pos+1):])[:webserver_pos-port_pos-1])
webserver = temp[:port_pos]
print("Connect to:", webserver, port)
# create a socket to connect to the web server
s = socket(AF_INET, SOCK_STREAM)
s.connect((webserver, port))
s.send(request.encode()) # send request to webserver
print(temp)
while 1:
# receive data from web server
data = s.recv(MAX_DATA_RECV)
if (len(data) > 0):
# send to browser
conn.send(data)
else:
break
s.close()
conn.close()
If someone is able to help me, thanks in advance
I want that my program close when I stop of send
At the network level there is no implicit indicator that the other side will not send anymore. If all requests would be done through a single TCP connection then the end of the connection might be treated as such an indicator. But you are using a new TCP connection for every request so you need to define your own condition(s) how the server should determine that the client will not send anymore.
This could for example a timeout, i.e. if the client has not sent any more request for 20 seconds then the client is treated as dead. Or it might be a special message by the client to signal the end - in which case your code needs to explicitly look for this message.

Python36 and sockets

So I'm connected to an IRC chat using socket.connec
I log in by passing my variables via socket.send
The log in is successful and then I sit in a while true loop using
Socket.recv(1024)
If I just continually print the response everything looks fine, but let's say I want to add to the end of the string... I noticed that socket.recv doesn't always get the full message (only grabs up to 1024 as expected) and the remainder of the message is in the next iteration of the loop.
This makes its it impossible to process the feedback line by line.
Is there a better way to constantly read the data without it getting trunked? Is it possible to figure out the size of the response before receiving it so the buffer can be set dynamically?
TCP is a stream-based protocol. Buffer the bytes received and only extract complete messages from the stream.
For complete lines, look for newline characters in the buffer.
Example server:
import socket
class Client:
def __init__(self,socket):
self.socket = socket
self.buffer = b''
def getline(self):
# if there is no complete line in buffer,
# add to buffer until there is one.
while b'\n' not in self.buffer:
data = self.socket.recv(1024)
if not data:
# socket was closed
return ''
self.buffer += data
# break the buffer on the first newline.
# note: partition(n) return "left of n","n","right of n"
line,newline,self.buffer = self.buffer.partition(b'\n')
return line + newline
srv = socket.socket()
srv.bind(('',5000))
srv.listen(1)
conn,where = srv.accept()
client = Client(conn)
print(f'Client connected on {where}')
while True:
line = client.getline()
if not line:
break
print(line)
Example client:
s=socket()
s.connect(('127.0.0.1',5000))
s.sendall(b'line one\nline two\nline three\nincomplete')
s.close()
Output on server:
Client connected on ('127.0.0.1', 2667)
b'line one\n'
b'line two\n'
b'line three\n'

Python 3.4 - How to 'run' another script python script continuously, How to pass http get / post to socket

This question is two-fold.
1. So I need to run code for a socket server that's all defined and created in another.py, Clicking run on PyCharm works just fine, but if you exec() the file it just runs the bottom part of the code.
There are a few answers here but they are conflicting and for Python 2.
From what I can gather there are three ways:
- Execfile(), Which I think is Python 2 code.
- os.system() (But I've seen it be said that it's not correct to pass to the OS for this)
- And subprocess.Popen (unsure how to use this either)
I need this to run in the background, it is used to create threads for sockets for the recv portion of the overall program and listen on those ports so I can input commands to a router.
This is the complete code in question:
import sys
import socket
import threading
import time
QUIT = False
class ClientThread(threading.Thread): # Class that implements the client threads in this server
def __init__(self, client_sock): # Initialize the object, save the socket that this thread will use.
threading.Thread.__init__(self)
self.client = client_sock
def run(self): # Thread's main loop. Once this function returns, the thread is finished and dies.
global QUIT # Need to declare QUIT as global, since the method can change it
done = False
cmd = self.readline() # Read data from the socket and process it
while not done:
if 'quit' == cmd:
self.writeline('Ok, bye. Server shut down')
QUIT = True
done = True
elif 'bye' == cmd:
self.writeline('Ok, bye. Thread closed')
done = True
else:
self.writeline(self.name)
cmd = self.readline()
self.client.close() # Make sure socket is closed when we're done with it
return
def readline(self): # Helper function, read up to 1024 chars from the socket, and returns them as a string
result = self.client.recv(1024)
if result is not None: # All letters in lower case and without and end of line markers
result = result.strip().lower().decode('ascii')
return result
def writeline(self, text): # Helper func, writes the given string to the socket with and end of line marker at end
self.client.send(text.strip().encode("ascii") + b'\n')
class Server: # Server class. Opens up a socket and listens for incoming connections.
def __init__(self): # Every time a new connection arrives, new thread object is created and
self.sock = None # defers the processing of the connection to it
self.thread_list = []
def run(self): # Server main loop: Creates the server (incoming) socket, listens > creates thread to handle it
all_good = False
try_count = 0 # Attempt to open the socket
while not all_good:
if 3 < try_count: # Tried more than 3 times without success, maybe post is in use by another program
sys.exit(1)
try:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Create the socket
port = 80
self.sock.bind(('127.0.0.1', port)) # Bind to the interface and port we want to listen on
self.sock.listen(5)
all_good = True
break
except socket.error:
print('Socket connection error... Waiting 10 seconds to retry.')
del self.sock
time.sleep(10)
try_count += 1
print('Server is listening for incoming connections.')
print('Try to connect through the command line with:')
print('telnet localhost 80')
print('and then type whatever you want.')
print()
print("typing 'bye' finishes the thread. but not the server",)
print("eg. you can quit telnet, run it again and get a different ",)
print("thread name")
print("typing 'quit' finishes the server")
try:
while not QUIT:
try:
self.sock.settimeout(0.500)
client = self.sock.accept()[0]
except socket.timeout:
time.sleep(1)
if QUIT:
print('Received quit command. Shutting down...')
break
continue
new_thread = ClientThread(client)
print('Incoming Connection. Started thread ',)
print(new_thread.getName())
self.thread_list.append(new_thread)
new_thread.start()
for thread in self.thread_list:
if not thread.isAlive():
self.thread_list.remove(thread)
thread.join()
except KeyboardInterrupt:
print('Ctrl+C pressed... Shutting Down')
except Exception as err:
print('Exception caught: %s\nClosing...' % err)
for thread in self.thread_list:
thread.join(1.0)
self.sock.close()
if "__main__" == __name__:
server = Server()
server.run()
print('Terminated')
Notes:
This is created in Python 3.4
I use Pycharm as my IDE.
One part of a whole.
2. So I'm creating a lightning detection system and this is how I expect it to be done:
- Listen to the port on the router forever
The above is done, but the issue with this is described in question 1.
- Pull numbers from a text file for sending text message
Completed this also.
- Send http get / post to port on the router
The issue with this is that i'm unsure how the router will act if I send this in binary form, I suspect it wont matter, the input commands for sending over GSM are specific. Some clarification may be needed at some point.
- Recieve reply from router and exception manage
- Listen for relay trip for alarm on severe or close strike warning.
- If tripped, send messages to phones in storage from text file
This would be the http get / post that's sent.
- Wait for reply from router to indicate messages have been sent, exception handle if it's not the case
- Go back to start
There are a few issues I'd like some background knowledge on that is proving hard to find via the old Google and here on the answers in stack.
How do I grab the receive data from the router from another process running in another file? I guess I can write into a text file and call that data but i'd rather not.
How to multi-process and which method to use.
How to send http get / post to socket on router, post needed occording to the router manual is as follows: e.g. "http://192.168.1.1/cgi-bin/sms_send?number=0037061212345&text=test"
Notes: Using Sockets, threading, sys and time on Python 3.4/Pycharm IDE.
Lightning detector used is LD-250 with RLO Relay attached.
RUT500 Teltonica router used.
Any direction/comments, errors spotted, anything i'm drastically missing would be greatly appreciated! Thank you very much in advance :D constructive criticism is greatly encouraged!
Okay so for the first part none of those suggested in the OP were my answer. Running the script as is from os.system(), exec() without declaring a new socket object just ran from __name__, this essentially just printed out "terminated", to get around this was simple. As everything was put into a classes already, all I had to do is create a new thread. This is how it was done:
import Socketthread2
new_thread = Socketthread2.Server() # Effectively declaring a new server class object.
new_thread.run()
This allowed the script to run from the beginning by initialising the code from the start in Socket, which is also a class of Clientthread, so that was also run too. Running this at the start of the parent program allowed this to run in the background, then continue with the new code in parent while the rest of the script was continuously active.

Resources