I am writing a messenger application in python and I have a problem. The problem is quite simple: I want the program to only receive data from the other computer if it was sent, otherwise, my program would wait for a data transfer infinitely. How would I write that piece of code? I imagine it'd be something like this:
try:
data = s.recv(1024).decode()
except:
data == None
See the select module. A socket can be monitored for readability with a timeout, so other process can proceed.
Example server:
import socket
import select
with socket.socket() as server:
server.bind(('',5000))
server.listen(3)
to_read = [server] # add server to list of readable sockets.
clients = {}
while True:
# check for a connection to the server or data ready from clients.
# readers will be empty on timeout.
readers,_,_ = select.select(to_read,[],[],0.5)
for reader in readers:
if reader is server:
client,address = reader.accept()
print('connected',address)
clients[client] = address # store address of client in dict
to_read.append(client) # add client to list of readable sockets
else:
# Simplified, really need a message protocol here.
# For example, could receive a partial UTF-8 encoded sequence.
data = reader.recv(1024)
if not data: # No data indicates disconnect
print('disconnected',clients[reader])
to_read.remove(reader) # remove from monitoring
del clients[reader] # remove from dict as well
else:
print(clients[reader],data.decode())
print('.',flush=True,end='')
A simple client, assuming your IP address is 1.2.3.4.
import socket
s = socket.socket()
s.connect(('1.2.3.4',5000))
s.sendall('hello'.encode())
s.close()
Related
In an ambitious attempt making a VNC with Python, I am trying to continuously send screenshots of one user (Server) to another (Client). After hours of trying and hitting Ballmer's peak, I managed to do it. However, now my problem is sending multiple images, a continuous stream of them. I first tried to write all the binary data to one file, which didn't work. When the second image was opened, it crashed. I thought this might be because the binary data somehow got corrupted, so instead I tried making a new file for every image, yet I have the same problem. I know that Tcp is a constant stream of data so that it would be hard to know the end of the first image and start of the next, but by creating another file, I thought I would be all good.
Any help in fixing this and/or increasing the efficiency of this is greatly appreciated :)
Server side:
import socket
from PIL import Image, ImageGrab
PORT = 10007
HOST = '127.0.0.1'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
counter = 3
while counter > 0:
image = ImageGrab.grab(bbox=None)
image.save('test.png')
f = open('test.png', 'rb')
l = f.read(1024)
while (l):
conn.send(l)
l = f.read(1024)
f.close()
print('Done sending curr image')
counter -= 1
conn.close()
Client side:
import socket
from PIL import Image
HOST = '127.0.0.1'
PORT = 10007
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
counter = 1
while counter != 3:
fname = 'image' + str(counter) + '.png'
with open(fname, 'wb') as file:
print('file opened')
while True:
data = s.recv(1024)
if not data:
break
else:
file.write(data)
file.close()
currImg = Image.open(fname)
currImg.show()
counter += 1
s.close()
Your receiver does not know when one file finishes and the next begins. The easiest way to fix that is to send the length of the file (perhaps as a 4-byte unsigned value) to the receiver before sending each file. Then the receiver can read the length, read the file, read the length, read the file, ...
To improve efficiency you can stop saving the file data into an actual file at both ends and instead save it into (and, obviously, read it from) an in-memory buffer. See this answer for explanations of how to do that. In Python 3 it looks like you would use the BytesIO module.
Another improvement would be to only send image data for the parts of the screen that have changed since the previous send. For that you'll need to figure out how to compare the current capture against the previous one. For a first pass you could use PIL.ImageChops.difference followed by PIL.Image.getbbox and then encode and send only that region of the current capture. For that to work, the sender will have to tell the receiver not only the size of the PNG but also the location in the output screen image where the new image patch should be painted. So you'll want to send a position in addition to the size and the encoded image data.
I recently finished a project using a mix of Django and Twisted and realized it's overkill for what I need which is basically just a way for my servers to communicate via TCP sockets. I turned to Trio and so far I'm liking what I see as it's way more direct (for what I need). That said though, I just wanted to be sure I was doing this the right way.
I followed the tutorial which taught the basics but I need a server that could handle multiple clients at once. To this end, I came up with the following code
import trio
from itertools import count
PORT = 12345
BUFSIZE = 16384
CONNECTION_COUNTER = count()
class ServerProtocol:
def __init__(self, server_stream):
self.ident = next(CONNECTION_COUNTER)
self.stream = server_stream
async def listen(self):
while True:
data = await self.stream.receive_some(BUFSIZE)
if data:
print('{} Received\t {}'.format(self.ident, data))
# Process data here
class Server:
def __init__(self):
self.protocols = []
async def receive_connection(self, server_stream):
sp: ServerProtocol = ServerProtocol(server_stream)
self.protocols.append(sp)
await sp.listen()
async def main():
await trio.serve_tcp(Server().receive_connection, PORT)
trio.run(main)
My issue here seems to be that each ServerProtocol runs listen on every cycle instead of waiting for data to be available to be received.
I get the feeling I'm using Trio wrong in which case, is there a Trio best practices that I'm missing?
Your overall structure looks fine to me. The issue that jumps out at me is:
while True:
data = await self.stream.receive_some(BUFSIZE)
if data:
print('{} Received\t {}'.format(self.ident, data))
# Process data here
The guarantee that receive_some makes is: if the other side has closed the connection already, then it immediately returns an empty byte-string. Otherwise, it waits until there is some data to return, and then returns it as a non-empty byte-string.
So your code should work fine... until the other end closes the connection. Then it starts doing an infinite loop, where it keeps checking for data, getting an empty byte-string back (data = b""), so the if data: ... block doesn't run, and it immediately loops around to do it again.
One way to fix this would be (last 3 lines are new):
while True:
data = await self.stream.receive_some(BUFSIZE)
if data:
print('{} Received\t {}'.format(self.ident, data))
# Process data here
else:
# Other side has gone away
break
What I’m Trying to Do:
I’m trying to create a client script that can listen for potential messages from the server and receive input from a user at any point that tells the script to send a message to the server all on a single socket. The server I’m working with will only connect to one client at a time, so everything is on one socket.
The script will facilitate interaction between a maya ui and a file with prewritten functions that determine what to send to the server.
How it works:
I have the script with two threads. The parent thread is the adding messages thread and there is a child thread that runs the listening in the background. The child thread has a constant background loop, listening for any messages from the server (for example an error message) and reads a message queue to see if anything has been added. If something is added to the queue, the listening loop stops, sends a message, then starts the listening loop again. The parent thread allows the user to add a command into the message queue using the add_message() attribute. The maya ui will have buttons that call functions to add commands to the message queue. I made this with python 2.7
Here is an simplified example of the client. I added a single message before starting the listening loop so you can see what it is supposed to look like.
import socket
import struct
import threading
import Queue
import time
class ThreadedClient(threading.Thread):
def __init__(self, host, port):
threading.Thread.__init__(self)
#set up queues
self.send_q = Queue.Queue(maxsize = 10)
#declare instance variables
self.host = host
self.port = port
#connect to socket
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.connect((self.host, self.port))
#LISTEN
def listen(self):
while True: #loop forever
try:
print 'checking for message...'
# stops listening if there's a message to send
if self.send_q.empty() == False:
self.send_message()
else:
print 'no message'
print 'listening...'
message = self.s.recv(4096)
print 'RECEIVED: ' + message
except socket.timeout:
pass
def start_listen(self):
t = threading.Thread(target = self.listen())
t.start()
#t.join()
print 'started listen'
#ADD MESSAGE
def add_message(self, msg):
#put message into the send queue
self.send_q.put(msg)
print 'ADDED MSG: ' + msg
#self.send_message()
#SEND MESSAGE
def send_message(self):
#send message
msg = self.get_send_q()
if msg == "empty!":
print "nothing to send"
else:
self.s.sendall(msg)
print 'SENDING: ' + msg
#restart the listening
#self.start_listen()
#SAFE QUEUE READING
#if nothing in q, prints "empty" instead of stalling program
def get_send_q(self):
if self.send_q.empty():
print "empty!"
return "empty!"
else:
msg = self.send_q.get()
return msg
if __name__ == '__main__':
port = 8001
address = 'localhost'
s = ThreadedClient(address, port)
s.start()
print('Server started, port: ', port)
s.add_message('hello world')
s.start_listen()
s.add_message('hello world')
And here is an example server for the client:
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 8001)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
# Receive the data in small chunks and retransmit it
while True:
data = connection.recv(16)
print >>sys.stderr, 'received "%s"' % data
if data:
print >>sys.stderr, 'sending data back to the client'
connection.sendall(data)
else:
print >>sys.stderr, 'no more data from', client_address
break
finally:
# Clean up the connection
connection.close()
Problem:
Once I start the listening loop thread, the script will no longer take any more input. Just as a test I added a line after starting the listening thread that would add a message to the queue and nothing happens.
Previous Approaches:
This question is essentially exactly what I’m trying to do as well, although it was never solved: How to handle chat client using threading and queues?
I have looked at Grab user input asynchronously and pass to an Event loop in python about creating a main loop that accepts user input and have tried implementing the queue system but am getting stuck.
I’ve also tried out How to use threading to get user input realtime while main still running in python approach but I can’t use raw_input() for my final usage of the code. I have also tried socket.timeout, but also loops without taking input.
One approach I was considering was making the script asynchronous, but from what I’ve read I don’t think it will resolve the issue.
TL/DR:
Is there a way to create a script that has a client listening to the server loop running in the background while at the same time being able to accept real time commands from the user?
I would greatly appreciate any help or nudge in the right direction, I've been stuck on this a while now.
The script works. There were just some errors throwing it off. In the start_listen() function
t = threading.Thread(target = self.listen())
should be
t = threading.Thread(target = self.listen)
and then in the __init__() after
self.s.connect((self.host, self.port))
add
self.s.settimeout(.1)
so that the script will cycle through the while loop.
I'm having difficulty understanding the behaviour of my altered echo server, which attempts to take advantage of python 3's asyncio module.
Essentially I have an infinite loop (lets say I want to stream some data from the server to the client indefinitely whilst the connection has been made) e.g. MyServer.py:
#! /usr/bin/python3
import asyncio
import os
import time
class MyProtocol(asyncio.Protocol):
def connection_made(self, transport):
peername = transport.get_extra_info('peername')
print('Connection from {}'.format(peername))
self.transport = transport
def connection_lost(self, exc):
asyncio.get_event_loop().stop()
def data_received(self, data):
i = 0
while True:
self.transport.write(b'>> %i' %i)
time.sleep(2)
i+=1
loop = asyncio.get_event_loop()
coro = loop.create_server(MyProtocol,
os.environ.get('MY_SERVICE_ADDRESS', 'localhost'),
os.environ.get('MY_SERVICE_PORT', 8100))
server = loop.run_until_complete(coro)
try:
loop.run_forever()
except:
loop.run_until_complete(server.wait_closed())
finally:
loop.close()
Next when I connect with nc ::1 8100 and send some text (e.g. "testing") I get the following:
user#machine$ nc ::1 8100
*** Connection from('::1', 58503, 0, 0) ***
testing
>> 1
>> 2
>> 3
^C
Now when I attempt to connect using nc again, I do not get any welcome message and after I attempt to send some new text to the server I get an endless stream of the following error:
user#machine$ nc ::1 8100
Is there anybody out there?
socket.send() raised exception
socket.send() raised exception
...
^C
Just to add salt to the wound the socket.send() raised exception message continues to spam my terminal until I kill the python server process...
As I'm new to web technologies (been a desktop dinosaur for far too long!), I'm not sure why I am getting the above behaviour and I haven't got a clue on how to produce the intended behaviour, which loosely looks like this:
server starts
client 1 connects to server
server sends welcome message to client
4 client 1 sends an arbitrary message
server sends messages back to client 1 for as long as the client is connected
client 1 disconnects (lets say the cable is pulled out)
client 2 connects to server
Repeat steps 3-6 for client 2
Any enlightenment would be extremely welcome!
There are multiple problems with the code.
First and foremost, data_received never returns. At the transport/protocol level, asyncio programming is single-threaded and callback-based. Application code is scattered across callbacks like data_received, and the event loop runs the show, monitoring file descriptors and invoking the callbacks as needed. Each callback is only allowed to perform a short calculation, invoke methods on transport, and arrange for further callbacks to be executed. What the callback cannot do is take a lot of time to complete or block waiting for something. A while loop that never exits is especially bad because it doesn't allow the event loop to run at all.
This is why the code only spits out exceptions once the client disconnects: connection_lost is never called. It's supposed to be called by the event loop, and the never-returning data_received is not giving the event loop a chance to resume. With the event loop blocked, the program is unable to respond to other clients, and data_received keeps trying to send data to the disconnected client, and logs its failure to do so.
The correct way to express the idea can look like this:
def data_received(self, data):
self.i = 0
loop.call_soon(self.write_to_client)
def write_to_client(self):
self.transport.write(b'>> %i' % self.i)
self.i += 1
loop.call_later(2, self.write_to_client)
Note how both data_received and write_to_client do very little work and quickly return. No calls to time.sleep(), and definitely no infinite loops - the "loop" is hidden inside the kind-of-recursive call to write_to_client.
This change reveals the second problem in the code. Its MyProtocol.connection_lost stops the whole event loop and exits the program. This renders the program unable to respond to the second client. The fix could be to replace loop.stop() with setting a flag in connection_lost:
def data_received(self, data):
self._done = False
self.i = 0
loop.call_soon(self.write_to_client)
def write_to_client(self):
if self._done:
return
self.transport.write(b'>> %i' % self.i)
self.i += 1
loop.call_later(2, self.write_to_client)
def connection_lost(self, exc):
self._done = True
This allows multiple clients to connect.
Unrelated to the above issues, the callback-based code is a bit tiresome to write, especially when taking into account complicated code paths and exception handling. (Imagine trying to express nested loops with callbacks, or propagating an exception occurring inside a deeply embedded callback.) asyncio supports coroutines-based streams as alternative to callback-based transports and protocols.
Coroutines allow writing natural-looking code that contains loops and looks like it contains blocking calls, which under the hood are converted into suspension points that enable the event loop to resume. Using streams the code from the question would look like this:
async def talk_to_client(reader, writer):
peername = writer.get_extra_info('peername')
print('Connection from {}'.format(peername))
data = await reader.read(1024)
i = 0
while True:
writer.write(b'>> %i' % i)
await writer.drain()
await asyncio.sleep(2)
i += 1
loop = asyncio.get_event_loop()
coro = asyncio.start_server(talk_to_client,
os.environ.get('MY_SERVICE_ADDRESS', 'localhost'),
os.environ.get('MY_SERVICE_PORT', 8100))
server = loop.run_until_complete(coro)
loop.run_forever()
talk_to_client looks very much like the original implementation of data_received, but without the drawbacks. At each point where it uses await the event loop is resumed if the data is not available. time.sleep(n) is replaced with await asyncio.sleep(n) which does the equivalent of loop.call_later(n, <resume current coroutine>). Awaiting writer.drain() ensures that the coroutine pauses when the peer cannot process the output it gets, and that it raises an exception when the peer has disconnected.
I am trying to send some data from android to python server then want to run some functions and send the data back to android. Android code is working fine, but my program ignore the lines where I am creating new variables and sending the data. I am using python3.
The particular line is num = random.randint(1,100)
serverSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
serverSock.bind(("", UDP_PORT))
def userrec(addr):
num = random.randint(1,100)
result = num.encode("utf-8")
serverSock.sendto(result, addr)
while True:
data, addr = serverSock.recvfrom(1024)
message = data.decode("utf-8")
if data:
userrec(addr)
serverSock.close()
Can't we create new variables or run some function after recieving the data?
Found the solution. I just have to remove ServerSock.close()