I trying to do a simple server. The server has to load data while client provides it. I want to use non-blocking 'recv' command, but whatever I do it still blocks. My code is above.
import socket
import fcntl
import os
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("localhost", 8888))
server_socket.listen()
client_socket, addr = server_socket.accept()
#fcntl.fcntl(client_socket, fcntl.F_SETFL, os.O_NONBLOCK) #mark1
client_socket.setblocking(False) #mark2
#client_socket.settimeout(0.0) #mark3
while True:
data = client_socket.recv(10)
if len(data) != 0:
print(data.decode("UTF8"))
else:
break
print("Exit")
I tried to use lines '#mark1', '#mark2', '#mark3'. These lines seem to be pretty good for me, but my program is still stuck on the second iteration with data = client_socket.recv(10) line.
Why do client_socket.setblocking(False) and others not affect?
Related
After a lot of trials with httpio, requests, socket I'm seeing that no matter how I approach the solution it still behaves like a synchronous program even with an async function. the code is -
import socket
import time
import asyncio
t1 = time.time()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
target = '72.221.171.130'
#async socket to connect to the target with a given port
async def connect(port):
try:
s.connect((target, port))
print(port)
except Exception as e:
print(e)
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait([connect(port) for port in range(4100, 4150)]))
#printing time taken to scan ports two decimal places
print('Time taken to scan ports:', round((time.time() - t1),3), 'seconds')
I looking for the open port which is 4145. but this takes around 20 seconds to execute. I'm a beginner in async programming so I would really appreciate your help.
Here is my problem : I have 2 programs communicating thanks to zmq on an arbitrary tcp port.
When the #1 receives message from #2 he has to call some function.
If #1 receives a message before the current function ends, I'd like #1 to interrupt the current function and call the new one.
I tried to use threading.Event to interrupt function.
I don't know if zmq is the right option for my needs or if the socket types fine.
To simplify I show the simplest version possible,here is what I tried :
p1.py
import zmq
from threading import Event
port_p2 = "6655"
context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.connect("tcp://localhost:%s" % port_p2)
print("port 6655")
__exit1 = Event()
__exit2 = Event()
def action1():
__exit1.clear()
__exit2.set()
while not __exit1.is_set():
for i in range(1, 20):
print(i)
time.sleep(1)
__exit1.set()
def action2():
__exit2.clear()
__exit1.set()
while not __exit2.is_set():
for i in range(1, 20):
print(i * 100)
time.sleep(1)
__exit2.set()
if __name__ == "__main__":
try:
while True:
try:
string = socket.recv(flags=zmq.NOBLOCK)
# message received, process it
string = str(string, 'utf-8')
if "Action1" in string:
action1()
if "Action2" in string:
action2()
except zmq.Again as e:
# No messages waiting to be processed
pass
time.sleep(0.1)
except(KeyboardInterrupt, SystemExit):
print("exit")
and p2.py
import time
import random
port_p1 = "6655"
context = zmq.Context()
socket_p1 = context.socket(zmq.PAIR)
socket_p1.bind("tcp://*:%s" % port_p1)
print("port 6655")
if __name__ == "__main__":
while True:
i = random.choice(range(1, 10))
print(i)
try:
if random.choice([True, False]):
print("Action 1")
socket_p1.send(b'Action1')
else:
socket_p1.send(b'Action2')
print("Action 2")
except zmq.Again as e:
pass
time.sleep(i)
For my purpose I didn't want / can't use system signals
I'd appreciate any input and don't hesitate to ask for precision, I have to confess that I had trouble writing this down.
Thank you
Q : …like #1 to interrupt the current function…
Given you have forbidden to use signals, #1 can but passively signal (be it over the present ZeroMQ infrastructure or not) the function, not to continue further and return in a pre-mature fashion ( so the fun() has to get suitably modified for doing that active re-checking, best in some reasonably granular progressive fashion, regularly checking actively the #1 if did passively signal ( "tell" the fun() ) to RET early, due to whatever reason and way the #1 had and used to do that ).
The other chance is to extend the already present ZeroMQ infrastructure ( the Context()-instance(s) ) with a socket-monitor artillery and make the fun() .connect()-directly to the socket-monitor resources to actively learn about any new message arriving to #1 ( i.e. autonomously, without #1's initiative ) and deciding to return in a pre-mature fashion, in those cases, where feasible according to your application logic.
For the socket-monitor case, the API documentation has all details needed for implementation, which would otherwise go way beyond the scope of the Stack Overflow post.
In an ambitious attempt making a VNC with Python, I am trying to continuously send screenshots of one user (Server) to another (Client). After hours of trying and hitting Ballmer's peak, I managed to do it. However, now my problem is sending multiple images, a continuous stream of them. I first tried to write all the binary data to one file, which didn't work. When the second image was opened, it crashed. I thought this might be because the binary data somehow got corrupted, so instead I tried making a new file for every image, yet I have the same problem. I know that Tcp is a constant stream of data so that it would be hard to know the end of the first image and start of the next, but by creating another file, I thought I would be all good.
Any help in fixing this and/or increasing the efficiency of this is greatly appreciated :)
Server side:
import socket
from PIL import Image, ImageGrab
PORT = 10007
HOST = '127.0.0.1'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
counter = 3
while counter > 0:
image = ImageGrab.grab(bbox=None)
image.save('test.png')
f = open('test.png', 'rb')
l = f.read(1024)
while (l):
conn.send(l)
l = f.read(1024)
f.close()
print('Done sending curr image')
counter -= 1
conn.close()
Client side:
import socket
from PIL import Image
HOST = '127.0.0.1'
PORT = 10007
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
counter = 1
while counter != 3:
fname = 'image' + str(counter) + '.png'
with open(fname, 'wb') as file:
print('file opened')
while True:
data = s.recv(1024)
if not data:
break
else:
file.write(data)
file.close()
currImg = Image.open(fname)
currImg.show()
counter += 1
s.close()
Your receiver does not know when one file finishes and the next begins. The easiest way to fix that is to send the length of the file (perhaps as a 4-byte unsigned value) to the receiver before sending each file. Then the receiver can read the length, read the file, read the length, read the file, ...
To improve efficiency you can stop saving the file data into an actual file at both ends and instead save it into (and, obviously, read it from) an in-memory buffer. See this answer for explanations of how to do that. In Python 3 it looks like you would use the BytesIO module.
Another improvement would be to only send image data for the parts of the screen that have changed since the previous send. For that you'll need to figure out how to compare the current capture against the previous one. For a first pass you could use PIL.ImageChops.difference followed by PIL.Image.getbbox and then encode and send only that region of the current capture. For that to work, the sender will have to tell the receiver not only the size of the PNG but also the location in the output screen image where the new image patch should be painted. So you'll want to send a position in addition to the size and the encoded image data.
I have a serial device (Arduino) regularly outputting log data, which shold be written in a Log file. Also the device takes spontaneous commands over serial. I send the commands to a Raspberry over Telegram, which are handled and sent to the arduino by Telepot, which runs in a separate thread.
How can I make sure that the two processes get along with each other?
I am a complete Beginner in Multithreading.
Here is a shortened version of my Code:
import time
import datetime
import telepot
import os
import serial
from time import sleep
ser = None
bot = None
def log(data):
with open('logfile', 'w') as f:
file.write("Timestamp" + data)
#The handle Function is called by the telepot thread,
#whenever a message is received from Telegram
def handle(msg):
chat_id = msg['chat']['id']
command = msg['text']
print( 'Command Received: %s' % command)
if command = '/start':
bot.sendMessage(chat_id, 'welcome')
elif command == 'close_door':
#This serial write could possibly happen while a
#ser.readline() is executed, which would crash my program.
ser.write("Close Door")
elif command == 'LOG':
#Here i should make sure that nothing
#is waiting from the Arduino
#so that the next two Serial lines are the Arduinos
#respoonce to the "LOG" command.
#and that hanlde is the only
#function talking to the Serial port now.
ser.write("LOG")
response = ser.readline()
response += "\0000000A" + ser.readline()
#The Arduinos response is now saved as one string
#and sent to the User.
bot.sendMessage(chat_id, response)
print("Command Processed.")
bot = telepot.Bot('BOT TOKEN')
bot.message_loop(handle)
ser = serial.Serial("Arduino Serial Port", 9600)
print( 'I am listening ...')
while True:
#anything to make it not run at full speed (Recommendations welcome)
#The log updates are only once an hour.
sleep(10)
#here i need to make sure it does not collide with the other thread.
while ser.in_waiting > 0:
data = ser.readline()
log(data)
This code is not my actual code, but it should represent exactly what I'm trying to do.
My last resort would be to put the serial code in the threads loop function, But this would require me to change the libary which would be ugly.
I looked up some stuff about Queues in Asincio, and locking functions. However i don't really understand how to apply that. Also I don't use the async telepot.
After reading more on locking and threads, I found an answer with help of the links provided in this Question: Locking a method in Python?
It was often recommended to use Queues, however I don't know how.
My solution (code may have errors, but the principle works)
import time
import random
import datetime
import telepot
import os
import serial
from time import sleep
#we need to import the Lock from threading
from threading import Lock
ser = None
bot = None
def log(data):
with open('logfile', 'w') as f:
file.write("Timestamp" + data)
#create a lock:
ser_lock = Lock()
#The handle Function is called by the telepot thread,
#whenever a message is received from Telegram
def handle(msg):
#let the handle function use the same lock:
global ser_lock
chat_id = msg['chat']['id']
command = msg['text']
print( 'Command Received: %s' % command)
if command == '/start':
bot.sendMessage(chat_id, 'welcome')
elif command == 'close_door':
#This serial write could possibly happen while a
#ser.readline() is executed, which would crash my program.
with ser_lock:
ser.write("Close Door")
elif command == 'LOG':
#Here i should make sure that nothing
#is waiting from the Arduino
#so that the next two Serial lines are the Arduinos
#respoonce to the "LOG" command.
#and that hanlde is the only
#function talking to the Serial port now.
#the lock will only be open when no other thread is using the port.
#This thread will wait untill it's open.
with ser_lock:
while ser.in_waiting > 0:
data = ser.readline()
log(data)
#Should there be any old data, just write it to a file
#now i can safely execute serial writes and reads.
ser.write("LOG")
response = ser.readline()
response += "\0000000A" + ser.readline()
#The Arduinos response is now saved as one string
#and sent to the User.
bot.sendMessage(chat_id, response)
print("Command Processed.")
bot = telepot.Bot('BOT TOKEN')
bot.message_loop(handle)
ser = serial.Serial("Arduino Serial Port", 9600)
print( 'I am listening ...')
while True:
#anything to make it not run at full speed (Recommendations welcome)
#The log updates are only once a
sleep(10)
#here i need to make sure it does not collide with the other thread.
with ser_lock:
while ser.in_waiting > 0:
data = ser.readline()
log(data)
i'm trying to build a program to send a string to process Tangki and Tangki2 then send a bit of array data each to process Outdata, but it seems not working correctly. but when i disable gate to the Outdata everything works flawlessly.
this is the example code:
import os
from multiprocessing import Process, Pipe
from time import sleep
import cv2
def outdata(input1,input2):
while(1):
room=input1.recv()
room2=input2.recv()
def tangki(keran1,selang1): ##============tangki1
a=None
x,y,degree,tinggi=0,0,0,0
dout=[]
while(1):
frame=keran1.recv()
dout.append([x,y,degree,tinggi])
selang1.send(dout)
print ("received from: {}".format(frame))
def tangki2(keran3,selang2): ##=================tangki2
x,y,degree,tinggi=0,0,0,0
dout2=[]
while(1):
frame=keran3.recv()
dout2.append([x,y,degree,tinggi])
selang2.send(dout2)
print("received from: {}".format(frame))
def pompa(gate1,gate2):
count=0
while(1):
count+=1
gate1.send("gate 1, val{}".format(count))
gate2.send("gate 2, val{}".format(count))
if __name__ == '__main__':
pipa1, pipa2 = Pipe()
pipa3, pipa4 = Pipe()
tx1,rx1 = Pipe()
tx2,rx2 = Pipe()
ptangki = Process(target=tangki, args=(pipa2, tx1))
ptangki2 = Process (target=tangki2, args=(pipa4, tx2))
ppompa = Process(target=pompa, args=(pipa1,pipa3))
keran = Process(target=outdata, args=(rx1,rx2))
ptangki.start()
ptangki2.start()
ppompa.start()
keran.start()
ptangki.join()
ptangki2.join()
ppompa.join()
keran.join()
at exact count reach 108 the process hang, not responding whatsoever. when i TOP it, the python3 process has gone, it seems that selang1 and selang2 causing the problem. i've search in google and it might be a Pipe Deadlock. so the question is how to prevent this from happening since i've already dump all data in pipe via repeated reading both in input1 and input2.
Edit: it seems that the only problem was the communication between tangki and tangki2 to outdata
it's actually because buffer size limit? but adding dout=[x,y,degree,tinggi] and dout=[x,y,degree,tinggi] reset the size of data to minimal, or by assigning dout=[0,0,0,0] and dout2=[0,0,0,0] right after selang1.send(dout) and selang2.send(dout2)