I discover FastAPI for the first time with which I try to discuss 2 services.
When a service asks "how are you?" the other service replies "Good!", and vice versa. The same requests are therefore made in both directions, they are crossed.... And this is repeated after a certain delay.
Here is my code for a service (it's the same code for the 2 services).
It works well, except when the delay is too short!
from fastapi import FastAPI
import requests
import uvicorn
import asyncio
app = FastAPI()
def get(url, headers, json_data=False):
if json_data:
r = requests.get(url, headers=headers, json=json_data)
else:
r = requests.get(url, headers=headers)
return r
#app.get("/")
async def how_are_you():
message = f"{service_name}: Good!"
print(message)
return message
async def toc_toc(message):
await asyncio.sleep(5)
url = f"http://{remote_host}:{remote_port}/"
headers = {
'accept': 'application/json',
}
count = 0
while True:
await asyncio.sleep(waiting)
print(f"{count}\t{service_name}: {message}!")
get(url, headers)
count += 1
async def run_webserver():
config = uvicorn.Config(app, host=host, port=port)
server = uvicorn.Server(config)
await server.serve()
async def main():
await asyncio.gather(
toc_toc("How are you"),
run_webserver()
)
Here are the input variables
of my first serve:
if __name__ == "__main__":
waiting = 0.1
service_name = "service_1"
host = "0.0.0.0"
port = 82
remote_host = "0.0.0.0"
remote_port = 83
loop = asyncio.get_event_loop()
tasks = main()
loop.run_until_complete(tasks)
And those of my second service:
if __name__ == "__main__":
waiting = 0.1
service_name = "service_2"
host = "0.0.0.0"
port = 83
remote_host = "0.0.0.0"
remote_port = 82
loop = asyncio.get_event_loop()
tasks = main()
loop.run_until_complete(tasks)
If I set this delay "waiting" to 1 second (between 2 requests "how are you?"), I have no problem. Everything works perfectly for minutes at least.
On the other hand, if I reduce this variable to 0.1 seconds (or even lower), the services send each other requests and respond to each other a hundred times and then the applications seem to freeze.
So I don't have any error message that could help me to debug.
I need to be able to lower this delay to 0.001 seconds to simulate a production load peak that my services will have to be able to support.
I have the impression that the problem comes from the fact that the requests are crossed between my 2 services. Because in the case where only one service can send requests to the other (by commenting out the get(url, headers) line for service 2), it works very well even with a delay of 0.001 seconds.
Do you have any ideas ?
Thanks in advance
PS: I run my services directly on my Mac M1 but also on my local Kubernets cluster, and I have the same result.
Related
I have currently written a code that has multiple threads (as example I used 50 threads) and for each thread only one proxy is allowed to be in one of these threads (meaning that 1 proxy cannot be in two threads).
import contextlib
import random
import threading
import time
import requests
my_proxies = [
'http://140.99.107.100:2100',
'http://140.99.107.101:2100',
'http://140.99.107.102:2100',
'http://140.99.107.103:2100',
'http://140.99.107.104:2100',
'http://140.99.107.105:2100',
'http://140.99.107.106:2100',
'http://140.99.107.107:2100',
'http://140.99.107.108:2100',
'http://140.99.107.109:2100',
'http://140.99.107.110:2100',
'http://140.99.107.111:2100',
'http://140.99.107.112:2100',
'http://140.99.107.113:2100',
'http://140.99.107.114:2100',
'http://140.99.107.115:2100',
'http://140.99.107.116:2100',
'http://140.99.107.117:2100',
'http://140.99.107.118:2100',
'http://140.99.107.119:2100',
'http://140.99.107.120:2100',
'http://140.99.107.121:2100',
'http://140.99.107.122:2100',
]
# --------------------------------------------------------------------------- #
class AvailableProxiesManager:
_proxy_lock: threading.Lock = threading.Lock()
def __init__(self):
self._proxy_dict = dict.fromkeys(my_proxies, True)
#property
#contextlib.contextmanager
def proxies(self):
"""
Context manager that yields a random proxy from the list of available proxies.
:return: dict[str, str] - A random proxy.
"""
proxy = None
with self._proxy_lock:
while not proxy:
if available := [att for att, value in self._proxy_dict.items() if value]:
proxy = random.choice(available)
self._proxy_dict[proxy] = False
else:
print('Waiting ... no proxies available')
time.sleep(.2)
yield proxy
self._proxy_dict[proxy] = True # Return the proxy to the list of available proxies
# --------------------------------------------------------------------------- #
available_proxies = AvailableProxiesManager()
def main():
while True:
with available_proxies.proxies as proxy:
response = requests.get('https://httpbin.org/ip', proxies={'https': proxy})
if response.status_code == 403:
print('Lets put proxy on cooldown for 10 minutes and try with new one!')
time.sleep(120)
if __name__ == '__main__':
threads = []
for i in range(50):
t = threading.Thread(target=main)
threads.append(t)
t.start()
time.sleep(1)
However my problem is that currently for every while True that is going on, it uses a new random proxy and instead what I am trying to achieve is that I want the same proxy to be used in the same thread until the response status is 403. That means that in the beginning if thread-1 gets the proxy: http://140.99.107.100:2100 then it should be used in thread-1 until it gets 403.
My question is, how can I be able to make the same proxy to be used until it hits response 403?
Expect:
Proxy to be the same until 403
Actual:
New proxy for every GET requests
What if you stop using a context manager,
(remove #contextlib.contextmanager)
and do something like this:
def main():
proxy = next(available_proxies.proxies)
while True:
response = requests.get('https://httpbin.org/ip', proxies={'https': proxy})
if response.status_code == 403:
proxy = next(available_proxies.proxies)
time.sleep(120)
Hope that helps, good luck !
I need to handle list of 2500 ip-addresses from csv file. So I need to create_task from coroutine 2500 times. Inside every coroutine firstly I need to fast-check access of IP:PORT via python module "socket" and it is a synchronous function want to be in loop.run_in_executor(). Secondly if IP:PORT is opened I need to connect to this socket via asyncssh.connect() for doing some bash commands and this is standart asyncio coroutine. Then I need to collect results of this bash commands to another csv file.
Additionaly there is an issue in Linux: system can not open more than 1024 connections at same time. I think it may be solved by making list of lists[1000] with asyncio.sleep(1) between or something like that.
I expected my tasks will be executed by 1000 in 1 second but it only 20 in 1 sec. Why?
Little working code snippet with comments here:
#!/usr/bin/env python3
import asyncio
import csv
import time
from pathlib import Path
import asyncssh
import socket
from concurrent.futures import ThreadPoolExecutor as Executor
PARALLEL_SESSIONS_COUNT = 1000
LEASES_ALL = Path("ip_list.csv")
PORT = 22
TIMEOUT = 1
USER = "testuser1"
PASSWORD = "123"
def is_open(ip,port,timeout):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(timeout)
try:
s.connect((ip, int(port)))
s.shutdown(socket.SHUT_RDWR)
return {"result": True, "error": "NoErr"}
except Exception as ex:
return {"result": False, "error": str(ex)}
finally:
s.close()
def get_leases_list():
# Minimal csv content:
# header must contain "IPAddress"
# every other line is concrete ip-address.
result = []
with open(LEASES_ALL, newline="") as csvfile_1:
reader_1 = csv.DictReader(csvfile_1)
result = list(reader_1)
return result
def split_list(some_list, sublist_count):
result = []
while len(some_list) > sublist_count:
result.append(some_list[:sublist_count])
some_list = some_list[sublist_count:]
result.append(some_list)
return result
async def do_single_host(one_lease_dict): # Function for each Task
# Firstly
IP = one_lease_dict["IPAddress"]
loop = asyncio.get_event_loop()
socket_check = await loop.run_in_executor(None, is_open, IP, PORT, TIMEOUT)
print(socket_check, IP)
# Secondly
if socket_check["result"] == True:
async with asyncssh.connect(host=IP, port=PORT, username=USER, password=PASSWORD, known_hosts=None) as conn:
result = await conn.run("uname -r", check=True)
print(result.stdout, end="") # Just print without write in file at this point.
def aio_root():
leases_list = get_leases_list()
list_of_lists = split_list(leases_list, PARALLEL_SESSIONS_COUNT)
r = []
loop = asyncio.get_event_loop()
for i in list_of_lists:
for j in i:
task = loop.create_task(do_single_host(j))
r.append(task)
group = asyncio.wait(r)
loop.run_until_complete(group) # At this line execute only by 20 in 1sec. Can't understand why :(
loop.close()
def main():
aio_root()
if __name__ == '__main__':
main()
loop.run_in_exectutor signature:
awaitable loop.run_in_executor(executor, func, *args)¶
The default ThreadPoolExecutor is used if executor is None.
ThreadPoolExecutor document:
Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.
Changed in version 3.8: Default value of max_workers is changed to min(32, os.cpu_count() + 4). This default value preserves at least 5 workers for I/O bound tasks. It utilizes at most 32 CPU cores for CPU bound tasks which release the GIL. And it avoids using very large resources implicitly on many-core machines.
Im making a bunch of GET requests to about a few hundred different API endpoints on different servers. In one of these endpoints there is some information that i want to fetch and return.
After any of these requests return something to me, i want to terminate the other threads and exit. Some requests are almost instant, some can take up to 20 seconds to finish.
If i happen to find the info in 2 seconds, i don't want for 20 seconds to elapse before i can resume work.
Currently I'm doing things like this:
threads = list()
for s in silos: #here i create all the requests
t = Thread(target=process_request, args=(my, args, here))
t.name = "{} - {}".format(some, name)
threads.append(t)
Then I do:
print("Threads: {}".format(len(threads))) # 100 - 250 of them
[ t.start() for t in threads ]
[ t.join() for t in threads ]
process_request() simply makes the get request and stores the result inside a dict if the status_code == 200.
I'm using the requests and threading modules.
If you use the multiprocess pool, then you can terminate the pool as soon as the first response arrives:
import multiprocessing as mp
import time
pool = None
def make_get_request(inputs):
print('Making get request with inputs ' + str(inputs))
time.sleep(2)
return 'dummy response for inputs ' + str(inputs)
def log_response(response):
print("Got response = " + response)
pool.terminate()
def main():
global pool
pool = mp.Pool()
for i in range(10):
pool.apply_async(make_get_request, args = (i,), callback = log_response)
pool.close()
pool.join()
if __name__ == '__main__':
main()
I want to initiate a PULL in a port and want to receive from other ports to my PULL port. In the case of a PULL port, it listens asynchronously and when it receives a message, it just prints the message in the console. So for that I have written a method inside a Push-class, which will send the message to the PULL port.
My code is as follows :
import random
import zmq
import time
import sys
import string
import asyncio
import zmq.asyncio
class Push():
def __init__(self, port, addr='localhost'):
self.port = port
self.addr = addr
self.ctx = zmq.Context()
self.scoket = self.ctx.socket(zmq.PUSH)
self.scoket.connect(f'tcp://{self.addr}:{selfa.port}')
def send(self):
chars = string.ascii_uppercase + string.ascii_lowercase
message = ''.join(random.choice(chars) for _ in range(4))
self.scoket.send(bytes(message, 'utf-8'))
print(f'sending: {message}')
class Pull():
def __init__(self, port, addr='*'):
self.port = port
self.addr = addr
self.ctx = zmq.Context()
self.socket = self.ctx.socket(zmq.PULL)
self.socket.bind(f'tcp://{self.addr}:{self.port}')
async def listen(self, listener):
while True:
string = await self.socket.recv()
listener(string)
if __name__ == '__main__':
push = Push('55501')
async def send():
while True:
await asyncio.sleep(5)
print('Sending...')
push.send()
pull = Pull('55501')
try:
asyncio.run(
pull.listen(print),
send(),
)
except KeyboardInterrupt:
print('exiting...')
exit()
The above code is not running. The code stops at the listen method.
#ADAPTED FROM PYMATA EXPRESS EXAMPLE CONCURRENTTAKS
#https://github.com/MrYsLab/pymata-express/
import asyncio
import zmq
import json
import zmq.asyncio as zmq_asyncio
from pymata_express.pymata_express import PymataExpress
class ConcurrentTasks:
def __init__(self, board):
self.loop = board.get_event_loop()
self.board = board
self.ctxsync = zmq.Context()
self.context = zmq.asyncio.Context()
self.rep = self.context.socket(zmq.REP)
self.rep.bind("tcp://*:5558")
self.trigger_pin = 53
self.echo_pin = 51
loop.run_until_complete(self.async_init_and_run())
### START: NEW CODE THAT RESOLVED THE ISSUE
async def pingsonar(self):
value = await self.board.sonar_read(self.trigger_pin)
return value
async def readsonar(self):
while True:
rep_recv = await self.rep.recv()
value = await asyncio.wait([self.pingsonar()])
valuesonar = list(value[0])[0].result()
json_data = json.dumps(valuesonar)
await self.rep.send(json_data.encode())
await asyncio.sleep(1 / 1000) #maybe this line isn't necessary
### END : NEW CODE THAT RESOLVED THE ISSUE
async def async_init_and_run(self):
await self.board.set_pin_mode_sonar(self.trigger_pin, self.echo_pin)
readsonar = asyncio.create_task(self.readsonar())
await readsonar
# OTHER CREATED_TASK GO HERE, (removed them in the MVE, but they work fine)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
my_board = PymataExpress()
try:
ConcurrentTasks(my_board)
except (KeyboardInterrupt, RuntimeError):
loop.run_until_complete(my_board.shutdown())
print('goodbye')
finally:
loop.close()
The above code is not running.
The code is running,yet there are mistakes ( in concept ) and typos in the source code.
The code as-is inhibits the Push-class from ever become .connect()-ed, thus the Pull-counterparty .bind()-s, yet has nobody to talk to.
SOLUTION
1 )Repair this typo ( + better, explicitly detect and handle all potential error-states )
self.scoket.connect(f'tcp://{self.addr}:{selfa.port}') # this will NEVER FLY
#--------------------------------------------^
self.scoket.connect(f'tcp://{self.addr}:{self.port}') # this will ... ( + detect Error-state(s)
2 )Correct the concept - mixing low-hanging fruits ( several asynchronous frameworks used at once, one working against all the others ) is a sign of shallow understanding of the responsibility of professional engineering in reliable and robust-enough distributed-computing ( flaws in the concept of control, like during landing Apollo-11 on Moon or, on the contrary, the Chernobyl-kind of mis-management system + mentality, are clean, inspiring and warning enough examples how bad practices ( if not prohibited + detected + corrected + penalised ) can & will harm again and again and again ).
The Best Next Step
If you are keen to get to the professional-level, start with Pieter Hintjens' book "Code Connected, Volume 1" - worth time, worth efforts, worth understanding their discussed concepts.
I am attempting to make use of concurrent.futures.ThreadPoolExecutor for the first time. One of my threads (level_monitor) consistently hangs on a call to datetime.now.strftime()—and on another hardware-specific function. For now I am assuming it is the same fundamental problem in both cases.
I've created a reproducible minimum example.
from concurrent.futures import ThreadPoolExecutor
import socket
from time import sleep
status = 'TRY AGAIN\n'
def get_level():
print('starting get_level()')
while True:
sleep(2)
now = datetime.now().strftime('%d-%b-%Y %H:%M:%S')
print('get_level woken...')
# report status when requested
def serve_level():
print('starting serve_level()')
si = socket.socket()
port = 12345
si.bind(('127.0.0.1',port))
si.listen()
print('socket is listening')
while True:
ci, addr = si.accept()
print('accepted client connection from ',addr)
with ci:
req = ci.recv(1024)
print( req )
str = status.encode('utf-8')
ci.send(str)
ci.close()
if __name__ == '__main__':
nthreads = 5
with ThreadPoolExecutor(nthreads) as executor:
level_monitor = executor.submit(get_level)
server = executor.submit(serve_level)
When I run it I see the serve_level thread works fine. I can talk to that thread using telnet. I can see the level_monitor thread starts too, but then it hangs before print('get_level woken...'). If I comment out the call to datetime then the thread behaves as expected.
I am sure that when I find out why I will have found out a lot.