I am trying to make a class, that will be eventually turned into a library. To do this, I am trying to do something like what discord.py made, and the idea comes from it.
The code that discord makes is:
#bot.event
async def on_ready():
print('discord bot is ready')
Where the '#bot' is just an object that I created before by doing
bot = discord()
And the '.event' is a preprogramed and ready to use method.
on_ready() is a function that is called already.
I want to have a way to create this from my own class, and from there mannage the entire code using async functions.
How to do it in my own code?
You need to implement a class whose public methods are decorators. For example, this class implements a scheduler that exposes scheduling through a decorator:
class Scheduler:
def __init__(self):
self._torun = []
def every_second(self, fn):
self._torun.append(fn)
async def _main(self):
while True:
for fn in self._torun:
asyncio.create_task(fn())
await asyncio.sleep(1)
def run(self):
asyncio.run(self._main())
You'd use it like this:
sched = Scheduler()
#sched.every_second
async def hello():
print('hello')
#sched.every_second
async def world():
print('world')
sched.run()
The class mimics discord in that it has a run method that calls asyncio.run for you. I would prefer to expose an async main instead, so the user can call asyncio.run(bot.main()) and have control over which event loop is used. But the example follows discord's convention, to make the API more familiar to users of discord.py.
Related
I have a class i created thats a websocket and it connects to my data endpoint with no issues. However, I wanted my socket to run forever. I am using the websockets python library. Heres a sample:
from websockets import connect
class Socket(metaclass=ABCMeta):
def __init__(self, url: str):
self.url = url
async def __aenter__(self):
self._conn = connect(self.url, ping_interval=None)
self.websocket = await self._conn.__aenter__()
return self
async def __aexit__(self, *args, **kwargs):
await self._conn.__aexit__(*args, **kwargs)
Now, i am able to write async with statement with no problems. My issue arises when i want my socket to remain connected.
reading in the library, it seems one suggested way is to do the following:
async for socket in websockets.connect(", ping_interval=None):
try:
your logic
except websockets.closedConnection as e:
continue
This allows me to keep trying to connect if there is an issue. How would i incorporate this into my class as an iterator? I tried the following but getting error:
TypeError: 'async for' received an object from __aiter__ that does not implement __anext__: coroutine
After i added the following code in the above class:
async def __aiter__(self):
return self
async def __anext__(self):
async for websocket in connect(self.url, ping_interval=None):
try:
self.websocket = await websocket
except StopIteration:
raise StopAsyncIteration
I am not posting my entire code here as the goal is to encapsulate a class around this socket class i created with the goal being
async for object in MyCustomClassSocketIterator(url):
try:
await object.send()
await object.receive()
except websockets.closedConnection as e:
etc....
where the encapsulated class has implemented receive() and send() functions. So each time program starts, object is instantiated asynchronously. If anything breaks...then it attemps to connect again if there is a socket.closedConnection. Thanks
It's clear for my how I use asyncio in python3.
import asyncio
import time
async def do_something(number):
print(f"do something no.{number} at timestamp : {time.perf_counter():6.2f}")
await asyncio.sleep(1)
print(f"do something no.{number} ended at timestamp: {time.perf_counter():6.2f}")
async def main():
await asyncio.gather(
do_something(1),
do_something(2)
)
asyncio.run(main() )
However, I have no idea how I could create an own "await"-able object like asyncio.sleep. In this "await"-able I could encapsulate urllib.request, isn't it?
Can someone post an example?
Many thanks.
Please, take a look at this answer it uses old yield for-based syntax, but the idea stays the same: using asyncio.Future and loop.call_later() you can cast callback-based code into coroutine-based code:
import asyncio
async def my_sleep(delay):
fut = asyncio.Future()
loop = asyncio.get_event_loop()
loop.call_later(
delay,
lambda *_: fut.set_result(True)
)
return await fut
async def main():
await asyncio.gather(
my_sleep(1),
my_sleep(2),
my_sleep(3)
)
print('ok')
asyncio.run(main())
I believe, urllib.request is blocking and doesn't provide callbacks so it can't be cast into coroutine-based form directly. A common way to handle the situation is to run it in async thread (see links in this answer).
But if you want just to make async http reqeust, forget all above and use aiohttp: it's created for the purpose.
You can await all coroutines, so any function with the prefix async is awaitable
For example:
import asyncio
async def some_function():
await asyncio.sleep(1)
print("Hello")
async def main():
await some_function()
asyncio.run(main())
You can find some more information about coroutines at the python docs:
https://docs.python.org/3/library/asyncio-task.html
Because urllib is blocking (does not allow other code to be running before it finishes) by default, it is not that easy to just "wrap" it in an awaitable function.
It is probably possible to offload it to something like another thread and then have an awaiable wait for that thread to finish, but it is probably easier to use an async web request library like aiohttp.
I am trying to interface some Api which requires an auth token using asyncio.
I have a method inside the ApiClient class for obtaining this token.
class ApiClient:
def __init__(self):
self._auth_token = None
# how to invoke _keep_auth_token_alive in the background?
async def _keep_auth_token_alive(self):
while True:
await self._set_auth_token()
await asyncio.sleep(3600)
The problem is, that every hour I need to recall this function in order to maintain a valid token because it refreshes every hour (without this I will get 401 after one hour).
How can I make this method invoke every hour in the background starting at the initializing of ApiClient?
(The _set_auth_token method makes an HTTP request and then self._auth_token = res.id_token)
To use an asyncio library, your program needs to run inside the asyncio event loop. Assuming that's already the case, you need to use asyncio.create_task:
class ApiClient:
def __init__(self):
self._auth_token = None
asyncio.create_task(self._keep_auth_token_alive())
Note that the auth token will not be available upon a call to ApiClient(), it will get filled no earlier than the first await, after the background task has had a chance to run. To fix that, you can make _set_async_token public and await it explicitly:
client = ApiClient()
await client.set_async_token()
To make the usage more ergonomic, you can implement an async context manager. For example:
class ApiClient:
def __init__(self):
self._auth_token = None
async def __aenter__(self):
await self._set_auth_token()
self._keepalive_task = asyncio.create_task(self._keep_auth_token_alive())
async def __aexit__(self, *_):
self._keepalive_task.cancel()
async def _keep_auth_token_alive(self):
# modified to sleep first and then re-acquire the token
while True:
await asyncio.sleep(3600)
await self._set_auth_token()
Then you use ApiClient inside an async with:
async with ApiClient() as client:
...
This has the advantage of reliably canceling the task once the client is no longer needed. Since Python is garbage-collected and doesn't support deterministic destructors, this would not be possible if the task were created in the constructor.
In discord.py 1.0.1 (Only version Repl.it has), the cogs are giving me a hard time.
import discord
from discord.ext import commands
class Coding(commands, Cog):
def __init__(self, client):
self.client = client
#commands.Cog.listener()
async def on_ready(self):
print("Kahoot Bot 0.1 ALPHA")
client.remove_command("help")
#commands.command()
async def clear(self, ctx, amount = 5):
await ctx.channel.purge(limit = amount + 1)
#commands.command()
async def ping(self, ctx):
await ctx.send(f"Pong! {round(client.latency * 1000)}ms.")
#client.command(pass_context = True, aliases = ["print"])
async def printing(ctx, *, what_to_print):
await ctx.send(what_to_print)
print(what_to_print)
def setup(client):
client.add_cog(Coding(client))
The gist of the errors is:
A) client is not defined
B) init() should return None, not coroutine
I've tried changing all my code to bot and back to client, but nothing's helped. No idea what's going on.
You do the inheritance wrong. You dont inherit from class: "commands" and "Cog". You inherit from class: "commands.Cog". thus changing: class Coding(commands, Cog): to class Coding(commands.Cog): will fix some of the errors.
You also do the following wrong (the "client does not exist" error):
#commands.Cog.listener()
async def on_ready(self):
print("Kahoot Bot 0.1 ALPHA")
client.remove_command("help") # this line
When we want to access a class variable we use self. in the beginning of that variable to indicate that we are using the class variable. This case you dont use self.client. but client.
As client is not defined in that function it will give an error. But it is defined as a class variable (the "init" function). To access it use: self.client.
For a project, I want to be able to have simultaneously a sync and async version of a library, the sync version has most logic parts and the async must call the sync version in a async way. For example I have a class which gets an http requester in constructor, this requester handle sync or async internally:
.
├── async
│ └── foo.py
├── foo.py
└── main.py
└── requester.py
# requester.py
class Requester():
def connect():
return self._connect('some_address')
class AsynRequester():
async def connect():
return await self._connect('some_address')
# foo.py
class Foo:
def __init__(self, requester):
self._requester = requester
def connect(self):
self.connect_info = self._requester.connect('host') # in async version it would be called by an await internally
# async/foo.py
from foo import Foo as RawFoo
class Foo(RawFoo):
async def connect(self):
return await super(RawFoo, self).connect()
# main.py
from async.foo import Foo # from foo import Foo
from requester import AsynRequester # from requester import Requester
def main():
f = Foo(AsyncRequester()) # or Foo(Requester()) where we use sync requester
await f.connect() # or f.connect() if we are using sync methods
But async connect finally calls the sync connect of sync class type of Foo (which is parent of async class) which internally calls requester.connect function. It is impossible because requester.connect internally has called await connect when it had being used in async mode but it is calling without any await.
All of my tests have been written for sync version, because async tests is not efficient as they must be, also I must write tests for one version and be sure that both versions would work correctly. How can I have both version at the same time which are using the same logic and just the I/O calls are separated.
the sync version has most logic parts and the async must call the sync version in a async way
It is possible, but it's a lot of work, as you're effectively fighting the function color mismatch. Your logic will have to be written in an async fashion, with some hacks to allow it to work in sync mode.
For example, a logic method would look like this:
# common code that doesn't assume it's either sync or async
class FooRaw:
async def connect(self):
self.connect_info = await self._to_async(self._requester.connect(ENDPOINT))
async def hello_logic(self):
await self._to_async(self.connect())
self.sock.write('hello %s %s\n' % (USERNAME, PASSWORD))
resp = await self._to_async(self.sock.readline())
assert resp.startswith('OK')
return resp
When running under asyncio, methods like connect and readline are coroutines, so their return value must be awaited. On the other hand, in blocking code self.connect and sock.readline are sync functions that return concrete values. But await is a syntactic construct which is either present or missing, you cannot switch it off at run-time without duplicating code.
To allow the same code to work in sync and async modes, FooRaw.hello_logic always awaits, leaving it to the _to_async method to wrap the result in an awaitable when running outside asyncio. In async classes _asincify awaits its argument and return the result, it's basically a no-op. In sync classes it returns the received object without awaiting it - but is still defined as async def, so it can be awaited. In that case FooRaw.hello_logic is still a coroutine, but one that never suspends (because the "coroutines" it awaits are all instances of _to_async which doesn't suspend outside asyncio.)
With that in place, the async implementation of hello_logic doesn't need to do anything except choose the right requester and provide the correct _to_async; its connect and hello_logic inherited from FooRaw do the right thing automatically:
class FooAsync(FooRaw):
def __init__(self):
self._requester = AsyncRequester()
#staticmethod
async def _to_async(x):
# we're running async, await X and return the result
result = await x
return result
The sync version will, in addition to implementing _to_async, need to wrap the logic methods to "run" the coroutine:
class FooSync(FooRaw):
def __init__(self):
self._requester = SyncRequester()
#staticmethod
async def _to_async(x):
# we're running sync, X is the result we want
return x
# the following can be easily automated by a decorator
def connect(self):
return _run_sync(super().connect())
def hello_logic(self):
return _run_sync(super().hello_logic())
Note that it is possible to run the coroutine outside the event loop only because the FooSync.hello_logic is a coroutine in name only; the underlying requester uses blocking calls, so FooRaw.connect and others never really suspend, they complete their execution in a single run. (This is similar to a generator that does some work without ever yielding anything.) This property makes the _run_sync helper straightforward:
def _run_sync(coro):
try:
# start running the coroutine
coro.send(None)
except StopIteration as e:
# the coroutine has finished; return the result
# stored in the `StopIteration` exception
return e.value
else:
raise AssertionError("coroutine suspended")