Our errbot is providing links to JIRA tickets when it sees the right ticket patterns. Unfortunately, in slack it is common for users to edit their posts, and if both edits contain the JIRA ticket pattern, errbot provides the link twice, it is annoying.
Can I detect when a message is an edit as opposed to the original message?
Now you can because from the .extra argument on the messages you can get the message ID from slack.
This is not currently possible (on any of the backends), no. For chat networks which allow editing of messages, errbot currently injects edited messages as a new message.
If you need this functionality, please open an issue on errbot's issue tracker so we can brainstorm about how to possibly introduce this functionality.
My bot watches for and expands references to Jira issues too; one of the things I did was keep track of how many messages had been seen in the channel, and when the last time was that I responded to any given issue. Issues expanded within the last N (I use 10) messages are ignored. That way, if they edit the issue key itself they usually get a new expansion, but not if they edit other parts of the message.
def activate(self):
self.messages_seen={} # room: messages seen
self.last_shown={} # issue_key : { room : message number last shown }
super().activate()
def callback_message(self, msg):
if msg.is_group:
try:
self.messages_seen[msg.to.id] += 1
except KeyError:
self.messages_seen[msg.to.id] = 1
def record_expanded(self, msg, key, orig_key=None):
if msg.is_group:
channel=msg.to.id
keys = [ key, orig_key ] if orig_key and orig_key != key else [ key ]
for k in keys:
if k in self.last_shown:
self.last_shown[ k ][ channel ] = self.messages_seen[ channel ]
else:
self.last_shown[ k ] = { channel : self.messages_seen[ channel ] }
def should_expand(self, msg, key):
expand=False
if msg.body.split()[0] == 'jira':
# User said 'jira <key>', so always expand
expand=True
if msg.is_group:
channel=msg.to.id
message_number=self.messages_seen.get(channel, 1)
expanded_in = self.last_shown.get(key, {})
if expanded_in:
if channel not in expanded_in: # key has been expanded, but not in this channel
expand=True
else:
expanded_last_here = message_number - expanded_in[channel]
if expanded_last_here >= self.bot_config.JIRA_EXPAND_ONLY_EVERY: # not recently enough
expand=True
else:
self.log.debug("not expanding '%s' because expanded %d messages ago" % (key, expanded_last_here))
else:
# this key has not been seen anywhere before
expand=True
else:
# direct IM - always expand
expand=True
if not expand:
self.log.debug("Recently expanded %s, suppressing" % key)
return expand
Related
I have made a telegram member scraper and inviter with Python. It was successful on some tests, but there are times my accounts get banned with upon seeing this error message:
AttributeError: 'ChatForbidden' object has no attribute 'access_hash'
I'm not sure why would it show ChatForbidden if I am already an admin of a group. It's hard to test these as I had to buy new phone numbers every time.
Here's a sample and explanation of my code to invite members to a group:
# Log in into the telegram account
client = TelegramClient('Tg_scraper', api_id, api_hash)
chats = []
last_date = None
chunk_size = 200
groups = []
hash_list = []
# Get all the groups/channel of the account
result = client(GetDialogsRequest(
offset_date=last_date,
offset_id=0,
offset_peer=InputPeerEmpty(),
limit=chunk_size,
hash=0
))
chats.extend(result.chats)
# Puts all the group/channel into a list
i = 0
print('Enter a NUMBER to choose a group where the members will be invited into:')
for chat in chats:
try:
groups.append(chat)
hash_list.append(chat.access_hash)
print(f"({i})" + ' - ' + chat.title)
i += 1
except:
continue
g_index = input("Enter a Number: ")
target_group = groups[int(g_index)]
target_group_entity = InputPeerChannel(target_group.id, target_group.access_hash)
Upon the last line, target_group_entity = InputPeerChannel(target_group.id, target_group.access_hash) is where I encounter the error I have stated above. Upon receiving that error, I get banned.
Does this have something to do with permissions? Do new accounts get banned for botting? It works on my first few tests, but then now I can't invite. Thank you so much for anyone who could help in advance.
I am already an admin of a group
This error is unrelated to your permission level.
Upon the last line is where I encounter the error
Wrong. you encounter this error because you're not coding it right with types in mind, expecting all your .chats are groups. Telegram doesn't tell you what fields exactly have, as you see in this error.
You must use type checking to limit your chats objects to only what you expect, your try block is appending then erroring, so, rather than a plain:
except:
continue
you need to actually confirm it won't error when accessing fields.
print('Enter a NUMBER to choose a group where the members will be invited into:')
i = 0
for chat in chats:
if isinstance(chat, telethon.types.Channel):
if chat.broadcast: continue # ignore non-group
groups.append(chat)
hash_list.append(chat.access_hash)
print(f"({i})" + ' - ' + chat.title)
i += 1
g_index = input("Enter a Number: ")
target_group = groups[int(g_index)]
I'm trying to make a discord bot and I've added functionality of the tasks repeating after 24 hours using tasks.loop after at an interval of 24 hours( 64800 seconds). Since the code is run on repl.it, I've used uptime robot to continuously send a ping after 5 minutes to make it running always in the repl.it.
My issue is that even after checking historyin the channel the message is getting send again as you can see in the below image. ( by checking here I mean the explanation which is the explanation that comes with the API). I'm using NASA api here.
Code snippet
def get_APOD():
image = requests.get("https://api.nasa.gov/planetary/apod?api_key="+apod)
json_data = json.loads(image.text)
description = json_data["url"]
explanation = "'" + json_data["explanation"]+ "'"
title = json_data["title"]
return description,explanation,title
#client.event
async def on_ready():
print('We have logged in as {0.user}'.format(client))
send_message.start()
#tasks.loop(seconds=64800.0)
async def send_message():
channel = client.get_channel(int(apod_channel))
messages = await channel.history(limit=1).flatten()
description,explanation,title = get_APOD()
if messages[0].content == explanation:
print("Its the same")
else:
await channel.send(description)
await channel.send(title)
await channel.send(explanation)
. Here you can see the messages repeating twice. It shouldn't by the code I've written here. Since it check for the history in the channel.
What is the issue here? Appreciate any help.
EDIT: When I restart the repl.it IDE the code will run alright and print "It's the same" as it should if the message is already sent but otherwise it's failing.
I'm working on a bot with the Python-Telegram-Bot API. Right now I'm updating the bot menu with InlineKeyboardButtons instead of ?menu, ?info type of commands (like on Discord). But I came across some trouble. Let me explain step by step what the command is supposed to do:
Step 1 - User opens up the menu (a commandhandler)
Step 2 - InlineKeyboardButtons load, one of them has an option called "Jokenpo" for the game
Step 3 - I set its callback_data to load a message greeting the player and showing the rules (def jkpMenu).
Step 4 - Then def jkpMenu is supposed to go to the next state, that is def jokenpo. During the function, there's a loop for receiving input like paper, rock, scissors that keeps repeting until user types stop or loses their lives.
varScore = 0
varLives = 5
varTie = 0
JKP = range(2)
...
def menuTest(update: Update, _: CallbackContext) -> int:
commands = [
[InlineKeyboardButton(text='📑 | Changelog', callback_data='clog'), InlineKeyboardButton(text='📑 | Info', callback_data='info')],
[InlineKeyboardButton(text='📑 | Jokenpo', callback_data='jokenpo')]]
update.message.reply_text("Menu:\n", reply_markup=InlineKeyboardMarkup(commands))
def comQuery(update: Update, context: CallbackContext) -> None:
query = update.callback_query
query.answer()
if query.data == 'clog': changelog(update, context)
if query.data == 'info': myInfo(update, context)
if query.data == 'jokenpo': jkpMenu(update, context)
#Jokenpo 1: greetings and rules
def jkpMenu(update: Update, _: CallbackContext) ->JKP:
update.callback_query.message.reply_text('Jokenpo mode. Please type "rock", "paper" or "scissors" to continue')
return JKP
#Jokenpo 2:
def jokenpo(update: Update, _:CallbackContext):
msgUser = update.callback_query.message.text.lower()
global varScore
global varLives
global varTie
while True:
computer = ("rock", "paper", "scissors")
computer = random.choice(computer)
if (msgUser== "rock" and computer == "paper") or (msgUser== "paper" and computer == "scissors") or (msgUser=="scissors" and computer == "rock"):
update.callback_query.message.reply_text("Computer chooses <i>" + computer + "</i>. <b>You lose!</b>", parse_mode ='HTML')
varLives -= 1
if (msgUser== "rock" and computer == "scissors") or (msgUser == "paper" and computer == "rock") or (msgUser == "scissors" and computer == "paper"):
update.callback_query.message.reply_text("Computer chooses <i>" + computer + "</i>. <b>You win!</b>", parse_mode ='HTML')
varScore +=1
if (msgUser== computer):
update.callback_query.message.reply_text("We both chose <i>" + computer +"</i>. <b>It's a tie!</b>", parse_mode ='HTML')
varTie +=1
if (msgUser== "status"):
update.callback_query.message.reply_text("Your current score: " + str(varScore) + " points.\nYou have "+ str(varLives)+ " lives left.\nWe tied " + str(varTie) +' times.')
if (msgUser== "sair") or (varLives == 0):
update.callback_query.message.reply_text("Game finished. You made <b>" + str(varScore) + "</b> points.\nWe tied " + str(varTie) +' times.\n', parse_mode ='HTML')
varScore = 0
varTie = 0
varLives = 5
return ConversationHandler.END
update.callback_query.message.reply_text('Please choose: paper, rock or scissors? ')
return
...
...
def(main):
convHandler = ConversationHandler(
entry_points=[jokenpo],
fallbacks=[],
states = {
JKP: [MessageHandler(Filters.text, jokenpo)]
})
dp.add_handler(convHandler)
Problem is: I managed to do that before, when I used CommandHandlers to access the function and it worked.
But now it only reaches the first function (Step 3), it seems it doesn't return to the state I wanted. When I changed return JKP for return jokenpo(update,_) it did access def jokenpo, but it didn't wait for any answer.
When I set the inline buttons, since they use CallbackContext, I'm confused on how to handle arguments.
I copied only some part of the code, not it complete, since the other parts are related to other functions. Any help is appreciated.
I see several issues in your code snippet & description:
using global variables is in general bad practice. If you want to store user/chat related data, I suggest to use PTBs built-in mechanism for that. see here.
The while True loop in jokenpo doesn't make any sense. In fact, at the end of the body of the loop, you return anyway, so the body is executed exactly once.
As entry points for your conversation you use entry_points = [jokenpo], but jokenpo is a function and not a handler
In the state JKP you have a handler MessageHandler(Filters.text, jokenpo). This means that the update that will be passed to jokenpo will have update.(edited_)message/channel_post, but never update.callback_query - yet you try to access this attribute within jokenpo.
"Then def jkpMenu is supposed to go to the next state, that is def jokenpo" - the callback jkpMenu is not part of your ConversationHandler at all and hence the return value will be ignored completely.
"When I changed return JKP for return jokenpo(update,_) it did access def jokenpo, but it didn't wait for any answer." Sure - you called the function. But waiting for input can only work if the conversationhandler is started, which this doesn't do
"When I set the inline buttons, since they use CallbackContext, I'm confused on how to handle arguments." This one I just don't understand. What do you mean by "arguments" in the context of inline buttons and what does that have to do with CallbackContext?
I have the impression that you need to deepen your understanding of how PTB is supposed to be used, especially ConversationHandler. I suggest to have a look at the introductory example. Following the flow chart & tracking the executed code while running the example should help to get a better understanding of what's going on. Also reading the documentation of ConversationHandler should clarify some things.
Disclaimer: I'm currently the maintainer of python-telegram-bot.
Also for completeness sake: This was already discussed to some extend in the usergroup of PTB, see https://t.me/pythontelegrambotgroup/513856
This is my first post, I've been lurking for a while.
Some context to my question;
I'm working with the Stripe API to pull transaction data and match these with booking numbers from another API source. (property reservations --> funds received for reconciliation)
I started by just making calls to the API and sorting the data in place using python 3, however it started to get very complicated and I thought I should persist the data in a mongodb stored on localhost. I began to do this, however I decided that storing the sorted data was still just as complicated and the request times were getting quite long, I thought, maybe I should pull all the stripe data and store it locally and then query whatever I needed.
So here I am, with a bunch of code I've written for both and still not alot of progress. I'm a bit lost with the next move. I feel like I should probably pick a path and stick with it. I'm a little unsure what is the "best practise" when working with API's, usually I would turn to YouTube, but I haven't been able to find a video which covers this specific scenario. The amount of data being pulled from the API would be around 100kb per request.
Here is the original code which would grab each query. Recently I've learnt I can use the expand method (I think this is what it's called) so I don't need to dig down so many levels in my for loop.
The goal was to get just the metadata which contains the booking reference numbers that can then be match against a response from my property management systems API. My code is a bit embarrassing, I've kinda just learnt it over the last little while in my downtime from work.
import csv
import datetime
import os
import pymongo
import stripe
"""
We need to find a Valid reservation_ref or reservation_id in the booking.com Metadata. Then we need to match this to a property ID from our list of properties in the book file.
"""
myclient = pymongo.MongoClient("mongodb://localhost:27017/")
mydb = myclient["mydatabase"]
stripe_payouts = mydb["stripe_payouts"]
stripe.api_key = "sk_live_thisismyprivatekey"
r = stripe.Payout.list(limit=4)
payouts = []
for data in r['data']:
if data['status'] == 'paid':
p_id = data['id']
amount = data['amount']
meta = []
txn = stripe.BalanceTransaction.list(payout=p_id)
amount_str = str(amount)
amount_dollar = str(amount / 100)
txn_len = len(txn['data'])
for x in range(txn_len):
if x != 0:
charge = (txn['data'][x]['source'])
if charge.startswith("ch_"):
meta_req = stripe.Charge.retrieve(charge)
meta = list(meta_req['metadata'])
elif charge.startswith("re_"):
meta_req = stripe.Refund.retrieve(charge)
meta = list(meta_req['metadata'])
if stripe_payouts.find({"_id": p_id}).count() == 0:
payouts.append(
{
"_id": str(p_id),
"payout": str(p_id),
"transactions": txn['data'],
"metadata": {
charge: [meta]
}
}
)
# TODO: Add error exception to check for po id already in the database.
if len(payouts) != 0:
x = stripe_payouts.insert_many(payouts)
print("Inserted into Database ", len(x.inserted_ids), x.inserted_ids)
else:
print("No entries made")
"_id": str(p_id),
"payout": str(p_id),
"transactions": txn['data'],
"metadata": {
charge: [meta]
This last section doesn't work properly, this is kinda where I stopped and starting calling all the data and storing it in mongodb locally.
I appreciate if you've read this wall of text this far.
Thanks
EDIT:
I'm unsure what the best practise is for adding additional information, but I've messed with the code below per the answer given. I'm now getting a "Key error" when trying to insert the entries into the database. I feel like It's duplicating keys somehow.
payouts = []
def add_metadata(payout_id, transaction_type):
transactions = stripe.BalanceTransaction.list(payout=payout_id, type=transaction_type, expand=['data.source'])
for transaction in transactions.auto_paging_iter():
meta = [transaction.source.metadata]
if stripe_payouts.Collection.count_documents({"_id": payout_id}) == 0:
payouts.append(
{
transaction.id: transaction
}
)
for data in r['data']:
p_id = data['id']
add_metadata(p_id, 'charge')
add_metadata(p_id, 'refund')
# TODO: Add error exception to check for po id already in the database.
if len(payouts) != 0:
x = stripe_payouts.insert_many(payouts)
#print(payouts)
print("Inserted into Database ", len(x.inserted_ids), x.inserted_ids)
else:
print("No entries made")```
To answer your high level question. If you're frequently accessing the same data and that data isn't changing much then it can make sense to try to keep your local copy of the data in sync and make your frequent queries against your local data.
No need to be embarrassed by your code :) we've all been new at something at some point.
Looking at your code I noticed a few things:
Rather than fetch all payouts, then use an if statement to skip all except paid, instead you can pass another filter to only query those paid payouts.
r = stripe.Payout.list(limit=4, status='paid')
You mentioned the expand [B] feature of the API, but didn't use it so I wanted to share how you can do that here with an example. In this case, you're making 1 API call to get the list of payouts, then 1 API call per payout to get the transactions, then 1 API call per charge or refund to get the metadata for charges or metadata for refunds. This results in 1 * (n payouts) * (m charges or refunds) which is a pretty big number. To cut this down, let's pass expand=['data.source'] when fetching transactions which will include all of the metadata about the charge or refund along with the transaction.
transactions = stripe.BalanceTransaction.list(payout=p_id, expand=['data.source'])
Fetching the BalanceTransaction list like this will only work as long as your results fit on one "page" of results. The API returns paginated [A] results, so if you have more than 10 transactions per payout, this will miss some. Instead, you can use an auto-pagination feature of the stripe-python library to iterate over all results from the BalanceTransaction list.
for transaction in transactions.auto_paging_iter():
I'm not quite sure why we're skipping over index 0 with if x != 0: so that may need to be addressed elsewhere :D
I didn't see how or where amount_str or amount_dollar was actually used.
Rather than determining the type of the object by checking the ID prefix like ch_ or re_ you'll want to use the type attribute. Again in this case, it's better to filter by type so that you only get exactly the data you need from the API:
transactions = stripe.BalanceTransaction.list(payout=p_id, type='charge', expand=['data.source'])
I'm unable to test because I lack the same database that you have, but wanted to share a refactoring of your code that you may consider.
r = stripe.Payout.list(limit=4, status='paid')
payouts = []
for data in r['data']:
p_id = data['id']
amount = data['amount']
meta = []
amount_str = str(amount)
amount_dollar = str(amount / 100)
transactions = stripe.BalanceTransaction.list(payout=p_id, type='charge', expand=['data.source'])
for transaction in transactions.auto_paging_iter():
meta = list(transaction.source.metadata)
if stripe_payouts.find({"_id": p_id}).count() == 0:
payouts.append(
{
"_id": str(p_id),
"payout": str(p_id),
"transactions": transactions,
"metadata": {
charge: [meta]
}
}
)
transactions = stripe.BalanceTransaction.list(payout=p_id, type='refund', expand=['data.source'])
for transaction in transactions.auto_paging_iter():
meta = list(transaction.source.metadata)
if stripe_payouts.find({"_id": p_id}).count() == 0:
payouts.append(
{
"_id": str(p_id),
"payout": str(p_id),
"transactions": transactions,
"metadata": {
charge: [meta]
}
}
)
# TODO: Add error exception to check for po id already in the database.
if len(payouts) != 0:
x = stripe_payouts.insert_many(payouts)
print("Inserted into Database ", len(x.inserted_ids), x.inserted_ids)
else:
print("No entries made")
Here's a further refactoring using functions defined to encapsulate just the bit adding to the database:
r = stripe.Payout.list(limit=4, status='paid')
payouts = []
def add_metadata(payout_id, transaction_type):
transactions = stripe.BalanceTransaction.list(payout=payout_id, type=transaction_tyep, expand=['data.source'])
for transaction in transactions.auto_paging_iter():
meta = list(transaction.source.metadata)
if stripe_payouts.find({"_id": payout_id}).count() == 0:
payouts.append(
{
"_id": str(payout_id),
"payout": str(payout_id),
"transactions": transactions,
"metadata": {
charge: [meta]
}
}
)
for data in r['data']:
p_id = data['id']
add_metadata('charge')
add_metadata('refund')
# TODO: Add error exception to check for po id already in the database.
if len(payouts) != 0:
x = stripe_payouts.insert_many(payouts)
print("Inserted into Database ", len(x.inserted_ids), x.inserted_ids)
else:
print("No entries made")
[A] https://stripe.com/docs/api/pagination
[B] https://stripe.com/docs/api/expanding_objects
I want to establish publish subscribe communication between to machines.
The two machines, that I have, are ryu-primary and ryu-secondary
The steps I follow in each of the machines are as follows.
In the initializer for ryu-primary (IP address is 192.168.241.131)
self.context = zmq.Context()
self.sub_socket = self.context.socket(zmq.SUB)
self.pub_socket = self.context.socket(zmq.PUB)
self.pub_port = 5566
self.sub_port = 5566
def establish_zmq_connection(self): # Socket to talk to server
print( "Connection to ryu-secondary..." )
self.sub_socket.connect( "tcp://192.168.241.132:%s" % self.sub_port )
def listen_zmq_connection(self):
print( 'Listen to zmq connection' )
self.pub_socket.bind( "tcp://*:%s" % self.pub_port )
def recieve_messages(self):
while True:
try:
string = self.sub_socket.recv( flags=zmq.NOBLOCK )
print( 'flow mod messages recieved {}'.format(string) )
return string
except zmq.ZMQError:
break
def push_messages(self,msg):
self.pub_socket.send( "%s" % (msg) )
From ryu-secondary (IP address - 192.168.241.132)
In the initializer
self.context = zmq.Context()
self.sub_socket = self.context.socket(zmq.SUB)
self.pub_socket = self.context.socket(zmq.PUB)
self.pub_port = 5566
self.sub_port = 5566
def establish_zmq_connection(self): # Socket to talk to server
print( "Connection to ryu-secondary..." )
self.sub_socket.connect( "tcp://192.168.241.131:%s" % self.sub_port )
def listen_zmq_connection(self):
print( 'Listen to zmq connection' )
self.pub_socket.bind( "tcp://*:%s" % self.pub_port )
def recieve_messages(self):
while True:
try:
string = self.sub_socket.recv( flags=zmq.NOBLOCK )
print( 'flow mod messages recieved {}'.format(string) )
return string
except zmq.ZMQError:
break
def push_messages(self,msg):
print( 'pushing message to publish socket' )
self.pub_socket.send( "%s" % (msg) )
These are the functions that I have.
I am calling on ryu-secondary:
establish_zmq_connections()
push_messages()
But I am not recieving those messages on ryu-primary, when I call
listen_zmq_connection()
recieve_messages()
Can someone point out to me what I am doing wrong?
Repair the PUB/SUB messaging pattern setup
There are several important steps in making the PUB/SUB pattern work.
All this is well described in the ZeroMQ documentation.
You need not repeat both pub & sub parts of code on both sides, the more that it masks, as A side-effect thereof, the case if you mix the pub and sub socket addresses/ports/calls/etc in an "opposite" node code and you do not see such a principal collision.
your code defines the initial form of PUB-archetype, that is expected to .push_messages()
your code defines the initial form of SUB-archetype, that is expected to .receive_messages()
your code does not show, how do you control who goes first on a connection setup -- whether .bind() or .connect() appears at random or before/after the other
your code does not show any subscription setup, after the SUB-archetype was instantiated. A default value upon a socket instantiation does need to be modified via a .setsockopt( zmq.SUBSCRIBE = '') method, otherwise there is a prohibitive filter that does not allow any ( yet unsubscribed ) message to pass through and got-output ( "received" ) on the SUB-side
Must modify a default SUB-side subscription filter, it is prohibitive
You may have noticed from the ZeroMQ documentation, that until setup otherwise, the sub-side does filter-out all incoming messages.
http://api.zeromq.org/2-1:zmq-setsockopt
"The ZMQ_SUBSCRIBE option shall establish a new message filter on a ZMQ_SUB socket. Newly created ZMQ_SUB sockets shall filter out all incoming messages, therefore you should call this option to establish an initial message filter.
An empty option_value of length zero shall subscribe to all incoming messages. A non-empty option_value shall subscribe to all messages beginning with the specified prefix. Multiple filters may be attached to a single ZMQ_SUB socket, in which case a message shall be accepted if it matches at least one filter."
Class-method pre-configuration of a Context instance possible
There is another possibility for a python code using pyzmq 13.0+. There you may also setup this via a Context class-method .setsockopt( zmq.SUBSCRIBE, "" ) et al, but such call has to precede the new socket instantiation from a Context-instance pre-configured this way.