I tried my subscriber which is written using Paho python client with HiveMQ broker and it worked just fine, but it is not working with IBM.
from Subscribing to application status messages, and this question, I implemented suscriber client as following (I got the "a:<ORG-ID>:<App-ID>" from the apps section of my IBM Watson platform):
def on_connect(client, userdata, flags, rc):
print("CONNACK received with code %d." % (rc))
(result, mid) = client.subscribe("iot-2/app/MyAppID/sensordata", 2)
print("result: ", result, ", mid: ", mid)
if result == paho.MQTT_ERR_SUCCESS:
print("success in subscribing.")
def on_subscribe(client, userdata, mid, granted_qos):
print("Subscribed: "+str(mid)+" "+str(granted_qos))
client = paho.Client("a:<ORG-ID>:<App-ID>")
# adding callbacks to client
client.on_connect = on_connect
client.on_subscribe = on_subscribe
client.on_message = on_message
client.username_pw_set("a-<ORG-ID>-<App-ID>","my authentication token")
client.tls_set( ca_certs=None, certfile=None, keyfile=None, cert_reqs=ssl.CERT_REQUIRED,
tls_version=ssl.PROTOCOL_TLS, ciphers=None)
client.connect("<ORG-ID>.messaging.internetofthings.ibmcloud.com", 8883, 60)
client.loop_start()
When I run the project, I get rc with value of 0 which means successful connection.
and this is the on_connect() callback prints:
CONNACK received with code 0.
result: 0 , mid: 2
success in subscribing.
And the on_subscribe() callback is not being called. what am I doing wrong?
If you want to Subscribe to application status messages then
An application can subscribe to monitor status of one or more applications, for example:
Subscribe to topic iot-2/app/appId/mon
Note: To subscribe to updates for all applications, use the MQTT "any" wildcard character (+) for the appId comp
Based on above, the line:
(result, mid) = client.subscribe("iot-2/app/MyAppID/sensordata", 2)
should be
(result, mid) = client.subscribe("iot-2/app/MyAppID/mon", 2)
or
(result, mid) = client.subscribe("iot-2/app/+/mon", 2)
If want to receive sensor data, then use the below line:
Subscribe to topic iot-2/type/device_type/id/device_id/evt/event_id/fmt/format_string
You would need to replace: device_type, device_id, event_id, format_string(could be json, txt)
For every possible event:
(result, mid) = client.subscribe("iot-2/type/+/id/+/evt/+/fmt/+",2)
Related
for my backend micro-service application i am using RabbitMq as the message broker.
For the existing services i am getting events well. My question is how to pull all the old events into a new micro-service which would launch in future.
Just consider if there are three services currently
Product
Notification
Order
if i created a new product it information will be broadcasted to Notification service as well as Order service. So a record of product will be there in both Notification and Order.
so, after a while(when around 500+ products where added), if i had created new service called Analytic’s and wanted all the product created events to be listened when it is initially up.
I am using Python, RabbitMQ & Pika library.
this is my sample code
Sample Publisher code
import pika
import sys, random
connection = pika.BlockingConnection(pika.URLParameters('<rabbitmq-link>'))
channel = connection.channel()
channel.exchange_declare(exchange='group', exchange_type='fanout')
message = "info: Hello World!"
channel.basic_publish(exchange='group', routing_key='', body=message)
print(" [x] Sent %r" % message)
connection.close()
Service one code
import pika
connection = pika.BlockingConnection(pika.URLParameters('<rabbitmq-link>'))
channel = connection.channel()
channel.exchange_declare(exchange='group', exchange_type='fanout')
result = channel.queue_declare(queue='group-1', exclusive=False)
queue_name = result.method.queue
channel.queue_bind(exchange='group', queue=queue_name)
print(' [*] Waiting for logs. To exit press CTRL+C')
def callback(ch, method, properties, body):
print(" [x] %r" % body)
channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
channel.start_consuming()
Service two code
import pika
connection = pika.BlockingConnection(pika.URLParameters('<rabbitmq-link>'))
channel = connection.channel()
channel.exchange_declare(exchange='group', exchange_type='fanout')
result = channel.queue_declare(queue='group-2', exclusive=False)
queue_name = result.method.queue
channel.queue_bind(exchange='group', queue=queue_name)
print(' [*] Waiting for logs. To exit press CTRL+C')
def callback(ch, method, properties, body):
print(" [x] %r" % body)
channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
channel.start_consuming()
So, i want a way where when i made the third service live, it should be able to triggered by old events
I am trying to build a publisher/subscriber system. In simple words, a publisher can publish messages to all subscribers via a broker. Subscribers can support 2 functionalities: 1) receive a message from the broker 2) send a message to the broker with input method. But there is a problem: Subscribers block while waiting the input from stdin and they receive the publishers messages after input, even though no input is needed.
Ι wοuld like to solve the problem in this direction: while waiting for input from stdin, a subscriber can receive a message from publish. I tried "curses" but i failed.
I post a part of my code
subscriber.py
while True:
try:
command = input("Enter a command: ")
#command = sys.stdin.readline()
if command == "quit":
break
command.strip()
command_to_broker = process_command(command, sub_id)
if command_to_broker == error_message:
print("Command with wrong format!")
continue
sock.sendall(bytes(command_to_broker, ENCODING))
received = str(sock.recv(BUFFER_SIZE), ENCODING)
print("Received from BROKER: " + received)
except:
print("Error/Disconnect")
broker.py (starts a publisher thread)
def publisher_thread(connection, topics_and_subscribers, subscribers_and_ports, subscribers):
while True:
data = connection.recv(BUFFER_SIZE)
print("Command from PUBLISHER {}".format(data.decode(ENCODING)))
response = 'OK'
command = data.decode(ENCODING).split(" ", 3)
pub_id = command[0]
topic = command[2]
message = command[3]
if topic in topics_and_subscribers:
for s in subscribers:
print(s.getpeername())
sum = s.send(bytes(message, ENCODING))
print(sum)
if not data:
break
connection.sendall(bytes(response, ENCODING))
connection.close()
I am following aws api to connect to mqtt over websockets. Below is my code:
credentials_provider = AwsCredentialsProvider.new_static(
access_key_id = auth_response_dictionary['user']['accessKeyId'],
secret_access_key = auth_response_dictionary['user']['secretKey'],
session_token = auth_response_dictionary['user']['sessionToken']
)
event_loop_group = io.EventLoopGroup(1)
host_resolver = io.DefaultHostResolver(event_loop_group)
client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver)
mqtt_connection = mqtt_connection_builder.websockets_with_default_aws_signing(
endpoint=auth_response_dictionary['user']['iotEndpoint'],
region=auth_response_dictionary['user']['region'],
credentials_provider=credentials_provider,
client_bootstrap=client_bootstrap,
client_id=clientId
)
print("Connecting to aws")
# Make the connect() call
connect_future = mqtt_connection.connect()
# Future.result() waits until a result is available
print('connect_future ' + str(connect_future))
x= connect_future.result()
print('connect_future ' + str(x))
print("Connected!")
future, packet_id = mqtt_connection.publish(topic=TOPIC, payload=json.dumps(message), qos=mqtt.QoS.AT_LEAST_ONCE)
future, packet_id = mqtt_connection.publish(topic='test/po', payload=json.dumps(message), qos=mqtt.QoS.AT_LEAST_ONCE)
print('future ' + str(future))
print('future ' + str(packet_id))
print('Publish End')
I am not getting any error while connecting and while publishing but I am not receiving any msgs on my aws mqtt broker when I subscribe to that topic there in 'Test' section.
I think that i have configured something wrong in either credentials_provider or client_bootstrap or both but dont know what.
Here are the printed logs
Connecting to aws
connect_future<Future at 0x7f605f942af0 state=pending>
connect_future{'session_present': False}
Connected!
future <Future at 0x7f605f8e54f0 state=pending>
future 3
Publish End
Can somebody please help?
mqtt_connection.subscribe(...) is used to subscribe to an MQTT topic for AWS IoT messages, which I can't see anywhere in your code.
mqtt_connection.subscribe is called like below, taking in the topic name, a Quality of Service level and a callback.
received_count = 0
received_all_event = threading.Event()
...
topic='test/po'
print("Subscribing to topic '{}'...".format(topic))
subscribe_future, packet_id = mqtt_connection.subscribe(
topic=topic,
qos=mqtt.QoS.AT_LEAST_ONCE,
callback=on_message_received)
subscribe_result = subscribe_future.result()
print("Subscribed with {}".format(str(subscribe_result['qos'])))
on_message_received can look like this:
def on_message_received(topic, payload, dup, qos, retain, **kwargs):
print("Received message from topic '{}': {}".format(topic, payload))
global received_count
received_count += 1
# Number of messages to wait for
if received_count = 10:
received_all_event.set()
Then in your main method, you can wait until you've received 10 messages:
# Wait for all messages to be received.
# This waits forever if count was set to 0.
if not received_all_event.is_set():
print("Waiting for all messages to be received...")
received_all_event.wait()
print("{} message(s) received.".format(received_count))
There's really good sample code provided by AWS, which I'd recommend you check out.
I'm trying to make translate bot using twitchdev python code: https://github.com/twitchdev/chat-samples/blob/master/python/chatbot.py
I was able to get messages from channels and send translated text, but I cannot join multiple channels.
What I did below is making list of channels and call using for loop, but it only join to the last channel.
I tried to make another list of channels in def on_welcome(self, c, e) but it also works on the last channel (when I print self.channels in def on_welcome(self, c, e) it printed blank list, and when I print self.channel it only printed last channel)
Any suggestions are welcome.
import sys
import irc.bot
import requests
import config
class TwitchBot(irc.bot.SingleServerIRCBot):
def __init__(self, username, client_id, token, channels):
for channel in channels
self.client_id = client_id
self.token = token
self.channel = '#' + channel
# Get the channel id, we will need this for v5 API calls
url = 'https://api.twitch.tv/kraken/users?login=' + channel
headers = {'Client-ID': client_id, 'Accept': 'application/vnd.twitchtv.v5+json'}
r = requests.get(url, headers=headers).json()
self.channel_id = r['users'][0]['_id']
# Create IRC bot connection
server = 'irc.chat.twitch.tv'
port = 6667
print 'Connecting to ' + server + ' on port ' + str(port) + '...'
irc.bot.SingleServerIRCBot.__init__(self, [(server, port, 'oauth:'+token)], username, username)
def on_welcome(self, c, e):
print 'Joining ' + self.channel
# You must request specific capabilities before you can use them
c.cap('REQ', ':twitch.tv/membership')
c.cap('REQ', ':twitch.tv/tags')
c.cap('REQ', ':twitch.tv/commands')
c.join(self.channel)
def on_pubmsg(self, c, e):
# If a chat message starts with an exclamation point, try to run it as a command
if e.arguments[0][:1] == '!':
cmd = e.arguments[0].split(' ')[0][1:]
print 'Received command: ' + cmd
self.do_command(e, cmd)
return
def do_command(self, e, cmd):
c = self.connection
# Provide basic information to viewers for specific commands
elif cmd == "raffle":
message = "This is an example bot, replace this text with your raffle text."
c.privmsg(self.channel, message)
elif cmd == "schedule":
message = "This is an example bot, replace this text with your schedule text."
c.privmsg(self.channel, message)
def main():
if len(sys.argv) != 5:
print("Usage: twitchbot <username> <client id> <token> <channel>")
sys.exit(1)
username = config.twitch['botname']
client_id = config.twitch['cliendID']
token = config.twitch['oauth']
channels = ["channel1", "channel2"]
bot = TwitchBot(username, client_id, token, channels)
bot.start()
if __name__ == "__main__":
main()
In your main() function, you are establishing an object of the bot:
bot = TwitchBot(username, client_id, token, channels)
Instead of passing multiple channels to the same bot object, create multiple bot objects with single channels. I haven't tested this yet because Twitch updated their OAuth protocol and I haven't been through the docs.
You may need to thread them, and depending on what functions you implement it may not be thread-safe (just test before going into production). That'd probably look something like:
import threading
t1 = threading.Thread(target=TwitchBot, args=(username, client_id, token, channel1))
t2 = threading.Thread(target=TwitchBot, args=(username, client_id, token, channel2))
t1.setDaemon(True)
t2.setDaemon(True)
t1.start()
t2.start()
Once again, I haven't tested any of this but will likely have time in a few weeks to patch this in my own code after the holiday.
I want to establish publish subscribe communication between to machines.
The two machines, that I have, are ryu-primary and ryu-secondary
The steps I follow in each of the machines are as follows.
In the initializer for ryu-primary (IP address is 192.168.241.131)
self.context = zmq.Context()
self.sub_socket = self.context.socket(zmq.SUB)
self.pub_socket = self.context.socket(zmq.PUB)
self.pub_port = 5566
self.sub_port = 5566
def establish_zmq_connection(self): # Socket to talk to server
print( "Connection to ryu-secondary..." )
self.sub_socket.connect( "tcp://192.168.241.132:%s" % self.sub_port )
def listen_zmq_connection(self):
print( 'Listen to zmq connection' )
self.pub_socket.bind( "tcp://*:%s" % self.pub_port )
def recieve_messages(self):
while True:
try:
string = self.sub_socket.recv( flags=zmq.NOBLOCK )
print( 'flow mod messages recieved {}'.format(string) )
return string
except zmq.ZMQError:
break
def push_messages(self,msg):
self.pub_socket.send( "%s" % (msg) )
From ryu-secondary (IP address - 192.168.241.132)
In the initializer
self.context = zmq.Context()
self.sub_socket = self.context.socket(zmq.SUB)
self.pub_socket = self.context.socket(zmq.PUB)
self.pub_port = 5566
self.sub_port = 5566
def establish_zmq_connection(self): # Socket to talk to server
print( "Connection to ryu-secondary..." )
self.sub_socket.connect( "tcp://192.168.241.131:%s" % self.sub_port )
def listen_zmq_connection(self):
print( 'Listen to zmq connection' )
self.pub_socket.bind( "tcp://*:%s" % self.pub_port )
def recieve_messages(self):
while True:
try:
string = self.sub_socket.recv( flags=zmq.NOBLOCK )
print( 'flow mod messages recieved {}'.format(string) )
return string
except zmq.ZMQError:
break
def push_messages(self,msg):
print( 'pushing message to publish socket' )
self.pub_socket.send( "%s" % (msg) )
These are the functions that I have.
I am calling on ryu-secondary:
establish_zmq_connections()
push_messages()
But I am not recieving those messages on ryu-primary, when I call
listen_zmq_connection()
recieve_messages()
Can someone point out to me what I am doing wrong?
Repair the PUB/SUB messaging pattern setup
There are several important steps in making the PUB/SUB pattern work.
All this is well described in the ZeroMQ documentation.
You need not repeat both pub & sub parts of code on both sides, the more that it masks, as A side-effect thereof, the case if you mix the pub and sub socket addresses/ports/calls/etc in an "opposite" node code and you do not see such a principal collision.
your code defines the initial form of PUB-archetype, that is expected to .push_messages()
your code defines the initial form of SUB-archetype, that is expected to .receive_messages()
your code does not show, how do you control who goes first on a connection setup -- whether .bind() or .connect() appears at random or before/after the other
your code does not show any subscription setup, after the SUB-archetype was instantiated. A default value upon a socket instantiation does need to be modified via a .setsockopt( zmq.SUBSCRIBE = '') method, otherwise there is a prohibitive filter that does not allow any ( yet unsubscribed ) message to pass through and got-output ( "received" ) on the SUB-side
Must modify a default SUB-side subscription filter, it is prohibitive
You may have noticed from the ZeroMQ documentation, that until setup otherwise, the sub-side does filter-out all incoming messages.
http://api.zeromq.org/2-1:zmq-setsockopt
"The ZMQ_SUBSCRIBE option shall establish a new message filter on a ZMQ_SUB socket. Newly created ZMQ_SUB sockets shall filter out all incoming messages, therefore you should call this option to establish an initial message filter.
An empty option_value of length zero shall subscribe to all incoming messages. A non-empty option_value shall subscribe to all messages beginning with the specified prefix. Multiple filters may be attached to a single ZMQ_SUB socket, in which case a message shall be accepted if it matches at least one filter."
Class-method pre-configuration of a Context instance possible
There is another possibility for a python code using pyzmq 13.0+. There you may also setup this via a Context class-method .setsockopt( zmq.SUBSCRIBE, "" ) et al, but such call has to precede the new socket instantiation from a Context-instance pre-configured this way.