I am trying to subscribe on three different topics using single subscriber client. But with the below mentioned code I am able to get data from only one server.
Please suggest ant modification in my code that can be implemented to get the desired data from the different publisher clients.
# Define Variables
MQTT_BROKER = "10.97.143.44"
MQTT_PORT = 11883
MQTT_TOPIC = [("Server1/kpi1"),("Server2/kpi2"),("Server3/kpi3")
def on_connect(client, userdata, flags, rc):
if rc == 0:
print("Connected to broker")
global Connected #Use global variable
Connected = True #Signal connection
else:
print("Connection failed")
def on_message(client, userdata, message):
data = message.payload
receive=data.decode("utf-8")
m_decode = json.loads(receive)
#print(m_decode)
#print (m_decode['Server_name'])
print ("Message received: " + str(m_decode))
Connected = False #global variable for the state of the connection
client = mqttClient.Client("Python") #create new instance
client.on_connect= on_connect #attach function to callback
client.on_message= on_message #attach function to callback
client.connect(MQTT_BROKER,MQTT_PORT) #connect to broker
client.loop_start() #start the loop
while Connected != True: #Wait for connection
time.sleep(0.1)
client.subscribe(MQTT_TOPIC)
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
print ("exiting")
List item
client.disconnect()
client.loop_stop()
Your MQTT_TOPIC array should contain QOS levels as well as topic names.
From the doc:
String and integer tuple
e.g. subscribe(("my/topic", 1))
topic
a tuple of (topic, qos). Both topic and qos must be present in
the tuple.
qos
not used.
e.g.
MQTT_TOPIC = [("Server1/kpi1",0),("Server2/kpi2",0),("Server3/kpi3",0)]
Related
I need to implement a bidirectional streaming in gRPC with python. What I am implementing is a bit different from the example on gRPC's tutorial. In my implementation I need to keep sending responses to the client after initial subscribe request till the client unsubscribes. I have provided a sample implementation below. The main problem I face is that for 1 client the implementation works normally, however the moment I run 2 clients parallely, the response is send to the clients alternatively. Is there any way to stop this behavior?
Server Code
class ChatServicer(route_guide_pb2_grpc.ChatServicer):
"""Provides methods that implement functionality of route guide server."""
def __init__(self):
self.db = None#route_guide_resources.read_route_guide_database()
self.prev_notes = []
self._watch_response_queue = SimpleQueue()
self._watch_response_queue2 = []
def process_request_note(self, note):
while True:
time.sleep(1.0)
self._watch_response_queue.put(note)
def RouteChat(self, request_iterator, context):
print("ROuteCHat")
temp = None
for new_note in request_iterator:
temp = new_note
for prev_note in self.prev_notes:
if prev_note.location == new_note.location:
yield prev_note
self.prev_notes.append(new_note)
process_request_thread = Thread(
target=self.process_request_note,
args=(temp,),
)
process_request_thread.start()
lock = Lock()
while True:
with lock:
while not self._watch_response_queue.empty():
yield self._watch_response_queue.get(block=True, timeout=2.0)
# while len(self._watch_response_queue2) > 0:
# yield self._watch_response_queue2.pop()
time.sleep(1.0)
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
route_guide_pb2_grpc.add_ChatServicer_to_server(
ChatServicer(), server)
server.add_insecure_port('[::]:50051')
server.start()
print("server started")
try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
server.stop(0)
Client Code
def generate_messages():
messages = [
make_route_note("First", 0, 0),
make_route_note("Second", 0, 1),
make_route_note("Third", 1, 0),
make_route_note("Fourth", 0, 0),
make_route_note("Fifth", 1, 0),
]
for msg in messages:
print("Sending %s at %s" % (msg.message, msg.location))
yield msg
def guide_route_chat(stub):
responses = stub.RouteChat(generate_messages())
ctr = 0
for response in responses:
ctr += 1
print(f"Received message{ctr} %s at %s" %
(response.message, response.location))
def run():
# NOTE(gRPC Python Team): .close() is possible on a channel and should be
# used in circumstances in which the with statement does not fit the needs
# of the code.
with grpc.insecure_channel('localhost:50051') as channel:
stub = route_guide_pb2_grpc.ChatStub(channel)
print("-------------- RouteChat --------------")
guide_route_chat(stub)
For the second client I use the same code as the client above.
TLDR: Grpc server is sending response alternatively to multiple clients whereas it is desired for only 1 client to receive message at a time.
I've got a Python script that implements the Paho library to subscribe to and republish MQTT topics coming from a Mosquitto server. My script works fine when I invoke it from the terminal. What is tripping me up is trying to daemonize the script to run unattended. I'm using the daemonize library and while I can get the script to start as daemon, it doesn't actually do anything. When I run the non-daemonized script, I see the MQTT messages received at the endpoint I've specified, but when the same script is running as a daemon, no messages are showing up at my receiving endpoint. I'm including both the non-daemonized and daemonized scripts for reference.
Any ideas as to why the non-daemonized script works but the daemonized script does not?
Non-Daemonized
import paho.mqtt.client as mqtt
import random
import json
import time
#replace [device-id] with your device you created in IoT Hub.
iothubmqtttopic = "devices/MoxieSensorsBHM/messages/events/"
#define on_connect function
def on_connect(client, userdata, flags, rc):
print("Connected with result code " + str(rc))
client.subscribe("MW/oneal/Tag/#")
#define on_message function - this function translates the incoming topic
#and publishes to the specified topic for IoT Hub
def on_message(client, userdata, message):
global iothubmqtttopic
topic = message.topic
if(topic != iothubmqtttopic):
#extract the sensor ID from the topic
splitTopic = topic.split("/") #split the topic into a list
sensorID = splitTopic[3] #the 3rd item in the list is the sensor ID
msgType = splitTopic[4] #the 4th item in the list is the type (e.g. status, UWB)
#convert the json response to a python dictionary object
m_decode = str(message.payload.decode("utf-8"))
m_in = json.loads(m_decode)
#get current time for timestamp
ts = time.gmtime()
tsFormatted = time.strftime("%Y-%m-%d %H:%M:%S", ts)
#add new elements to the dictionary for topic and sensor ID
m_in["topic"] = topic
m_in["sensorID"] = sensorID
m_in["messageType"] = msgType
m_in["timestamp"] = tsFormatted
#set the propertybag for IoT Hub
propertyBag = "topic=" + topic + "&sensorID=" + sensorID + "&messageType=" + msgType
print(propertyBag)
#convert back to JSON
m_encode = json.dumps(m_in)
#create an array of all the possible sensor IDs
#print to screen and publish to IoT Hub
print("Topic: ", topic)
print("Message received: ", m_encode)
print("IoT Hub Topic: ", iothubmqtttopic)
client.publish(iothubmqtttopic, m_encode)
# replace <broker address> with the FQDN or IP address of your MQTT broker
broker_address="localhost"
#create the client and connect to the broker
print("creating new instance")
client = mqtt.Client("iottopicxlate" + str(random.randrange(1,1000000))) #create new instance
client.on_message=on_message #attach function to callback
print("connecting to broker")
client.connect(broker_address) #connect to broker
client.on_connect = on_connect #attach function to callback
client.loop_forever() #stop the loop
Daemonized
import paho.mqtt.client as mqtt
import random
import json
import time
import os, sys
from daemonize import Daemonize
def main():
#replace [device-id] with your device you created in IoT Hub.
iothubmqtttopic = "devices/MoxieSensorsBHM/messages/events/"
#define on_connect function
def on_connect(client, userdata, flags, rc):
#print("Connected with result code " + str(rc))
client.subscribe("MW/oneal/Tag/#")
#define on_message function - this function translates the incoming topic
#and publishes to the specified topic for IoT Hub
def on_message(client, userdata, message):
global iothubmqtttopic
topic = message.topic
if(topic != iothubmqtttopic):
#extract the sensor ID from the topic
splitTopic = topic.split("/") #split the topic into a list
sensorID = splitTopic[3] #the 3rd item in the list is the sensor ID
msgType = splitTopic[4] #the 4th item in the list is the type (e.g. status, UWB)
#convert the json response to a python dictionary object
m_decode = str(message.payload.decode("utf-8"))
m_in = json.loads(m_decode)
#get current time for timestamp
ts = time.gmtime()
tsFormatted = time.strftime("%Y-%m-%d %H:%M:%S", ts)
#add new elements to the dictionary for topic and sensor ID
m_in["topic"] = topic
m_in["sensorID"] = sensorID
m_in["messageType"] = msgType
m_in["timestamp"] = tsFormatted
#set the propertybag for IoT Hub
propertyBag = "topic="+topic+"&sensorID="+sensorID+"&messageType="+msgType
#print(propertyBag)
#convert back to JSON
m_encode = json.dumps(m_in)
#print to screen and publish to IoT Hub
print("Topic: ", topic + propertyBag)
print("Message received: ", m_encode)
client.publish(iothubmqtttopic, m_encode)
# replace <broker address> with the FQDN or IP address of your MQTT broker
broker_address="localhost"
#create the client and connect to the broker
#print("creating new instance")
client = mqtt.Client("iottopicxlate" + str(random.randrange(1,1000000))) #create new instance
client.on_message=on_message #attach function to callback
#print("connecting to broker")
client.connect(broker_address) #connect to broker
client.on_connect = on_connect #attach function to callback
client.loop_forever() #stop the loop
#start the daemon
topictransdpid = os.path.basename(sys.argv[0])
pidfile = "topictransd.pid"
daemon = Daemonize(app = "topictransd", pid = pidfile, action = main)
daemon.start()
Update
Vincent, I'm trying to implement your suggestion to write to a file for debug. Please bear with me as I'm learning Python on the fly. I've added the following code snip to the on_message function in the non-daemonized (i.e. working) version of the script and I see the messages written to text files in my "debug" directory. When I implement the same snip in the daemonized version, no files are written. So while my daemon is running, it's not actually doing anything.
f = open("debug/" + sensorID+tsFormatted + ".txt", "x")
f.write(m_encode)
f.close
Any ideas as to what I'm missing?
Update 2
I've implemented a simple logger to write a debug message when the script starts and and another to write a message when daemonize calls the main() function. My log file has an entry for the script starting, but there is not an entry for when main() is called - it's as though daemonize isn't executing the main() function? Here is my updated daemon script with the logger enabled:
import paho.mqtt.client as mqtt
import random
import json
import time
import logging
import os, sys
from daemonize import Daemonize
def main():
logger.warning("main has been started") # write a log file entry to indicate main has been called by daemonize
#replace [device-id] with your device you created in IoT Hub.
iothubmqtttopic = "devices/MoxieSensorsBHM/messages/events/"
#define on_connect function
def on_connect(client, userdata, flags, rc):
#print("Connected with result code " + str(rc))
client.subscribe("MW/oneal/Tag/#")
#define on_message function - this function translates the incoming topic
#and publishes to the specified topic for IoT Hub
def on_message(client, userdata, message):
global iothubmqtttopic
topic = message.topic
if(topic != iothubmqtttopic):
#extract the sensor ID from the topic
splitTopic = topic.split("/") #split the topic into a list
sensorID = splitTopic[3] #the 3rd item in the list is the sensor ID
msgType = splitTopic[4] #the 4th item in the list is the type (e.g. status, UWB)
#convert the json response to a python dictionary object
m_decode = str(message.payload.decode("utf-8"))
m_in = json.loads(m_decode)
#get current time for timestamp
ts = time.gmtime()
tsFormatted = time.strftime("%Y-%m-%d %H:%M:%S", ts)
#add new elements to the dictionary for topic and sensor ID
m_in["topic"] = topic
m_in["sensorID"] = sensorID
m_in["messageType"] = msgType
m_in["timestamp"] = tsFormatted
#set the propertybag for IoT Hub
propertyBag = "topic="+topic+"&sensorID="+sensorID+"&messageType="+msgType
#print(propertyBag)
#convert back to JSON
m_encode = json.dumps(m_in)
#print to screen and publish to IoT Hub
#print("Topic: ", topic + propertyBag)
#print("Message received: ", m_encode)
client.publish(iothubmqtttopic, m_encode)
#write the message to a debug file
f = open("debug/" + sensorID+tsFormatted + ".txt", "w")
f.write(m_encode)
f.close
# replace <broker address> with the FQDN or IP address of your MQTT broker
broker_address="localhost"
#create the client and connect to the broker
#print("creating new instance")
client = mqtt.Client("iottopicxlate" + str(random.randrange(1,1000000))) #create new instance
#create a logger
#logging.basicConfig(level=logging.DEBUG, filename="topictransd.log", format="%(asctime)s %(message)s", filemode="w")
#logger = logging.getLogger()
#client.enable_logger(logger)
client.on_message=on_message #attach function to callback
#print("connecting to broker")
client.connect(broker_address) #connect to broker
client.on_connect = on_connect #attach function to callback
#start the loop
client.loop_forever()
#start the daemon
logging.basicConfig(level=logging.DEBUG, filename="topictransd.log", format="%(asctime)s %(message)s", filemode="w")
logger = logging.getLogger()
pidfile = "topictransd.pid"
logger.warning("successfully started the script") # write a log file entry to indicate the script has successfully started
daemon = Daemonize(app = "topictransd", pid = pidfile, action = main)
daemon.start()
#!/usr/bin/env python
import sys
from arduino.Arduino import Arduino
import paho.mqtt.client as mqtt
import serial
import time
pin = 13
broker_adress = "10.0.2.190"
sys.path.append("/home/hu/Schreibtisch/Arduino_BA_2.0/Probe_Programmierung/Python-Arduino-Proto-API-v2/arduino")
ser = serial.Serial('/dev/ttyACM0', 9600)
b = Arduino('/dev/ttyACM0')
b.output([pin])
b.setLow(pin)
gassensor_value = "no default_value"
sensor_value = [['/Deutschland/Osnabrueck/Coffee-bike-1/Sensor_1',gassensor_value]]
#########################################################################
# Callback_1 for relay
#on_connect1,on_disconnect1,on_subscribe１on_message_1
#########################################################################
def on_connect(mqttrelay, obj, flags, rc):
if rc != 0:
exit(rc)
else:
mqttrelay.subscribe("qos0/test", 0)
def on_disconnect(mqttrelay, obj, rc):
obj = rc
def on_subscribe(mqttrelay, obj, mid, granted_qos):
print(mqttrelay.subscribe("qos0/test", 0))
print("Waiting for the subscribed messages")
def on_message(mqttrelay,userdata, message):
a = str(message.payload.decode("utf-8"))
print(a)
if (a == "1" or a == "0"):
if (a == "1"):
b.setHigh(13)
time.sleep(10)
else:
b.setLow(13)
time.sleep(10)
else:
print("please publish the message 1 or 0")
#########################################################################
# Callback_2 for gassensor
# on_connect2,on_publish2
#########################################################################
def on_publish(mqttgassensor, obj, mid):
print("mid: " + str(mid))
def on_connect(mqttgassensor, userdata, flags, rc):
print("Connected with result code " + str(rc))
#create new instance to subscribe the sitution of relay
mqttrelay = mqtt.Client("relay_K_12", 1)
#create new instance to publish the situation of gassensor
mqttgassensor = mqtt.Cleint("gassensor",1)
#the events and callbacks of instance mqttrelais associate with each other:
mqttrelay.on_message = on_message
mqttrelay.on_connect = on_connect
mqttrelay.on_subscribe = on_subscribe
mqttrelay.on_disconnect = on_disconnect
mqttrelay.connect(broker_adress)
#the events and callbacks of instance gassensor associate with each other:
mqttgassensor.on_connect = on_connect
mqttgassensor.on_publish = on_publish
mqttgassensor.connect(broker_adress)
while True:
mqttrelay.loop_start()
time.sleeps(2)
mqttrelay.loop_stop()
print("relay 開始循環")
mqttgassensor.loop_start()
mqttgassensor.loop()
time.sleep(1)
sensor_value[0][1] = ser.readline()
if (sensor_value[0][1] != "no default_value" or sensor_value[0][1] != b''):
print(sensor_value[0])
mqttgassensor.publish("/Deutschland/Osnabrueck/Coffee-bike-1/Sensor_1", sensor_value[0][1])
mqttgassensor.loop_stop()
Hello, everyone. I want to accomplish two instances in this script.
Through the publish we can get the data from the gassensor. Because at the top of this script i have imported the serial modul with that we can accomplish the communication between the arduino and raspberry pi.
I want to use the subscribe to get the command(1 or 0) from the server. the number 1 can active the Relay and the 0 can deactive the relay.
I have tried to accomplish the two thoughts lonely and successfully. But the Combination gives me no reply.
You only need 1 instance of the MQTT client since there is only 1 broker. You can subscribe and publish to multiple topics from a single client.
You should connect this 1 instance and then start the network loop in the background with client.start_loop()
You can then run your own loop to read from the serial port.
this is my code and I need add,
(1) loop(thread1) which can publish data when receive specific data by UART(ser.read())
(2) loop(thread2) which can run report_temp_humidity() with timed delay
so I put thread for test but this runs only one time, not loop.
Is this cause by "mqttclient.loop_forever()" at the end of line?
Need some help on this.
import paho.mqtt.client as mqtt
import Adafruit_DHT
import Adafruit_BBIO.UART as UART
import serial
import threading
import time
# DHT-11 Temperature & Humidity
sensor = Adafruit_DHT.DHT11
sensor_pin = 'P8_11'
# UART
UART.setup("UART1")
ser = serial.Serial(port = "/dev/ttyO1", baudrate=9600)
ser.close()
ser.open()
# Define event callbacks
def on_connect(self, client, userdata, rc):
if rc == 0:
print("Connected successfully.")
else:
print("Connection failed. rc= "+str(rc))
def on_publish(client, userdata, mid):
print("Message "+str(mid)+" published.")
def on_subscribe(client, userdata, mid, granted_qos):
print("Subscribe with mid "+str(mid)+" received.")
def on_message(client, userdata, msg):
print("Message received on topic "+msg.topic+" with QoS "+str(msg.qos)+"and payload "+msg.payload)
if msg.payload == "check_status":
report_temp_humidity()
if msg.payload == "motor_run!":
print "Send command to RF"
ser.write("11.1111[S]")
# Define my function prototypes
def report_temp_humidity():
hum, temp = Adafruit_DHT.read_retry(sensor, sensor_pin)
mqttclient.publish("/my_topic1/", "Temperature "+ str(temp) + "`C", qos=0)
mqttclient.publish("/my_topic1/", "Humidity " + str(hum) + "%", qos=0)
# Define thread(loop) test
def loop_test(delay):
print(loop!)
time.sleep(1)
t1 = threading.Thread(target=loop_test, args=('1,',))
t1.start()
mqttclient = mqtt.Client()
# Assign event callbacks
mqttclient.on_connect = on_connect
mqttclient.on_publish = on_publish
mqttclient.on_subscribe = on_subscribe
mqttclient.on_message = on_message
# Connect
mqttclient.username_pw_set("dioty_user", "1234567")
mqttclient.connect("mqtt.dioty.co", 1883)
# Start subscription
mqttclient.subscribe("/my_topic1/")
# Publish a message
mqttclient.publish("/my_topic1/", "Hello World Message!")
# Loop
mqttclient.loop_forever()
This has nothing to do with the MQTT library.
Your thread t1 will run the loop_test function, which only has 2 statements in it. It will run these (once) then exit, there is nothing to tell it to continue to do anything else.
If you want that thread to continue to run for ever then you will need to include a loop statement of some sort (e.g. while True).
Im trying to make a small 3 connected robots. They have to send and receive sensor data from each other and take an action to make the reads tends to zero again .
Iam trying to exchange the angle of the movements to let the other one follow him . Iam using the MQTT and it works very well .
But when i try to get the data from def on_message to use it in the next while the variable isn't global .
the code as follow :
############### MQTT section ##################
# when connecting to mqtt do this;
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe(sub_topic)
# when receiving a mqtt message do this;
def on_message(client, userdata, msg):
masterangel = int(msg.payload)
print ( masterangel )
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(Broker, 1883, 60)
client.loop_start()
# Start the Program
psm.screen.termPrintAt(1, "Press Go to stop program")
while(not doExit):
oldAngel = Angel
Angel = gyro.readValue()
Angelshow = "Current Angel ="+" "+str (Angel)
if ( oldAngel != Angel):
psm.screen.termPrintAt(5, Angelshow)
if (Angel < masterangel) :
psm.BBM1.setSpeedSync(20)
psm.BAM1.floatSync()
elif (masterangel < Angel ) :
psm.BAM1.setSpeedSync(20)
psm.BBM1.floatSync()
client.publish(pub_topic, "test")
now has anyone an Idea how can i use the variable "masterangel" in the while loop ?
by the way the print order of works fine .
print ( masterangel )
thank you in Advance and i appreciate your Help
You need to initialise the variable outside the scope of the onMessage function and then mark it as a global variable in the onMessage functions.
Take a look at the python global keyword.
############### MQTT section ##################
# when connecting to mqtt do this;
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe(sub_topic)
# when receiving a mqtt message do this;
def on_message(client, userdata, msg):
# use global version not local
global masterangel
masterangel = int(msg.payload)
print ( masterangel )
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(Broker, 1883, 60)
client.loop_start()
# initialise variable to starting value
masterangel = 0
# Start the Program
psm.screen.termPrintAt(1, "Press Go to stop program")
while(not doExit):
oldAngel = Angel
Angel = gyro.readValue()
Angelshow = "Current Angel ="+" "+str (Angel)
if ( oldAngel != Angel):
psm.screen.termPrintAt(5, Angelshow)
if (Angel < masterangel) :
psm.BBM1.setSpeedSync(20)
psm.BAM1.floatSync()
elif (masterangel < Angel ) :
psm.BAM1.setSpeedSync(20)
psm.BBM1.floatSync()
client.publish(pub_topic, "test")
To avoid the problems that may come with scope of variables later on (i.e. mixing local variables with global variables), instead of using the global keyword, take advantage of the return statement.
In other words, return allows you to actually use the arguments you pass into the return function and store it into another variable, whereas print() just prints to the screen.
############### MQTT section ##################
# when connecting to mqtt do this;
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe(sub_topic)
# when receiving a mqtt message do this;
def on_message(client, userdata, msg):
masterangel = int(msg.payload)
return (masterangel) # Use return() instead of print ()
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(Broker, 1883, 60)
client.loop_start()
# Start the Program
psm.screen.termPrintAt(1, "Press Go to stop program")
#Assign the value of masterangel being returned by calling the on_message
#function and storing it into its own variable.
new_var = on_message() # This stores the returned value of masterangel
while(not doExit):
oldAngel = Angel
Angel = gyro.readValue()
Angelshow = "Current Angel ="+" "+str (Angel)
if ( oldAngel != Angel):
psm.screen.termPrintAt(5, Angelshow)
if (Angel < new_var) :
psm.BBM1.setSpeedSync(20)
psm.BAM1.floatSync()
elif (new_var < Angel ) :
psm.BAM1.setSpeedSync(20)
psm.BBM1.floatSync()
client.publish(pub_topic, "test")