I am trying to create an example in which a client issues a message to a thing and the thing replies back to the client. The thing is connected to Eclipse Ditto via a MQTT connection and the client is simulated via a curl command. In order to do so, I took some parts from the following two tutorials: https://www.eclipse.org/ditto/2018-12-05-example-command-and-control.html and https://github.com/eclipse/ditto-examples/tree/master/mqtt-bidirectional.
Besides the fact that Ditto does not route the messages from the client to the thing in a very reliable way (I would say that one out of three messages is not delivered to the client), the client is not able to receive the response message from the thing, even with a very high timeout value.
This is my Python code that acts as the thing:
import logging
import time
import random
import json
import paho.mqtt.client as mqtt
def on_message(client, userdata, message):
response = json.dumps({
"topic": "org.eclipse.ditto/teapot/things/live/messages/brew",
"headers": {
"content-type": "application/json",
"correlation-id": "command-and-control"
},
"path": "/inbox/messages/brew",
"value": {
"eta": 58
},
"status": 200
})
client.publish(outTopic, response)
inTopic = "ditto-tutorial/org.eclipse.ditto:teapot/#";
outTopic = "ditto-tutorial/";
thingId = "teapot";
interval = 10
broker_address = "test.mosquitto.org"
client = mqtt.Client(thingId) #create new instance
client.on_message = on_message #attach function to callback
client.connect(broker_address) #connect to broker
client.loop_start() #start the loop
client.subscribe(inTopic)
while True:
time.sleep(interval)
This is my curl command that simulates the client:
curl -i -X POST 'http://localhost:8080/api/2/things/org.eclipse.ditto:teapot/inbox/messages/brew?timeout=60' \
-u ditto:ditto \
-H 'x-correlation-id: command-and-control' \
-d '{"targetTemperature":85}'
At the beginning I thought that I was doing something wrong with the Ditto protocol, but I believe this is not the case, since the correlation-id is the same in both request and responde and the other fields seem to be OK.
I think the problem might be that you are using a fixed correlation-id for your commands and responses. Try to use a random, e.g. a UUID.
Related
i want to connect to a websocket server, when my graphql server starts, and inside a resolver i want to use send and recv functions of the connected websocket for data communication.
brief about my backend,
i have a python rest client that also have a websocket server, i can fetch solo product details and product list via the websocket server.
(in graphql resolver i want to collect my products data and also an inventory data and merge them both for UI. as node is async programming, and all the example are connect to server then, use a then block and communicate, i dont want to do that i want to use the connection object in resolver and the connection should be done once.)
import { WebSocketClient } from './base';
const productWebSocket = new WebSocketClient("ws://192.168.0.109:8000/abi-connection");
productWebSocket.connect({reconnect: true});
export { productWebSocket };
now i will import this productWebSocket and want to use it in resolvers.
this websocket and graphql shi* isnt that popular, but designing this shi* this way gives me a performance boost, as i use utility functions for both of my restapis and websocket server in core-apis. i call this maksuDII+ way of programming.
i couldnt do this with nodejs and got no help. so i implemented graphql with python and got more better control in websocket client.
i search for ws, websocket some other shitty packages in nodejs none worked
websocket.on('connect', ws=> {
websocket.on('message', data => {
// this shit is a argument how am i suppose to get this value in a express api end point.
// search total 5 pages in google got nothing so have to shift to python.
})
})
python Version
from graphql import GraphQLError
from ..service_base import query
from app.websocket.product_websocket import product_ws_connection
from app.websocket.inventory_websocket import inventory_ws_connection
import json
from app.utils.super_threads import SuperThreads
def get_websocket_data(socket_connection, socket_send_data):
socket_connection.send(json.dumps(socket_send_data))
raw_data = socket_connection.recv()
jsonified_data = json.loads(raw_data)
return jsonified_data
#query.field("productDetails")
def product_details(*_, baseCode: str):
product_ws = product_ws_connection.get_connection() # connected client, with proper connection to my websocket server
inventory_ws = inventory_ws_connection.get_connection()
if not product_ws:
raise GraphQLError("Product Api Down")
if not inventory_ws:
raise GraphQLError("Inventory Api Down")
product_ws_data = {
"operation": "PRODUCT_FETCH",
"baseCode": baseCode
}
inventory_ws_data = {
"operation": "STOCK_FETCH",
"baseCode": baseCode
}
# super threads here is a diffrent topic, it a wrapper around standard python Threads.
ws_product_thread = SuperThreads(target=get_websocket_data, args=(product_ws, product_ws_data))
ws_inventory_thread = SuperThreads(target=get_websocket_data, args=(inventory_ws, inventory_ws_data))
ws_product_thread.start() # asking one of my websocket server for data.
ws_inventory_thread.start() # same with this thread.
product_data_payload = ws_product_thread.join() # i get the what websocket gives me as response
inventory_data_payload = ws_inventory_thread.join() # needed this type of shit in nodejs could not have it.
if "Fetched" in product_data_payload["abi_response_info"]["message"] and \
"Fetched" in inventory_data_payload["abi_response_info"]["message"]:
# data filtering here
return product_data
else:
raise GraphQLError("Invalid Product Code")
I am trying to create a Strava webhook subscription to recieve events from users who have authorised my Strava application. I was able to successfully create a subscription using the code in this Strava tutorial. However, I don't understand Javascript, so can't adapt the code to my needs. I am therefore trying to replicate it's functionality within Python using Django.
My basic approach has been to follow the setup outlined in the Django section of this webpage, then replace the code in the file views.py with the code below. I have tried to make the code function as similarly as possible to the Javascript function within the tutorial I linked above. However, I'm very new to web applications, so have taken shortcuts / 'gone with what works without understang why' in several places.
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.clickjacking import xframe_options_exempt
import json
#csrf_exempt
#xframe_options_exempt
def example(request):
if request.method == "GET":
verify_token = "STRAVA"
str_request = str(request)
try:
mode = str_request[str_request.find("hub.mode=") + 9 : len(str_request) - 2]
token = str_request[str_request.find("hub.verify_token=") + 17 : str_request.find("&hub.challenge")]
challenge = str_request[str_request.find("hub.challenge=") + 14 : str_request.find("&hub.mode")]
except:
return HttpResponse('Could not verify. Mode, token or challenge not valid.')
if (mode == 'subscribe' and token == verify_token):
resp = json.dumps({"hub.challenge":challenge},separators=(',', ':'))
return HttpResponse(resp, content_type='application/json; charset=utf-8')
else:
return HttpResponse('Could not verify mode or token.')
The Strava documentation says that the callback url must respond to a GET request within 2 seconds with a status of 200 and an echo of the hub.challenge json string. This function seems to do that. Yet when I try to create a POST request equivalent to the one below:
$ curl -X POST https://www.strava.com/api/v3/push_subscriptions \
-F client_id=[MY-CLIENT-ID] \
-F client_secret=[MY-CLIENT-SECRET] \
-F 'callback_url=http://[MY-IP-ADDRESS]:8000/webhooks/example/' \
-F 'verify_token=STRAVA'
I get the following response:
{
"message": "Bad Request",
"errors": [
{
"resource": "PushSubscription",
"field": "callback url",
"code": "not verifiable"
}
]
}
Does anyone have any idea what might be going wrong?
P.S. Please let me know if there's anything I can do to make this example more reproducible. I don't really understand this area well enough to know whether I'm leaving out some crucial info!
consumer.py
# accept websocket connection
def connect(self):
self.accept()
# Receive message from WebSocket
def receive(self, text_data):
text_data_json = json.loads(text_data)
command = text_data_json['command']
job_id = text_data_json['job_id']
if command == 'subscribe':
self.subscribe(job_id)
elif command == 'unsubscribe':
self.unsubscribe(job_id)
else:
self.send({
'error': 'unknown command'
})
# Subscribe the client to a particular 'job_id'
def subscribe(self, job_id):
self.channel_layer.group_add(
'job_{0}'.format(job_id),
self.channel_name
)
# call this method from rest api to get the status of a job
def send_job_notification(self, message, job_id):
channel_layer = get_channel_layer()
group_name = 'job_{0}'.format(job_id)
channel_layer.group_send(
group_name,
{
"type": "send.notification",
"message": message,
}
)
# Receive message from room group
def send_notification(self, event):
message = event['message']
# Send message to WebSocket
self.send(text_data=json.dumps(
message))
In the above code what I am trying to do is connect clients to the socket and subscribe clients to a particular "job_id" by creating a group called "job_1" using "subscribe" method and add it to the channel layer. Creation of groups are dynamic.
I am using below "simple websocket client extension" from Google to connect to the above websocket. I am able to make a connection with the websocket and send request to it as shown in the picture below.
Now since the client is connected and subscribed to a particular "job_id",
I am using "Postman" to send notification to the above connected client(simple websocket client extension) subscribed to particular "job_id" by passing in the job_id in the request as highlighted in yellow below.
when I do a post to the "REST-API" I am calling "send_job_notification(self, message, job_id)" method of "consumer.py" file along with the "job_id" as '1' shown in the picture below in yellow
After doing all this I don't see any message sent to the connected client subscribed to a "job_id" from the "REST-API" call.
Any help would be highly appreciated as it has been dragging on for a long time.
Edit:
thank you for the suggestion Ken its worth to make the method as "#staticmethod" but Ken how do I make the API send job status updates to the connected Clients because my long running jobs would run in some process and send update messages back to the backend via REST-API and the updates then need to be sent to the correct Client (via websockets).
My API call to the socket consumer is as below:
from websocket_consumer import consumers
class websocket_connect(APIView):
def post(self, request, id):
consumers.ChatConsumer.send_job_notification("hello",id)
My socket consumer code is as below:
Edit
`CHANNEL_LAYERS = {
"default": {
"BACKEND": "channels_redis.core.RedisChannelLayer",
"CONFIG": {
"hosts": [("localhost", 6379)],
},
},
}`
As you can see 'Redis' service is also running
Edit-1
You cannot call the method in the consumer directly from an external code because you need to get the particular consumer instance connected to your client. This is the job of the channel layer achieved by using a message passing system or broker as reddis.
From what I can see, you're already going towards the right direction, except that the send_job_notification is an instance method which will require instantiating the consumer. Make it a static method instead, so you can call it directly without a consumer instance
#staticmethod
def send_job_notification(message, job_id):
channel_layer = get_channel_layer()
group_name = 'job_{0}'.format(job_id)
channel_layer.group_send(
group_name,
{
"type": "send.notification",
"message": message,
}
And in your API view, you can simply call it as:
ChatConsumer.send_job_notification(message, job_id)
I have created a messenger chatbot with flask, pymessenger and wit.ai.
I want to add facebook provided templates (like buttons, adding images and sound media)(https://developers.facebook.com/docs/messenger-platform/reference/template/button/)
There using some curl and json thing which I do not understand. Can some one help me, where should I put these snippet in my python code.
#app.route('/', methods=['POST'])
def webhook():
data = request.get_json()
log(data)
if data['object'] == 'page':
for entry in data['entry']:
for messaging_event in entry['messaging']:
sender_id = messaging_event['sender']['id']
recipient_id = messaging_event['recipient']['id']
if messaging_event.get('message'):
if 'text' in messaging_event['message']:
messaging_text = messaging_event['message']['text']
else:
messaging_text = 'no text'
response = None
entity, value = wit_response(messaging_text)
if entity == 'newstype':
response = "OK. I will send you {} news".format(str(value))
elif entity == 'cust_greet':
response = get_message()
elif entity == 'cust_bye':
response = "Bye! Have a Good Day!".format(str(value))
elif entity == 'cust_option':
response = "Option 1: Option 2:"
bot.send_text_message(sender_id, response)
return "ok", 200
def log(message):
print(message)
sys.stdout.flush()
HTTP requests use one of these two formats:
GET: All the request information is in the url
POST: Some information is sent via a JSON format to the url
What we see in the Facebook API is a POST request: the url is defined as
https://graph.facebook.com/v2.6/me/messages?access_token=<PAGE_ACCESS_TOKEN>
...and there is POST request information in the JSON section underneath
Curl is a program used to send HTTP requests from the terminal. If you install Curl, you can fill in the JSON/url information, run the command (which sends the POST request), and see the buttons pop up for the recipient. Just as you want your chatbot to do!
However, Curl is a tool, not a Python library!
To do this in Python, you can send the request through Python's built in libraries, or install a package which makes this easier (such as requests), look into "sending http requests via python".
Below is an example (adapted from this question):
from urllib.parse import urlencode
from urllib.request import Request, urlopen
# the url we are sending the request to
url = "https://graph.facebook.com/v2.6/me/..."
# the POST request data
request_data = {
"recipient": {
"id": "<PSID>"
},
"message": {
"attachment": {
...
}
}
}
# create the request with the url and the data
post_request = Request(url, urlencode(request_data).encode())
# send it to Facebook! Response is the API response from Facebook
response = urlopen(post_request).read().decode()
I need help with this.. Basically i need to use this command. This is the example given using CURL. All i need to do is simply paste this in cmd and it does its job.
curl \
-H "Content-Type: application/json" -X POST \
-u "{username}":"{password}" \
-d "{\"dialog_node\":\"greeting\",\"conditions\":\"#hello\",\"output\":{\"text\":\"Hi! How can I help you?\"},\"title\":\"greeting\"}" "https://gateway-s.watsonplatform.net/conversation/api/v1/workspaces/bec28d8f-18c1-4e97-8d08-9c842c658b51/dialog_nodes?version=2017-05-26"
The URL documentation can be found here:
https://www.ibm.com/watson/developercloud/conversation/api/v1/?curl#create_dialognode
The problem now is that i want to run this in a python script instead of in CMD.. i have searched google and stackOverflow for a few hours now.. but i cant seem to find the right answer..
So far i seen ppl using
1.requests
2.urllib
3.urllib2
4.pycurl
5.subprocess
I want to do it the right way. What is the best way to run the above command in a python script and how do i do it?
Also i am using python 3
Likes Simon O'Doherty said, you can use the Python SDK for using Conversation service. It is really the best practice to use the service, using the SDK or http requests.
"If something's worth doing, it's worth doing right, right?". So what
you're asking is "how do I run this other program, from within my
program, just to make a measly little web request?".
You can use cURL command, yes, you can. But it hardly looks very
Pythonic. That's a lot of work just for one little request. Python's
is more than it.
Author from the phrases here.
But, your question looks like you really want to use the cURL command inside your Python code, so, here is one example. In this case, you can use subprocess.
Call the Converstation:
import subprocess
subprocess.call(['curl', '-x', 'POST', '-H', '"Accept: application/json"', '-u', '{"userNameFromServiceCredentials:PasswordFromServiceCredentials" }', '"https://gateway-s.watsonplatform.net/conversation/api/v1/workspaces/bec28d8f-18c1-4e97-8d08-9c842c658b51/dialog_nodes?version=2017-05-26"'])
Important: For send the message and getting the output, you need to use the function subprocess.check_output(); like this example. And send the message for the right router, your cURL command needs to looks like this example from #German Atannasio and #Pridkkett.
Note: This answer is just to tell what is the better way to you follow, and if you really wants to use, one "Stone path" for you follow.
API Reference for using Watson Conversation Service with Python.
Requests documentation here.
If you are using Watson Conversation, then you can just use the Python WDC SDK.
https://github.com/watson-developer-cloud/python-sdk
For your example above it would be:
from watson_developer_cloud import ConversationV1
username = 'USERNAME',
password = 'PASSWORD',
version = '2017-05-26',
workspace_id = 'bec28d8f-18c1-4e97-8d08-9c842c658b51'
url = 'https://gateway-s.watsonplatform.net/conversation/api'
conversation = ConversationV1(
username=username
password=password,
version=version,
url=url
}
dialog_nodes = []
welcome_node = {
'output': {
'text': { 'values': [ 'Welcome!' ],
'selection_policy': 'sequential'
}
},
'parent': None,
'context': None,
'metadata': None,
'next_step': None,
'conditions': 'welcome',
'dialog_node': 'Welcome',
'previous_sibling': None
}
dialog_nodes.append(welcome_node)
# this appends to the previous node above, set by previous_sibling
node = {
'dialog_node': 'greeting',
'conditions': '#hello',
'context': None,
'metadata': None,
'next_step': None,
'output':{
'text': { 'values': [ 'Hi! How can I help you?' ]},
'selection_policy': 'sequential'
}
},
'title': 'greeting ',
'previous_sibling': 'Welcome',
'parent': None
}
dialog_nodes.append(node)
## Update the workspace.
response = conversation.update_workspace(
workspace_id=workspace_id,
dialog_nodes=dialog_nodes
)
print(response)
This call is an all or nothing, so if you have existing nodes it will delete them. The reason being is the SDK doesn't have the individual node editing. But this is a faster way to do it, rather then editing a single node (if you have more then one node).
If you want to make the individual call, then you will need to use something like requests, until the SDK is updated.
Example (using same variables from above):
import requests
from requests.auth import HTTPBasicAuth
endpoint = '{}/v1/workspaces/{}/dialog_nodes?version={}'.format(url,workspace_id,version)
basic_auth = HTTPBasicAuth(username, password)
# Change the condition to always execute.
node['conditions'] = 'true'
response = requests.post(url=endpoint, auth=basic_auth, json=node)
In python3 when sending files with pycurl, I want to understand why do I have to send the binary file directly
instead of providing its path
IBM Watson
#coding:utf-8
import certifi
import pycurl
import io
response = io.BytesIO()
c = pycurl.Curl()
#"apikey" is a key word
user = "apikey"
pwd = "YouRaPiKey"
#the file you want to submit
with open("path_to_file/audio-file.flac","rb") as f:
audiofile = f.read()
c.setopt(c.URL, "https://api. .... /.... /.. /v1/recognize")
c.setopt(pycurl.USERPWD, "{}:{}".format(user, pwd))
c.setopt(c.WRITEFUNCTION, response.write)
c.setopt(c.HTTPHEADER, ['Content-Type: audio/flac','Transfer-Encoding: chunked'])
c.setopt(c.POSTFIELDS, audiofile)
c.setopt(c.CAINFO,certifi.where())
c.perform()
c.close()
body = response.getvalue()
body = body.decode('iso-8859-1')
print(body)