Nexmo-Vonage: Place call from Vonage VoIP device to phone, initiate at server - voip

My client has a Vonage business account, which includes a small staff which works from home, using Vonage hard VoIP phones to call customers.
The calls from the staff to customers are automatically initiated from the server by using a webhook URL. This webhook API is old (something Vonage acquired from Vocalocity), and Vonage is making noises that they will discontinue it. They are pushing Nexmo instead.
The Nexmo and GitHub documentation includes examples of how to place an outbound call. However, the examples seem to all be written around playing a text-to-speech recording from a Nexmo soft phone#. This is not what my client needs. We need to have the call initiated from my client's server so that the staff does not have to hand-dial a phone# to speak to a customer.
How can I configure Nexmo to place a call from an already-existing Vonage account / phone#?
I am using Python, and have written the following code:
from nexmo import Client, Voice
import sys
APPLICATION_ID = sys.argv[1]
PRIVATE_KEY = sys.argv[2]
callFrom = sys.argv[3]
callTo = sys.argv[4]
answer_url = 'https://developer.nexmo.com/ncco/tts.json'
client = Client(application_id=APPLICATION_ID, private_key=PRIVATE_KEY)
voice = Voice(client)
response = voice.create_call({
'to': [{'type': 'phone', 'number': callTo}],
'from': {'type': 'phone', 'number': callFrom},
'answer_url': [answer_url]
})
print(response)

In order to connect an existing VBC extension into a new Vonage Voice API Conversation, you would first initiate the conversation to the PSTN using a Vonage virtually provisioned phone number as the from number (more info here).
Then, you can use the connect action in the NCCO to connect the VBC extension into the conversation.
The subsequent connect NCCO would look like this:
[
{
"action": "talk",
"voiceName": "Russell",
"text": "Hi there, connecting you to your Vonage Business Cloud Extension"
},
{
"action": "connect",
"endpoint": [
{
"type": "vbc",
"extension": "111" // supply your VBC extension here
}
]
}
]

Related

CoinbaseProAPI/CBpro. Authentication. Websocket. Shouldn't be this hard?

Been at this for a week: every, single, day. Nothing is working.
How hard is it for a minimal authentication? I bet so many people are looking for this answer too.
Here is rough-draft of what we've got so far:
import cbpro, hmac, hashlib, time,
api_key = "[YOUR_API_KEY]"
secret_key = "[YOUR_B64SECRET]"
passphrase = "[YOUR_PASSPHRASE]"
message = message.encode('ascii')
hmac_key = base64.b64decode(secret_key)
signature = hmac.new(hmac_key, message, hashlib.sha256)
signature_b64 = base64.b64encode(signature.digest()).decode('utf-8')
auth_client = cbpro.AuthenticatedClient(api_key, signature_b64, passphrase)
auth_client.get_accounts()
or while subscribing with websockets, something with this:
{
"type": "subscribe",
"product_ids": [
"BTC-USD"
],
"channels": ["full"],
"signature": "...",
"key": "...",
"passphrase": "...",
"timestamp": "..."
}
using:
socket = "wss://ws-feed.pro.coinbase.com"
ws = websocket.WebSocketApp(socket, on_open=on_open, on_message=on_message)
ws.run_forever()
I dont understand why a simply few lines of code to authenticate and view open orders/account is so hard, while subscribing to the websocket ticker and receiving price changes is so easy. Encoding the secret key shouldnt be too hard? Please help us with a super short and simple example (without all the "init"s and imports and different functions with passing). Sheesh
"I dont understand why a simply few lines of code to authenticate and view open orders/account is so hard, while subscribing to the websocket ticker and receiving price changes is so easy."
This is because your code will not produce an authenticated websockets session. The session is secured using 'wss' aka TLS, but inside of this session, you are not authenticated to their server and therefore you do not receive enhanced data elements for trade messages in which your account is involved. Receiving price data does not require being authenticated.
This is just like you are visiting a website at its "https" address, but are not logged in. "Public" data is being sent to you across a secure connection.
The cbpro module creates an AuthenticatedClient instance by passing to it the unaltered api key, base64-encoded secret, and passphrase. You should really review any module's code base before trusting it with your authentication data, or even installing it on your machine, but to each their own.
import cbpro #, hmac, hashlib, time,
api_key = YOUR_API_KEY
secret_key = YOUR_B64SECRET
passphrase = YOUR_PASSPHRASE
# message = message.encode('ascii')
# hmac_key = base64.b64decode(secret_key)
# signature = hmac.new(hmac_key, message, hashlib.sha256)
# signature_b64 = base64.b64encode(signature.digest()).decode('utf-8')
auth_client = cbpro.AuthenticatedClient(api_key, secret_key, passphrase)
auth_client.get_accounts()
You can test out what I am saying about how your websocket code is not properly authenticating either by running your wss client and subscribing to only the "user" channel for a product before placing a trade in for that product. With no other channels subscribed, you will receive a message indicating that the server received your new order. You can then cancel the order after testing. If your code does somehow allow you to receive messages in the "user" channel, then CoinbasePro is doing something different from what is published in its documentation.

How to create Quick Replies using MS Bot Framework on Facebook Messenger?

I have been using Node.js and the MS Bot Framework(3.0) for my bot development needs for quite some time now.
One of my needs is to request a user to share its e-mail address with the bot.
Facebook offers a Quick Replies API exactly for that.
I am having a hard time understanding how should i utilize the framework to create a custom message with the quick reply option.
One of my first attempts was to pass native metadata to a channel using custom channel data
I have succeeded implementing various templates which are supported by Messenger platform, but quick replies are sort of other beast compared to buttons, lists and other templates. currently i struggle to create a quick reply message using the framework provided tools.
Please point me in the right direction.
You can send Facebook quick replies either through the source data in V3 of the BotFramework or through the channel data in V4 of the framework. See the two examples below:
Node
V4
await turnContext.sendActivity({
text: 'What is your email?',
channelData: {
"quick_replies":[
{
"content_type": "user_email"
}
]
}
});
V3
var message = new botbuilder.Message(session)
.text('What is your email?')
.sourceEvent({
facebook: {
"quick_replies":[
{
"content_type": "user_email"
}
]
}
});
session.send(message);
CSharp
V4
Activity reply = turnContext.Activity.CreateReply();
reply.Text = "What is your location?";
reply.ChannelData = JObject.FromObject( new {
quick_replies = new object[]
{
new
{
content_type = "location",
},
},
});
await turnContext.SendActivityAsync(reply, cancellationToken);
Hope this helps!
On v3 you can just add the JSON of the quick_reply template defined by facebook to the channeldata as JSON object (JObject)
reply.channelData = new JOBject("[JSON HERE]");

How do I get notifications when a bot sends a message in Teams?

I developed a bot for Microsoft Teams using the Microsoft Bot Framework v4 Nodejs SDK (botbuilder-sdk for nodejs). We have implemented the bot in such a way that, when we receive data using a REST API call from one of our CRMs, the data is posted to the channels on Microsoft Teams. However, when I do that, we do not receive a notification on the devices. Has anyone faced such an issue?
I am saving the context state initially. Everytime we receive data from a CRM, I am incrementing the activity id of the message (to send it as a new message and not a reply) and sending it to Microsoft Teams using context.sendActivity().
When we receive that adaptive card, we do not receive a notification in the activity feed or on any of the devices.
I have gone through all the steps as you described above. I've also gone through the troubleshooting steps. However, it still doesn't give me a notification for the card. However, when I initiate a conversation with the bot, I get a notification when the bot responds.
https://i.stack.imgur.com/Bi4fc.png
https://i.stack.imgur.com/ab6uP.png
In this image, I get a notification when I get the TMS Bot started! message. However, I don't get a notification for the next two messages.
Edit: OP and I have exchanged a few emails to get this answered. This answer, as a whole, is good information for accomplishing Teams Proactive messaging, in general, but the main answer is in the last section, Simplified Code.
This is a long answer that covers many areas, simply because I'm not 100% sure I know what kind of notification you aren't receiving.
Troubleshooting
Troubleshooting Guide
Pay special attention to the many areas where notifications need to be enabled
In particular, the user may need to "Follow" and/or "Favorite" the channel to receive notifications from it
If a user has the desktop app open, they will receive the notification there and will not receive one on their phone unless they have been inactive on the desktop app for 3+ minutes. Otherwise, it's likely a bug in Teams.
Chat Notifications
If you've followed the Troubleshooting Guide linked above, your users should receive chat notifications. If not, you can try updating your MS Teams desktop or mobile client. As #KyleDelaney mentioned, it may be helpful to # mention users and/or channels
Activity Feed Notifications
You can also create Activity Feed Notifications. The gist of it is that you need to:
Include text and summary in the message
Include channelData that sets notifications.alert to true
This code will accomplish that:
const msg = MessageFactory.text('my message');
msg.summary = 'my summary';
msg.channelData = {
notification: {
alert: true,
},
};
return await dc.context.sendActivity(msg);
Result:
Note: If your bot only creates notifications and doesn't have conversations, you may benefit from creating a notifications-only bot.
Full Implementation Code
import * as adaptiveCard from '../src/adaptiveCard.json';
...
const card = CardFactory.adaptiveCard(adaptiveCard);
const activity = {
attachments: [card],
text: 'Test Card',
summary: 'my summary',
channelData: {
notification: {
alert: true,
},
},
};
await turnContext.sendActivity(activity);
Result:
Using the Teams Extension
There's a Teams Extension for BobBuilder V4 that's currently in Beta, but seems to accomplish what you need. I believe the reason you weren't getting notifications while using the above is because your bot is creating a new reply chain in the channel and not replying directly to a user. I believe you can do all of this without the extension (by manually editing activity/context properties), but the extension should make it easier.
Here's the code I used to get working notifications within a channel:
In index.js (or app.js):
import * as teams from 'botbuilder-teams';
[...]
// Change existing to use "new teams.TeamsAdapter..."
const adapter = new teams.TeamsAdapter({
appId: endpointConfig.appId || process.env.microsoftAppID,
appPassword: endpointConfig.appPassword || process.env.microsoftAppPassword,
});
Wherever you're sending the message:
import * as teams from 'botbuilder-teams';
import * as adaptiveCard from '../src/adaptiveCard.json';
...
const card = CardFactory.adaptiveCard(adaptiveCard);
const activity = {
attachments: [card],
text: 'Test Card',
summary: 'my summary',
channelData: {
notification: {
alert: true,
},
},
};
const adapter = context.adapter as teams.TeamsAdapter;
await adapter.createReplyChain(context, [activity]);
Simplified Code
OP and I have emailed back and forth a bit and the key issue is that he needed to add the trustServiceUrl code from below. Normally, this manifests itself with a 500 error, but in this case, it appears to not create notifications.. After significant testing, here's all you really have to do to send different notifications to different channels. It basically amounts to setting a couple of properties of turncontext.activity and trusting the serviceUrl. No touching activity ID or using the Teams Extension at all. My code below is how I sent messages from Emulator that could then send cards to different Teams channels:
public onTurn = async (turnContext: TurnContext) => {
const dc = await this.dialogs.createContext(turnContext);
const dialogResult = await dc.continueDialog();
// Route message from Emulator to Teams Channel - I can send "1", "2", or "3" in emulator and bot will create message for Channel
let teamsChannel;
switch (turnContext.activity.text) {
// You can get teamsChannel IDs from turnContext.activity.channelData.channel.id
case '1':
teamsChannel = '19:8d60061c3d104exxxxxxxxxxxxxxxxxx#thread.skype';
break;
case '2':
teamsChannel = '19:0e477430ebad4exxxxxxxxxxxxxxxxxx#thread.skype';
break;
case '3':
teamsChannel = '19:55c1c5fb0d304exxxxxxxxxxxxxxxxx0#thread.skype';
break;
default:
break;
}
if (teamsChannel) {
const card = CardFactory.adaptiveCard(adaptiveCard);
const activity = {
attachments: [card],
summary: 'my summary',
text: 'Test Card',
};
const serviceUrl = 'https://smba.trafficmanager.net/amer/';
turnContext.activity.conversation.id = teamsChannel;
turnContext.activity.serviceUrl = serviceUrl;
// This ensures that your bot can send to Teams
MicrosoftAppCredentials.trustServiceUrl(serviceUrl);
await turnContext.sendActivity(activity);
} else {
[...Normal onTurn Code...]
await this.conversationState.saveChanges(turnContext);
}
Note: To receive notifications, you and your users must follow the channel.
I have gone through all the steps as you described above. I've also gone through the troubleshooting steps. However, it still doesn't give me a notification for the card. However, when I initiate a conversation with the bot, I get a notification when the bot responds.
https://i.stack.imgur.com/Bi4fc.png
https://i.stack.imgur.com/ab6uP.png
In this image, I get a notification when I get the TMS Bot started! message. However, I don't get a notification for the next two messages.

How to receive my own telegram messages in node.js without bot

I would like to have a very simple client in nodejs (an example) that can receive messages from my contacts in telegram. I just searched in internet but I only get bot samples. I want to receive group messages in what I don't have access to give privileges to my bot so I would like to know if I can receive my own messages with no bot as intermediary.
Well... Other answers give examples from unmaintained libraries. Hence, you should not rely on these libraries.
See: telegram.link is dead
You should use the newest Telegram client library which is telegram-mtproto
1. Obtain your api_id and api_hash from:
Telegram Apps
2. Install the required client library:
npm install telegram-mtproto#beta --save
3. Initialize your node.js application with api_id and api_hash you got from Telegram Apps and with your phone number:
import MTProto from 'telegram-mtproto'
const phone = {
num : '+90555555555', // basically it is your phone number
code: '22222' // your 2FA code
}
const api = {
layer : 57,
initConnection : 0x69796de9,
api_id : 111111
}
const server = {
dev: true //We will connect to the test server.
} //Any empty configurations fields can just not be specified
const client = MTProto({ server, api })
async function connect(){
const { phone_code_hash } = await client('auth.sendCode', {
phone_number : phone.num,
current_number: false,
api_id : 111111, // obtain your api_id from telegram
api_hash : 'fb050b8fjernf323FDFWS2332' // obtain api_hash from telegram
})
const { user } = await client('auth.signIn', {
phone_number : phone.num,
phone_code_hash: phone_code_hash,
phone_code : phone.code
})
console.log('signed as ', user);
}
connect();
4. Receive messages (The fun part! 👨🏻‍💻)
const telegram = require('./init') // take a look at the init.js from the examples repo
const getChat = async () => {
const dialogs = await telegram('messages.getDialogs', {
limit: 50,
})
const { chats } = dialogs;
const selectedChat = await selectChat(chats);
return selectedChat;
}
Furthermore, Take a look at the examples from the original repo:
Usage Examples
Reading Chat History
Updating Profile Info
If you want to interact with Telegram data outside one of the official applications (website, mobile or desktop app etc...) you will have to create an App, so you will be required to generate API key and/or use any already existing App which fit your requirement (bots in your case).
Let me underline that with API system it appear difficult to get access to something which is restricted, if you don't have previously grant or add privileges to access it ... Nobody want that anyone could access any data...
Regards
You can use the following libraries.
https://github.com/zerobias/telegram-mtproto
https://github.com/dot-build/telegram-js
They provide abstractions to build applications to interact with telegram. for an example on how to use telegram-js, you can use https://github.com/dot-build/telegram-js/blob/master/sample.js.
(Thank you #gokcand for the feedback)

GoogleActions Account not linked yet error

I'm trying to implement oauth2 authentication on my nodejs Google Assistant app developed using (DialogFlow or API.ai and google actions).
So I followed this answer. But I'm always getting "It looks like your test oauth account is not linked yet. " error. When I tried to open the url shown on the debug tab, it shows 500 broken url error.
Dialogflow fullfillment
index.js
'use strict';
const functions = require('firebase-functions'); // Cloud Functions for Firebase library
const DialogflowApp = require('actions-on-google').DialogflowApp; // Google Assistant helper library
const googleAssistantRequest = 'google'; // Constant to identify Google Assistant requests
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
console.log('Request headers: ' + JSON.stringify(request.headers));
console.log('Request body: ' + JSON.stringify(request.body));
// An action is a string used to identify what needs to be done in fulfillment
let action = request.body.result.action; // https://dialogflow.com/docs/actions-and-parameters
// Parameters are any entites that Dialogflow has extracted from the request.
const parameters = request.body.result.parameters; // https://dialogflow.com/docs/actions-and-parameters
// Contexts are objects used to track and store conversation state
const inputContexts = request.body.result.contexts; // https://dialogflow.com/docs/contexts
// Get the request source (Google Assistant, Slack, API, etc) and initialize DialogflowApp
const requestSource = (request.body.originalRequest) ? request.body.originalRequest.source : undefined;
const app = new DialogflowApp({request: request, response: response});
// Create handlers for Dialogflow actions as well as a 'default' handler
const actionHandlers = {
// The default welcome intent has been matched, welcome the user (https://dialogflow.com/docs/events#default_welcome_intent)
'input.welcome': () => {
// Use the Actions on Google lib to respond to Google requests; for other requests use JSON
//+app.getUser().authToken
if (requestSource === googleAssistantRequest) {
sendGoogleResponse('Hello, Welcome to my Dialogflow agent!'); // Send simple response to user
} else {
sendResponse('Hello, Welcome to my Dialogflow agent!'); // Send simple response to user
}
},
// The default fallback intent has been matched, try to recover (https://dialogflow.com/docs/intents#fallback_intents)
'input.unknown': () => {
// Use the Actions on Google lib to respond to Google requests; for other requests use JSON
if (requestSource === googleAssistantRequest) {
sendGoogleResponse('I\'m having trouble, can you try that again?'); // Send simple response to user
} else {
sendResponse('I\'m having trouble, can you try that again?'); // Send simple response to user
}
},
// Default handler for unknown or undefined actions
'default': () => {
// Use the Actions on Google lib to respond to Google requests; for other requests use JSON
if (requestSource === googleAssistantRequest) {
let responseToUser = {
//googleRichResponse: googleRichResponse, // Optional, uncomment to enable
//googleOutputContexts: ['weather', 2, { ['city']: 'rome' }], // Optional, uncomment to enable
speech: 'This message is from Dialogflow\'s Cloud Functions for Firebase editor!', // spoken response
displayText: 'This is from Dialogflow\'s Cloud Functions for Firebase editor! :-)' // displayed response
};
sendGoogleResponse(responseToUser);
} else {
let responseToUser = {
//richResponses: richResponses, // Optional, uncomment to enable
//outputContexts: [{'name': 'weather', 'lifespan': 2, 'parameters': {'city': 'Rome'}}], // Optional, uncomment to enable
speech: 'This message is from Dialogflow\'s Cloud Functions for Firebase editor!', // spoken response
displayText: 'This is from Dialogflow\'s Cloud Functions for Firebase editor! :-)' // displayed response
};
sendResponse(responseToUser);
}
}
};
// If undefined or unknown action use the default handler
if (!actionHandlers[action]) {
action = 'default';
}
// Run the proper handler function to handle the request from Dialogflow
actionHandlers[action]();
// Function to send correctly formatted Google Assistant responses to Dialogflow which are then sent to the user
function sendGoogleResponse (responseToUser) {
if (typeof responseToUser === 'string') {
app.ask(responseToUser); // Google Assistant response
} else {
// If speech or displayText is defined use it to respond
let googleResponse = app.buildRichResponse().addSimpleResponse({
speech: responseToUser.speech || responseToUser.displayText,
displayText: responseToUser.displayText || responseToUser.speech
});
// Optional: Overwrite previous response with rich response
if (responseToUser.googleRichResponse) {
googleResponse = responseToUser.googleRichResponse;
}
// Optional: add contexts (https://dialogflow.com/docs/contexts)
if (responseToUser.googleOutputContexts) {
app.setContext(...responseToUser.googleOutputContexts);
}
app.ask(googleResponse); // Send response to Dialogflow and Google Assistant
}
}
// Function to send correctly formatted responses to Dialogflow which are then sent to the user
function sendResponse (responseToUser) {
// if the response is a string send it as a response to the user
if (typeof responseToUser === 'string') {
let responseJson = {};
responseJson.speech = responseToUser; // spoken response
responseJson.displayText = responseToUser; // displayed response
response.json(responseJson); // Send response to Dialogflow
} else {
// If the response to the user includes rich responses or contexts send them to Dialogflow
let responseJson = {};
// If speech or displayText is defined, use it to respond (if one isn't defined use the other's value)
responseJson.speech = responseToUser.speech || responseToUser.displayText;
responseJson.displayText = responseToUser.displayText || responseToUser.speech;
// Optional: add rich messages for integrations (https://dialogflow.com/docs/rich-messages)
responseJson.data = responseToUser.richResponses;
// Optional: add contexts (https://dialogflow.com/docs/contexts)
responseJson.contextOut = responseToUser.outputContexts;
response.json(responseJson); // Send response to Dialogflow
}
}
});
// Construct rich response for Google Assistant
const app = new DialogflowApp();
const googleRichResponse = app.buildRichResponse()
.addSimpleResponse('This is the first simple response for Google Assistant')
.addSuggestions(
['Suggestion Chip', 'Another Suggestion Chip'])
// Create a basic card and add it to the rich response
.addBasicCard(app.buildBasicCard(`This is a basic card. Text in a
basic card can include "quotes" and most other unicode characters
including emoji 📱. Basic cards also support some markdown
formatting like *emphasis* or _italics_, **strong** or __bold__,
and ***bold itallic*** or ___strong emphasis___ as well as other things
like line \nbreaks`) // Note the two spaces before '\n' required for a
// line break to be rendered in the card
.setSubtitle('This is a subtitle')
.setTitle('Title: this is a title')
.addButton('This is a button', 'https://assistant.google.com/')
.setImage('https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png',
'Image alternate text'))
.addSimpleResponse({ speech: 'This is another simple response',
displayText: 'This is the another simple response 💁' });
// Rich responses for both Slack and Facebook
const richResponses = {
'slack': {
'text': 'This is a text response for Slack.',
'attachments': [
{
'title': 'Title: this is a title',
'title_link': 'https://assistant.google.com/',
'text': 'This is an attachment. Text in attachments can include \'quotes\' and most other unicode characters including emoji 📱. Attachments also upport line\nbreaks.',
'image_url': 'https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png',
'fallback': 'This is a fallback.'
}
]
},
'facebook': {
'attachment': {
'type': 'template',
'payload': {
'template_type': 'generic',
'elements': [
{
'title': 'Title: this is a title',
'image_url': 'https://developers.google.com/actions/images/badges/XPM_BADGING_GoogleAssistant_VER.png',
'subtitle': 'This is a subtitle',
'default_action': {
'type': 'web_url',
'url': 'https://assistant.google.com/'
},
'buttons': [
{
'type': 'web_url',
'url': 'https://assistant.google.com/',
'title': 'This is a button'
}
]
}
]
}
}
}
};
Actually I deployed the code exists in the dialog flow inline editor. But don't know how to implement an oauth endpoint, whether it should be a separate cloud function or it has to be included within the existsing one. And also I am so confused with how oauth authorization code flow will actually work.. Let's assume we are on the Assistant app, once the user say "talk to foo app", does it automatically opens a web browser for oauth code exchange process?
The answer you referenced had an update posted on October 25th indicating they had taken action to prevent you from entering in a google.com endpoint as your auth provider for Account Linking. It seems possible that they may have taken other actions to prevent using Google's auth servers in this way.
If you're using your own auth server, the error 500 would indicate an error on your oauth server, and you should check your oauth server for errors.
Update to answer some of your other questions.
But don't know how to implement an oauth endpoint
Google provides guidance (but not code) on what you need to do for a minimal OAuth service, either using the Implicit Flow or the Authorization Code Flow, and how to test it.
whether it should be a separate cloud function or it has to be included within the existing one
It should be separate - it is even arguable that it must be separate. In both the Implicit Flow and the Authorization Code Flow, you need to provide a URL endpoint where users will be redirected to log into your service. For the Authorization Code Flow, you'll also need an additional webhook that the Assistant will use to exchange tokens.
The function behind these needs to be very very different than what you're doing for the Dialogflow webhook. While someone could probably make a single function that handles all of the different tasks - there is no need to. You'll be providing the OAuth URLs separately.
However, your Dialogflow webhook does have some relationship with your OAuth server. In particular, the tokens that the OAuth server hands to the Assistant will be handed back to the Dialogflow webhook, so Dialogflow needs some way to get the user's information based on that token. There are many ways to do this, but to list just a few:
The token could be a JWT and contain the user information as claims in the body. The Dialogflow webhook should use the public key to verify the token is valid and needs to know the format of the claims.
The OAuth server and the Dialogflow webhook could use a shared account database, and the OAuth server store the token as a key to the user account and delete expired keys. The Dialogflow webhook could then use the token it gets as a key to look up the user.
The OAuth server might have a(nother) webhook where Dialogflow could request user information, passing the key as an Authorization header and getting a reply. (This is what Google does, for example.)
The exact solutions depends on your needs and what resources you have available to you.
And also I am so confused with how oauth authorization code flow will actually work.. Let's assume we are on the Assistant app, once the user say "talk to foo app", does it automatically opens a web browser for oauth code exchange process?
Broadly speaking - yes. The details vary (and can change), but don't get too fixated on the details.
If you're using the Assistant on a speaker, you'll be prompted to open the Home app which should be showing a card saying what Action wants permission. Clicking on the card will open a browser or webview to the Actions website to begin the flow.
If you're using the Assistant on a mobile device, it prompts you directly and then opens a browser or webview to the Actions website to begin the flow.
The auth flow basically involves:
Having the user authenticate themselves, if necessary.
Having the user authorize the Assistant to access your resources on the user's behalf.
It then redirects to Google's servers with a one-time code.
Google's servers then take the code... and close the window. That's the extent of what the user's see.
Behind the scenes, Google takes this code and, since you're using the Authorization Code Flow, exchanges it for an auth token and a refresh token at the token exchange URL.
Then, whenever the user uses your Action, it will send an auth token along with the rest of the request to your server.
Plz suggest the necessary package for OAuth2 configuration
That I can't do. For starters - it completely depends on your other resources and requirements. (And this is why StackOverflow doesn't like people asking for suggestions like this.)
There are packages out there (you can search for them) that let you setup an OAuth2 server. I'm sure someone out there provides OAuth-as-a-service, although I don't know any offhand. Finally, as noted above, you can write a minimal OAuth2 server using the guidance from Google.
Trying to create a proxy for Google's OAuth is... probably possible... not as easy as it first seems... likely not as secure as anyone would be happy with... and possibly (but not necessarily, IANAL) a violation of Google's Terms of Service.
can't we store the user's email address by this approach?
Well, you can store whatever you want in the user's account. But this is the user's account for your Action.
You can, for example, access Google APIs on behalf of your user to get their email address or whatever else they have authorized you to do with Google. The user account that you have will likely store the OAuth tokens that you use to access Google's server. But you should logically think of that as separate from the code that the Assistant uses to access your server.
My implementation of a minimal oauth2 server(works for the implicit flow but doesn't store the user session).
taken from https://developers.google.com/identity/protocols/OAuth2UserAgent.
function oauth2SignIn() {
// Google's OAuth 2.0 endpoint for requesting an access token
var oauth2Endpoint = 'https://accounts.google.com/o/oauth2/v2/auth';
// Create element to open OAuth 2.0 endpoint in new window.
var form = document.createElement('form');
form.setAttribute('method', 'GET'); // Send as a GET request.
form.setAttribute('action', oauth2Endpoint);
//Get the state and redirect_uri parameters from the request
var searchParams = new URLSearchParams(window.location.search);
var state = searchParams.get("state");
var redirect_uri = searchParams.get("redirect_uri");
//var client_id = searchParams.get("client_id");
// Parameters to pass to OAuth 2.0 endpoint.
var params = {
'client_id': YOUR_CLIENT_ID,
'redirect_uri': redirect_uri,
'scope': 'email',
'state': state,
'response_type': 'token',
'include_granted_scopes': 'true'
};
// Add form parameters as hidden input values.
for (var p in params) {
var input = document.createElement('input');
input.setAttribute('type', 'hidden');
input.setAttribute('name', p);
input.setAttribute('value', params[p]);
form.appendChild(input);
}
// Add form to page and submit it to open the OAuth 2.0 endpoint.
document.body.appendChild(form);
form.submit();
}
This implementation isn't very secure but it's the only code I've gotten to work as OAuth server for the Assistant.
I am able to make it work after a long time. We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first. Then follow the steps mentioned below for the Account Linking in actions on google:-
Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there -> Download JSON - from json note down the project id, auth_uri, token_uri -> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL -> Save the changes
Actions on Google -> Account linking setup 1. Grant type = Authorisation code 2. Client info 1. Fill up client id,client secrtet, auth_uri, token_uri 2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token 3. Save and run 4. It will show an error while running on the google assistant, but dont worry 5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth and token_uri as https://accounts.google.com/o/oauth2/token 6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email and weare good to go. 7. Save the changes.
In the hosting server(heroku)logs, we can see the access token value and through access token, we can get the details regarding the email address.
Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.
`accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])`

Resources