I am trying to handle the carousel item selection using DeepLink inside the Android app, using Dialogflow of actions-on-google.
The code is:
'use strict';
const dialogflow = require('actions-on-google').ActionsSdkApp;
const {
dialogflow,
DeepLink,
Image,
Carousel
} = require('actions-on-google');
const agent = dialogflow({
debug: true,
});
agent.intent(INTENT_OPTION, (conv, input, option) => {
console.log("carousel clicked input " + input + " option " + option);
deepLink(conv, option);
});
function deepLink(conv, option) {
// example url = 'examplescheme://www.someexample.in/content/id/110212';
const options = {
destination: 'Sample App',
url: option,
package: 'com.example.deeplink',
reason: 'Launch App',
};
conv.ask('Great! looks like maybe we can do that in the app.');
conv.ask(new DeepLink(options));
}
But when I follow this, the title of carousel item is displayed on the screen and getting a prompt dialog with the provided description to actually redirect to the app to handle the action.
I observed the Youtube app that it is possible to eliminate the title printed & this prompt dialog and directly launch the app with a DeepLink.
Is this can be possible with other apps?
Thanks in advance.
Related
Hello I want to handle popups in puppeteer and node.js
await page.goto('https://twitter.com');
await sleep(3000);
const frame = page.frames().find(f => f.url().startsWith('https://accounts.google.com/gsi/button'));
const acceptBtn = await frame.$('#container > div > div.nsm7Bb-HzV7m-LgbsSe-bN97Pc-sM5MNb.oXtfBe-l4eHX > span.nsm7Bb-HzV7m-LgbsSe-BPrWId');
await acceptBtn.click();
So scheme is:
Go to page.com
Click "Sign in via Google" (here we send click to iframe, code above)
New window popup appear, and I wanna send puppeteer actions in this popup
So how to tell puppeteer that he will use this popup as page, or wait it for open, and declare it?
I was trying do this way, but sadly doesnt work :(
browser.on('targetcreated', async (target) => { //
if (target.type() === 'page') { // if it tab/page
const page = await target.page(); // declare it
const url = page.url(); // example, look at her url
if (url.search('accounts.google.com/v3/signin') == -1) { // if url is not like site.com (pop-up window whith ads to anower site)
await page.close(); // close this page
}
}
});
I was trying things from this topic and still can't find fix.
Puppeteer - Handle basic authentication popup (that appears after clicking on a link)
I am using the node-notifier package for notification but I have one problem. notifier.on click event is not fired in Windows Action Center
notifier.notify(
{
appId: "com.electron.demo", // Absolute path (doesn't work on balloons),
title: "Alert Message",
message: "Click to view",
icon: path.join(__dirname, 'msicon.png'),
wait: true,
timeout: false,
id: "demo app"
},
function(err, response) {
// Response is response from notification
if (err) {
reject(err);
} else {
resolve(response);
}
}
);
notifier.on("click", function(notifierObject, options, event) {
// Triggers if `wait: true` and user clicks notification
var request = new sql.Request();
let username = userName;
request.query("update alerts.dbo.groupmessages set isRead = 1\
where pk = '"+ recordset.recordset[i].pk + "'\
and userFK = (select pk from alerts.dbo.users where username = '"+ username + "')\
and groupFK = (select groupFK from alerts.dbo.users where username = '"+ username + "')", function (err, recordset) {
if (err) throw err
console.log(err, recordset, 'User shit');
});
function createWindow() {
// Create the browser window.
win = new BrowserWindow({
width: 400,
height: 400,
frame: false,
webPreferences: {
nodeIntegration: true
}
});
// and load the index.html of the app.
ejse.data('message', recordset.recordset[i].message);
win.loadFile('index.ejs')
}
createWindow();
});
});
});
notifier.on click event is not fired in only Windows Action Center. please let me know the reason and solution as well. Thanks.
Action Center requires separate implementation in native code, which node-notifier doesn't have.
You can try node-powertoast instead and use the onActivated callback:
npm i node-powertoast
const toast = require('powertoast');
toast({
title: "Hello",
message: "world",
callback: {
timeout: 5000, //keep-a-live in ms
onActivated: ()=>{ console.log("activated") },
onDismissed: (reason)=>{ console.log(reason) }
})
.then(()=> console.log("Notified")
.catch(err => console.error(err));
As for the reason:
You can see for other projects it's a significant undertaking and the following project, for example, seems to have Action Center support stuck in limbo for 2 years now: https://github.com/mohabouje/WinToast/issues/35 People trying to implement it themselves appear to have gotten stuck, and have had difficulty getting it implemented correctly.
It's just hard to do.
Notification doesn't get clicks because not all Action Center requirements are fulfilled.
In general Windows notifications have three types of activation: foreground, background and protocol.
In any case you will need to have the following:
Have a shortcut in Start Menu with AppUserModelId property. Please consult your installer on how to do it or use shell.readShortcutLink and shell.writeShortcutLink Electron API. Specifically WiX allows you to specify appUserModelId and toastActivatorClsid in options.
Specify AppUserModelId in the app.
app.setAppUserModelId('AppUserModelId');
Register a custom protocol and some reaction to it. For example like this:
app.setAsDefaultProtocolClient('example');
To use foreground or background you need to:
Have a shortcut in Start Menu with ToastActivatorCLSID property. See above on how to do it.
Use a project like electron-windows-interactive-notifications to register a COM server with the specified earlier ToastActivatorCLSID and a custom protocol of your choice and call:
registerComServer();
registerActivator();
I won't get deep into details on how to react to custom protocol but you will need to parse command line args in search for example: and also expect it in the second-instance hook if the app is already launched.
To use protocol type no extra actions needed.
After that even standard Electron notifications will get clicks from Action Center.
But how to use these protocols and get feature rich notifications?
Electron allows you to specify toastXml directly!
Let's say here's your notification. It will open https://google.com even if clicked from Action Center.
const notification = new Notification({
toastXml: `
<toast launch="https://google.com" activationType="protocol">
<visual>
<binding template="ToastGeneric">
<text>Wonderman meets Superwoman</text>
<text>In the eve of the new millennium, Wonderman challenges Superwoman to a bliniking contest.</text>
<text placement="attribution">Mars Press</text>
</binding>
</visual>
</toast>
`,
});
notification.show();
This notification simply opens Google. You can get more ideas at Notification Visualizer App
Goal: I'm trying to scrape pictures from instagram using Puppeteer to programmatically log in to my account and start mining data.
The issue: I can log in fine but then I get hit with a popup asking if I want notifications (I turned headless off to see this in action). I'm following the example code for this found here: https://github.com/checkly/puppeteer-examples/blob/master/3.%20login/instagram.js which uses the below try block to find the notification popup and click the 'Not now' button.
//check if the app asks for notifications
try {
await loginPage.waitForSelector(".aOOlW.HoLwm",{
timeout:5000
});
await loginPage.click(".aOOlW.HoLwm");
} catch (err) {
}
The problem is it doesn't actually click the 'Not now' button so my script is stuck in limbo. The selector is pointing to the right div so what gives?
can you please try enabling "notifications" using browserContext.overridePermissions?
you can override notification. This is the code that would, for example, disable the Allow Notifications popup when logging into facebook.
let crawl = async function(){
let browser = await puppeteer.launch({ headless:false });
const context = browser.defaultBrowserContext();
// URL An array of permissions
context.overridePermissions("https://www.facebook.com", ["geolocation", "notifications"]);
let page = await browser.newPage();
await page.goto("https://www.facebook.com");
await page.type("#email", process.argv[2]);
await page.type("#pass", process.argv[3]);
await page.click("#u_0_2");
await page.waitFor(1000);
await page.waitForSelector("#pagelet_composer");
let content2 = await page.$$("#pagelet_composer");
console.log(content2); // .$$ An array containing elementHandles .$ would return 1 elementHandle
}
crawl();
I am using NodeJs API of Microsoft Bot Framework v4. And my dialogs are not hardcoded in the ActivityHandler, I just call from there. I am using Waterfall dialogs. So when I try to show Carousel card on messenger (which is HeroCard on Microsoft bot framework), it shows up successfully but when I click any button on cards, there is no response for next dialog.
I tried to handle on onMessage hook but, it just tries to validate the response and throw errors.
....
ListItems extends ComponentDialog {
constructor(userProfileState, conversionStateAccessor) {
super(LIST_ITEMS);
this.userState = userProfileState;
this.conversionState = conversionStateAccessor;
this.addDialog(new WaterfallDialog(LIST_ITEMS_DIALOG, [
this.sendItems.bind(this),
this.handleItems.bind(this)
]
));
this.initialDialogId = LIST_ITEMS_DIALOG;
}
async sendItems(step) {
.......
await step.context.sendActivity({ attachments: cards }); // this line is working
}
async handleItems(step) {
console.log(step.result) // the response is not in 'step.result.value' if i click a button on cards
}
Thanks for your help
---- I added more detail -----
I am using this template to create cards
const card = CardFactory.heroCard('title', 'subtitle', ['imagelink'], [
{ type: ActionTypes.PostBack,
title: 'product 1',
value: 'product_1'
},
{ type: ActionTypes.PostBack,
title: 'product 1',
value: 'product_1'
},
]);
await context.sendActivity({ attachments: [card] });
Cars can be created successfully but the problem is, after that, I send a prompt to let the user turn the main menu if the user wants.
so I send them like that
await context.sendActivity({ attachments: [card] });
await step.prompt(LIST_ITEMS_MENU_PROMPT, menuPromptPayload);
And if the user clicks a button on cards, an error is thrown because I think the framework waits for prompt's answers. Couldn't catch the card's button's payload/
You need to instruct your bot to wait for and return a response following the sendActivity in sendItems. Similarly, you will encounter an error in the handleItems if you don't tell the bot to continue on since there is nothing happening other than console logging.
async sendItems(step) {
.......
await step.context.sendActivity({ attachments: cards });
// Tells the bot to wait. Normally, on `sendActivity`, it continues on to the next step.
// "DialogTurnStatus" can be found in the `botbuilder-dialogs` package.
return { status: DialogTurnStatus.waiting };
}
async handleItems(step) {
console.log(step.result);
return step.next(); // Tells the bot to continue to the next step.
}
Hope of help!
How do I end my conversation from the webhook ?
Marking it within Dialogflow does nothing , basically does not stop it as I am using the webhook for fulfillment .
And if I add it to the code as below then it does not play the media.
// Import the Dialogflow module from the Actions on Google client library.
// https://github.com/actions-on-google/actions-on-google-nodejs
const {dialogflow, Suggestions, MediaObject, Image} = require('actions-on-google');
// Import the firebase-functions package for Cloud Functions for Firebase fulfillment.
const functions = require('firebase-functions');
// Node util module used for creating dynamic strings
const util = require('util');
// Instantiate the Dialogflow client with debug logging enabled.
const app = dialogflow({
debug: true
});
// Do common tasks for each intent invocation
app.middleware((conv, framework) => {
console.log(`Intent=${conv.intent}`);
console.log(`Type=${conv.input.type}`);
//kng
console.log(`Arguments=${conv.arguments}`);
console.log(`Arguments=${typeof(conv.arguments)}`);
// Determine if the user input is by voice
conv.voice = conv.input.type === 'VOICE';
if (!(conv.intent === 'Default Fallback Intent' || conv.intent === 'No-input')) {
// Reset the fallback counter for error handling
conv.data.fallbackCount = 0;
}
});
app.intent('Play Sound', (conv, {SoundType,duration}) => {
const suggestions1 = new Suggestions('do this ', 'do that', 'do nothing');
simple_response = 'this is a response from the webhook'
conv.ask(simple_response)
conv.ask(new MediaObject({
name: SoundType,
url: some_mp3file_url,
icon: new Image({
url: some_image_url,
alt: 'Media icon'
})
}));
conv.ask( suggestions1);
//if I close from the code it doesnot play the sound
conv.close();
//if I comment out the close statement above then it does not close and toggling on the "set this intent as the end of convesation does not seem to help."
}
)
Update - This was intact a bug as pointed out by one of the comments . Reported to google and they fixed the same in April or May
I can duplicate the issue, but it appears to be a bug - playing audio as part of the response and having it close after the audio finishes used to work. It is clearly supposed to be supported - the documentation and the simulator state that Suggestions aren't required if this is a final response.
The workaround is to create an additional Intent that handles the Action actions_intent_MEDIA_STATUS. This Intent would then close the conversation.