I'm working on a cast web receiver application that shows an advert when the user pauses. On a Google TV, when the user presses the Google Assistant button on the remote, it shows dialogue at the top and pauses the current media. Because this isn't a true pause and the dialogue obscures a portion of the advert, we don't want the advert to appear (often because the voice command takes the user elsewhere and they don't see the ad, causing a false impression).
The web receiver is contained within the Google TV and cannot access OS functions. I can detect whether it's on a Google TV by seeing whether the media UI controls are an element in the DOM (<touch-controls>...</touch-controls>). However, the DOM doesn't contain the Google Assistant dialogue, so I can't use that to detect whether the Google Assistant is active.
In this answer, CygnusOlor described a method to tell whether the Google TV is pausing by adding information to the customData, but that doesn't solve the problem of determining when the Google Assistant is active. CygnusOlor also said that the pause commands from the remote and Google Assistant are indistinguishable, so that solution won't work for my needs.
Is there any way to check whether the Google Assistant dialogue is active? I am afraid that this is not possible, but someone may have some info. Thank you for your time!
Related
I have two google assistant responses:
simple response, cause I must to make it for google assistant
link out suggestion response, which I need to display
When I test it, a have just simple response.
Can you prompt please, what should I do to get linked out suggestion response?
You have sent a screenshot of the speech interactions of your conversation. Suggestion chips are only shown in the visual display section of the simulator. This can be found on the left side of the web page, either under the Suggestion section or in the visual display of your device.
If you do not see anything on the left side, check if you have set your simulator to a platform that supports visuals during its conversation, for instance:
Phone
Smart Display ( Only normal suggestions will show on smart displays)
I have built an action with Actions-on-Google(2.5.0) and dialogflow-fulfillment(0.6.1) Node.js Library. I cannot test my app on dialogflow test console because I return conv object which is not supported there. Now, I cannot test it in the google action simulator, either. This is the error I get:
Invocation Error
You cannot use standard Google Assistant features in the Simulator. If you want to try them, use Google Assistant on your phone or other compatible devices.
I'd like to use the simulator, so I can debug better.
It is how the error message says: The simulator lacks many features that normal Assistant surfaces (speaker, Assistant app) have and can even sometimes give you completely wrong error messages. There is really no way around testing your app on real devices.
You can however view the same logs that you see in the simulator in Google Stackdriver Logging. To activate this go to the settings of your Dialogflow agent, select the "General" tab and activate the "Log interactions to Google Cloud" option. Then click on the link below the button to get to the logs. The default view will probably show you only the Actions-on-Google logs, i.e. the requests between your users and AoG. To see the requests between Dialogflow and your webhook click on the dropdown arrow in the filter box, select "Convert to advanced filter" and set the filter to resource.type="global".
If you have multiple Actions projects that use the same display name, the simulator chooses one at random. For consistent testing results, use unique names or release channels for each Action.
Reference Link: https://support.google.com/actions-console/answer/9613473?hl=en
Now how to give a display name or change the display name.
Go to develop tab and give display name or change display name as follows
You should definitely be able to test your action in the Actions simulator. Note that the interaction model b/w Dialogflow and Actions simulators are different. In Dialogflow, you can send commands directly to your agent. In the Actions simulator, you first need to invoke your Action.
At the bottom of the screen, you'll see a suggested input like "talk to my test app".
You'll need to send this, or a similar command, first. That will then invoke your action, and you'll be able to send commands to it after. You will see it is invoked by a banner at the top of the simulator.
Initially, I built up a test agent on DialogFlow console according to the document, and it works well on the Actions On Google which is a simulator of Google Assistant on mobile devices.
Then, I deployed it through the Release in the left menu as you can see the pic attached.
deployed successfully
After that, I added some Alpha Testers including my colleague and sent my opt-in link to my colleague, besides, I granted them all the viewer permission in IAM.
However, problem appeared. It didn't work well on my testers’ phone(IOS 10+) but only worked well on developer's account(mine). When they opened the link I sent to them, and clicked send to devices, then clicked the notification on top of the screen.
send to device
The result is shown as below.
Google Assistant didn't respond to "Talk to mytest app"
In my case the command was set as "Talk to hello qad", and it did work well on my phone used the developer account.
developer account works well
If my tester input the text "Talk to hello qad", it just replied some direct searching results not hello qad diaglog.
To recap:
My action has already been in "deployed" status for couple of days
I've added the tester accounts in whitelist and give them "Viewer" permission in IAM
Testers could see the action directory page in devices by open opt-in link, but they couldn't see the "I'm In" button and couldn't access the action
Appreciate for any help or advice
During my project development, I also faced a similar issue. This is how managed to do testing:
Made sure all Google Accounts were created with the US as the country.
Through IAM, share AoG project with the tester's Google accounts.
All testers to open the shared simulator link in their browsers. This is important!
Test the app on browser first using the simulator.
Once tested, use any device with the whitelisted Google Accounts.
US country was required for my use case as I was having Transaction API in my Assistant. I had to also mock my location to the US on mobile for testing US specific features.
See if the above steps help you.
I contacted Google and this is what they said:
You have to copy the link to the notes app, and then click it and in the prompt, choose “Open in Assistant”.
Full text:
Please ensure that the opt-in link is opened in Google Assistant app and not in a browser. At this time, Google Assistant app is only available in USA. For opening an opt-in link in iPhone or any iOS device, please follow the steps below:
1. Download the Assistant App in App Store
2. Log in using the included account for Beta testing
3. Copy the opt-in URL to Notes app
4. Hold press the opt-in link then select 'Open in "Assistant"'. Google Assistant and App page in the Assistant Directory will be displayed.
5. Scroll down the page until you see the "Become a Beta tester" section
6. Click the I'M IN button
7. Test the Action
This does not work for me, probably because I am not in the US. However, the app is of course available here.
I ran into a similar issue with Google Actions/Dialog Flow. Here is how I resolved it...
Share the app from within DialogFlow to the test user
Copy the current DialogFlow URL from the address bar
Launch an Incognito Window
Paste the DialogFlow URL from Step 2 in the address bar
Log into DialogFlow using the test account
At the Standard Google Account Access Prompt Allow the access so your Google Action can talk to DialogFlow.
I couldn't find this documented anywhere and wasted about 8 hours figuring it out.
For me it only worked after I shared with the user the link to test on desktop that look likes: https://console.actions.google.com/project/XXX/simulatorcreate?isDeepLink the one you can get the the console menu on the sharing icon.
I asked him to login with same google email he has opted-in in Google Assistant and
to verify if when mouse over on the devices icon on the menu he would see "Testing on Device: Enabled
You currently are able to test your Actions on all
Assistant devices connected to "xxxx#gmail.com". "
Then he could invoque almost right away the alpha version of the BOT.
All that considering that he has already clicked on the opt-in link, I had added his emails as a Alpha tester and I had also added him as a Viewer on Permissions at the console admin https://console.cloud.google.com/iam-admin.
If you are a developer of the project, the test version is enabled by default on your device. If you want to access the alpha and beta versions, make sure to disable the ‘Testing on device’ option on the Actions Console simulator.
Add users before deploying alpha release
I feel obligated to copy dedman's comment on your question here, since his solution worked for me and is the only one directly addressing the problem at hand: an alpha release is not available to testers you have added after deploying the release.
As stated in question, you first deployed the test version and then added alpha testers. You might have to publish new alpha version "after adding the testers" and then wait till you get "deployed" status on the new alpha release too... – dedman
I was encountering this exact problem with the Alpha release of my Action, i.e. assistant not responding on alpha users devices, even though I had deployed the release, shared the opt-in link with users and had them click on it to opt-in. Deploying a new alpha release after users had clicked the opt-in link solved the problem and the action is now responding correctly for all users.
IAM Viewer status is not mandatory
By the way, I can also confirm your suspicion that "Viewer" permission in IAM is not needed for alpha users to have access to your release - they only need to opt-in before you deploy a new release.
I am trying to test my agent on a real device. Following instruction from
Official Google video
However, my panel for integrating Actions on Google doesn't look similar to the one shown in the video.
I see neither AUTHORIZE nor PREVIEW button. I can not set invocation name and TTS voice as well.
I attached my panel that I see. Is there anything missing?
My Action on Google dialog:
That video predates recent changes in the API.AI Actions on Google screen.
The name and voice are now set in the Actions Console, but neither are required to do testing.
If you're willing to accept the default voice for testing, you can
Click on the "Test" button in the screen you're referencing.
You can then go to the Simulator (there will be a link provided) or ask any Assistant device (such as Home) to start your action with "Talk to my test app".
They launched this new platform on the Google I/O.
Now you have the invocation name and everything else in the Actions on Google Console.
It's a pretty intuitive platform, the only annoying thing is that you need to fulfill all of the App information before testing. You can see that the simulator is in this console as well.
Whenever you modify things in API.ai, click the UPDATE button (the one in your print screen), then TEST. Then you can test in the console simulator.
Link to previous issue: Gmail contextual gadget broken
Yesterday we received a couple of customer complaints regarding our Gmail gadget. They claimed that it had gone missing from their account. This morning, several of our employees have reported the same although, this isn't happening for everyone.
We haven't made any changes to the gadget and since it loads for some, I don't think this was caused by us.
What happens:
The window of the contextual gadget does not appear in the main Gmail interface
The window of the contextual gadget does not appear when opening an email in a new window (shift + click). This differs from when this happened previously.
We've also contacted Google and they have responded that they are looking into the issue. We've also seen that the shift-click to open an email in a new window doesn't work as a work around this time.
The last time this happened our monitoring showed that about 10% of users lost their access to the gadget per-day. We seeing a similar drop off in activity this time around.
Here are the details of the response I received from Google for this issue
Thank you for contacting Google for Work Support. I understand that
that you Gmail Contextual Gadgets have disappeared again from the
Gmail web interface.
I notice there are other user reporting a similar behavior hence he
have requested an update from our Eng team and I am currently waiting
for their response. As soon as I have an official response, I will be
contacting you back directly with the next steps.
Also I notice you mentioned that the previous workaround is no longer
working. Previously the workaround provided was to press "Shift" and
then click on the email. The email will be opened in a new window and
then the gadgets should be displayed. Can you please confirm if the
above workaround is working or not?
I’ll keep this case open while I wait for your reply. If you have any
other questions or additional comments, don’t hesitate to reply and
I'll be happy to continue helping you.
Sincerely,
Wilmer Google for Work Support
Here is the latest update from Google
As before this issue appears to be UI based as opposed to any problem with the Contextual Gadget API itself, we've gathered some troubleshooting information from affected test environments and are working on identifying a clear root cause and then remedying the behaviour at the moment.
There isn't any further you need to do on your end, I'll update you as I learn of any changes in the status of this behaviour.
Regards,
Patrick