Is there a way to play back a media objects for a specified period using nodejs actions sdk ?
For example in the code below in the code below the file is for 1 minutes and 42 seconds. But suppose I had a file that was 60 minutes long, I want to be able to pass a parameter to the object that will specify the duration of the play back, which would be less than the total duration .
function yourFunctionHandler(agent) {
let conv = agent.conv();
conv.ask(new SimpleResponse("Here is a funky Jazz tune"));
conv.ask(new Suggestions(['suggestion 1', 'suggestion 2']));
conv.close(new MediaObject({
name: 'Jazz in Paris',
url: 'https://storage.googleapis.com/automotive-media/Jazz_In_Paris.mp3',
description: 'A funky Jazz tune',
icon: new Image({
url: 'https://storage.googleapis.com/automotive-media/album_art.jpg',
alt: 'Media icon',
}),
})); }
No, there is currently no way to limit the playback from a Media Response to a limited time, to specify a start point in the media, or to specify an end point in the media.
Related
I'm building a MERN stack web app and I want the users to be able to upload post with an image and post data.
The data looks like this:
title: 'The war in Ukraine continues',
text: 'Volodymyr Horbenko is the latest official to lose his job after Mr Zelensky said bosses failed to root out pro-Russian elements in the agency',
sources: [
'https://www.bbc.com/news/world-62223264',
'https://www.theguardian.com/world/live/2022/jul/19/russia-ukraine-war-live-news-putin-and-erdogan-to-meet-us-weaponry-stabilising-frontlines-ukraine-military-chief-says',
],
locations: [
{
description: 'boutcha',
type: 'Point',
coordinates: [-122.479887, 38.510312],
},
{
description: 'kyiv',
type: 'Point',
coordinates: [-122.582948, 38.585707],
},
],
};
now I also want to upload a picture this is just a photo that comes from an input.
I know that I should use formData to send the picture and i've tried uploading the data as well like this:
const form = new FormData();
form.append('data', JSON.stringify(data));
form.append('imageCover', fileInputRef.current.files[0]);
axios.post('/api/v1/stories/', form);
but when it comes to the backend I haven't been able to figure out how to retrieve the data into the request body. I've tried using multer app.use(multer.single('data'));(which works fine for the pictures) and app.use(express.json({ type: 'multipart/form-data' }));
but I get Error: Unexpected end of form, and Error:Unexpected token - in JSON at position 0 errors respectively.
also, if there is another way of sending both form data and files i'd like to hear about it, keep in mind that because there are nested objects I can't simply write form.append(key,value) for each pair in the data
app.post("/api/v1/stories", multer().single("imageCover"), function(req, res) {
var file = req.file;
var data = JSON.parse(req.body.data);
});
This is the response I'm sending from the webhook:
conv.ask(new Carousel({
title: 'Carousel Title',
items: {
'AU': {
title: 'Australia',
subtitle: '',
image: new Image({
url: 'https://blocky-host-files.web.app/dc_graphics/au.png',
alt: 'Australia graphic',
}),
},
'CA_US': {
title: 'Americas',
subtitle: 'Canada / United States',
image: new Image({
url: 'https://blocky-host-files.web.app/dc_graphics/ca_us.png',
alt: 'Canada United States graphic',
}),
},
'EU': {
title: 'Europe',
subtitle: '',
image: new Image({
url: 'https://blocky-host-files.web.app/dc_graphics/eu.png',
alt: 'Europe graphic',
}),
}
}
}));
This is how it looks on my phone:
I cannot figure out why the images aren't shown. They are not being displayed on my Google Home Hub too.
However, the Actions on Google simulator displays my images.
I think your issue is coming due to two possible reasons:
There might be a problem with the platform you are using (blocky host). It is possible that this platform is not supported for displaying the carousel, Please check, if you are able to display a card in this platform. If not - then there is some problem with the platform you are are using
It can be due to a network issue
You can try seeing the surface capability in the request. It can help
It works now after moving the images to a different host (imgur).
I've used Firebase Hosting to host the images before and somehow Google couldn't get these images from it's own platform, even though they are public accessible.
I'm building an AOG (actions on google) project that will do basic transaction functionality. Since I'm still a bit new to AOG, I'm completely stuck on how to take what the user selects (whether it be a carousel, a basic card etc.) and pass that argument value/key that they selected into the proposed order or the order preview before they finish their transaction.
Here is basically what I have tried (This isn't the actual code because it's rather long, but it still gets the idea across)
app.intent('delivery_address_complete', (conv) => {
const arg = conv.arguments.get('DELIVERY_ADDRESS_VALUE');
if (arg.userDecision ==='ACCEPTED') {
conv.ask('Ok, what would you like to order?');
conv.ask(new Suggestions(intentSuggestions));
conv.ask(new Carousel({
items: {
// Add the first item to the carousel
SELECTION_KEY_COFFEE: {
synonyms: [
'Coffee'
],
title: 'Coffee',
description: 'Sweet cream and sugar coffee.',
image: new Image({
url: IMG_URL_COFFEE,
alt: 'Image alternate text',
}),
},
}));
}
});
const yesOrno = [
'Yes',
'No'
];
app.intent('actions.intent.OPTION', (conv ) => {
conv.ask('Okay, are you ready to proceed?');
conv.ask(new Suggestions(yesOrno));
});
app.intent('transaction_decision_action', (conv) => {
const order = {
id: UNIQUE_ORDER_ID,
cart: {
merchant: {
id: 'coffee',
name: 'Coffee Store',
},
lineItems: [
{
name: 'My Memoirs',
id: 'coffee_1',
price: {
amount: {
currencyCode: 'USD',
nanos: 990000000,
units: 3,
},
type: 'ACTUAL',
},
quantity: 1,
subLines: [
{
note: 'coffee',
},
],
type: 'REGULAR',
},
otherItems: [
{
name: 'Subtotal',
id: 'subtotal',
price: {
amount: {
currencyCode: 'USD',
nanos: 220000000,
units: 32,
},
type: 'ESTIMATE',
},
type: 'SUBTOTAL',
},
{
name: 'Tax',
id: 'tax',
price: {
amount: {
currencyCode: 'USD',
nanos: 780000000,
units: 2,
},
type: 'ESTIMATE',
},
type: 'TAX',
},
],
totalPrice: {
amount: {
currencyCode: 'USD',
nanos: 0,
units: 35,
},
type: 'ESTIMATE',
},
};
Please note: This is mostly dummy code, so if some things like over charging or prices not making sense is happening, it's not the problem I'm trying to fix.
How can I take what the user selected from whatever method, and get it so it will appear on the order preview or proposed order? I do not need help with anything regarding making carousels or basic cards ect. Just how to get this selected information to the order preview.
To be more specific:
I can create an order object that is required, and I know how to send it to Google (and then to the user) as part of a ProposedOrder object that becomes part of the TransactionDecision object. (The "transaction_decision_action" Intent handler in the code above.)
What I don't understand is how to build the order based on the user saying things or by selecting on carousel or list items that I've shown them. (What do I do in the "actions.intent.OPTION" Intent handler above, for example?)
edit: This also may clear up any confusion. This is a video representation of what I'm attempting to do (mentioned in comments below):
youtube.com/watch?v=LlgMcJBnNN8 from 1:02 to 1:29 I know how to do, I'm confused (In the video example) how they were able to get the 'turkey sandwich' and the 'Green smoothie' added to the order preview at 1:35 ish from the carousel selections
What you're looking to do is what Google refers to as building the order. As it notes at that link
Once you have the user information you need, you'll build a "cart
assembly" experience that guides the user to build an order. Every
Action will likely have a slightly different cart assembly flow as
appropriate for your product or service.
You could build a cart assembly experience that enables the user to
re-order their most recent purchase via a simple yes or no question.
You could also present the user a carousel or list card of the top
"featured" or "recommended" items. We recommend using rich responses
to present the user's options visually, but also design the
conversation such that the user can build their cart using only their
voice.
For more information on how to build a high-quality cart assembly
experience, see the Transactions Design Guidelines.
So there is no one way to do what you're asking about. However, there are a few tips of things you can and should be doing to build the proposed order.
Managing the order
The big thing you need to do is to keep track of all the things that the user is ordering as you go through the process. There are a number of ways you can store this information:
In a Dialogflow Context
In the user session store
In a database or data store for the session
In short, any of the current ways you have to store session information. All of the information below assumes you've picked some way to do this.
Since everything will become one of the lineItems, an easy solution is to build this array as you go along, and then you can just copy the array directly into the order object. Another approach is to just store a list of item IDs, and then populate the rest of the information later when we build the order.
For this example, we're going to go with this latter scheme (because its easier to show) and store it in the session storage object using the actions-on-google library.
So for starters, when we start the Action, or when we know we'll be taking the order, we need to initialize our list of items being ordered with something like
conv.user.data.items = [];
Now that we have our initial item list, we can explore different ways to add to this list.
Adding an item: "my regular"
For some types of orders, it may make sense for the user to be able to say "I'll have my usual". In cases like this, we want an Intent that handles this phrase (or handles a "yes" response to our prompting), and an Intent Handler that looks up the user's regular order and adds it to the items. Perhaps something like this:
app.intent('order.usual', conv => {
// Get their user profile from our database
// The "loadUser" function is up to you, and has little to do with AoG
return loadUser( conv )
.then( user => {
// Add each item in their usual order to the current items
let usualOrder = user.usualOrder;
usualOrder.forEach( item => conv.user.data.items.push( item ) );
// Send a message back to the user
conv.add( "You got it! Do you want anything else?" );
});
});
Adding an item from a list
If you've presented a carousel or a list to the user of possible items, your life is a little easier (although you may not think it at the moment). You do need to setup a Dialogflow Intent that handles the actions_intent_OPTION event (which I'll call order.option in this case).
In the handler for this, we'll assume that the key you used for the option also happens to be the item ID, so you can just add it to the list
app.intent('order.option', (conv, params, option) => {
// The item is the option sent
let item = option;
// Add the item to the list of items
conv.user.data.items.push( item );
// Send a message back to the user
conv.add( "I've put that in your cart. Anything else?" );
});
Adding an item by name
But remember, the user can take the conversation in any direction at any time. So they may ask for an item that you currently aren't showing in the carousel. The best way to handle this is by creating an Entity Type in Dialogflow (which I'll call item, as an example)
And then an Intent that captures some phrases that expresses the user asking to add them (which I'll call order.name and which has an itemName parameter that the user has to include).
[
In the handler, you need to get the name that they spoke, look up what the item is, and add this to the list of items they've ordered.
app.intent('order.name', (conv, params) => {
// Get the name
let itemName = params['itemName'];
// Look it up to find out what they ordered
// You need to implement the itemFromName function
return itemFromName( itemName )
.then( item => {
// Add the item
conv.user.data.items.push( item );
// And reply
conv.add( "You got it! Anything else?" );
});
});
Finish building the order
Once you've finished collecting everything they want, your Intent Handler should put the order together, assembling the full list of lineItems from the conv.user.data.items array that we've been putting together, calculating tax, totals, and all the other parts of the order.
We then need to propose the order by sending a TransactionDecision object that contains our order in the proposedOrder parameter. Clever, no? Possibly something like this:
app.intent('review', conv => {
// Get the items the user has saved
let items = conv.user.data.items;
// Turn these into more complete lineItems
// You will need to provide the "itemToLineItem" function
let lineItems = items.map( itemToLineItem );
// Get some other objects we need
// You'll need to define these functions, too
let orderId = generateOrderId();
let subtotal = computeSubtotal( lineItems );
let tax = computeTax( lineItems );
let total = computerTotal( subtotal, tax );
// Build the order object
let order = buildOrder( lineItems, subtotal, tax, total );
conv.ask(new TransactionDecision({
orderOptions: {
requestDeliveryAddress: false,
},
paymentOptions: {
googleProvidedOptions: {
prepaidCardDisallowed: false,
supportedCardNetworks: ['VISA', 'AMEX'],
// These will be provided by payment processor,
// like Stripe, Braintree, or Vantiv.
tokenizationParameters: {
tokenizationType: 'PAYMENT_GATEWAY',
parameters: {
'gateway': 'stripe',
'stripe:publishableKey': (conv.sandbox ? 'pk_test_key' : 'pk_live_key'),
'stripe:version': '2017-04-06'
},
},
},
},
proposedOrder: order,
}));
});
I broke most of the stuff out as a function since there is nothing specific about them, except the format of the order (which you illustrate in your example). You can really build it any way you want.
Conclusion
Much of what you need to do really boils down to
Collecting the information of what the user wants to order, mostly storing the IDs of these items
Turning this list of items into the complete order object
Sending this order for the user to review
I am using Square Payment API, I couldn't find anything in documents for the same,
Let say I want to charge 52.50 CAD.
As we all know It does not allow decimal amount (I am using Node.js SDK v2).
When I go through the documentation it says that amount must be in the smaller denomination.
I have own custom form, I want to know, do I need to pass amount in smaller denomination when the passing value from payment form (generating card nonce), or only in the API side, or both.
I am asking amount from the user without smaller denomination i.e. (52.50 CAD), How do I pass amount in SDK and in the frontend as well.
The smallest denomination of the Canadian Dollar (CAD), like the US Dollar (USD), is cents.
Your custom/front-end form may elect to show the amount in dollars, but the API call must be in the smallest denomination.
As you can see in the below example code, the nonce is taken from the payment form and the charged amount is hard coded. If you wanted to take the charged amount from the form, you would need to sanitize it, convert it to cents (if in dollars) and make sure that the currency ID is correct.
Example Code
Source: Payment processing example: Node JS
router.post('/process-payment', function(req,res,next){
var request_params = req.body;
var idempotency_key = require('crypto').randomBytes(64).toString('hex');
// Charge the customer's card
var transactions_api = new squareConnect.TransactionsApi();
var request_body = {
card_nonce: request_params.nonce,
amount_money: {
amount: 100, // $1.00 charge
currency: 'USD'
},
idempotency_key: idempotency_key
};
transactions_api.charge(config.squareLocationId, request_body).then(function(data) {
console.log(util.inspect(data, false, null));
res.render('process-payment', {
'title': 'Payment Successful',
'result': "Payment Successful (see console for transaction output)"
});
}, function(error) {
console.log(util.inspect(error.status, false, null));
res.render('process-payment', {
'title': 'Payment Failure',
'result': "Payment Failed (see console for error output)"
});
});
});
Relevant documentation:
SquareConnect.TransactionsApi
SquareConnect.ChargeRequest
SquareConnect.Money
Code to replicateIn my app I have the following code.
var seed = "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP";
sp.core.getMetadata(seed, {
onSuccess: function (metadata) {
console.log(metadata);
},
onFailure: function () {}
});
Steps to view the behaviour
If spotify is already open close it and then reopen it, after opening spotify run the app and view the console Which displays as
Object
name: "Whitesnake"
portrait: ""
type: "artist"
uri: "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP"
__proto__: Object
Then click on the radio app and create a station based on the the band Whitesnake.
Rerun the original app and view the console output which will display.
Object
name: "Whitesnake"
portrait: "spotify:image:3c4aa30d845dd456d750cf16bef8e2cadb2af342"
type: "artist"
uri: "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP"
__proto__: Object
Quit spotify and rerun the original application and view the console which will display
Object
name: "Whitesnake"
portrait: ""
type: "artist"
uri: "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP"
__proto__: Object
Question
Why is the radio app able to access a portrait uri that is then accessible from my app after using the radio app? Is there some sort of initialization, that needs to be done first or is this a bug?
Some information on an artist (and album, etc) isn't available until you do a "browse" on that URI - until then, Spotify only downloads basic information on an item to keep performance up.
What you're seeing is the Radio doing a browse, which causes the Spotify client to fill in the missing information.
An update to Spotify is coming soon that will improve this behaviour.
If I understood you correctly, you've noticed that after you get the result with a portrait URI from the radio, you get that object data populated when using it in your app?
On another note; you should be using the Artist class instead of sp.core.
var sp = getSpotifyApi(1);
var models = sp.require('sp://import/scripts/api/models');
models.Artist.fromURI("spotify:artist:3UbyYnvNIT5DFXU4WgiGpP", function(album) {
console.log(album);
});
This should give you the correct output with a portrait uri string.
Artist
data: Object
name: "Whitesnake"
portrait: "spotify:image:3c4aa30d845dd456d750cf16bef8e2cadb2af342"
type: "artist"
uri: "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP"
As iKenndac mentioned, try doing a 'browse':
var seed = "spotify:artist:3UbyYnvNIT5DFXU4WgiGpP";
sp.core.browseUri(seed, {
onSuccess: function (metadata) {
console.log(metadata);
},
onFailure: function () {}
});
Examples here: https://github.com/ptrwtts/kitchensink