I have implemented an Alexa Audio Player skill which plays the audio just fine, but when played on an Echo Show, the name of the song does not show on the display.
I see the documents on Amazon (https://amzn.to/2xzpH4u) refer to a play directive which includes MetaData such as the background image and such, but I'm not sure how to set this up in node.js.
This is the code snippet from my Play intent handler:
if (this.event.request.type === 'IntentRequest' || this.event.request.type === 'LaunchRequest') {
var cardTitle = streamDynamic.subtitle;
var cardContent = streamDynamic.cardContent;
var cardImage = streamDynamic.image;
this.response.cardRenderer(cardTitle, cardContent, cardImage);
}
this.response.speak('Enjoy.').audioPlayerPlay('REPLACE_ALL', streamDynamic.url, streamDynamic.url, null, 0);
this.emit(':responseReady');
In your If statement meaning video rendering is supported you build the content of the metadata for the card that is rendered on the device.
So following the documentation cardTitle, Content and Image all have got to be what you are looking for the device to render as a card. You are returning it to be rendered in the this.response statement when all the resources have been provided.
In the example code from Amazon https://github.com/alexa/skill-sample-nodejs-audio-player/blob/mainline/single-stream/lambda/src/audioAssets.js notice how the card assets are specified. Follow this example and look at the whole project for any other pieces you may be missing.
Related
I have a site where users can upload videos to be encoded and viewed in azure media player. Some of the videos uploaded do not have audio tracks which azure media player can't play. How can I encode an empty audio track with these videos? I'm using v3 of the REST api.
My current code for transforms is:
private async Task<string> CreateTransformAsync(string transform)
{
JObject body = new JObject(
new JProperty("properties",
new JObject(
new JProperty("description", "Basic Transform using an Adaptive Streaming encoding preset from the libray of built-in Standard Encoder presets"),
new JProperty("outputs",
new JArray(
new JObject(
new JProperty("onError", "StopProcessingJob"),
new JProperty("relativePriority", "Normal"),
new JProperty("preset",
new JObject(
new JProperty("#odata.type", "#Microsoft.Media.BuiltInStandardEncoderPreset"),
new JProperty("presetName", "H264MultipleBitrate720p")
)
)
)
)
)
)
)
);
var jsonBody = new StringContent(body.ToString(), Encoding.UTF8, "application/json");
HttpResponseMessage responseMsg = await _httpClient.PutAsync($"subscriptions/{_config.Value.SubscriptionId}/resourceGroups/{_config.Value.ResourceGroup}/providers/Microsoft.Media/mediaServices/{_config.Value.MediaAccountName}/transforms/{transform}/?api-version={_config.Value.ApiVersion}", jsonBody);
string responseContent = await responseMsg.Content.ReadAsStringAsync();
var response = JObject.Parse(responseContent);
if (response["error"] == null)
{
return response["name"].ToString();
} else
{
throw new Exception(response["error"].ToString());
}
}
UPDATE:
After scouring the documentation, I've gotten a little further with this: https://learn.microsoft.com/en-us/azure/media-services/latest/custom-preset-rest-howto#define-a-custom-preset
I now define a custom preset, read it in and send that in the body instead. Problem now is I can't find a similiar option for "condition": "InsertSilenceIfNoAudio" like in v2 of the API. I've opened a github issue about it here: https://github.com/MicrosoftDocs/azure-docs/issues/28133
What's your target encoding settings? Do you need a custom preset?
If not, and you just need a standard adaptive streaming profile preset, you can use the
AdaptiveStreaming preset. It handles the insert silence.
It's not finally announced, but as we tested for our project, Azure Media Player got complete support of Video Only content, starting with version 2.3.0 (April 30, 2019).
Officially there is a mention in feature list that the feature is already implemented ("Video Only" feature with comment "Supported in AzureHtml5JS", here) and it's said in Change list of 2.3.0 release that "Added support for video-only assets for DASH" (here), but we personally tested for SMOOTH and HLS as well - no issues, so video-only assets start playing without any issues starting with version 2.3.0.
At the same time the issue is still mentioned in Known Issues: "Assets that are audio or video only will not play back via the AzureHtml5JS tech.", but I guess they just didn't update the docs. Another option, probably they didn't test it completely, but as I say from our internal testing it looks like it completely works.
I'm trying to get a multi-audio HLS stream working on a v3 Google Cast custom receiver app. The master playlist of the stream refers to several video renditions of different resolution and two alternative audio tracks:
#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",LANGUAGE="de",NAME="TV Ton",DEFAULT=YES, AUTOSELECT=YES,URI="index_1_a.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",LANGUAGE="de",NAME="Audiodeskription",DEFAULT=NO, AUTOSELECT=NO,URI="index_2_a.m3u8"
#EXT-X-STREAM-INF:AUDIO="aac",BANDWIDTH=383000,RESOLUTION=320x176,CODECS="avc1.4d001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
index_0_av.m3u8
...more renditions
#EXT-X-STREAM-INF:AUDIO="aac",BANDWIDTH=3697000,RESOLUTION=1280x720,CODECS="avc1.4d001f, mp4a.40.2",CLOSED-CAPTIONS=NONE
index_6_av.m3u8
The video plays fine in both the sender and receiver app, I can see both audio tracks in the sender app, but when casting to the receiver there are no controls for changing the audio tracks.
When accessing the AudioTracksManager's getTracks() method while intercepting the LOAD message like so...
playerManager.setMessageInterceptor(
cast.framework.messages.MessageType.LOAD, loadRequestData => {
loadRequestData.media.hlsSegmentFormat = cast.framework.messages.HlsSegmentFormat.TS
const audioTracksManager = playerManager.getAudioTracksManager();
console.log(audioTracksManager.getTracks())
console.log('Load request: ', loadRequestData);
return loadRequestData;
});
I get an error saying:
Uncaught Error: Tracks info is not available.
Maybe unrelated, but super weird: I can console.log the request's media prop and see its tracks prop (an array with the expected 1 video and 2 audio tracks), however, if I try to access the tracks property in the LOAD message interceptor I get undefined.
I currently cannot look into the iOS sender code yet, so I tried to eliminate error sources on the receiver end. The thing is:
I always assumed that the receiver identifies alternative audio tracks on its own when loading HLS playlists. Is this assumption correct or can the AudioTracksManager only access tracks that have been previously defined in a sender app?
I couldn't find a clear statement on that in the Google Cast reference...
Ok, feeling stupid for the time I spent on this, but I'm finally able to answer my own question. I didn't realize that I was accessing the AudioTracksManager in the wrong place - namely in the LOAD message interceptor instead of in a PLAYER_LOAD_COMPLETE event listener (as it is properly documented here)
After placing my logic into this event listener I was able to access and programmatically set my audio tracks.
So to answer my original question: Yes, the receiver app automatically identifies alternative audio tracks from an HLS playlist.
Google has somewhat recently rolled out the ability to insert audio files from your Drive into Slides with various playback options.
I cannot find any documentation on how to insert a file through Google Scripts but can do so going through the available menu options. I tried using the insertVideo method but got an error
"Exception: The parameters (DriveApp.File) don't match the method signature for SlidesApp.Slide.insertVideo."
Here is a general function I'm trying to get to work (NOOB disclaimer goes here):
function uploadAudioToCurrentSlide(){
var presentation = SlidesApp.getActivePresentation();
var currentSlide = presentation.getSlides()[0];
var audioFile = DriveApp.getFileById('idofaudiofileindrive');
currentSlide.insertVideo(audioFile);
}
Any help is most appreciated!
You want to insert a audio file in Google Drive to Google Slides using Google Apps Script.
Issue and workaround:
I think that the reason of your issue is that the file object is directly used to the method of insertVideo. The argument of insertVideo is the URL and the video object which is not the file object. By this, such error occurs.
In the current stage, when the method of insertVideo is used, the video content is required to be the publicly shared YouTube URL.
And also, it seems that the audio file cannot be directly inserted.
Unfortunately, it seems that these are the current specification. So as a workaround, how about the following flow?
At first, convert the audio file to a video file like MP4. As a test, this can be done at other site. But I'm not sure about the file type of your audio file.
Insert the converted MP4 file on Google Drive using Slides API.
When the Slides API is used, you can insert the video file in Google Drive to the Google Slides. In this sample script, "CreateVideoRequest" of the batchUpdate method of Slides API is used.
Sample script:
Before you run the script, please enable Slides API at Advanced Google services.
function myFunction() {
var fileId = "###"; // Please set the file ID of the converted video file on Google Drive.
var presentation = SlidesApp.getActivePresentation();
var currentSlide = presentation.getSlides()[0];
var resource = {requests: [{createVideo: {source: "DRIVE", id: fileId, elementProperties: {pageObjectId: currentSlide.getObjectId()}}}]};
Slides.Presentations.batchUpdate(resource, presentation.getId());
}
Note:
When you can upload the audio file to YouTube and publicly share it, you can use your script using the URL of the YouTube.
References:
insertVideo(videoUrl)- Advanced Google services
Method: presentations.batchUpdate
CreateVideoRequest
I'm using the SpotifyAPI-NET on GitHub from JohnnyCrazy to play and pause songs on my Spotify desktop client. This works fine.
Now I want to change the playing position of the currently playing song. So I only want to say something like "SetPlayingPosition(64)" to play the current song from position "01:04". It seems that the SpotifyLocalAPI didn't support this feature.
To play and pause a song the API uses a message with the following format:
http://127.0.0.1:4381/remote/pause.json?pause=true&ref=&cors=&_=1520448230&oauth=oauth&csrf=csrf
I tried to find a summary of possible commands in this format, but I didn't find anything.
Is there something like http://127.0.0.1:4381/remote/seek.json... that I can use to seek to a specific position?
EDIT:
I tried to write my own method in the RemoteHandler class in the local portion of the SpotifyAPI. With this method I can set the position in the current playback.
Here's my code:
internal async Task SendPositionRequest(double playingPositionSec) //The desired playback position in seconds
{
StatusResponse status = GetNewStatus(); //Get the current status of the local desktop API
string trackUri = "spotify:track:" + status.Track.TrackResource.ParseUri().Id; //The URI of the current track
TimeSpan playingPositionTimeSpan = TimeSpan.FromSeconds(playingPositionSec);
string playingPosStr = playingPositionTimeSpan.ToString(#"mm\:ss"); //Convert the playingPosition to a string (Format mm:ss)
string playingContext = "spotify:artist:1EfwyuCzDQpCslZc8C9gkG";
await SendPlayRequest(trackUri + "#" + playingPosStr, playingContext);
if (!status.Playing) { await SendPauseRequest(); }
}
I need to call the SendPlayRequest() method with the correct playingContext because when the current song is part of a playlist and you call SendPlayRequest() without the context, the next song isn't from the playlist anymore.
But you can see that I use a fixed context at the moment.
So my question is now: How can I get the context (playlist, artist, ...) of the currently played song with the SpotifyLocalAPI?
The SeekPlayback method of the library you mentioned lets you seek through playback on whatever device your user is listening on. You can find the docs here.
Seeking playback is not currently possible using the Spotify Local API portion of that library.
I would like to play a short sound for a more amusing output. If I understand the documentation correctly it should be possible with a reply in api.ai of something like this SSML:
<speak>Okay here we go: <audio src="http://example.com/boing.wav">boing</audio>. You are welcome!</speak>
Just for reference SSML means Speech Synthesis Markup Language.
The web simulator don't play this sound instead all tags seems to be stripped out. Is that not supported yet or did I do something wrong?
The src URL must also be an https URL (Google Cloud Storage can host your audio files on an https URL).
https://developers.google.com/actions/reference/ssml
Without seeing your source, there are a few possible reasons:
The audio file must be served publicly via HTTPS, not HTTP. See the description for <audio> on https://developers.google.com/actions/reference/ssml
The audio file should be in a correct format (see https://developers.google.com/actions/reference/ssml again).
If you're returning it via the webhook response, you need to make sure you set the data.google.is_ssml property in the JSON to true in https://developers.google.com/actions/reference/webhook-format#response
I have the following for my node.js server which works (well, except for the URL):
var msg = `
<speak>
Tone one
<audio src="https://examaple.com/wav/Dtmf-1.wav"></audio>
Tone two
<audio src="https://example.com/wav16/Dtmf-2.wav"></audio>
Foghorn
<audio src="https://example.com/mp3/foghorn.mp3"></audio>
Done
</speak>
`;
var reply = {
speech: msg,
data:{
google:{
"expect_user_response": true,
"is_ssml": true
}
}
};
res.send( reply );
So here is what I have for the code. It is in the Text responds field one my intent.
<speak> One second <break time="3s"/> OK, I have used the best quantum processing algorithms known to computer science! Your silly name is $color $number. I hope you like it. <audio src="https://www.partnersinrhyme.com/files/sounds1/WAV/sports/baseball/Ball_Hit_Cheer.wav"></audio> </speak>
It does not work in the testing area of the api(dot)ai field, but does work when I turn on the integration and try it at the Google simiulator. here: https://developers.google.com/actions/tools/web-simulator