Azure Media Services - encoding 4K UHD video with v3 - azure

I made a library that encodes video in Azure using v3 API (.NET Core). I successfully made encoding up to FHD.
But then I tried to encode 4k UHD video (based on How to encode with a custom Transform and H264 Multiple Bitrate 4K articles).
So, here's my code to create this Transform:
private static async Task<Transform> Ensure4kTransformExistsAsync(IAzureMediaServicesClient client,
string resourceGroupName,
string accountName)
{
H264Layer CreateH264Layer(int bitrate, int width, int height)
{
return new H264Layer(
profile: H264VideoProfile.Auto,
level: "auto",
bitrate: bitrate, // Note that the units is in bits per second
maxBitrate: bitrate,
//bufferWindow: TimeSpan.FromSeconds(5), // this is the default
width: width.ToString(),
height: height.ToString(),
bFrames: 3,
referenceFrames: 3,
adaptiveBFrame: true,
frameRate: "0/1"
);
}
// Does a Transform already exist with the desired name? Assume that an existing Transform with the desired name
// also uses the same recipe or Preset for processing content.
Transform transform = await client.Transforms.GetAsync(resourceGroupName, accountName, TRANSFORM_NAME_H264_MULTIPLE_4K_S);
if (transform != null) return transform;
// Create a new Transform Outputs array - this defines the set of outputs for the Transform
TransformOutput[] outputs =
{
// Create a new TransformOutput with a custom Standard Encoder Preset
new TransformOutput(
new StandardEncoderPreset(
codecs: new Codec[]
{
// Add an AAC Audio layer for the audio encoding
new AacAudio(
channels: 2,
samplingRate: 48000,
bitrate: 128000,
profile: AacAudioProfile.AacLc
),
// Next, add a H264Video for the video encoding
new H264Video(
// Set the GOP interval to 2 seconds for both H264Layers
keyFrameInterval: TimeSpan.FromSeconds(2),
// Add H264Layers
layers: new[]
{
CreateH264Layer(20000000, 4096, 2304),
CreateH264Layer(18000000, 3840, 2160),
CreateH264Layer(16000000, 3840, 2160),
CreateH264Layer(14000000, 3840, 2160),
CreateH264Layer(12000000, 2560, 1440),
CreateH264Layer(10000000, 2560, 1440),
CreateH264Layer(8000000, 2560, 1440),
CreateH264Layer(6000000, 1920, 1080),
CreateH264Layer(4700000, 1920, 1080),
CreateH264Layer(3400000, 1280, 720),
CreateH264Layer(2250000, 960, 540),
CreateH264Layer(1000000, 640, 360)
}
),
// Also generate a set of PNG thumbnails
new PngImage(
start: "25%",
step: "25%",
range: "80%",
layers: new[]
{
new PngLayer(
"50%",
"50%"
)
}
)
},
// Specify the format for the output files - one for video+audio, and another for the thumbnails
formats: new Format[]
{
// Mux the H.264 video and AAC audio into MP4 files, using basename, label, bitrate and extension macros
// Note that since you have multiple H264Layers defined above, you have to use a macro that produces unique names per H264Layer
// Either {Label} or {Bitrate} should suffice
new Mp4Format(
"Video-{Basename}-{Label}-{Bitrate}{Extension}"
),
new PngFormat(
"Thumbnail-{Basename}-{Index}{Extension}"
)
}
),
OnErrorType.StopProcessingJob,
Priority.Normal
)
};
const string DESCRIPTION = "Multiple 4k";
// Create the custom Transform with the outputs defined above
transform = await client.Transforms.CreateOrUpdateAsync(resourceGroupName, accountName, TRANSFORM_NAME_H264_MULTIPLE_4K_S,
outputs,
DESCRIPTION);
return transform;
}
But the job ends up with the following error:
Job ended with error: Fatal service error, please contact support.
An error has occurred. Stage: ProcessSubtaskRequest. Code: System.Net.WebException.
And I did use S3 Media Reserved Unit for encoding. So, is there any way to make it work?

Posting back the solutions to this thread for completeness
There was a bug in the sample code (RunAsync() method), which resulted in Jobs using an incorrect output Asset. The bug has now been fixed.
There was a related bug in error handling that is being addressed

Related

How can I load an exported Tileset (image collection) from Tiled in Phaser 3?

I want to load an image collection tileset into my phaser game. I know that with tilesets that are just one image you can just load that image into phaser, but what about an image collection? In Tiled I saw the options to export that tileset as either .tsx or .json, but I don't know if that helps in my case. The reason I need this is because I have some objects that are too big to be used as tiles. I can load them into Tiled and place them like I want to, but obviously they don't show up in my game, unless I can import that tileset into phaser. Does anyone know how to do that, or maybe you know a better option than using an image collection?
Well after some tests, and updating my Tiled version to 1.9.2, it seems there is an pretty simple way.
As long as the tileset collection is marked as "Embeded in map"
(I could have sworn, this checkbox was hidden/deactivated, when selecting "Collection of Images", in my early Tiled version)
Export the map as json
Load map and tile-images
preload() {
this.load.tilemapTiledJSON('map', 'export.tmj');
this.load.image('tile01', 'tile01.png');
this.load.image('tile02', 'tile02.png');
...
}
create Phaser TileSets, just use the filename from the json as the tilsetName (this is the "tricky" part, atleast for me)
create() {
var map = this.make.tilemap({ key: 'map' });
var img1 = map.addTilesetImage( 'tile01.png', 'tile01');
var img2 = map.addTilesetImage( 'tile02.png', 'tile02');
...
// create the layer with all tilesets
map.createLayer('Tile Layer 1', [img1, img2, ...]);
...
}
This should work, atleast with a "Collection of images", with images that have a size of 8x8 pixel (since I don't know the needed/intended Images size, I didn't want to waste time testing various images-sizes needlessly).
Here a small demo:
(due to CORS-issues the map data is inserted as jsonObject and the textures are generate and not loaded)
const mapJsonExport = {"compressionlevel":-1,"height":10,"infinite":false,"layers":[{"compression":"","data":"AQAAAAEAAAACAAAAAgAAAAEAAAACAAAAAgAAAAIAAAACAAAAAgAAAAIAAAACAAAAAgAAAAEAAAABAAAAAQAAAAEAAAABAAAAAQAAAAIAAAABAAAAAQAAAAEAAAABAAAAAQAAAAEAAAABAAAAAQAAAAEAAAACAAAAAQAAAAEAAAACAAAAAgAAAAEAAAABAAAAAgAAAAIAAAABAAAAAgAAAAIAAAACAAAAAgAAAAEAAAACAAAAAQAAAAEAAAABAAAAAQAAAAEAAAABAAAAAQAAAAIAAAACAAAAAgAAAAEAAAACAAAAAgAAAAEAAAACAAAAAQAAAAEAAAACAAAAAQAAAAEAAAABAAAAAQAAAAEAAAACAAAAAgAAAAEAAAABAAAAAgAAAAEAAAABAAAAAQAAAAEAAAACAAAAAQAAAAIAAAACAAAAAgAAAAEAAAABAAAAAgAAAAEAAAACAAAAAgAAAAIAAAACAAAAAgAAAAEAAAABAAAAAgAAAAIAAAACAAAAAgAAAAEAAAACAAAAAQAAAA==","encoding":"base64","height":10,"id":1,"name":"Tile Layer 1","opacity":1,"type":"tilelayer","visible":true,"width":10,"x":0,"y":0}],"nextlayerid":2,"nextobjectid":1,"orientation":"orthogonal","renderorder":"right-down","tiledversion":"1.9.2","tileheight":8,"tilesets":[{"columns":0,"firstgid":1,"grid":{"height":1,"orientation":"orthogonal","width":1},"margin":0,"name":"tiles","spacing":0,"tilecount":2,"tileheight":8,"tiles":[{"id":0,"image":"tile01.png","imageheight":8,"imagewidth":8},{"id":1,"image":"tile02.png","imageheight":8,"imagewidth":8}],"tilewidth":8}],"tilewidth":8,"type":"map","version":"1.9","width":10};
var config = {
width: 8 * 10,
height: 8 * 10,
zoom: 2.2,
scene: { preload, create }
};
function preload() {
// loading inline JSON due to CORS-issues with the code Snippets
this.load.tilemapTiledJSON('map', mapJsonExport);
// generating texture instead of loading them due to CORS-issues with the code Snippets
let graphics = this.make.graphics({add: false});
graphics.fillStyle(0xff0000);
graphics.fillRect(0, 0, 8, 8);
graphics.generateTexture('tile01', 8, 8);
graphics.fillStyle(0x000000);
graphics.fillRect(0, 0, 8, 8);
graphics.generateTexture('tile02', 8, 8);
}
function create () {
let map = this.make.tilemap({ key: 'map' });
let img1 = map.addTilesetImage( 'tile01.png', 'tile01');
let img2 = map.addTilesetImage( 'tile02.png', 'tile02');
map.createLayer('Tile Layer 1', [img1, img2], 0, 0);
}
new Phaser.Game(config);
<script src="https://cdn.jsdelivr.net/npm/phaser#3.55.2/dist/phaser.js"></script>

Ogg opus granule position to timestamp

With an ultimate aim to crop/cut/trim the Ogg file containing a single opus stream,
I'm trying to retrieve and filter ogg pages form the file and those which sit between the crop window of startTimeMs and endTimeMs I'll append them to the 2 ogg heads resulting in trimmed opus without transcoding
I have reached a stage where I have access to the ogg pages but I'm confused on how to determine if the page lies in crop window or not
OggOpusStream oggOpusStream = OggOpusStream.from("audio/technology.opus");
// Get ID Header
IdHeader idHeader = oggOpusStream.getIdHeader();
// Get Comment Header
CommentHeader commentHeader = oggOpusStream.getCommentHeader();
while (true) {
AudioDataPacket audioDataPacket = oggOpusStream.readAudioPacket();
if (audioDataPacket == null) break;
for (OpusPacket opusPacket : audioDataPacket.getOpusPackets()) {
if(packetLiesWithinTrimRange(opusPacket )){ writeToOutput(opusPacket); }
}
}
// Create an output stream
OutputStream outputStream = ...;
// Create a new Ogg page
OggPage oggPage = OggPage.empty();
// Set header fields by calling setX() method
oggPage.setSerialNum(100);
// Add a data packet to this page
oggPage.addDataPacket(audioDataPacket.dump());
// Call dump() method to dump the OggPage object to byte array binary
byte[] binary = oggPage.dump();
// Write the binary to stream
outputStream.write(binary);
It should work if I would be able to complete this method
private boolean packetLiesWithinTrimRange(OpusPacket packet){
if(????????){ return true;}
return false;
}
or maybe
private boolean pageLiesWithinTrimRange(OggPage page){
if(????????){ return true;}
return false;
}
Any ogg/opus help is appreciated
https://github.com/leonfancy/oggus/issues/2
OggPage.java with private long granulePosition;
https://github.com/leonfancy/oggus/blob/master/src/main/java/org/chenliang/oggus/ogg/OggPage.java
Ogg Encapsulation for the Opus Audio Codec
https://datatracker.ietf.org/doc/html/rfc7845
An audio page's end time can be calculated with using the stream's pre-skip and first/initial granule position. The page's start time can be obtained using the previous page's end time. See pseudocode below:
sampleRate = 48_000
streamPreskip = ...
streamGranulePosFirst = ...
isPageWithinTimeRange(page, prevPage, msStart, msEnd) {
pageMsStart = getPageMsEnd(prevPage)
pageMsEnd = getPageMsEnd(page)
return (pageMsStart >= msStart && pageMsEnd <= msEnd)
}
getPageMsEnd(page) {
return (page.granulePos - streamGranulePosFirst - streamPreskip) / sampleRate
}

How to crop image after upload Cloudinary?

How i can to crop image after upload and send edited response ulr to frontent?
I will be grateful for the answer
MY CODE:
const stream = cloudinary.uploader.upload_stream(
{
folder,
},
(error: UploadApiErrorResponse | undefined, result: UploadApiResponse | undefined): void => {
console.log(error, result)
if (result) {
resolve({
url: result.url,
size: Math.round(result.bytes / 1024),
height: result.height,
width: result.width,
})
} else {
reject(error)
}
}
)
streamifier.createReadStream(buffer).pipe(stream)
The most common method of integrating Cloudinary is that you upload the original file to your Cloudinary account and store the Upload API response to your database which contains the details for the image: https://cloudinary.com/documentation/image_upload_api_reference#sample_response
If you don't want to store the entire response, you should store at least the fields needed to create a URL for that image later: https://cloudinary.com/documentation/image_transformations#transformation_url_structure
(Which are public_id, resource_type, type, format, and timestamp) though strictly speaking, most of those are optional if your assets are images of type 'upload' - certainly you need the public_id though.
Then, in your frontend code, when adding the image to your page or to your application, you add transformation parameters when building the URL, asking that the image is returned with transformations applied to match it to where/how you're using the image.
A common option is to set the width and height to exactly match the image tag or image view, then apply automatic cropping if the aspect ratio of the original doesn't match, with the crop selection being automatic: https://cloudinary.com/documentation/resizing_and_cropping
A Javascript example to add those parameters, if the image should be 500x500 is:
cloudinary.url( public_id,
{
resource_type: 'image', //these are the defaults and can be ommitted
type: 'upload', //these are the defaults and can be ommitted
height: 500,
width: 500,
crop: 'lfill', // try to fill the requested width and height without scaling up, crop if needed
gravity: 'auto', // select which area to keep automatically
fetch_format: 'auto',
quality: 'auto',
format: 'jpg', // sets the file extension on the URL, and will convert to that format if needed, and no fetch_format was set to override that
});
The resulting URL will be something like: http://res.cloudinary.com/demo/image/upload/c_lfill,f_auto,g_auto,h_500,q_auto,w_500/sample.jpg

ALSA Hooks -- Modifying audio as it's being played

I'm at a loss with this — I have several personal projects in mind that essentially require that I "tap" into the audio stream: read the audio data, do some processing and modify the audio data before it is finally sent to the audio device.
One example of these personal projects is a software-based active crossover. If I have an audio device with 6 channels (i.e., 3 left + 3 right), then I can read the data, apply a LP filter (×2 – left + right), a BP filter, and a HP filter and output the streams through each of the six channels.
Notice that I know how to write a player application that does this — instead, I would want to do this so that any audio from any source (audio players, video players, youtube or any other source of audio being played by the web browser, etc.) is subject to this processing.
I've seen some of the examples (e.g., pcm_min.c from the alsa-project web site, play and record examples in the Linux Journal article by Jeff Tranter from Sep 2004) but I don't seem to have enough information to do something like what I describe above.
Any help or pointers will be appreciated.
You can implement your project as a LADSPA plugin, test it with audacity or any other program supporting LADSPA plugins, and when you like it, insert it into alsa/pulseaudio/jack playback chain.
"LADSPA" is a single header file defining a simple interface to write audio processing plugins. Each plugin has its input/output/control ports and run() function. The run() function is executed for each block of samples to do actual audio processing — apply "control" arguments to "input" buffers and write result to "output" buffers.
Example LADSPA stereo amplifier plugin (single control argument: "Amplification factor", two input ports, two output ports):
///gcc -shared -o /full/path/to/plugindir/amp_example.so amp_example.c
#include <stdlib.h>
#include "ladspa.h"
enum PORTS {
PORT_CAMP,
PORT_INPUT1,
PORT_INPUT2,
PORT_OUTPUT1,
PORT_OUTPUT2
};
typedef struct {
LADSPA_Data *c_amp;
LADSPA_Data *i_audio1;
LADSPA_Data *i_audio2;
LADSPA_Data *o_audio1;
LADSPA_Data *o_audio2;
} MyAmpData;
static LADSPA_Handle myamp_instantiate(const LADSPA_Descriptor *Descriptor, unsigned long SampleRate)
{
MyAmpData *data = (MyAmpData*)malloc(sizeof(MyAmpData));
data->c_amp = NULL;
data->i_audio1 = NULL;
data->i_audio2 = NULL;
data->o_audio1 = NULL;
data->o_audio2 = NULL;
return data;
}
static void myamp_connect_port(LADSPA_Handle Instance, unsigned long Port, LADSPA_Data *DataLocation)
{
MyAmpData *data = (MyAmpData*)Instance;
switch (Port)
{
case PORT_CAMP: data->c_amp = DataLocation; break;
case PORT_INPUT1: data->i_audio1 = DataLocation; break;
case PORT_INPUT2: data->i_audio2 = DataLocation; break;
case PORT_OUTPUT1: data->o_audio1 = DataLocation; break;
case PORT_OUTPUT2: data->o_audio2 = DataLocation; break;
}
}
static void myamp_run(LADSPA_Handle Instance, unsigned long SampleCount)
{
MyAmpData *data = (MyAmpData*)Instance;
double amp = *data->c_amp;
size_t i;
for (i = 0; i < SampleCount; i++)
{
data->o_audio1[i] = data->i_audio1[i]*amp;
data->o_audio2[i] = data->i_audio2[i]*amp;
}
}
static void myamp_cleanup(LADSPA_Handle Instance)
{
MyAmpData *data = (MyAmpData*)Instance;
free(data);
}
static LADSPA_Descriptor myampDescriptor = {
.UniqueID = 123, // for public release see http://ladspa.org/ladspa_sdk/unique_ids.html
.Label = "amp_example",
.Name = "My Amplify Plugin",
.Maker = "alsauser",
.Copyright = "WTFPL",
.PortCount = 5,
.PortDescriptors = (LADSPA_PortDescriptor[]){
LADSPA_PORT_INPUT | LADSPA_PORT_CONTROL,
LADSPA_PORT_INPUT | LADSPA_PORT_AUDIO,
LADSPA_PORT_INPUT | LADSPA_PORT_AUDIO,
LADSPA_PORT_OUTPUT | LADSPA_PORT_AUDIO,
LADSPA_PORT_OUTPUT | LADSPA_PORT_AUDIO
},
.PortNames = (const char * const[]){
"Amplification factor",
"Input left",
"Input right",
"Output left",
"Output right"
},
.PortRangeHints = (LADSPA_PortRangeHint[]){
{ /* PORT_CAMP */
LADSPA_HINT_BOUNDED_BELOW | LADSPA_HINT_BOUNDED_ABOVE | LADSPA_HINT_DEFAULT_1,
0, /* LowerBound*/
10 /* UpperBound */
},
{0, 0, 0}, /* PORT_INPUT1 */
{0, 0, 0}, /* PORT_INPUT2 */
{0, 0, 0}, /* PORT_OUTPUT1 */
{0, 0, 0} /* PORT_OUTPUT2 */
},
.instantiate = myamp_instantiate,
//.activate = myamp_activate,
.connect_port = myamp_connect_port,
.run = myamp_run,
//.deactivate = myamp_deactivate,
.cleanup = myamp_cleanup
};
// NULL-terminated list of plugins in this library
const LADSPA_Descriptor *ladspa_descriptor(unsigned long Index)
{
if (Index == 0)
return &myampDescriptor;
else
return NULL;
}
(if you prefer "short" 40-lines version see https://pastebin.com/unCnjYfD)
Add as many input/output channels as you need, implement your code in myamp_run() function. Build the plugin and set LADSPA_PATH environment variable to the directory where you've built it, so that other apps could find it:
export LADSPA_PATH=/usr/lib/ladspa:/full/path/to/plugindir
Test it in audacity or any other program supporting LADSPA plugins. To test it in terminal you can use applyplugin tool from "ladspa-sdk" package:
applyplugin input.wav output.wav /full/path/to/plugindir/amp_example.so amp_example 2
And if you like the result insert it into your default playback chain. For plain alsa you can use a config like (won't work for pulse/jack):
# ~/.asoundrc
pcm.myamp {
type plug
slave.pcm {
type ladspa
path "/usr/lib/ladspa" # required but ignored as `filename` is set
slave.pcm "sysdefault"
playback_plugins [{
filename "/full/path/to/plugindir/amp_example.so"
label "amp_example"
input.controls [ 2.0 ] # Amplification=2
}]
}
}
# to test it: aplay -Dmyamp input.wav
# to point "default" pcm to it uncomment next line:
#pcm.!default "myamp"
See also:
ladspa.h - answers to most technical questions are there in comments
LADSPA SDK overview
listplugins and analyseplugin tools from "ladspa-sdk" package
alsa plugins : "type ladspa" syntax, and alsa configuration file syntax
ladspa plugins usage examples
If you want to get your hands dirty with some code, you could check out some of these articles by Paul Davis (Paul Davis is a Linux audio guru). You'll have to combine the playback and capture examples to get live audio. Give it a shot, and if you have problems you can post a code-specific problem on SO.
Once you get the live audio working, you can implement an LP filter and go from there.
There are plenty of LADSPA and LV2 audio plugins that implement LP, HP and BP filters but I'm not sure if any are available for your particular channel configuration. It sounds like you want to roll your own anyway.

How to export audio-media from a MOV-file with QuickTime-API?

I want to export the audio-media of a MOV-File with the QuickTime-API and save it to an WAV-File (or something equivalent). How can I do that? I use Windows XP.
Sorry if I am stating the obvious, but you should be able to achieve this the same way as described here:
Export every frame as image from a Movie-File (QuickTime-API)
Or do you need to do this in a non-interactive way?
Edit
To export the audio media of a Movie file to WAVE non-interactively using a Movie Exporter, use the following code:
"include "QuickTimeComponents.h"
...
// aquire Movie
...
ComponentDescription desc;
MovieExportComponent exporter;
char filename[255];
FSSpec fsspec;
int flags;
// first we have to find a Movie Exporter capable of eporting the format we want
desc.componentType = MovieExportType;
desc.componentSubType = kQTFileTypeWave; // 'WAVE'
desc.componentManufacturer = SoundMediaType;
desc.componentFlags = 0;
desc.componentFlagsMask = 0;
// and create an instance of it
exporter = OpenComponent( FindNextComponent( 0, &desc ) );
// then set up a FSSpec for our output file
sprintf( outfilename, "C:/test.wav" );
c2pstr( outfilename );
FSMakeFSSpec( 0, 0L, (ConstStr255Param)outfilename, &fsspec );
// if you do error handling take care to ignore fnfErr being returned
// by FSMakeFSSpec - we're about to create a new file after all
// then finally initiate the conversion
flags= createMovieFileDeleteCurFile | movieToFileOnlyExport;
ConvertMovieToFile( movie, 0, &fsspec, kQTFileTypeWave, 'TVOD', 0, 0, flags, exporter );
CloseComponent( exporter );
...
// Clean up
...
ffmpeg is easily capable of this and can be compiled under the LGPL if needed for commercial software. If you need more customizable hooks you can use libavcodec and libavformat from the same project.
with mplayer
mplayer.exe -ao pcm:file=output.wav -vo null -vc dummy input.mov

Resources