I'm very new to gstreamer and rust and am trying to render a video made from sections of other videos. Based on the docs, gstreamer-rs examples, and this question about doing the same thing in python, I think my code looks pretty good, but throws errors.
This is my code:
use gstreamer as gst;
use gstreamer::{ElementExt, ElementExtManual, GstObjectExt};
use gstreamer_editing_services as ges;
use gstreamer_editing_services::{GESPipelineExt, LayerExt, TimelineExt};
use gstreamer_pbutils as gst_pbutils;
use gstreamer_pbutils::{EncodingProfileBuilder};
pub fn clip_video() {
match gst::init() {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
match ges::init() {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
let timeline = ges::Timeline::new_audio_video();
let layer = timeline.append_layer();
let pipeline = ges::Pipeline::new();
match pipeline.set_timeline(&timeline) {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
let video_profile = gst_pbutils::EncodingVideoProfileBuilder::new()
.name("h.264")
.description("h.264-profile")
.format(&gst::caps::Caps::new_simple("video/x-h264", &[]))
.build()
.unwrap();
let audio_profile = gst_pbutils::EncodingAudioProfileBuilder::new()
.name("mp3")
.description("mp3-profile")
.format(&gst::caps::Caps::new_simple(
"audio/mpeg",
&[("mpegversion", &"1"), ("layer", &"3")],
))
.build()
.unwrap();
let contianer_profile = gst_pbutils::EncodingContainerProfileBuilder::new()
.name("default-mp4-profile")
.description("mp4-with-h.264-mp3")
.format(&gst::caps::Caps::new_simple(
"video/quicktime",
&[("variant", &"iso")],
))
.enabled(true)
.add_profile(&video_profile)
.add_profile(&audio_profile)
.build()
.unwrap();
let asset = ges::UriClipAsset::request_sync("file:///home/ryan/repos/auto-highlighter-processing-service/input/test-video.mp4").expect("Failed to create asset");
match layer.add_asset(
&asset,
0 * gst::SECOND,
10 * gst::SECOND,
10 * gst::SECOND,
ges::TrackType::CUSTOM,
) {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
match pipeline.set_render_settings("file:///home/ryan/repos/auto-highlighter-processing-service/output/test-video.mp4", &contianer_profile){
Err(e) => eprintln!("{:?}", e),
_ => (),
}
match pipeline.set_mode(ges::PipelineFlags::RENDER) {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
match pipeline.set_state(gst::State::Playing) {
Err(e) => eprintln!("{:?}", e),
_ => (),
}
let bus = pipeline.get_bus().unwrap();
for msg in bus.iter_timed(gst::CLOCK_TIME_NONE) {
use gst::MessageView;
match msg.view() {
MessageView::Eos(..) => break,
MessageView::Error(err) => {
println!(
"Error from {:?}: {} ({:?})",
err.get_src().map(|s| s.get_path_string()),
err.get_error(),
err.get_debug()
);
break;
}
_ => (),
}
}
}
The errors that I am getting:
BoolError { message: "Failed to set render settings", filename: "/home/ryan/.cargo/registry/src/github.com-1ecc6299db9ec823/gstreamer-editing-services-0.16.5/src/auto/pipeline.rs", function: "gstreamer_editing_services::auto::pipeline", line: 228 }
StateChangeError
I'm struggling to find what to do about these errors or what the problem could be. From what I know I'm using the set_render_settings() and set_mode() functions correctly.
I didn't try running your code, but one problem I found when reading was the following
.format(&gst::caps::Caps::new_simple(
"audio/mpeg",
&[("mpegversion", &"1"), ("layer", &"3")],
))
The "mpegversion" and "layer" fields of the caps are not strings but integers. If you use them as such it should work (or at least work better)
.format(&gst::caps::Caps::new_simple(
"audio/mpeg",
&[("mpegversion", &1i32), ("layer", &3i32)],
))
Everything else looks correct to me.
You can find more details about such errors by making use of the GStreamer debugging system. You can enable that via the GST_DEBUG environment variable, e.g. by setting that to 6.
Although this answer is over a year late, I thought I'd post anyway, as examples in Rust for GES are sparse, with only a single (though good) example of applying a 'agingtv' effect on the gstreamer-rs repo. Additionally, the OP's sample code above will result in rendering 10 seconds of black video, and does not, as the OP mentioned, result in the desired output.
Using the example listed in the original question above:
On gstreamer-rs 0.19:
Build video profile (note the caps now must be passed via builder() ):
let video_profile = gstreamer_pbutils::EncodingVideoProfile::builder(
&gst::Caps::builder("video/x-h264").build(),
)
.name("video_profile")
.build();
Build the Audio profile:
let audio_profile = gstreamer_pbutils::EncodingAudioProfile::builder(
&gstreamer::Caps::new_simple("audio/x-aac", &[]),
)
.name("audio_profile")
.build();
Build the Container profile:
let container_profile = gstreamer_pbutils::EncodingContainerProfile::builder(
&gstreamer::Caps::new_simple("video/x-matroska", &[]),
)
.name("container_profile")
.add_profile(&audio_profile)
.add_profile(&video_profile)
.build();
Note: as an alternative you can build the whole encoding profile in one go from the DiscovererInfo if you ran the gst-discover on the media first. This will result in an output file very similar to the input file in it's encoding settings.
let encoding_profile =
gstreamer_pbutils::EncodingProfile::from_discoverer(&m_info.discover_info)?;
The following example will clip the video, add a transition, and merge in an additional clip to fade to:
let timeline = ges::Timeline::new_audio_video();
timeline.set_auto_transition(true);
let layer = timeline.append_layer();
let pipeline = ges::Pipeline::new();
pipeline.set_timeline(&timeline)?;
let audio_profile = gstreamer_pbutils::EncodingAudioProfile::builder(
&gstreamer::Caps::new_simple("audio/x-aac", &[]),
)
.name("audio_profile")
.build();
let video_profile = gstreamer_pbutils::EncodingVideoProfile::builder(
&gst::Caps::builder("video/x-h264").build(),
)
.name("video_profile")
.build();
let container_profile = gstreamer_pbutils::EncodingContainerProfile::builder(
&gstreamer::Caps::new_simple("video/x-matroska", &[]),
)
.name("container_profile")
.add_profile(&audio_profile)
.add_profile(&video_profile)
.build();
/* alternatively
let encoding_profile = gstreamer_pbutils::EncodingProfile::from_discoverer(&m_info.discover_info)?;
*/
/* original video */
let clip = ges::UriClip::new("file:///home/ryan/repos/auto-highlighter-processing-service/input/test-video.mp4")?;
layer.add_clip(&clip)?;
clip.set_inpoint(gst::ClockTime::from_seconds(0));
clip.set_duration(gst::ClockTime::from_seconds(10));
/* video to transition to with a fade */
let clip_transition_to = ges::UriClip::new("/some/2/second/video/file.mp4")?;
clip_transition_to.set_start(gst::ClockTime::from_seconds(9)); //this should overlap the original video clip, but not completely
clip_transition_to.set_inpoint(gst::ClockTime::from_seconds(0));
clip_transition_to.set_duration(gst::ClockTime::from_seconds(2));
layer.add_clip(&clip_transition_to)?;
pipeline.set_render_settings("file:///home/ryan/repos/auto-highlighter-processing-service/output/test-video.mp4", &container_profile)?; //or &encoding_profile
pipeline.set_mode(ges::PipelineFlags::RENDER)?;
pipeline.set_state(gst::State::Playing)?;
let bus = pipeline
.bus()
.expect("Pipeline without bus. Shouldn't happen!");
for msg in bus.iter_timed(gst::ClockTime::NONE) {
use gst::MessageView;
match msg.view() {
MessageView::Eos(..) => break,
MessageView::Error(err) => {
pipeline.set_state(gst::State::Null)?;
match err.details() {
Some(details) if details.name() == "error-details" => details
.get::<&ErrorValue>("error")
.unwrap()
.clone()
.0
.lock()
.unwrap()
.take()
.map(Result::Err)
.expect("error-details message without actual error"),
_ => Err({
let err_src = msg
.src()
.map(|s| String::from(s.path_string()))
.unwrap_or_else(|| String::from("None"));
log!(
Level::Error,
"A GStreamer Error was Encountered {}",
&err_src
);
ErrorMessage {
src: err_src,
error: err.error().to_string(),
debug: err.debug(),
source: err.error(),
}
.into()
}),
}?;
}
MessageView::StateChanged(state_changed) => {
// if state_changed.src().map(|s| s == decodebin).unwrap_or(false)
// && state_changed.current() == gst::State::Playing
// {
// // Generate a dot graph of the pipeline to GST_DEBUG_DUMP_DOT_DIR if defined
// let bin_ref = decodebin.downcast_ref::<gst::Bin>().unwrap();
// bin_ref.debug_to_dot_file(gst::DebugGraphDetails::all(), "PLAYING");
// }
let msg = format!(
"State changed from {:?}: {:?} -> {:?} ({:?})",
state_changed.src().map(|s| s.path_string()),
state_changed.old(),
state_changed.current(),
state_changed.pending()
);
log!(Level::Debug, "{}", msg)
}
_ => (),
}
}
let log_msg = "Play run complete, changing state...";
log!(Level::Info, "{}", &log_msg);
pipeline.set_state(gst::State::Null)?;
The result will be a 10 second video, with a fade out to a 2 second video (e.g. something that says "the end", etc).
I hope this helps someone. It took a some reading and research to achieve the desired effect, and hopefully this will help someone else on the same journey.
gstreamer is excellent software and, although it's been a bit of work to get things functional, it's been great to work with the rust bindings.
Related
I'm attempting to do something that I feel is pretty basic: I have a pulldown, and I'd like the onchange event for that pulldown to cause the program to fetch some data from the backend based on the user's input. (And then, you know, give the user more options based on the first thing they picked. Really simple, and seems like I ought to have been able to find an easy way to do this.)
Full code for this minimal (failing) example is at: https://github.com/djmcmath/broken-yew
But the relevant bit, which doesn't behave correctly, is below:
The view function renders, delightfully, an iterated list. I pass in a callback, so it knows what to do on the "onchange" event.
The callback gets executed, which makes me very happy. But it isn't calling the Msg::GetData. This compiles, which is nice, but it doesn't work, which is less nice.
I've spent, I'm ashamed to admit, several weeks of my spare time fighting with this. I think it has something to do with scopes and lifetimes. I think that the way I'm making this compile -- by cloning the context and using "move" disconnects it from the actual context that I need to make this work. But every variation on the theme that I've been able to find in examples and references complains about scope or lifetimes.
Thanks in advance for the help.
fn update(&mut self, ctx: &Context<Self>, msg: Self::Message) -> bool {
match msg {
Msg::GetData(value) => {
log::info!("Start 'fetch' with user-selected value: {}", value);
ctx.link().send_future(async {
match fetch_markdown("url_shortened_for_clarity").await {
Ok(md) => Msg::SetMarkdownFetchState(FetchState::Success(md)),
Err(err) => Msg::SetMarkdownFetchState(FetchState::Failed(err)),
}
});
false
},
Msg::SetMarkdownFetchState(fetch_state) => {
let mut wr = WebReturn { term_id: 0, dow: 0, dep_time_num: 0 };
match fetch_state {
FetchState::Success(s) => { wr = serde_json::from_str(&s).expect(&format!("Poorly formatted JSON! {}", s).to_string()); },
FetchState::Failed(f) => { log::info!("Fetch failed: {}", f); },
FetchState::NotFetching => {},
FetchState::Fetching => {}
};
log::info!("term_id (3) : {}, dep_time_num (12000) : {}, and dow (3) : {}", wr.term_id, wr.dep_time_num, wr.dow);
true
}
}
}
fn view(&self, ctx:&Context<Self>) -> Html {
let ctx_link = ctx.link().clone();
let my_callback: Callback<String> = Callback::from(move |value: String| {
let val_as_num = value.parse::<i32>().unwrap_or(0);
log::info!("Returned value: {}", val_as_num);
ctx_link.callback(|val_as_num: i32| Msg::GetData(val_as_num));
});
html! {
<div>
{ self.render_list(&self.props.term_list, my_callback) }
</div>
}
}
This line does not "call back" to your component, it creates a callback and then doesn't call it:
ctx_link.callback(|val_as_num: i32| Msg::GetData(val_as_num));
You need to instead call .send_message() in your callback or, better yet, create your original callback with .callback():
let my_callback = ctx_link.callback(|value: String| {
let val_as_num = value.parse::<i32>().unwrap_or(0);
log::info!("Returned value: {}", val_as_num);
Msg::GetData(val_as_num)
});
I wanted to be able to retrieve the content from an attribute like this:
#[foreign_key(table = "some_table", column = "some_column")]
This is how I am trying:
impl TryFrom<&&Attribute> for EntityFieldAnnotation {
type Error = syn::Error;
fn try_from(attribute: &&Attribute) -> Result<Self, Self::Error> {
if attribute.path.is_ident("foreign_key") {
match attribute.parse_args()? {
syn::Meta::NameValue(nv) =>
println!("NAME VALUE: {:?}, {:?}, {:?}",
nv.path.get_ident(),
nv.eq_token.to_token_stream(),
nv.lit.to_token_stream(),
),
_ => println!("Not interesting")
}
} else {
println!("No foreign key")
}
// ... More Rust code
}
Everything works fine if I just put in there only one NameValue. When I add the comma,
everything brokes.
The only error:
error: unexpected token
How can I fix my logic to enable the possibility of have more than just one NameValue?
Thanks
UPDATE: While writing this answer, I had forgotten that Meta has List variant as well which gives you NestedMeta. I would generally prefer doing that instead of what I did in the answer below for more flexibility.
Although, for your particular case, using Punctuated still seems simpler and cleaner to me.
MetaNameValue represents only a single name-value pair. In your case it is delimited by ,, so, you need to parse all of those delimited values as MetaNameValue instead.
Instead of calling parse_args, you can use parse_args_with along with Punctuated::parse_terminated:
use syn::{punctuated::Punctuated, MetaNameValue, Token};
let name_values: Punctuated<MetaNameValue, Token![,]> = attribute.parse_args_with(Punctuated::parse_terminated).unwrap(); // handle error instead of unwrap
Above name_values has type Punctuated which is an iterator. You can iterate over it to get various MetaNameValue in your attribute.
Updates based on comments:
Getting value out as String from MetaNameValue:
let name_values: Result<Punctuated<MetaNameValue, Token![,]>, _> = attr.parse_args_with(Punctuated::parse_terminated);
match name_values {
Ok(name_value) => {
for nv in name_value {
println!("Meta NV: {:?}", nv.path.get_ident());
let value = match nv.lit {
syn::Lit::Str(v) => v.value(),
_ => panic!("expeced a string value"), // handle this err and don't panic
};
println!( "Meta VALUE: {:?}", value )
}
},
Err(_) => todo!(),
};
I've attempted to create a pipeline for receiving RTP video/audio streams via Gstreamer using the gstreamer-rs crate, but I am not having much luck. Here is a quick distillation of my approach:
let udp_src = ElementFactory::make("udpsrc", Some("udp_src"))?;
udp_src.set_property("port", &5004);
udp_src.set_property("caps", &"application/x-rtp".to_string());
let rtpbin = ElementFactory::make("rtpbin", Some("rtp_bin"))?;
let pipeline = Pipeline::new(Some("RTP Pipeline"));
pipeline.add_many(&[&udp_src, &rtpbin]);
udp_src.link(&rtpbin)?;
rtpbin.connect_pad_added(move |src, src_pad| {
println!("Received new pad {} from {}", src_pad.get_name(), src.get_name());
let new_pad_caps = src_pad.get_current_caps().expect("Failed to get caps of new pad");
let new_pad_struct = new_pad_caps.get_structure(0).expect("Failed to get first structure of caps");
let new_pad_type = new_pad_struct.get_name();
println!("{}", new_pad_type);
});
But I'm not getting anything from the connect_pad_added when I run the code and send UDP signals to the specified port. I'm not even sure if this is the right approach, and would appreciate any directions on where to find examples or pointers on how to use rtpbin with udpsrc.
You're not specifying to which pad of the rtpbin you're linking your udpsrc, so it probably selects the wrong one here (maybe the one for a sender-rtpbin).
Try with udp_src.link_pads(Some("src"), &rtpbin, Some("recv_rtp_sink_%u")) instead. Then you should get a pad-added with a "recv_rtp_src_%u_%u_%u" name. The first number will be 0, the other two will be payload type and the ssrc.
You can link a udp source to an rtpbin via the following:
fn get_static_pad(element: &Element, pad_name: &'static str) -> Result<Pad, Error> {
let pad = element.get_static_pad(pad_name);
pad.ok_or(Error::msg(format!("Failed to get pad: {}", pad_name)))
}
fn get_request_pad(element: &Element, pad_name: &'static str) -> Result<Pad, Error> {
match element.get_request_pad(pad_name) {
Some(pad) => Ok(pad),
None => {
let element_name = element.get_name();
Err(Error::msg(pad_name))
}
}
}
let rtp_udp_src = ElementFactory::make("udpsrc", Some("rtp_udp_src"))?;
let caps = Caps::new_simple("application/x-rtp", &[("clock-rate", &90000i32)]);
rtp_udp_src.set_property("port", &5006);
rtp_udp_src.set_property("caps", &caps);
let rtpbin = ElementFactory::make("rtpbin", Some("rtp_bin"))?;
rtpbin.connect("request-pt-map", false, |values| {
let pt = values[2].get::<u32>().expect("Invalid argument");
match pt {
Some(100) => Some(
Caps::new_simple(
"application/x-rtp",
&[
("media", &"video"),
("clock-rate", &90000i32),
],
)
.to_value(),
),
Some(96) => Some(
Caps::new_simple(
"application/x-rtp",
&[
("media", &"video"),
("clock-rate", &90000i32),
("encoding-name", &"VP8"),
],
)
.to_value(),
),
_ => None,
}
})?;
let rtp_udp_src_pad = get_static_pad(&rtp_udp_src, "src")?;
let rtp_recv_sink_pad = get_request_pad(&rtpbin, "recv_rtp_sink_0")?;
rtp_udp_src_pad.link(&rtp_recv_sink_pad).expect("Failed to link rtp_udp_src_pad and rtpbin");
This will correctly fire off the connect_pad_added sequence and let you use it with other elements.
I'm using Yew to program a theme switcher that by clicking cycles through different themes.
This is my update function. It gets the current theme which is stored in shared state, depending on what would come next inside theme_cycle the theme value inside the shared state gets set to it.
fn update(&mut self, msg: Self::Message) -> ShouldRender {
match msg {
Msg::ChangeTheme => {
let theme_cycle: [&str; 3] = ["light", "dark", "rust"];
let current_theme = self.props.handle.state().theme.clone();
// eval next theme
let next_theme = match theme_cycle.iter().position(|x| x == ¤t_theme) {
None => theme_cycle[0].to_string(),
Some(i) => {
if i >= (current_theme.len() - 1) {
theme_cycle[0].to_string()
} else {
theme_cycle[i + 1].to_string()
}
},
};
// set next theme
self.props.handle.reduce(move |state| state.theme = next_theme.clone());
// store it inside localstorage
},
Msg::ToggleLangDropdown => self.show_dropdown = !self.show_dropdown,
};
true
}
But if the theme val inside shared state is "rust" and I click the button again that calls Msg::ChangeTheme, the theme should be set to "light" but instead my code panics and I get an "Uncaught Error: undefined" inside the Browser Console.
I've found a workaround; instead of using an array and accessing the values, I've tried to do the same task but just with iterators and made sure that the update function takes no ownership of any variable outside the function itself (I don't really know if that is really necessary though...)
fn update(&mut self, msg: Self::Message) -> ShouldRender {
match msg {
Msg::ChangeTheme => {
let theme_cycle = ["light".to_string(), "dark".to_string(), "rust".to_string()];
let current_theme = self.props.handle.state().theme.clone();
let indexof_current_theme = match theme_cycle.iter().position(|x| x.to_string() == current_theme) {
None => 0,
Some(x) => x.clone(),
};
let next_theme = match theme_cycle.iter().nth(indexof_current_theme + 1) {
None => theme_cycle.iter().nth(0).unwrap().clone(),
Some(x) => x.clone(),
};
self.props.handle.reduce(move |state| state.theme = next_theme.to_string());
},
Msg::ToggleLangDropdown => self.show_lang_dropdown = !self.show_lang_dropdown,
Msg::ToggleThemeDropdown => self.show_theme_dropdown = !self.show_theme_dropdown,
};
true
}
Still would be cool if anyone knows what I did wrong in my first attempt.
I'm learning rust and trying to make a find like utility (yes another one), im using clap and trying to support command line and config file for the program's parameters(this has nothing to do with the clap yml file).
Im trying to parse the commands and if no commands were passed to the app, i will try to load them from a config file.
Now I don't know how to do this in an idiomatic way.
fn main() {
let matches = App::new("findx")
.version(crate_version!())
.author(crate_authors!())
.about("find + directory operations utility")
.arg(
Arg::with_name("paths")
...
)
.arg(
Arg::with_name("patterns")
...
)
.arg(
Arg::with_name("operation")
...
)
.get_matches();
let paths;
let patterns;
let operation;
if matches.is_present("patterns") && matches.is_present("operation") {
patterns = matches.values_of("patterns").unwrap().collect();
paths = matches.values_of("paths").unwrap_or(clap::Values<&str>{"./"}).collect(); // this doesn't work
operation = match matches.value_of("operation").unwrap() { // I dont like this
"Append" => Operation::Append,
"Prepend" => Operation::Prepend,
"Rename" => Operation::Rename,
_ => {
print!("Operation unsupported");
process::exit(1);
}
};
}else if Path::new("findx.yml").is_file(){
//TODO: try load from config file
}else{
eprintln!("Command line parameters or findx.yml file must be provided");
process::exit(1);
}
if let Err(e) = findx::run(Config {
paths: paths,
patterns: patterns,
operation: operation,
}) {
eprintln!("Application error: {}", e);
process::exit(1);
}
}
There is an idiomatic way to extract Option and Result types values to the same scope, i mean all examples that i have read, uses match or if let Some(x) to consume the x value inside the scope of the pattern matching, but I need to assign the value to a variable.
Can someone help me with this, or point me to the right direction?
Best Regards
Personally I see nothing wrong with using the match statements and folding it or placing it in another function. But if you want to remove it there are many options.
There is the ability to use the .default_value_if() method which is impl for clap::Arg and have a different default value depending on which match arm is matched.
From the clap documentation
//sets value of arg "other" to "default" if value of "--opt" is "special"
let m = App::new("prog")
.arg(Arg::with_name("opt")
.takes_value(true)
.long("opt"))
.arg(Arg::with_name("other")
.long("other")
.default_value_if("opt", Some("special"), "default"))
.get_matches_from(vec![
"prog", "--opt", "special"
]);
assert_eq!(m.value_of("other"), Some("default"));
In addition you can add a validator to your operation OR convert your valid operation values into flags.
Here's an example converting your match arm values into individual flags (smaller example for clarity).
extern crate clap;
use clap::{Arg,App};
fn command_line_interface<'a>() -> clap::ArgMatches<'a> {
//Sets the command line interface of the program.
App::new("something")
.version("0.1")
.arg(Arg::with_name("rename")
.help("renames something")
.short("r")
.long("rename"))
.arg(Arg::with_name("prepend")
.help("prepends something")
.short("p")
.long("prepend"))
.arg(Arg::with_name("append")
.help("appends something")
.short("a")
.long("append"))
.get_matches()
}
#[derive(Debug)]
enum Operation {
Rename,
Append,
Prepend,
}
fn main() {
let matches = command_line_interface();
let operation = if matches.is_present("rename") {
Operation::Rename
} else if matches.is_present("prepend"){
Operation::Prepend
} else {
//DEFAULT
Operation::Append
};
println!("Value of operation is {:?}",operation);
}
I hope this helps!
EDIT:
You can also use Subcommands with your specific operations. It all depends on what you want to interface to be like.
let app_m = App::new("git")
.subcommand(SubCommand::with_name("clone"))
.subcommand(SubCommand::with_name("push"))
.subcommand(SubCommand::with_name("commit"))
.get_matches();
match app_m.subcommand() {
("clone", Some(sub_m)) => {}, // clone was used
("push", Some(sub_m)) => {}, // push was used
("commit", Some(sub_m)) => {}, // commit was used
_ => {}, // Either no subcommand or one not tested for...
}