I'm trying to create a custom videosink for playbin in gstreamer 1.6.3
The final idea is to have some videomixer inside the videosink to be able to do.. stuff.
At the moment i would like to simply create a custom Bin that incapsulates a videosink.
The relevant parts of the code at the moment are:
def get_videomix_bin(self):
mix_bin = Gst.Bin.new('sink')
sink = Gst.ElementFactory.make('glimagesink')
gp = Gst.GhostPad.new('vs', sink.get_static_pad('sink'))
mix_bin.add(sink)
mix_bin.add_pad(gp)
return mix_bin
def get_pipeline(self, videosink):
"""A basic playbin pipeline pipeline"""
self.pipeline = Gst.ElementFactory.make('playbin')
videosink = self.get_videomix_bin()
self.pipeline.set_property('video-sink', videosink)
self.fireEvent('pipeline-created')
This code is part of a bigger software that I cannot post whole. But if i comment out the self.pipeline.set_property('video-sink', videosink) part, it works, so i tend to think that the problem is somwhere there.
It... well it basically don't work. The pipeline won't start.
At GST_DEBUG=2 i get this warning
0:00:00.758103367 15560 0x7f81000050a0 WARN uridecodebin gsturidecodebin.c:939:unknown_type_cb:<uridecodebin0> warning: No decoder available for type 'video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe1001c674d401fe8802802dd80b501010140000003004000000c83c60c448001000468ebaf20, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1, parsed=(boolean)true'.
You have to call the ghostpad on the videosink bin "sink", not "vs". The pad names are part of the API, and sink elements are expected to have a pad called "sink".
Related
I am working on an Gstreamer application that renders decoded frames. The input is from another application (gets frames from a network camera) that gives .H264 encoded frames.
The gstreamer pipeline I use is as follows:
appsrc ! h264parse ! avdec_h264 ! videoconvert ! ximagesink
The appsrc creates the GstBuffer and timestamps it, starting from 0.
The rendered output seems approx. 2seconds delayed.
How do I reduce the latency in this case?
Any help is appreciated.
The appsrc's properties are set (using gst_object_set() )as below:
stream-type = 0
format = GST_FORMAT_TIME
is-live = true
max-latency = 0
min-latency = 0
Update:
I tried sending a latency event of -2 seconds (experimental) to the pipeline
GstClockTime latency = (-2 * gst_util_uint64_scale_int (1, GST_SECOND, 1));
GstEvent *event = gst_event_new_latency (latency);
gst_element_send_event (pipeline, event);
This did not help, it made the output really choppy.
As of now, this is my best answer.
Use do-timestamp property of appsrc Gstreamer element.
This eliminated the latency to almost <200ms.
I'm a student teaching myself Drake, specifically pydrake with Dr. Russ Tedrake's excellent Underactuated Robotics course. I am trying to write a combined energy shaping and lqr controller for keeping a cartpole system balanced upright. I based the diagram on the cartpole example found in Chapter 3 of Underactuated Robotics [http://underactuated.mit.edu/acrobot.html], and the SwingUpAndBalanceController on Chapter 2: [http://underactuated.mit.edu/pend.html].
I have found that due to my use of the cart_pole.sdf model I have to create an abstract input port due receive FramePoseVector from the cart_pole.get_output_port(0). From there I know that I have to create a control signal output of type BasicVector to feed into a Saturation block before feeding into the cartpole's actuation port.
The problem I'm encountering right now is that I'm not sure how to get the system's current state data in the DeclareVectorOutputPort's callback function. I was under the assumption I would use the LeafContext parameter in the callback function, OutputControlSignal, obtaining the BasicVector continuous state vector. However, this resulting vector, x_bar is always NaN. Out of desperation (and testing to make sure the rest of my program worked) I set x_bar to the controller's initialization cart_pole_context and have found that the simulation runs with a control signal of 0.0 (as expected). I can also set output to 100 and the cartpole simulation just flies off into endless space (as expected).
TL;DR: What is the proper way to obtain the continuous state vector in a custom controller extending LeafSystem with a DeclareVectorOutputPort?
Thank you for any help! I really appreciate it :) I've been teaching myself so it's been a little arduous haha.
# Combined Energy Shaping (SwingUp) and LQR (Balance) Controller
# with a simple state machine
class SwingUpAndBalanceController(LeafSystem):
def __init__(self, cart_pole, cart_pole_context, input_i, ouput_i, Q, R, x_star):
LeafSystem.__init__(self)
self.DeclareAbstractInputPort("state_input", AbstractValue.Make(FramePoseVector()))
self.DeclareVectorOutputPort("control_signal", BasicVector(1),
self.OutputControlSignal)
(self.K, self.S) = BalancingLQRCtrlr(cart_pole, cart_pole_context,
input_i, ouput_i, Q, R, x_star).get_LQR_matrices()
(self.A, self.B, self.C, self.D) = BalancingLQRCtrlr(cart_pole, cart_pole_context,
input_i, ouput_i,
Q, R, x_star).get_lin_matrices()
self.energy_shaping = EnergyShapingCtrlr(cart_pole, x_star)
self.energy_shaping_context = self.energy_shaping.CreateDefaultContext()
self.cart_pole_context = cart_pole_context
def OutputControlSignal(self, context, output):
#xbar = copy(self.cart_pole_context.get_continuous_state_vector())
xbar = copy(context.get_continuous_state_vector())
xbar_ = np.array([xbar[0], xbar[1], xbar[2], xbar[3]])
xbar_[1] = wrap_to(xbar_[1], 0, 2.0*np.pi) - np.pi
# If x'Sx <= 2, then use LQR ctrlr. Cost-to-go J_star = x^T * S * x
threshold = np.array([2.0])
if (xbar_.dot(self.S.dot(xbar_)) < 2.0):
#output[:] = -self.K.dot(xbar_) # u = -Kx
output.set_value(-self.K.dot(xbar_))
else:
self.energy_shaping.get_input_port(0).FixValue(self.energy_shaping_context,
self.cart_pole_context.get_continuous_state_vector())
output_val = self.energy_shaping.get_output_port(0).Eval(self.energy_shaping_context)
output.set_value(output_val)
print(output)
Here are two things that might help:
If you want to get the state of the cart-pole from MultibodyPlant, you probably want to be connecting to the continuous_state output port, which gives you a normal vector instead of the abstract-type FramePoseVector. In that case, your call to get_input_port().Eval(context) should work just fine.
If you do really want to read the FramePoseVector, then you have to evaluate the input port slightly differently. You can find an example of that here.
I am creating a MAW well and want to use it as an observation well to compare it later to field data, it should be screened over multiple layers. However, I am only getting the head value in the well of the very last timestep in my output file. Any ideas on how to get all timesteps in the output?
The FloPy manual says something about it needing to be in Output Control, but I can't figure out how to do that:
print_head (boolean) – print_head (boolean) keyword to indicate that the list of multi-aquifer well heads will be printed to the listing file for every stress period in which “HEAD PRINT” is specified in Output Control. If there is no Output Control option and PRINT_HEAD is specified, then heads are printed for the last time step of each stress period.
In the MODFLOW6 manual I see that it is possible to make a continuous output:
modflow6
My MAW definition looks like this:
maw = flopy.mf6.ModflowGwfmaw(gwf,
nmawwells=1,
packagedata=[0, Rwell, minbot, wellhead,'MEAN',OBS1welllayers],
connectiondata=OBS1connectiondata,
perioddata=[(0,'STATUS','ACTIVE')],
flowing_wells=False,
save_flows=True,
mover=True,
flow_correction=True,
budget_filerecord='OBS1wellbudget',
print_flows=True,
print_head=True,
head_filerecord='OBS1wellhead',
)
My output control looks like this:
oc = flopy.mf6.ModflowGwfoc(gwf,
budget_filerecord=budget_file,
head_filerecord=head_file,
saverecord=[('HEAD', 'ALL'), ('BUDGET', 'ALL'), ],
)
Hope this is all clear and someone can help me, thanks!
You need to initialise the MAW observations file... it's not done in the OC package.
You can find the scripts for the three MAW examples in the MF6 documentation here:
https://github.com/MODFLOW-USGS/modflow6-examples/tree/master/notebooks
It looks something like this:
obs_file = "{}.maw.obs".format(name)
csv_file = obs_file + ".csv"
obs_dict = {csv_file: [
("head", "head", (0,)),
("Q1", "maw", (0,), (0,)),
("Q2", "maw", (0,), (1,)),
("Q3", "maw", (0,), (2,)),
]}
maw.obs.initialize(filename=obs_file, digits=10, print_input=True, continuous=obs_dict)
I wish to generate a lot of --enable-*/--disable-* options by something like:
COMPONENTS([a b c], [yes])
where the second argument is the default value of the automatic enable_* variable. My first attempt was to write an AC_ARG_ENABLE(...) within an m4_foreach_w, but so far, I'm only getting the first component to appear in the ./configure --help output.
If I add hand-written AC_ARG_ENABLEs, they work as usual.
Regardless, the --enable-*/--disable-* options work as they should, just the help text is missing.
Here's the full code to reproduce the problem:
AC_INIT([foo], 1.0)
AM_INIT_AUTOMAKE([foreign])
AC_DEFUN([COMPONENTS],
[
m4_foreach_w([component], [$1], [
AS_ECHO(["Processing [component] component with default enable=$2"])
AC_ARG_ENABLE([component],
[AS_HELP_STRING([--enable-[]component], [component] component)],
,
[enable_[]AS_TR_SH([component])=$2]
)
])
AC_ARG_ENABLE([x],
[AS_HELP_STRING([--enable-[]x], [component x])],
,
[enable_[]AS_TR_SH([x])=$2]
)
AC_ARG_ENABLE([y],
[AS_HELP_STRING([--enable-[]y], [component y])],
,
[enable_[]AS_TR_SH([y])=$2]
)
])
COMPONENTS([a b c], [yes])
for var in a b c x y; do
echo -n "\$enable_$var="
eval echo "\$enable_$var"
done
AC_CONFIG_FILES(Makefile)
AC_OUTPUT
And an empty Makefile.am. To verify that the options work:
$ ./configure --disable-a --disable-b --disable-d --disable-x
configure: WARNING: unrecognized options: --disable-d
...
Processing component a with default enable=yes
Processing component b with default enable=yes
Processing component c with default enable=yes
$enable_a=no
$enable_b=no
$enable_c=yes
$enable_x=no
$enable_y=yes
After I poked around in autoconf sources, I figured out this has to do with the m4_divert_once call in the implementation of AC_ARG_ENABLE:
# AC_ARG_ENABLE(FEATURE, HELP-STRING, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
# ------------------------------------------------------------------------
AC_DEFUN([AC_ARG_ENABLE],
[AC_PROVIDE_IFELSE([AC_PRESERVE_HELP_ORDER],
[],
[m4_divert_once([HELP_ENABLE], [[
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]]])])dnl
m4_divert_once([HELP_ENABLE], [$2])dnl
_AC_ENABLE_IF([enable], [$1], [$3], [$4])dnl
])# AC_ARG_ENABLE
# m4_divert_once(DIVERSION-NAME, CONTENT)
# ---------------------------------------
# Output CONTENT into DIVERSION-NAME once, if not already there.
# An end of line is appended for free to CONTENT.
m4_define([m4_divert_once],
[m4_expand_once([m4_divert_text([$1], [$2])])])
I'm guessing that the HELP-STRING argument is remembered in it's unexpanded form, so it is added just once for all components. Manually expanding the AC_HELP_STRING does what I want:
AC_DEFUN([COMPONENTS],
[
m4_foreach_w([comp], [$1], [
AS_ECHO(["Processing component 'comp' with default enable=$2"])
AC_ARG_ENABLE([comp],
m4_expand([AS_HELP_STRING([--enable-comp], enable component comp)]),
,
[enable_[]AS_TR_SH([comp])=$2]
)
])
])
COMPONENTS([a b c x y], [yes])
I couldn't find a way to properly quote components so that it appears as a string, after being used as the loop variable in m4_foreach_w, so I just renamed it to spare me the trouble.
I'm developing a GStreamer plugin following the GStreamer Plugin Writer's Guide and using gst-element-maker from the gst-plugins-bad repository with the base class set to basetransform. As a starting point I have developed a plugin named MyFilter that simply passes the data along the chain. The plugin is working, but when I run gst-launch with the debug level set to 2, I get the following error:
alsa gstalsa.c:124:gst_alsa_detect_formats: skipping non-int format.
I am executing the command:
gst-launch --gst-debug-level=2 --gst-plugin-load=./src/libgstmyfilter.la filesrc location=./song.mp3 ! flump3dec ! audioconvert ! audioresample ! myfilter ! alsasink
From the base class that was created by gst-element-maker I have removed calls to gst_pad_new_from_static_template() because the calls were returning errors reporting that the sink and source pad were already created, I have set the chain function using gst_pad_set_chain_function(), have implemented the function gst_myfilter_transform_caps(), and have added code to handle the GST_EVENT_NEWSEGMENT event. The STATIC_CAPS string I am using for source and sink are:
"audio/x-raw-int, "
"rate = (int) { 16000, 32000, 44100, 48000 }, "
"channels = (int) [ 1, 2 ], "
"endianness = (int) BYTE_ORDER, "
"signed = (boolean) true, "
"width = (int) 16, "
"depth = (int) 16"
I return the caps from gst_myfilter_transform_caps() using gst_pad_get_fixed_caps_func(GST_BASE_TRANSFORM_SRC[[/SINK]]_PAD(trans)). The pad caps are set using the default code created by gst-element-maker in gst_myfilter_base_init() using:
gst_element_class_add_pad_template(element_class, gst_static_pad_template_get(&gst_myfilter_sink_template));
Is there a problem with the GstBaseTransform class? I have another custom filter which does not use the GstBaseTransform class and does not have this problem. I am using GStreamer v0.10.36 with Ubuntu 12.04.