Requesting a new tee's src pad temporary pauses the pipeline

[Sebastian Greco]

Hello!
I have been working on this c program for the past weeks and even tho I
have learned a lot, I have been hitting a wall lately, and I really cant
solve this problem with tee.

So my idea is to have a running pipeline, and on demand add sinks to the
app. For what I have been reading around this should be done something like
this.

mysrc -> tee -> queue -> fakesink


So when I want to add a new sink, I simply have to request a src_% pad to
the tee, connect it to my new bin, in case should be something like this:

tee -> [queue -> flvmux -> rtmpsink ] ([] is a bin I have created)


I have simplified the pipeline to make it easier to explain, but I have
been working with different possibilities (having the flvmux placed before
the tee for example, and using only one tee, and now I changed to having
one tee for audio and another for video and then adding it to the flvmux
inside my bin)

The problem is all seems to be working, but when looking at the endpoint
(in this case youtube live) I can't see any image even if it says it's
receiving data... just weird.

So looking at my debug output, I can see that my pipeline is passing to
PAUSED state right after requesting the src pads to the tee. And then after
all is elements are in play state, the pipeline changes to PLAYING again.
So I'm wondering if that's normal and if it should be like that.

Some code:
Example of how I add the fake sink to a tee:

GstElement *audioFakesink = gst_element_factory_make("fakesink",
"main-audio-fakesink");
GstElement *Aqueue = gst_element_factory_make("queue", "fakeAQueue");
mainAudioTee = gst_element_factory_make("tee", "mainAudioTee");
gst_bin_add_many (GST_BIN (pipeline), webrtcbin, mainAudioTee,
mainVideoTee, Aqueue, Vqueue,audioFakesink, videoFakesink, NULL);

gst_element_link ( Aqueue, audioFakesink);

GstPad *queueAudioPad = gst_element_get_static_pad (Aqueue, "sink");
g_assert_nonnull (queueAudioPad);

GstPadTemplate *templ = gst_element_class_get_pad_template
(GST_ELEMENT_GET_CLASS (mainAudioTee), "src_%u");
GstPad *teeAudioSrcPad = gst_element_request_pad (mainAudioTee, templ,
NULL, NULL);
g_assert_nonnull (teeAudioSrcPad);

ret = gst_pad_link (teeAudioSrcPad, queueAudioPad);
g_assert_cmpint (ret, ==, GST_PAD_LINK_OK);

Example of how I add a new source

gst_bin_add (GST_BIN (pipeline), myBinRTMP);

std::cout << "Info: RtmpBin added " << std::endl;
// AUDIO
GstPad *sinkAudioTargetPad = gst_element_get_static_pad (myBinRTMP, "audio");
g_assert_nonnull (sinkAudioTargetPad);

std::cout << "Info: RtmpBin got AUDIO sinkPad " << std::endl;


GstPadTemplate *templAudio = gst_element_class_get_pad_template
(GST_ELEMENT_GET_CLASS (mainAudioTee), "src_%u");
GstPad *teeAudioSrcPad = gst_element_request_pad (mainAudioTee,
templAudio, NULL, NULL);
g_assert_nonnull (teeAudioSrcPad);

std::cout << "Info: RtmpBin got AUDIO Tee new sourcePad " << std::endl;
/* Start displaying video */
gst_element_sync_state_with_parent (myBinRTMP);
std::cout << "Info: RtmpBin sync " << std::endl;

gst_pad_link (teeAudioSrcPad, sinkAudioTargetPad);

std::cout << "Info: RtmpBin pad linked " << std::endl;


What I'áž¿ seeing in my logs

Debug: Checking to start pending
Debug: AUDIO AND VIDEO READY lets roll
Debug: putting pipeline to work>>>>>>>> STATE CHANGED:
main-video-fakesink NEW: PAUSED PENDING: VOID_PENDING>>>>>>>> STATE
CHANGED: main-video-fakesink NEW: PLAYING PENDING:
VOID_PENDING>>>>>>>> STATE CHANGED: main-audio-fakesink NEW: PLAYING
PENDING: VOID_PENDING*>>>>>>>> STATE CHANGED: my-pipeline NEW: PLAYING
PENDING: VOID_PENDING
*DEBUG: onMessage: called with message_id: streamRTMP
Info: Adding RTMP: FUKyqA3xY:
rtmp://a.rtmp.youtube.com/live2//xxxx-cccc-ssss-wwww
Info: Add when ready - sending to: FUKyqA3xY:
rtmp://a.rtmp.youtube.com/live2//xxxx-cccc-ssss-wwww
Info: RtmpBin added
Info: RtmpBin got AUDIO sinkPad
Info: RtmpBin got AUDIO Tee new sourcePad
Info: RtmpBin got VIDEO sinkPad
Info: RtmpBin got VIDEO Tee new sourcePad>>>>>>>> STATE CHANGED:
rtmpsink-FUKyqA3xY NEW: READY PENDING: VOID_PENDING>>>>>>>> STATE
CHANGED: flvmux-FUKyqA3xY NEW: READY PENDING: VOID_PENDING>>>>>>>>
STATE CHANGED: queue-audio-FUKyqA3xY NEW: READY PENDING:
VOID_PENDING>>>>>>>> STATE CHANGED: queue-video-FUKyqA3xY NEW: READY
PENDING: VOID_PENDING>>>>>>>> STATE CHANGED: rtmpBin-FUKyqA3xY NEW:
READY PENDING: PLAYING*>>>>>>>> STATE CHANGED: my-pipeline NEW: PAUSED
PENDING: PAUSED
*>>>>>>>> STATE CHANGED: flvmux-FUKyqA3xY NEW: PAUSED PENDING:
VOID_PENDING>>>>>>>> STATE CHANGED: queue-audio-FUKyqA3xY NEW: PAUSED
PENDING: VOID_PENDING>>>>>>>> STATE CHANGED: queue-video-FUKyqA3xY
NEW: PAUSED PENDING: VOID_PENDING
Info: RtmpBin sync
Info: RtmpBin pad linked>>>>>>>> STATE CHANGED: rtmpsink-FUKyqA3xY
NEW: PAUSED PENDING: VOID_PENDING>>>>>>>> STATE CHANGED:
rtmpBin-FUKyqA3xY NEW: PAUSED PENDING: VOID_PENDING*>>>>>>>> STATE
CHANGED: my-pipeline NEW: PAUSED PENDING: PLAYING
*>>>>>>>> STATE CHANGED: rtmpsink-FUKyqA3xY NEW: PLAYING PENDING:
VOID_PENDING>>>>>>>> STATE CHANGED: flvmux-FUKyqA3xY NEW: PLAYING
PENDING: VOID_PENDING>>>>>>>> STATE CHANGED: queue-audio-FUKyqA3xY
NEW: PLAYING PENDING: VOID_PENDING>>>>>>>> STATE CHANGED:
queue-video-FUKyqA3xY NEW: PLAYING PENDING: VOID_PENDING>>>>>>>> STATE
CHANGED: rtmpBin-FUKyqA3xY NEW: PLAYING PENDING: VOID_PENDING*>>>>>>>>
STATE CHANGED: my-pipeline NEW: PLAYING PENDING: VOID_PENDING
*


So I know it's difficult to read my code, but maybe I have a concept error
and some of you can guide me on the right direction... I have done my
homework and looked into the doc and examples like this:
https://github.com/sdroege/gst-snippets/blob/217ae015aaddfe3f7aa66ffc936ce93401fca04e/dynamic-tee-vsink.c

Or reading things like:
https://gstreamer.freedesktop.org/documentation/tutorials/basic/dynamic-pipelines.html

Maybe I need to use some probe blocking (which to be honest I don't fully
understand) but I was hoping to have to deal with that only when removing
the sink pads... but looks like I'm not getting there yet hehe

Something maybe matters is that all this is a stream, I'm receiving a
stream from webrtc and pushing it to rtmp (audio being converted to acc,
video trying to sent it directly in h264 as it comes out from the webrtcbin)

Thanks a lot even if you took the time to read 1/3 of this huge post.

Cheers!

---
Sebastian A. Greco

[Tim Müller]

On Sat, 2018-11-24 at 00:57 +0100, Sebastian Greco wrote:

Hi Sebastian,

I admit I only skim-read your post so apologies if I missed something,
but the reason the pipeline temporarily loses its PLAYING state is
likely the addition of the new sink (and setting its state to PAUSED or
PLAYING), not the requesting of the tee pad.

You may want to set the "async" property on the sink to FALSE before
adding it to the already-running pipeline to prevent this.

Cheers
Tim

[sebasgre]

Hi Tim!
well even tho you skim-read it you actually hit the problem at the first
try! haha once I put async to false the pipeline continued in PLAYING state.

I'm still having a problem on myt rtmp sink I guess, because I'm not getting
any valid data on the other side... I guess I will have to use some rtmp
server that tells me in more detail what is going on there.

Anyway! that's a problem for another week of debugging ahah

thanks a lot... I need to go to read the doc to know why that changed the
behavior.

Again! Thanks a lot, you saved me tons of hours of headache!

Cheers!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/