Video sync via GstClock

[Andreas Rödig]

Hi,

i want to synchronize to videos.

I found that i can use GstClock (gst_clock_set_calibration) to jump
forward. So i transport the position from the master to every client
over network.

Then i set the client with gst_clock_set_calibration to the master
position. That works if master time bigger than client time.
But i can not jump backwards with this function. And sometimes (at far
jumps) my audio plugin crashed.

Are there something i forgot? Is this the right function to change the
pipeline clock?

Here are my steps:
- start pipeline (playbin2)
- get clock (gst_pipeline_get_clock)
- get basetime (gst_clock_get_time)
.
.
running
.
.
- incomming master position (0 - n milliseconds)
- set clock (basetime + master position)

Thanks
--
Andreas Rödig

null821 media services
Weberstr. 2
86462 Langweid/Foret

Tel.: 0821 / 800 60 -34
Fax.: 0821 / 800 60 -55
Email: ***@null821.de
Web: www.null821.de

________________________________________________________________
null821 media services gmbh & co. kg
Inhaber: Ralf Walz
Weberstraße 2, 86462 Langweid/Foret
Steuernummer: 102/170/52004

[Tim-Philipp Müller]

On Mon, 2013-03-11 at 17:03 +0100, Andreas Rödig wrote:

Hi,

Post by Andreas Rödig
i want to synchronize to videos.
I found that i can use GstClock (gst_clock_set_calibration) to jump
forward. So i transport the position from the master to every client
over network.
Then i set the client with gst_clock_set_calibration to the master
position. That works if master time bigger than client time.
But i can not jump backwards with this function. And sometimes (at far
jumps) my audio plugin crashed.
Are there something i forgot? Is this the right function to change the
pipeline clock?
- start pipeline (playbin2)
- get clock (gst_pipeline_get_clock)
- get basetime (gst_clock_get_time)
.
.
running
.
.
- incomming master position (0 - n milliseconds)
- set clock (basetime + master position)

Have you seen GstNetTimeProvider and GstNetClientClock ?

Cheers
-Tim

[Andreas Rödig]

Hi Tim-Philipp,

yes i try this before. But it does not work.

I use it at that way:

Server (If state changed to GST_STATE_PLAYING):
netclock = (GstClock *)gst_pipeline_get_clock((GstPipeline*)pipeline);
gst_net_time_provider_new(netclock, NULL, 40000);



Client (If state changed to GST_STATE_PLAYING):

localClock = (GstClock)gst_pipeline_get_clock((GstPipeline*)pipeline);
netclock = gst_net_client_clock_new("clientclock", "localhost", 40000,
gst_clock_get_time(localClock));
gst_clock_set_master(localClock, netclock);


Thanks,
Andreas Rödig

Post by Tim-Philipp Müller
Hi,

Post by Andreas Rödig
i want to synchronize to videos.
I found that i can use GstClock (gst_clock_set_calibration) to jump
forward. So i transport the position from the master to every client
over network.
Then i set the client with gst_clock_set_calibration to the master
position. That works if master time bigger than client time.
But i can not jump backwards with this function. And sometimes (at far
jumps) my audio plugin crashed.
Are there something i forgot? Is this the right function to change the
pipeline clock?
- start pipeline (playbin2)
- get clock (gst_pipeline_get_clock)
- get basetime (gst_clock_get_time)
.
.
running
.
.
- incomming master position (0 - n milliseconds)
- set clock (basetime + master position)

Have you seen GstNetTimeProvider and GstNetClientClock ?
Cheers
-Tim
_______________________________________________
gstreamer-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller]

On Tue, 2013-03-12 at 15:11 +0100, Andreas Rödig wrote:

Hi,

Post by Andreas Rödig
yes i try this before. But it does not work.

What exactly/how does it not work?

Post by Andreas Rödig
netclock = (GstClock *)gst_pipeline_get_clock((GstPipeline*)pipeline);
gst_net_time_provider_new(netclock, NULL, 40000);
localClock = (GstClock)gst_pipeline_get_clock((GstPipeline*)pipeline);
netclock = gst_net_client_clock_new("clientclock", "localhost", 40000,
gst_clock_get_time(localClock));
gst_clock_set_master(localClock, netclock);

The usual usage pattern is to have a net time provider (usually based on
a system clock), and then do gst_pipeline_use_clock() with net client
clocks on all pipelines to be synchronised. You will also have to handle
distribution of the base time of the master by some external means.

You might find some useful code in https://github.com/thaytan/aurena

Cheers
-Tim

Post by Andreas Rödig

Post by Tim-Philipp Müller
Hi,

Post by Andreas Rödig
i want to synchronize to videos.
I found that i can use GstClock (gst_clock_set_calibration) to jump
forward. So i transport the position from the master to every client
over network.
Then i set the client with gst_clock_set_calibration to the master
position. That works if master time bigger than client time.
But i can not jump backwards with this function. And sometimes (at far
jumps) my audio plugin crashed.
Are there something i forgot? Is this the right function to change the
pipeline clock?
- start pipeline (playbin2)
- get clock (gst_pipeline_get_clock)
- get basetime (gst_clock_get_time)
.
.
running
.
.
- incomming master position (0 - n milliseconds)
- set clock (basetime + master position)

Have you seen GstNetTimeProvider and GstNetClientClock ?
Cheers
-Tim
_______________________________________________
gstreamer-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Andreas Rödig]

Hi,

Post by Tim-Philipp Müller

Post by Andreas Rödig
yes i try this before. But it does not work.

What exactly/how does it not work?

What i try to do: Two videos (master and slave) should play to the
millisecond. But the play always at their own clock...

I think: If i play the master and then start the slave, the slave should
jump to the current position of the master.

Post by Tim-Philipp Müller
The usual usage pattern is to have a net time provider (usually based on
a system clock), and then do gst_pipeline_use_clock() with net client
clocks on all pipelines to be synchronised. You will also have to handle
distribution of the base time of the master by some external means.

I have the possibility to transfer the basetime of the master (the
gst_clock_get_time(clock_of_pipeline)) to the slave.

I set it then with
netclock=gst_net_client_clock_new("clientclock", ip_adress, 5637,
basetime_of_master);
gst_pipeline_use_clock((GstPipeline*)pipeline,netclock);

Is this correct?

Thanks,
Andreas

[Marc Leeman]

Post by Andreas Rödig
What i try to do: Two videos (master and slave) should play to the
millisecond. But the play always at their own clock...

You need to provide more details, I've done a setup like that several
times and it just works; both form network and from file.

e.g. just the other day; we had 4 TS files playing in sync and another
test where decoders lock on to a network source and play off in sync.

[Andreas Rödig]

Hi Marc,

ok, more details:

I have one or many videos. All of this videos have the same length.
Every video runs on another pc.

Then i start all of this videos at the same time. But sometimes they
will not be at the same frame.

So, my idea was: I have one master or time provider and all other
pc/gstreamer pipelines have to sync to the same frame position as the
master pc.

At the moment i try to transfer the position of on master/gstreamer
pipeline via socket to another gstreamer application.

With gst_clock_set_calibration i can set the slave gstreamer clock to
forward to the received master position. But not backwards.


Thanks,
Andreas Rödig

Post by Marc Leeman

Post by Andreas Rödig
What i try to do: Two videos (master and slave) should play to the
millisecond. But the play always at their own clock...

You need to provide more details, I've done a setup like that several
times and it just works; both form network and from file.
e.g. just the other day; we had 4 TS files playing in sync and another
test where decoders lock on to a network source and play off in sync.
_______________________________________________
gstreamer-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Marc Leeman]

I think these were the scripts I used for the demo (the devices are
powered off, so I cannot check atm).

the setup was a 2x2 of Blood and Chrome

the top left was the original and the three others were files that
were converted with some effect of videoconvert (sepia, xray, ...).

These files were all stored on the HDDs driving the monitor (4
monitors, 4 devices (AD10)). Oh, and the files are TS, I had an issue
with MKV but not yet look into it as to why.

***@drd1812:~$ cat play-master-file.py
#!/usr/bin/env python

import sys

import pygst
pygst.require('0.10')
import gst

def main(args):
_, uri, port, crop = args
port = int(port)
crop = crop.split(':')

# make the pipeline
# pipeline = gst.element_factory_make("playbin2", "player")
# pipeline.set_property("uri", uri)

# gstline = "filesrc name=src ! decodebin ! videocrop left=%s
right=%s top=%s bottom=%s ! autovideosink" % (crop[0], crop[1],
crop[2], crop[3])
# make pipeline more explicit for older code (require parser)
gstline = "filesrc name=src ! mpegtsdemux ! mpeg4videoparse !
ffdec_mpeg4 ! videocrop left=%s right=%s top=%s bottom=%s !
timeoverlay ! ffmpegcolorspace ! videoscale !
video/x-raw-rgb,height=1200,width=1920 ! ximagesink" % (crop[0],
crop[1], crop[2], crop[3])
print gstline

pipeline = gst.parse_launch (gstline)
pipeline.get_by_name('src').set_uri(uri) # uri interface

# make sure some other clock isn't autoselected
clock = pipeline.get_clock()
print 'Using clock: ', clock
pipeline.use_clock(clock)

# this will start a server listening on a UDP port
clock_provider = gst.NetTimeProvider(clock, None, port)

# we explicitly manage our base time
base_time = clock.get_time()
print ('Start slave as: python ./play-slave.py %s [IP] %d %s:%s:%s:%s %d'
% (uri, port, crop[0], crop[1], crop[2], crop[3], base_time))

# disable the pipeline's management of base_time -- we're going
# to set it ourselves.
pipeline.set_new_stream_time(gst.CLOCK_TIME_NONE)
pipeline.set_base_time(base_time)

# now we go :)
pipeline.set_state(gst.STATE_PLAYING)

# wait until things stop
pipeline.get_bus().poll(gst.MESSAGE_EOS | gst.MESSAGE_ERROR, -1)
pipeline.set_state(gst.STATE_NULL)

if __name__ == '__main__':
main(sys.argv)


slave:

***@drd1812:~$ cat play-slave-file.py
#!/usr/bin/env python

import sys

import pygst
pygst.require('0.10')
import gst

def main(args):
_, uri, ip, port, crop, base_time = args
port, base_time, crop= int(port), int(base_time), crop.split(':')

# make the pipeline
# pipeline = gst.element_factory_make("playbin2", "player")
# pipeline.set_property("uri", uri)

# gstline = "filesrc name=src ! decodebin ! videocrop left=%s
right=%s top=%s bottom=%s ! autovideosink" % (crop[0], crop[1],
crop[2], crop[3])
# make pipeline more explicit for older code (require parser)
gstline = "filesrc name=src ! mpegtsdemux ! mpeg4videoparse !
ffdec_mpeg4 ! videocrop left=%s right=%s top=%s bottom=%s !
timeoverlay ! ffmpegcolorspace ! videoscale !
video/x-raw-rgb,height=1200,width=1920 ! ximagesink" % (crop[0],
crop[1], crop[2], crop[3])
print gstline

pipeline = gst.parse_launch (gstline)
pipeline.get_by_name('src').set_uri(uri) # uri interface
# disable the pipeline's management of base_time -- we're going
# to set it ourselves.
pipeline.set_new_stream_time(gst.CLOCK_TIME_NONE)

# make a clock slaving to the network
clock = gst.NetClientClock(None, ip, port, base_time)

# use it in the pipeline
pipeline.set_base_time(base_time)
pipeline.use_clock(clock)

# now we go :)
pipeline.set_state(gst.STATE_PLAYING)

# wait until things stop
pipeline.get_bus().poll(gst.MESSAGE_EOS | gst.MESSAGE_ERROR, -1)
pipeline.set_state(gst.STATE_NULL)

if __name__ == '__main__':
main(sys.argv)

[Andreas Rödig]

Hi Marc,

yes your script seems what i want.

I try it in c, but i can not get videos in sync.
I got very good hints from your script (base_time and so on...), but
finally it doesn't has any effect.

Here is what i do:
Server:
netclock=gst_pipeline_get_clock((GstPipeline*)pipeline);
gst_pipeline_use_clock((GstPipeline*)pipeline,netclock);
basetime = gst_clock_get_time(netclock);
provider=gst_net_time_provider_new(netclock, "192.168.1.69", 5637);
gst_pipeline_set_new_stream_time((GstPipeline*)pipeline,GST_CLOCK_TIME_NONE);
gst_element_set_start_time(pipeline,basetime);
gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_PLAYING);

Client (basetime comes over network):
gst_pipeline_set_new_stream_time((GstPipeline*)pipeline,GST_CLOCK_TIME_NONE);
netclock = gst_net_client_clock_new(NULL, "192.168.1.69", 5637,
(GstClockTime)position);
gst_element_set_start_time(pipeline,position);
gst_pipeline_use_clock((GstPipeline*)pipeline,netclock);
gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_PLAYING);

Do i miss something?

Thanks,
Andreas Rödig

Post by Marc Leeman
I think these were the scripts I used for the demo (the devices are
powered off, so I cannot check atm).
the setup was a 2x2 of Blood and Chrome
the top left was the original and the three others were files that
were converted with some effect of videoconvert (sepia, xray, ...).
These files were all stored on the HDDs driving the monitor (4
monitors, 4 devices (AD10)). Oh, and the files are TS, I had an issue
with MKV but not yet look into it as to why.
#!/usr/bin/env python
import sys
import pygst
pygst.require('0.10')
import gst
_, uri, port, crop = args
port = int(port)
crop = crop.split(':')
# make the pipeline
# pipeline = gst.element_factory_make("playbin2", "player")
# pipeline.set_property("uri", uri)
# gstline = "filesrc name=src ! decodebin ! videocrop left=%s
right=%s top=%s bottom=%s ! autovideosink" % (crop[0], crop[1],
crop[2], crop[3])
# make pipeline more explicit for older code (require parser)
gstline = "filesrc name=src ! mpegtsdemux ! mpeg4videoparse !
ffdec_mpeg4 ! videocrop left=%s right=%s top=%s bottom=%s !
timeoverlay ! ffmpegcolorspace ! videoscale !
video/x-raw-rgb,height=1200,width=1920 ! ximagesink" % (crop[0],
crop[1], crop[2], crop[3])
print gstline
pipeline = gst.parse_launch (gstline)
pipeline.get_by_name('src').set_uri(uri) # uri interface
# make sure some other clock isn't autoselected
clock = pipeline.get_clock()
print 'Using clock: ', clock
pipeline.use_clock(clock)
# this will start a server listening on a UDP port
clock_provider = gst.NetTimeProvider(clock, None, port)
# we explicitly manage our base time
base_time = clock.get_time()
print ('Start slave as: python ./play-slave.py %s [IP] %d %s:%s:%s:%s %d'
% (uri, port, crop[0], crop[1], crop[2], crop[3], base_time))
# disable the pipeline's management of base_time -- we're going
# to set it ourselves.
pipeline.set_new_stream_time(gst.CLOCK_TIME_NONE)
pipeline.set_base_time(base_time)
# now we go :)
pipeline.set_state(gst.STATE_PLAYING)
# wait until things stop
pipeline.get_bus().poll(gst.MESSAGE_EOS | gst.MESSAGE_ERROR, -1)
pipeline.set_state(gst.STATE_NULL)
main(sys.argv)
#!/usr/bin/env python
import sys
import pygst
pygst.require('0.10')
import gst
_, uri, ip, port, crop, base_time = args
port, base_time, crop= int(port), int(base_time), crop.split(':')
# make the pipeline
# pipeline = gst.element_factory_make("playbin2", "player")
# pipeline.set_property("uri", uri)
# gstline = "filesrc name=src ! decodebin ! videocrop left=%s
right=%s top=%s bottom=%s ! autovideosink" % (crop[0], crop[1],
crop[2], crop[3])
# make pipeline more explicit for older code (require parser)
gstline = "filesrc name=src ! mpegtsdemux ! mpeg4videoparse !
ffdec_mpeg4 ! videocrop left=%s right=%s top=%s bottom=%s !
timeoverlay ! ffmpegcolorspace ! videoscale !
video/x-raw-rgb,height=1200,width=1920 ! ximagesink" % (crop[0],
crop[1], crop[2], crop[3])
print gstline
pipeline = gst.parse_launch (gstline)
pipeline.get_by_name('src').set_uri(uri) # uri interface
# disable the pipeline's management of base_time -- we're going
# to set it ourselves.
pipeline.set_new_stream_time(gst.CLOCK_TIME_NONE)
# make a clock slaving to the network
clock = gst.NetClientClock(None, ip, port, base_time)
# use it in the pipeline
pipeline.set_base_time(base_time)
pipeline.use_clock(clock)
# now we go :)
pipeline.set_state(gst.STATE_PLAYING)
# wait until things stop
pipeline.get_bus().poll(gst.MESSAGE_EOS | gst.MESSAGE_ERROR, -1)
pipeline.set_state(gst.STATE_NULL)
main(sys.argv)
_______________________________________________
gstreamer-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Post by Andreas Rödig
gst_element_set_start_time(pipeline,basetime);

You need to set that _start_time() to GST_TIME_CLOCK_NONE, and set
_base_time() to your basetime. Otherwise the bin will overwrite these
values later.

Nicolas

[Andreas Rödig]

Hi Nicolas,

thank you very much!

This was the missing link ;-)

Now it is working!


Thanks,
Andreas Rödig

null821 media services
Weberstr. 2
86462 Langweid/Foret

Tel.: 0821 / 800 60 -34
Fax.: 0821 / 800 60 -55
Email: ***@null821.de
Web: www.null821.de

________________________________________________________________
null821 media services gmbh & co. kg
Inhaber: Ralf Walz
Weberstraße 2, 86462 Langweid/Foret
Steuernummer: 102/170/52004

Post by Nicolas Dufresne

Post by Andreas Rödig
gst_element_set_start_time(pipeline,basetime);

You need to set that _start_time() to GST_TIME_CLOCK_NONE, and set
_base_time() to your basetime. Otherwise the bin will overwrite these
values later.
Nicolas