How to get the last frame of a video file ?

[adrien_sch]

Hello guys,

My goal here is to get few snapshots of a video file. My plan is to use the
pngenc element like this :

filesrc ! matroskademux ! h264parse ! avdec_h264 ! videoconvert ! pngenc !
appsink.

My first step is to navigate inside the stream using the seek query, and my
goal is to get 3 snapshot, the first frame of the video file, middle ( =>
duration / 2), and the last. I observed that when I seek to the last frame
(position = duration) the demux element throw a EOS event and I didn't get
the last frame. My question is, how can I achieve this ?

I tried to use step event, but I had the sample result.

My second question is, can I actually use the pngenc in preroll to create my
snapshot ? Indeed, the pipeline is in PAUSED state during the navigation,
and I'm not sure if the pngenc element can work in this state ( videoencoder
class also).

Thx in advance for your help.

Gstreamer version : 1.2
Video sample :
http://www.auby.no/files/video_tests/h264_1080p_hp_4.1_10mbps_dts_unstyled_subs_monsters.mkv
Source Code : http://pastebin.com/0KqiRQyk



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[Ian Davidson]

It seems to me .... If you have a video file which is exactly 1 second
(say 30 frames) long and you try to seek to a position of 1 second, you
will be trying to seek to the 31st frame, so an EOS would be in order.

If the file consists of Key Frames each followed by a number of Delta
frames, a Seek will often look for the next Key Frame, rather than
stopping on a Delta.

So, I think I would start by aiming for beginning, Middle and 'Just
before the end'.

Ian

Post by adrien_sch
Hello guys,
My goal here is to get few snapshots of a video file. My plan is to use the
filesrc ! matroskademux ! h264parse ! avdec_h264 ! videoconvert ! pngenc !
appsink.
My first step is to navigate inside the stream using the seek query, and my
goal is to get 3 snapshot, the first frame of the video file, middle ( =>
duration / 2), and the last. I observed that when I seek to the last frame
(position = duration) the demux element throw a EOS event and I didn't get
the last frame. My question is, how can I achieve this ?

[adrien_sch]

I tried with to target duration - frame duration or to use
GST_SEEK_FLAG_FLUSH | GST_SEEK_FLAG_KEY_UNIT | GST_SEEK_FLAG_SNAP_BEFORE
without a better result.





--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664854.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[adrien_sch]

In my case, The video file duration is 60146000000, the frame duration :
41708398 and the last seekable position is 59893000000.

So, 60146000000 - 59893000000 = 253000000 is about 6.06 times the frame
duration. How can I calculate the last frame timecode ?



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664859.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[Sebastian Dröge]

Post by adrien_sch
41708398 and the last seekable position is 59893000000.
So, 60146000000 - 59893000000 = 253000000 is about 6.06 times the frame
duration. How can I calculate the last frame timecode ?

This strongly depends on the accuracy of the duration in the container.
I think the most reliable way to get the very last frame is to catch the
frames from a pad probe in front of the sink. Seek to near the end, let
it run until EOS and always remember the last frame you saw. In the end
you'll definitely have the last frame.

[adrien_sch]

I'm sorry, but I think that there is something wrong.

This is an example :

Create a MKV file using the following command

GST_DEBUG=matroskamux:5 gst-launch-1.0 videotestsrc ! timeoverlay ! x264enc
! matroskamux ! filesink location=test.mkv -e

This is the output :
http://pastebin.com/j74Nrgh4

So reading this output the last frame of the video file must be the
0:00:56.500000000 and the total duration is 0:00:56.533333333 with a stable
frame duration at 33333333. This frame is actually before the last simple
block in the cluster. The resulting mkv file can be verified with
mkvinfo-gui (don't forget to show all elements in the UI menu).

At this point, we are sure that this frame exist in the video file, at
position 56500000000.

Now, let seek in this file using the matroskademux.
gst_element_seek_simple(data->sink,GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH,
56500000000);

I put a buffer probe on on the demux src probe, this is the output :
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56433000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56367000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56333000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56400000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
*Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56500000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615*
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56467000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Receive EOS on the probe

With my understanding, the demux element find the frame but this frame isn't
correctly recognized or forwarded to the downstream. Am I right ?


The full GST trace is available here :
http://www.fileswap.com/dl/WZm9tbpf3/out.tar.gz.html











--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664872.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[Sebastian Dröge]

Post by adrien_sch
[...]
At this point, we are sure that this frame exist in the video file, at
position 56500000000.
Now, let seek in this file using the matroskademux.
gst_element_seek_simple(data->sink,GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH,
56500000000);
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56433000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56367000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56333000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56400000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
*Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56500000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615*
Buffer 0 GST_BUFFER_FLAG_DELTA_UNIT Duration 33333333PTS : 56467000000
Target : 56500000000 DTS : -1 Offset : 18446744073709551615
Receive EOS on the probe
With my understanding, the demux element find the frame but this frame isn't
correctly recognized or forwarded to the downstream. Am I right ?

You mean because it starts outputting buffers before the seek position?
That happens because it has to start at the last keyframe before the
seek position. A decoder will (should) drop all buffers before the seek
position after decoding.

Or you mean it's wrong because the last buffer to be displayed is the
second to last buffer, and another one with a smaller PTS is output
afterwards and only then EOS? That happens because you're using a codec
that uses frame reordering. The decoder needs the frames in this order
for proper decoding, and the last to be displayed frame has to be
decoded before another frame that would be displayed before that. The
decoder will reorder the frames to display order.

[adrien_sch]

I understand these mecanisms, one thing that I don't, is why the frame
(56500000000) is found and throw to the decoder, but never displayed by the
video sink.

I validate this point by setting a buffer probe after the decoder
(avdec_h264).



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664875.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[Sebastian Dröge]

Post by adrien_sch
I understand these mecanisms, one thing that I don't, is why the frame
(56500000000) is found and throw to the decoder, but never displayed by the
video sink.
I validate this point by setting a buffer probe after the decoder
(avdec_h264).

Ah that's the problem, sorry I missed that in your previous mails. Do
you get the decoded frame after the decoder as the very last frame...
but it never arrives in the sink? It should be possible from the debug
logs to trace this frame further downstream and see where it last
appears. That should help to limit the areas of code where to look for
the bug...

[adrien_sch]

It appears that frames arrive in the avdec_h264 element, but nothing go out
(on the src pad).

I also tried also to drop the EOS event on the demux src pad with no better
results.

This is my code : http://pastebin.com/YdqdgbTb

Should I file a bug ?



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664877.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.

[Sebastian Dröge]

Post by adrien_sch
It appears that frames arrive in the avdec_h264 element, but nothing go out
(on the src pad).
I also tried also to drop the EOS event on the demux src pad with no better
results.
This is my code : http://pastebin.com/YdqdgbTb

But all previous frames come out of the decoder?

Post by adrien_sch
Should I file a bug ?

Yes please, with the testcase and a summary of the discussion we had on
the mailing list.

[adrien_sch]

Bug filed : https://bugzilla.gnome.org/show_bug.cgi?id=723076

When I seek to the last frame, nothing is coming out of the decoder element.





--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-get-the-last-frame-of-a-video-file-tp4664849p4664901.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.