Discussion:
[gst-devel] Re: [gst-cvs] wtay gstreamer: gstreamer/ gstreamer/docs/design/ gstreamer/gst/base/
(too old to reply)
Ronald S. Bultje
2005-05-02 16:26:12 UTC
Permalink
Hi,
Added draft for new query API.
[..]
Purpose
Queries are used to get information about the stream.
A query is started on a specific pad and travels up or downstream.
Types of queries
- get length of stream
- get position in stream
- get seeking capabilities
- get latency
- convert one value to another
- query supported formats
- query internal links.
Current implementation
The current implementation of query requires pads to implement the
gboolean (*GstPadConvertFunction) (GstPad *pad,
GstFormat src_format, gint64 src_value,
GstFormat *dest_format, gint64 *dest_value);
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQueryType type,
GstFormat *format, gint64 *value);
GList* (*GstPadIntLinkFunction) (GstPad *pad);
const GstFormat* (*GstPadFormatsFunction) (GstPad *pad);
const GstEventMask* (*GstPadEventMaskFunction) (GstPad *pad);
const GstQueryType* (*GstPadQueryTypeFunction) (GstPad *pad);
Most of these functions are not very extensible in particular,
the queryfunction can only return one value.
Requirements
- multiple return values, grouped together when they make sense.
- one pad function to perform the query
- extensible queries.
Proposition
- define GstQuery extending GstData and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query);
pad returns result in query structure and TRUE as result or FALSE when
query is not supported.
Proposed types
- get info on how seeking can be done
- getrange, with/without offset/size
- ranges where seeking is efficient (for caching network sources)
- flags describing seeking behaviour (forward, backward, segments,
play backwards, ...)
- get info on current position of the stream
- start position
- current position
- end position
- length
- get amount of buffering
- convert format/value to another format/value pair.
- return list of supported formats.
- return list of internal link pads.
Also????
Apart from the cvs messages still being horribly broken, let's go into
it by using inline comments, still.

First, you're throwing together a whole lot of stuff that, if you ask
me, are completely unrelated. Is that good? What's wrong with the old
get_internal_links() API? What's wrong with GST_PAD_CAPS(pad) (or
whatever that was renamed to)? Etc.

OK, more important part. Tim complained that the polling is very
suboptimal for applications, which makes sense. So instead of:

bool idle_func (gpointer data) {
time = query_position();
update_slider(time);
return TRUE;
}
int main (void) {
g_timeout_add (500, idle_func, NULL);
}

It'd be nice to have:

void data_func (pipeline, data) {
update_slider (time (data));
}
int main (void) {
g_signal_connect (pipeline, "data-passed", data_func, NULL);
}

This may or may not work out; in the end, it should work for
transcoding, playback, capture, etc, and some of those may be more
difficult with this method, but there are probably awys to fix it. Some
people also argue that length should be reported as tags
(GST_TAG_DURATION) instead of a query. It has some advantages and
disadvantages, I honestly don't care.

Ronald

PS whoever has access to CVSROOT, can you please fix the inline cvs diff
messages? They're not useful and misleading.
--
Ronald S. Bultje <***@ronald.bitfreak.net>
Wim Taymans
2005-05-02 16:51:29 UTC
Permalink
Post by Ronald S. Bultje
Hi,
Added draft for new query API.
[..]
Purpose
Queries are used to get information about the stream.
A query is started on a specific pad and travels up or downstream.
Types of queries
- get length of stream
- get position in stream
- get seeking capabilities
- get latency
- convert one value to another
- query supported formats
- query internal links.
Current implementation
The current implementation of query requires pads to implement the
gboolean (*GstPadConvertFunction) (GstPad *pad,
GstFormat src_format, gint64 src_value,
GstFormat *dest_format, gint64 *dest_value);
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQueryType type,
GstFormat *format, gint64 *value);
GList* (*GstPadIntLinkFunction) (GstPad *pad);
const GstFormat* (*GstPadFormatsFunction) (GstPad *pad);
const GstEventMask* (*GstPadEventMaskFunction) (GstPad *pad);
const GstQueryType* (*GstPadQueryTypeFunction) (GstPad *pad);
Most of these functions are not very extensible in particular,
the queryfunction can only return one value.
Requirements
- multiple return values, grouped together when they make sense.
- one pad function to perform the query
- extensible queries.
Proposition
- define GstQuery extending GstData and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query);
pad returns result in query structure and TRUE as result or FALSE when
query is not supported.
Proposed types
- get info on how seeking can be done
- getrange, with/without offset/size
- ranges where seeking is efficient (for caching network sources)
- flags describing seeking behaviour (forward, backward, segments,
play backwards, ...)
- get info on current position of the stream
- start position
- current position
- end position
- length
- get amount of buffering
- convert format/value to another format/value pair.
- return list of supported formats.
- return list of internal link pads.
Also????
Apart from the cvs messages still being horribly broken, let's go into
it by using inline comments, still.
First, you're throwing together a whole lot of stuff that, if you ask
me, are completely unrelated. Is that good? What's wrong with the old
get_internal_links() API? What's wrong with GST_PAD_CAPS(pad) (or
whatever that was renamed to)? Etc.
Many function names while they basically all get info from the pad in
question. Moving to a generic message based system to get info might
give us better extensibility and might allow for specific new queries
without having to change an API. It's and idea (hence the draft status
of the document). Having only processing, an event and a query function
to a pad could simplify a lot of API.
Post by Ronald S. Bultje
OK, more important part. Tim complained that the polling is very
bool idle_func (gpointer data) {
time = query_position();
update_slider(time);
return TRUE;
}
int main (void) {
g_timeout_add (500, idle_func, NULL);
}
void data_func (pipeline, data) {
update_slider (time (data));
}
int main (void) {
g_signal_connect (pipeline, "data-passed", data_func, NULL);
}
Well, data-passed (if it happens when I think it would happen, like
with every buffer passed on a pad) is certainly even more suboptimal...

My proposal includes 1 query to get both length, position and other
stuff. For this particular query there is not much more you can do to
make it more optimal IMO.
Post by Ronald S. Bultje
This may or may not work out; in the end, it should work for
transcoding, playback, capture, etc, and some of those may be more
difficult with this method, but there are probably awys to fix it. Some
people also argue that length should be reported as tags
(GST_TAG_DURATION) instead of a query. It has some advantages and
disadvantages, I honestly don't care.
If the position info is reported in a GstStructure we're not far away
from it being tags.. What's the advantage of it being a real tag?
merging with metadata is easier as it can use existing APIs?

Wim
Post by Ronald S. Bultje
Ronald
PS whoever has access to CVSROOT, can you please fix the inline cvs diff
messages? They're not useful and misleading.
--
Wim Taymans <***@fluendo.com>
Ronald S. Bultje
2005-05-02 18:00:22 UTC
Permalink
Post by Wim Taymans
Well, data-passed (if it happens when I think it would happen, like
with every buffer passed on a pad) is certainly even more suboptimal...
It doesn't matter, this is not a limiting factor. The point is, this is
easier, because it does not require polling but apps get notified
instead.

Ronald
--
Ronald S. Bultje <***@ronald.bitfreak.net>
Thomas Vander Stichele
2005-05-03 07:07:15 UTC
Permalink
Hi,
Post by Ronald S. Bultje
PS whoever has access to CVSROOT, can you please fix the inline cvs diff
messages? They're not useful and misleading.
You all have access to it :) just check it out from CVS. feel free to
fix it.

Thomas

Dave/Dina : future TV today ! - http://www.davedina.org/
<-*- thomas (dot) apestaart (dot) org -*->
It's only love that stops you from walking out the door
<-*- thomas (at) apestaart (dot) org -*->
URGent, best radio on the net - 24/7 ! - http://urgent.fm/
Benjamin Otte
2005-05-03 11:41:02 UTC
Permalink
Purpose
Queries are used to get information about the stream.
A query is started on a specific pad and travels up or downstream.
Types of queries
- get length of stream
- get position in stream
- get seeking capabilities
- get latency
- convert one value to another
- query supported formats
- query internal links.
I think all of these are wrong as a query and can be implemented better
without the need for a querying framework. I'll elaborate:
- get length of stream
This is currently polled when instead it should just be delivered when
available. The current mechanism to provide metainformation about a data
stream is tags. I don't see why using the GST_TAG_DURATION tag (and
probably a GST_TAG_SIZE for the size in bytes) tag won't work. This of
course requires adapting some elements to modify this tag, but in the end
people wanting to get tags are better off.
- get position in stream
The position inside the stream equals the timestamp + duration of the last
buffer. Or in finer grained environments (like audiosink) you can
get it directly from the element. In both cases there's no need to send a
query through the stream. Especially since queries that end up in other
threads tend to return widely differing positions anyway.
- get seeking capabilities
Seeking capabilities need to be negotiated, since it should not be
possible to link a non-seekable sink with an AVI muxer for example.
- get latency
I'll pass on this since I have no clue what the exact requirements for
this are. but I've thought that sending a PING event somewhere and
receiving a PONG event from the endpoint is a much better idea than
requiring elements to calculate their latency. Especially in this case:
src ! ... ! tcpserversink tcpclientsrc ! ... ! sink
- convert one value to another
This should be part of the caps media type meta information stuff (where
we put the fixate functions). Converting a timestamp to an offset is the
same as long as the format is the same. So it's better to just do
gst_convert (caps, format, value, &target_format, &target_value) or even
gst_pad_convert (pad, format, value, &target_format, &target_value) which
would use the pad's current negotiated caps. Especially because this
really eases the work of writing a new element, since conversions don't
need to be explicitly supported by each and every element.
- query supported formats
If that's the conversion formats, this would easily be solved by the stuff
I outlined above.
- query internal links
Is there any reason why someone would want to query it from outside the
element? (This is neglecting the fact that internal links are just a poor
hack to get default event handlers doing something that most likely will
look as if it'll work and therefore should go away.)
Proposition
- define GstQuery extending GstData and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query);
pad returns result in query structure and TRUE as result or FALSE when
query is not supported.
So you propose to make a query be an event, and even have the same
signature?
Wouldn't it make sense to just define some new events?


Benjamin
Wim Taymans
2005-05-04 09:36:09 UTC
Permalink
Post by Benjamin Otte
Purpose
Queries are used to get information about the stream.
A query is started on a specific pad and travels up or downstream.
Types of queries
- get length of stream
- get position in stream
- get seeking capabilities
- get latency
- convert one value to another
- query supported formats
- query internal links.
I think all of these are wrong as a query and can be implemented better
- get length of stream
This is currently polled when instead it should just be delivered when
available. The current mechanism to provide metainformation about a data
stream is tags. I don't see why using the GST_TAG_DURATION tag (and
probably a GST_TAG_SIZE for the size in bytes) tag won't work. This of
course requires adapting some elements to modify this tag, but in the end
people wanting to get tags are better off.
Posting a size message on the bus whenever the length of the stream
changes is indeed something usefull. Problem is that many elements will
have some idea about the length of the stream in
bytes/samples/frames/etc while the application is mostly interested in
the length as it will be played.
Post by Benjamin Otte
- get position in stream
The position inside the stream equals the timestamp + duration of the last
buffer. Or in finer grained environments (like audiosink) you can
get it directly from the element. In both cases there's no need to send a
query through the stream. Especially since queries that end up in other
threads tend to return widely differing positions anyway.
Queuing can screw this up, both in things like an audiosink or a queue.
Current possition before and after the queue is different. It's rather
hard to figure out what goes on by looking at the buffers that pass by.
The query is more convenient for the programmer as there is only a
single API to get the info.
Post by Benjamin Otte
- get seeking capabilities
Seeking capabilities need to be negotiated, since it should not be
possible to link a non-seekable sink with an AVI muxer for example.
The query is exactly how the information is obtained to perform this
negotiation.
Post by Benjamin Otte
- get latency
I'll pass on this since I have no clue what the exact requirements for
this are. but I've thought that sending a PING event somewhere and
receiving a PONG event from the endpoint is a much better idea than
src ! ... ! tcpserversink tcpclientsrc ! ... ! sink
Latency is usefull for alsasrc ! alsasink, a clock adjust has to be
aplied to make sure no samples arrive too late in the sink. In the case
you state above you mix up the method of obtaining the latency and
eventually using it.
Post by Benjamin Otte
- convert one value to another
This should be part of the caps media type meta information stuff (where
we put the fixate functions). Converting a timestamp to an offset is the
same as long as the format is the same. So it's better to just do
gst_convert (caps, format, value, &target_format, &target_value) or even
gst_pad_convert (pad, format, value, &target_format, &target_value) which
would use the pad's current negotiated caps. Especially because this
really eases the work of writing a new element, since conversions don't
need to be explicitly supported by each and every element.
Caps only provide info about the media type, not the processing state of
the element. Caps don't give enough info to do a conversion to get the
actual bitrate of encoder, for example.
Post by Benjamin Otte
- query supported formats
If that's the conversion formats, this would easily be solved by the stuff
I outlined above.
Exposing that a seek can be done on a chapter in a DVD cannot be derived
from the caps, let alone that the caps provide info on the number of
chapters.
Post by Benjamin Otte
- query internal links
Is there any reason why someone would want to query it from outside the
element? (This is neglecting the fact that internal links are just a poor
hack to get default event handlers doing something that most likely will
look as if it'll work and therefore should go away.)
For shutting down part of a pipeline one must be able to follow the flow
of data, hence the internal links. This is currently used to dispatch
events but also in playbin to mute streams.
Post by Benjamin Otte
Proposition
- define GstQuery extending GstData and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query);
pad returns result in query structure and TRUE as result or FALSE when
query is not supported.
So you propose to make a query be an event, and even have the same
signature?
Wouldn't it make sense to just define some new events?
It would have the same datastructure (as GstMessage) but with a
different intend, hence a different name for the structure.

Wim
Post by Benjamin Otte
Benjamin
-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r=20
_______________________________________________
gstreamer-devel mailing list
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
--
Wim Taymans <***@fluendo.com>
Continue reading on narkive:
Loading...