Discussion:
Problem setting WebRTC connection between GStreamer and FreeSwitch
d***@poradnik-webmastera.com
2018-03-27 13:48:39 UTC
Permalink
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.

I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.

My pipeline is simplified to use audio only, and no STUN:
pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
"audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay ! "
"queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
&error);

This is SDP created by GST, with ICE candidates added:

v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates

FreeSwitch accepts it, and sends following one in reply:

v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0

I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.

GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
entries there:

0:00:01.611202908 542 0x1b5b4a0 WARN structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'


0:00:05.670806044 542 0x1bd1190 DEBUG GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
chainfunction &gst_proxy_pad_chain_default with buffer buffer:
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393 542 0x1bd1190 DEBUG GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink> calling
chainfunction &sink_chain with buffer buffer: 0x7f6c10005040, pts
0:00:04.151691973, dts 0:00:04.151691973, dur 99:99:99.999999999, size
92, offset none, offset_end none, flags 0x40
0:00:05.670856500 542 0x1bd1190 LOG dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870 542 0x1bd1190 WARN dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066 542 0x1bd1190 WARN srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242 542 0x1bd1190 WARN srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906 542 0x1bd1190 DEBUG GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink> called
chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503 542 0x1bd1190 DEBUG GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer 0x7f6c10005040,
returned ok

Any ideas how to fix this?

My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.

Regards,
Daniel
Matthew Waters
2018-03-28 08:27:26 UTC
Permalink
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay ! "
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE connection
and thus is not generally signalled through the SDP.
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
calling chainfunction &sink_chain with buffer buffer: 0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur 99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer 0x7f6c10005040,
returned ok
This usually means that something's not quite right with what GStreamer
expects.
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).

Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
d***@poradnik-webmastera.com
2018-03-28 10:14:15 UTC
Permalink
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay !
"
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE
connection
and thus is not generally signalled through the SDP.
The same IP and port for RTCP are implied by a=rtcp-mux line above, so
this line looks redundant for me. Or am I missing something?
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's
currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
calling chainfunction &sink_chain with buffer buffer: 0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur 99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer 0x7f6c10005040,
returned ok
This usually means that something's not quite right with what GStreamer
expects.
In the meantime I found
https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
. It looks that GStreamer for some reason does not perform DTSL
handshake. BTW, it seems that FreeSwitch started its part - in its log I
found this:

ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
switch_rtp.c:3752 Changing audio DTLS state from OFF
to HANDSHAKE

Here is one of STUN Binding Request packets sent by FreeSwitch. It was
captured during other test, so its contents does not match SDP pasted
earlier:

Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
User Datagram Protocol, Src Port: 24012, Dst Port: 45557
Session Traversal Utilities for NAT
[Response In: 91]
Message Type: 0x0001 (Binding Request)
Message Length: 132
Message Cookie: 2112a442
Message Transaction ID: 364e4d535879554441354a61
Attributes
USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
Attribute Type: USERNAME (0x0006)
Attribute Length: 49
Username: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
Padding: 3
PRIORITY
Attribute Type: PRIORITY (0x0024)
Attribute Length: 4
Priority: 2013266428
SOFTWARE
Attribute Type: SOFTWARE (0x8022)
Attribute Length: 19
Software: FreeSWITCH ( 64bit)
Padding: 1
ICE-CONTROLLED
Attribute Type: ICE-CONTROLLED (0x8029)
Attribute Length: 8
Tie breaker: 3967545348314432
MESSAGE-INTEGRITY
Attribute Type: MESSAGE-INTEGRITY (0x0008)
Attribute Length: 20
HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
FINGERPRINT
Attribute Type: FINGERPRINT (0x8028)
Attribute Length: 4
CRC-32: 0xbd08d97e

I hope it will help you.

Regards,
Daniel
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).
Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
d***@poradnik-webmastera.com
2018-03-28 11:55:44 UTC
Permalink
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
I think FreeSwitch can work without them. I recall that I had problems
with local ip addresses - FS was rejecting my SIP INVITE with "488 Not
Acceptable Here" until I modified ACLs to accept them. I expect that
this also would happen in this case, if FS would require them.

Regards,
Daniel
d***@poradnik-webmastera.com
2018-03-28 15:05:22 UTC
Permalink
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay !
"
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE
connection
and thus is not generally signalled through the SDP.
The same IP and port for RTCP are implied by a=rtcp-mux line above, so
this line looks redundant for me. Or am I missing something?
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
calling chainfunction &sink_chain with buffer buffer: 0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur 99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer
0x7f6c10005040,
returned ok
This usually means that something's not quite right with what
GStreamer
expects.
In the meantime I found
https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
. It looks that GStreamer for some reason does not perform DTSL
handshake. BTW, it seems that FreeSwitch started its part - in its log
ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
switch_rtp.c:3752 Changing audio DTLS state from OFF
to HANDSHAKE
Here is one of STUN Binding Request packets sent by FreeSwitch. It was
captured during other test, so its contents does not match SDP pasted
Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
User Datagram Protocol, Src Port: 24012, Dst Port: 45557
Session Traversal Utilities for NAT
[Response In: 91]
Message Type: 0x0001 (Binding Request)
Message Length: 132
Message Cookie: 2112a442
Message Transaction ID: 364e4d535879554441354a61
Attributes
USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
Attribute Type: USERNAME (0x0006)
Attribute Length: 49
Username: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
Padding: 3
PRIORITY
Attribute Type: PRIORITY (0x0024)
Attribute Length: 4
Priority: 2013266428
SOFTWARE
Attribute Type: SOFTWARE (0x8022)
Attribute Length: 19
Software: FreeSWITCH ( 64bit)
Padding: 1
ICE-CONTROLLED
Attribute Type: ICE-CONTROLLED (0x8029)
Attribute Length: 8
Tie breaker: 3967545348314432
MESSAGE-INTEGRITY
Attribute Type: MESSAGE-INTEGRITY (0x0008)
Attribute Length: 20
HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
FINGERPRINT
Attribute Type: FINGERPRINT (0x8028)
Attribute Length: 4
CRC-32: 0xbd08d97e
I hope it will help you.
I am out of ideas today. It looks that DTLS-SRTP handshake is not
started at all. I would expect that GStreamer would initiate it - maybe
it also assumes that it has server role and expects that the other side
will initiate handshake?

I have filtered dtlssrtp-related lines from log and uploaded them here.
Please take a look, maybe you will find something there. First entry
after sending remove SDP to GST is at 0:00:01.399761438.
https://pastebin.com/uMncN6jn

Regards,
Daniel
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).
Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Matthew Waters
2018-03-29 01:27:57 UTC
Permalink
Post by d***@poradnik-webmastera.com
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay
! "
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE connection
and thus is not generally signalled through the SDP.
The same IP and port for RTCP are implied by a=rtcp-mux line above, so
this line looks redundant for me. Or am I missing something?
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
calling chainfunction &sink_chain with buffer buffer: 0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur 99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer 0x7f6c10005040,
returned ok
This usually means that something's not quite right with what GStreamer
expects.
In the meantime I found
https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
. It looks that GStreamer for some reason does not perform DTSL
handshake. BTW, it seems that FreeSwitch started its part - in its log
ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
switch_rtp.c:3752 Changing audio DTLS state from OFF
 to HANDSHAKE
Here is one of STUN Binding Request packets sent by FreeSwitch. It was
captured during other test, so its contents does not match SDP pasted
Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
User Datagram Protocol, Src Port: 24012, Dst Port: 45557
Session Traversal Utilities for NAT
    [Response In: 91]
    Message Type: 0x0001 (Binding Request)
    Message Length: 132
    Message Cookie: 2112a442
    Message Transaction ID: 364e4d535879554441354a61
    Attributes
        USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Attribute Type: USERNAME (0x0006)
            Attribute Length: 49
            Username: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Padding: 3
        PRIORITY
            Attribute Type: PRIORITY (0x0024)
            Attribute Length: 4
            Priority: 2013266428
        SOFTWARE
            Attribute Type: SOFTWARE (0x8022)
            Attribute Length: 19
            Software: FreeSWITCH ( 64bit)
            Padding: 1
        ICE-CONTROLLED
            Attribute Type: ICE-CONTROLLED (0x8029)
            Attribute Length: 8
            Tie breaker: 3967545348314432
        MESSAGE-INTEGRITY
            Attribute Type: MESSAGE-INTEGRITY (0x0008)
            Attribute Length: 20
            HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
        FINGERPRINT
            Attribute Type: FINGERPRINT (0x8028)
            Attribute Length: 4
            CRC-32: 0xbd08d97e
I hope it will help you.
I am out of ideas today. It looks that DTLS-SRTP handshake is not
started at all. I would expect that GStreamer would initiate it -
maybe it also assumes that it has server role and expects that the
other side will initiate handshake?
I have filtered dtlssrtp-related lines from log and uploaded them
here. Please take a look, maybe you will find something there. First
entry after sending remove SDP to GST is at 0:00:01.399761438.
https://pastebin.com/uMncN6jn
Ah, that's because it errors out :)

0:00:01.651764557   542      0x1b5b450 ERROR              webrtcbin gstwebrtcbin.c:2294:_set_description_task:<sendrecv> media 0 is missing or contains an empty 'mid' attribute

IIRC, mid is a required field for webrtc.

Cheers
-Matt
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).
Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
d***@poradnik-webmastera.com
2018-03-29 11:20:10 UTC
Permalink
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay
! "
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE
connection
and thus is not generally signalled through the SDP.
The same IP and port for RTCP are implied by a=rtcp-mux line above, so
this line looks redundant for me. Or am I missing something?
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found
following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer
0x7f6c10005040,
returned ok
This usually means that something's not quite right with what GStreamer
expects.
In the meantime I found
https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
. It looks that GStreamer for some reason does not perform DTSL
handshake. BTW, it seems that FreeSwitch started its part - in its log
ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
switch_rtp.c:3752 Changing audio DTLS state from OFF
 to HANDSHAKE
Here is one of STUN Binding Request packets sent by FreeSwitch. It was
captured during other test, so its contents does not match SDP pasted
Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
User Datagram Protocol, Src Port: 24012, Dst Port: 45557
Session Traversal Utilities for NAT
    [Response In: 91]
    Message Type: 0x0001 (Binding Request)
    Message Length: 132
    Message Cookie: 2112a442
    Message Transaction ID: 364e4d535879554441354a61
    Attributes
        USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Attribute Type: USERNAME (0x0006)
            Attribute Length: 49
cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Padding: 3
        PRIORITY
            Attribute Type: PRIORITY (0x0024)
            Attribute Length: 4
            Priority: 2013266428
        SOFTWARE
            Attribute Type: SOFTWARE (0x8022)
            Attribute Length: 19
            Software: FreeSWITCH ( 64bit)
            Padding: 1
        ICE-CONTROLLED
            Attribute Type: ICE-CONTROLLED (0x8029)
            Attribute Length: 8
            Tie breaker: 3967545348314432
        MESSAGE-INTEGRITY
            Attribute Type: MESSAGE-INTEGRITY (0x0008)
            Attribute Length: 20
            HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
        FINGERPRINT
            Attribute Type: FINGERPRINT (0x8028)
            Attribute Length: 4
            CRC-32: 0xbd08d97e
I hope it will help you.
I am out of ideas today. It looks that DTLS-SRTP handshake is not
started at all. I would expect that GStreamer would initiate it -
maybe it also assumes that it has server role and expects that the
other side will initiate handshake?
I have filtered dtlssrtp-related lines from log and uploaded them
here. Please take a look, maybe you will find something there. First
entry after sending remove SDP to GST is at 0:00:01.399761438.
https://pastebin.com/uMncN6jn
Ah, that's because it errors out :)
0:00:01.651764557   542      0x1b5b450 ERROR              webrtcbin
gstwebrtcbin.c:2294:_set_description_task:<sendrecv> media 0 is
missing or contains an empty 'mid' attribute
IIRC, mid is a required field for webrtc.
Thanks for hint! I have manually added "a=mid:audio0" at the end of
FreeStreamer SDP, and things moved forward. However GStreamer started
complaining about another thing:

0:00:04.290276668 9069 0x16ec450 ERROR default
webrtcsdp.c:511:_get_final_direction: Abnormal situation!

I checked source code and it turned out that GStreamer was complaining
about missing direction attribute. I added "a=sendrecv" to FreeStreamer
SDP and audio started going to FS! Hooray!

I checked RFC 4566 and found this note in description of a=sendrecv:
"If none of the attributes "sendonly", "recvonly", "inactive",
and "sendrecv" is present, "sendrecv" SHOULD be assumed as the
default for sessions that are not of the conference type
"broadcast" or "H332" (see below)."

Looks that GStreamer does not follow RFC in this case. I will log a bug
for this.

BTW, what do you think about making GStreamer more robust and making
a=mid optional, especially in case when only one stream (e.g. audio
only) is present?

Thanks for your help!

Regards,
Daniel
Post by Matthew Waters
Cheers
-Matt
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).
Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Matthew Waters
2018-03-29 11:47:20 UTC
Permalink
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Post by d***@poradnik-webmastera.com
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Hi,
I am trying to establish WebRTC connection between GStreamer and
FreeSwitch. FreeSwitch itself works - I am able to connect to it using
Blink VoIP client, and with WebRTC using Chrome+SIP.js.
I started with
https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst/webrtc-sendrecv.c
example, and modified it to talk with JS script running on node.js,
which uses SIP.js to communicate with FreeSwitch.
    pipe1 = gst_parse_launch("webrtcbin name=sendrecv "
        "audiotestsrc wave=red-noise ! queue ! opusenc ! rtpopuspay
! "
        "queue ! " RTP_CAPS_OPUS "97 ! sendrecv. ",
        &error);
v=0
o=- 8118924877098511175 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
m=audio 9 UDP/TLS/RTP/SAVPF 97
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:uAfKhCDTPsXL7o0pB1K2AZ6sTtqbmFt+
a=ice-pwd:RD0/zeks78Ix5jmeb7qKuPA14tRd1GNG
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=mid:audio0
a=fingerprint:sha-256
CA:3D:48:EC:90:39:83:2C:C4:99:4B:1E:16:0A:6E:B4:82:A4:F9:8D:2A:E5:61:0C:E2:22:AA:3A:25:53:64:A3
a=candidate:10 1 UDP 2013266428 192.168.100.20 38669 typ host
a=candidate:11 1 TCP 1015022591 192.168.100.20 9 typ host tcptype active
a=candidate:12 1 TCP 1010828287 192.168.100.20 33547 typ host tcptype
passive
a=candidate:10 2 UDP 2013266427 192.168.100.20 42780 typ host
a=candidate:11 2 TCP 1015022590 192.168.100.20 9 typ host tcptype active
a=candidate:12 2 TCP 1010828286 192.168.100.20 37994 typ host tcptype
passive
a=end-of-candidates
v=0
o=FreeSWITCH 1522134060 1522134061 IN IP4 192.168.100.20
s=FreeSWITCH
c=IN IP4 192.168.100.20
t=0 0
a=msid-semantic: WMS 2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
m=audio 19410 UDP/TLS/RTP/SAVPF 97 101
a=rtpmap:97 OPUS/48000/2
a=fmtp:97 useinbandfec=1; minptime=10; maxptime=40
a=rtpmap:101 telephone-event/8000
a=ptime:20
a=fingerprint:sha-256
95:66:C4:94:4B:DD:8B:BD:06:29:DD:0B:96:9C:C2:D2:57:81:B3:61:8A:D9:4E:42:36:17:BB:9D:E3:BF:A8:B7
a=setup:active
a=rtcp-mux
a=rtcp:19410 IN IP4 192.168.100.20
This a=rtcp line is problematic if FreeSwitch is expecting RTCP packets
to appear on port 19410.  RTCP is transmitted through the ICE connection
and thus is not generally signalled through the SDP.
The same IP and port for RTCP are implied by a=rtcp-mux line above, so
this line looks redundant for me. Or am I missing something?
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
a=ice-ufrag:lR5j1yY3DFPhRJNO
a=ice-pwd:WWDathYxuGwIKbOVTK5K9UDD
a=candidate:5323758004 1 udp 659136 192.168.100.20 19410 typ host
generation 0
a=end-of-candidates
a=ssrc:1792117166 cname:6niZnGNXemS9odOg
a=ssrc:1792117166 msid:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G a0
a=ssrc:1792117166 mslabel:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4G
a=ssrc:1792117166 label:2TtM3ayYJHI44bvP2idZ8jCOcfQVNm4Ga0
Does FreeSwitch require the msid/cname handling because that's currently
not implemented by webrtcbin.
I do not know yet, I am checking this now.
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
I send it in set-remote-description as in example code, but GST is not
able to establish SRTP session. In Wireshark capture I see that
FreeSwitch sent few STUN Binding Requests (GST responded to them) and
RTCP packets with Receiver Report and Source description.
GST debug logs are very verbose and it was hard for me to find some
useful info there. I tried to filter them a bit, and found following
0:00:01.611202908   542      0x1b5b4a0 WARN               structure
gststructure.c:1832:priv_gst_structure_append_to_gstring: No value
transform to serialize field 'srtp-key' of type 'buffer'
0:00:05.670806044   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> calling
0x7f6c10005040, pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999, size 92, offset none, offset_end none, flags 0x40
0:00:05.670825393   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4277:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
0x7f6c10005040,
pts 0:00:04.151691973, dts 0:00:04.151691973, dur
99:99:99.999999999,
size 92, offset none, offset_end none, flags 0x40
0:00:05.670856500   542      0x1bd1190 LOG            dtlssrtpdemux
gstdtlssrtpdemux.c:129:sink_chain:<dtlssrtpdemux0> pushing rtp packet
0:00:05.671088870   542      0x1bd1190 WARN             dtlssrtpdec
gstdtlssrtpdec.c:412:on_decoder_request_key:<dtlssrtpdec0> no srtp key
available yet
0:00:05.671104066   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:818:request_key_with_signal:<srtpdec0> Could not get caps
for stream with SSRC 1792117166
0:00:05.671115242   542      0x1bd1190 WARN                 srtpdec
gstsrtpdec.c:1262:gst_srtp_dec_chain:<srtpdec0> Invalid buffer, dropping
0:00:05.671154906   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdemux0:sink>
called chainfunction &sink_chain with buffer 0x7f6c10005040, returned ok
0:00:05.671166503   542      0x1bd1190 DEBUG         GST_SCHEDULING
gstpad.c:4283:gst_pad_chain_data_unchecked:<dtlssrtpdec0:sink> called
chainfunction &gst_proxy_pad_chain_default with buffer
0x7f6c10005040,
returned ok
This usually means that something's not quite right with what GStreamer
expects.
In the meantime I found
https://lists.freedesktop.org/archives/gstreamer-devel/2018-March/067319.html
. It looks that GStreamer for some reason does not perform DTSL
handshake. BTW, it seems that FreeSwitch started its part - in its log
ccba5a1c-31b9-11e8-a1d0-adc4cccadd8b 2018-03-27 14:24:30.860282 [INFO]
switch_rtp.c:3752 Changing audio DTLS state from OFF
 to HANDSHAKE
Here is one of STUN Binding Request packets sent by FreeSwitch. It was
captured during other test, so its contents does not match SDP pasted
Frame 90: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.100.20, Dst: 192.168.100.20
User Datagram Protocol, Src Port: 24012, Dst Port: 45557
Session Traversal Utilities for NAT
    [Response In: 91]
    Message Type: 0x0001 (Binding Request)
    Message Length: 132
    Message Cookie: 2112a442
    Message Transaction ID: 364e4d535879554441354a61
    Attributes
        USERNAME: cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Attribute Type: USERNAME (0x0006)
            Attribute Length: 49
cTbSKF2IVx3uDYZcJ0cCI+nuNS8eMHJn:uU7o0ZMkTQxjxEpF
            Padding: 3
        PRIORITY
            Attribute Type: PRIORITY (0x0024)
            Attribute Length: 4
            Priority: 2013266428
        SOFTWARE
            Attribute Type: SOFTWARE (0x8022)
            Attribute Length: 19
            Software: FreeSWITCH ( 64bit)
            Padding: 1
        ICE-CONTROLLED
            Attribute Type: ICE-CONTROLLED (0x8029)
            Attribute Length: 8
            Tie breaker: 3967545348314432
        MESSAGE-INTEGRITY
            Attribute Type: MESSAGE-INTEGRITY (0x0008)
            Attribute Length: 20
            HMAC-SHA1: 4ad3e5f5ef40ee8793d7c5a278043db73b36b2ca
        FINGERPRINT
            Attribute Type: FINGERPRINT (0x8028)
            Attribute Length: 4
            CRC-32: 0xbd08d97e
I hope it will help you.
I am out of ideas today. It looks that DTLS-SRTP handshake is not
started at all. I would expect that GStreamer would initiate it -
maybe it also assumes that it has server role and expects that the
other side will initiate handshake?
I have filtered dtlssrtp-related lines from log and uploaded them
here. Please take a look, maybe you will find something there. First
entry after sending remove SDP to GST is at 0:00:01.399761438.
https://pastebin.com/uMncN6jn
Ah, that's because it errors out :)
0:00:01.651764557   542      0x1b5b450 ERROR              webrtcbin
gstwebrtcbin.c:2294:_set_description_task:<sendrecv> media 0 is
missing or contains an empty 'mid' attribute
IIRC, mid is a required field for webrtc.
Thanks for hint! I have manually added "a=mid:audio0" at the end of
FreeStreamer SDP, and things moved forward. However GStreamer started
0:00:04.290276668  9069      0x16ec450 ERROR                default
webrtcsdp.c:511:_get_final_direction: Abnormal situation!
I checked source code and it turned out that GStreamer was complaining
about missing direction attribute. I added "a=sendrecv" to
FreeStreamer SDP and audio started going to FS! Hooray!
Awesome!
Post by d***@poradnik-webmastera.com
"If none of the attributes "sendonly", "recvonly", "inactive",
and "sendrecv" is present, "sendrecv" SHOULD be assumed as the
default for sessions that are not of the conference type
"broadcast" or "H332" (see below)."
Looks that GStreamer does not follow RFC in this case. I will log a
bug for this.
The RFC we're following is JSEP
(https://tools.ietf.org/html/draft-ietf-rtcweb-jsep-24) which says that
for WebRTC the offerer must add mid and direction sdp attributes.

That being said, I don't think that taking a missing direction attribute
to mean sendrecv is going to cause obvious problems.
Post by d***@poradnik-webmastera.com
BTW, what do you think about making GStreamer more robust and making
a=mid optional, especially in case when only one stream (e.g. audio
only) is present?
Could probably be done.  I don't think mid is required for anything
major until you start RTP bundling (which we don't support yet).

Cheers
-Matt
Post by d***@poradnik-webmastera.com
Thanks for your help!
Regards,
Daniel
Post by Matthew Waters
Cheers
-Matt
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by d***@poradnik-webmastera.com
Regards,
Daniel
Post by Matthew Waters
Post by d***@poradnik-webmastera.com
Any ideas how to fix this?
Depends very much on what FreeSwitch is expecting and producing :).
Cheers
-Matt
Post by d***@poradnik-webmastera.com
My FreeSwitch is installed on CentOS from official RPMs, and (mostly)
with default config. GStreamer 1.14 is compiled from sources there.
Regards,
Daniel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Loading...