RTMP playback fails when using videotoolbox

Operating System: macOS 10.13.6

VLC Version 3.0.3 and vlc-3.0.4-20180805-0710 nightly

Hardware: MacBook Pro 15-inch, 2017

Logs are attached in file upload, logs created from command line output with -vvv switch.

Repro step:

VLC displays only a single frame when playing back an RTMP stream encoded with Apple's VideoToolbox framework. Audio works fine.

For a server, I'm using nginx with the RTMP plugin from https://github.com/arut/nginx-rtmp-module. It can be installed with Docker (https://hub.docker.com/r/jasonrivers/nginx-rtmp/) if containers are your thing. I have not tried other servers, but can confirm other stream providers work with this setup, so I don't think the server is of consequence.

The stream is generated using HaishinKit (https://github.com/shogo4405/HaishinKit.swift). The HaishinKit repo provides a sample project you can use on iOS. The destination of the stream can be quickly set in Examples/iOS/Preference.swift (https://github.com/shogo4405/HaishinKit.swift/blob/master/Examples/iOS/Preference.swift). When the sample project is started, there are two blue buttons in the lower right of the window. The one with a white cicle starts the stream.

With HashinKit streaming I open a connection in VLC using File -> Open Network... and using rtmp://ipaddress/live/steamname. The -vvv console output is attached. The thing that stands out to me in the console output is:

[00007fc397666800] main decoder error: Timestamp conversion failed (delay 1000000, buffering 100000, bound 9000000)
[00007fc397666800] main decoder error: Could not convert timestamp 30545801958 for videotoolbox

RTMP includes some timestamp delta in the headers, but, as far as I can tell the error message VLC is producing in the console relates to a timestamp in the encoded video. As mentioned above HaishinKit is using Apple's VideoToolbox to encode the video. The relevant code is in https://github.com/shogo4405/HaishinKit.swift/blob/master/Sources/Codec/H264Encoder.swift. The encoder calls VTCompressionSessionEncodeFrame around line 250. The VTCompressionSessionEncodeFrame call requires a CMTime value as the third parameter. This seems to be the timestamp that's appearing in the console output. I've fiddled around with the VTCompressionSession parameters but haven't had any luck getting playback working in VLC.

I'm pretty sure the root of the problem is somewhere in vlc/modules/codec/videotoolbox.m. A stream created with HaishinKit will playback fine on VLC 2.x for macOS which, as far as I can tell, does not include the videotoolbox module. The stream will also playback fine on a VLC 3.0.3 install on Linux, which also does not have access to videotoolbox.

Other notes:

I've tried using OBS (https://obsproject.com) to stream and it works fine. I believe OBS is using ffmpeg to encode the video, and it decodes fine using the VLC videotoolbox module. I used tcpdump to log the network traffic and did notice that OBS sends some config information in the @setDataFrame() call as an array, where HaishinKit, and the RTMP ‘spec’ say it should be a Javascrip object.

I thought there may have been an issue with the audio and video timestamps being out of sync from the source of the stream, but commenting out the lines to send audio has the same result.

This does happen every time.

Please let me know if there is any other information I can provide that would be helpful.

log

Added attachment:

• vlc_log.txt

[flv @ 0x7fc3920bca00] invalid dts/pts combination 723984

Your muxing seems invalid.

Please share the flv file.

added Component::Demuxers label

rtmpdump output of stream that does not play in VLC

Added attachment:

• rtmpdump.flv

If I use rtmpdump the file does play in VLC. I've attached a .flv file that I got with rtmpdump. Is there another way I should be exporting the file?

Playback of the stream currently works with twitch streaming, and a few others.

Thanks!

I was able to get VLC playback to work by making a change to the client. The client was passing zero for what RTMP calls the Composition Time, which is a metadata field on the packets. Composition Time is not in the frame data.

VLC's videotoolbox.m has a function, VTSampleBufferCreate, that receives a block_t struct. That struct has local variable pts, that is calculated as:

if(!p_sys->b_poc_based_reorder && p_block->i_pts == VLC_TICK_INVALID)
  pts = CMTimeMake(p_block->i_dts, CLOCK_FREQ);
else
  pts = CMTimeMake(p_block->i_pts, CLOCK_FREQ);

I added some print lines to videotoolbox.m in the VTSampleBufferCreate function after the timeInfoArray variable is set, the results are:

[00007fd39ab0bb00] videotoolbox decoder warning: timeInfoArray duration: 33000
[00007fd39ab0bb00] videotoolbox decoder warning: timeInfoArray pts     : 0
[00007fd39ab0bb00] videotoolbox decoder warning: timeInfoArray dts     : 7499001

when the Composition Time in the RTMP packet is zero. On the client side I can see the frame buffer being sent to VLC has a pts, but no dts. In the above conditional the else case is being hit, and the pts value is being set from p_block->i_pts, which is zero.

I don't know enough about VLC's architecture to speak to what the correct solution might be, but I can see two options, 1) changing the way the block_t struct calculates i_pts when streaming RTMP or 2) make a change in videotoolbox.m.

=== 1) changing block_t I haven't gone into looking at how the block_t struct is created, but it seems like it prefers the to calculate the i_pts value from the Composition Time, rather than the video frame. I would think if the Composition Time results in a zero value it should fall back to the frame provided pts value, and if that's zero, to use the current time. But, of course, a Composition Time can legitimately be zero in a few cases (first or last frame, I don't remember). So this doesn't seem like a great option.

Another way to change block_t is to prefer the frame provided pts value. I'm assuming the client is encoding it correctly in this scenario, I can see it before it's dumped in the data buffer. But that seems like it could have some unintended consequences in other decoders.

=== 2) changing videotoolbox.m The if statement above (in videotoolbox.m) could be reworked to make sure the pts value is always non-zero. In the scenario I'm talking about here, the p_block->i_dts value is present, I would think that could be used. It could also be that the !p_sys->b_poc_based_reorder check is wrong too, I'm not sure what it is.

I know the goto response for this is probably, "the client was wrong, not our problem." But, as I mentioned above, when using decoders other than videotoolbox the stream works, even when the Composition Time is wrong. VLC has a great reputation for playing anything and it would be nice if it could also handle this admittedly wrong scenario.

=== footnote The Adobe Flash Video File Format Specification Version 10.1 section E.4.3.1 outlines what the Composition Time is.

Thanks for the details!

Could be a change to do in libavformat module handling too.

changed milestone to %Bugs paradize