[MPlayer-users] Parsing whole Youtube mp4 file before playing?

[Rasz]

Mplayer (libavcodec?) parses some mp4 files encoded by google/YT all
the way to the end preventing streaming. Here is an example:

--------------------------------------------
D:\_learning>youtube-dl.exe

-F
[youtube] M3IgHiWq5x4: Downloading webpage
[youtube] M3IgHiWq5x4: Extracting video information
[youtube] M3IgHiWq5x4: Downloading DASH manifest
[info] Available formats for M3IgHiWq5x4:
format code extension resolution note
140 m4a audio only DASH audio 129k , m4a_dash
container, aac @128k (44100Hz), 27.84MiB
171 webm audio only DASH audio 149k , ***@128k
(44100Hz), 24.75MiB
141 m4a audio only DASH audio 257k , m4a_dash
container, aac @256k (44100Hz), 55.89MiB
160 mp4 256x144 DASH video 115k , 15fps, video
only, 23.71MiB
278 webm 256x144 DASH video 144k , webm container,
VP9, 15fps, video only, 22.78MiB
133 mp4 426x240 DASH video 252k , 30fps, video
only, 53.20MiB
242 webm 426x240 DASH video 293k , 30fps, video
only, 44.73MiB
243 webm 640x360 DASH video 546k , 30fps, video
only, 85.39MiB
134 mp4 640x360 DASH video 634k , 30fps, video
only, 95.42MiB
244 webm 854x480 DASH video 1045k , 30fps, video
only, 157.52MiB
135 mp4 854x480 DASH video 1134k , 30fps, video
only, 187.37MiB
247 webm 1280x720 DASH video 2192k , 30fps, video
only, 326.61MiB
136 mp4 1280x720 DASH video 2262k , 30fps, video
only, 362.02MiB
248 webm 1920x1080 DASH video 3968k , 30fps, video
only, 605.60MiB
137 mp4 1920x1080 DASH video 4219k , 30fps, video
only, 668.94MiB
17 3gp 176x144
36 3gp 320x240
5 flv 400x240
43 webm 640x360
18 mp4 640x360
22 mp4 1280x720 (best)

Format 22 is ordinary mp4 video+audio file and streams perfectly
(thats what I use daily)
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 22 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A"

Format 248 is video only webm and also plays fine
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 248 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A"

Problem lies in Format 137 video only mp4. Instead of playing
immediately mplayer starts parsing this file piece by piece reading
chunks every couple of megabytes
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 137 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A" -v

....
[https @ 0000000001544180]header='Content-Range: bytes 0-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 701448022'
....
[https @ 0000000001544180]header='Content-Range: bytes
1228800-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 700219222'
....
[https @ 0000000001544180]header='Content-Range: bytes
1302528-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 700145494'
....
[https @ 0000000001544180]header='Content-Range: bytes
1957888-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 699490134'
....
[https @ 0000000001544180]header='Content-Range: bytes
4718592-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 696729430'
....
[https @ 0000000001544180]header='Content-Range: bytes
7479296-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 693968726'
....
[https @ 0000000001544180]header='Content-Range: bytes
10235904-701448021/701448022'
[https @ 0000000001544180]header='Accept-Ranges: bytes'
[https @ 0000000001544180]header='Content-Length: 691212118'
....and so on

it will eventually start playing after parsing whole file this way. I
would simply guess its one of those weird containers with metadata all
over the place and there is no way around it, except both Youtube
HTML5 and Flash video players stream this file fine without reading
all over the place. Same goes for m4a (formats 140, 141). Is this
mplayer or libavcodec issue? any way to do something about it?

[wm4]

On Fri, 5 Jun 2015 03:39:31 +0200

Post by Rasz
Mplayer (libavcodec?) parses some mp4 files encoded by google/YT all
--------------------------------------------
D:\_learning>youtube-dl.exe http://youtu.be/M3IgHiWq5x4 -F
[youtube] M3IgHiWq5x4: Downloading webpage
[youtube] M3IgHiWq5x4: Extracting video information
[youtube] M3IgHiWq5x4: Downloading DASH manifest
format code extension resolution note
140 m4a audio only DASH audio 129k , m4a_dash
(44100Hz), 24.75MiB
141 m4a audio only DASH audio 257k , m4a_dash
160 mp4 256x144 DASH video 115k , 15fps, video
only, 23.71MiB
278 webm 256x144 DASH video 144k , webm container,
VP9, 15fps, video only, 22.78MiB
133 mp4 426x240 DASH video 252k , 30fps, video
only, 53.20MiB
242 webm 426x240 DASH video 293k , 30fps, video
only, 44.73MiB
243 webm 640x360 DASH video 546k , 30fps, video
only, 85.39MiB
134 mp4 640x360 DASH video 634k , 30fps, video
only, 95.42MiB
244 webm 854x480 DASH video 1045k , 30fps, video
only, 157.52MiB
135 mp4 854x480 DASH video 1134k , 30fps, video
only, 187.37MiB
247 webm 1280x720 DASH video 2192k , 30fps, video
only, 326.61MiB
136 mp4 1280x720 DASH video 2262k , 30fps, video
only, 362.02MiB
248 webm 1920x1080 DASH video 3968k , 30fps, video
only, 605.60MiB
137 mp4 1920x1080 DASH video 4219k , 30fps, video
only, 668.94MiB
17 3gp 176x144
36 3gp 320x240
5 flv 400x240
43 webm 640x360
18 mp4 640x360
22 mp4 1280x720 (best)
Format 22 is ordinary mp4 video+audio file and streams perfectly
(thats what I use daily)
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 22 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A"
Format 248 is video only webm and also plays fine
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 248 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A"
Problem lies in Format 137 video only mp4. Instead of playing
immediately mplayer starts parsing this file piece by piece reading
chunks every couple of megabytes
D:\_learning>for /f "delims=" %A in ('youtube-dl.exe
http://youtu.be/M3IgHiWq5x4 -f 137 -g') do
@C:\Users\bofh\AppData\Roaming\smplayer\mplayer\mplayer "%A" -v
....
....
1228800-701448021/701448022'
....
1302528-701448021/701448022'
....
1957888-701448021/701448022'
....
4718592-701448021/701448022'
....
7479296-701448021/701448022'
....
10235904-701448021/701448022'
....and so on
it will eventually start playing after parsing whole file this way. I
would simply guess its one of those weird containers with metadata all
over the place and there is no way around it, except both Youtube
HTML5 and Flash video players stream this file fine without reading
all over the place. Same goes for m4a (formats 140, 141). Is this
mplayer or libavcodec issue? any way to do something about it?

libavcodec. It's because the mp4 file is "fragmented". It works if you
disable indexing.

[Rasz]

Post by wm4
libavcodec. It's because the mp4 file is "fragmented". It works if you
disable indexing.

how? idx/forceidx/noidx doesnt do anything
and like I said before both html5 and flash YT players have no problem
with those files and play them without fetching whole thing, including
quick seeking.

[wm4]

On Fri, 5 Jun 2015 11:14:05 +0200

Post by Rasz

Post by wm4
libavcodec. It's because the mp4 file is "fragmented". It works if you
disable indexing.

how? idx/forceidx/noidx doesnt do anything
and like I said before both html5 and flash YT players have no problem
with those files and play them without fetching whole thing, including
quick seeking.

Well, I don't remember how mplayer does it, but setting the
AVFMT_FLAG_IGNIDX flag should stop the libavformat demuxer
from reading the whole file. But other things break.

[Rasz]

Post by wm4
libavcodec. It's because the mp4 file is "fragmented". It works if you
disable indexing.

"fragmented" was a good clue
found http://stackoverflow.com/questions/18178405/mpeg-dash-and-fragmented-mp4
now I get it, its like hosting a bunch of mp4 files glued ass to
mouth, and mplayer insists on parsing every single one of them before
decoding anything.

I found a bug, or at least very undesirable result of the way mplayer
handles streams like that currently.
I tried VLC, it will also parse every single fragment, but VLC is
"clever" about it, it only fetches header data, 8MB out of 56MB file,
before it starts playing ("clever" in quotation marks because despite
fetching only essential data its doing it very slowly).
Mplayer on the other hand first fetches _whole_ file to parse
individual fragments .. and then it fetches that very same data again
while playing it, result is 112MB downloaded to watch 56MB stream.

It seems libavcodec is not at fault here, VLC uses it and acts differently?
I would still love to know how YT players are able to seamlessly start
playing without preloading anything :( Even better if there was a way
of incorporating it into mplayer, or at the very least copy VLC
"clever" method of only fetching essential (not that essential if YT
flash player can live without it) metadata.

[wm4]

On Fri, 5 Jun 2015 13:49:19 +0200

Post by Rasz

Post by wm4
libavcodec. It's because the mp4 file is "fragmented". It works if you
disable indexing.

"fragmented" was a good clue
found http://stackoverflow.com/questions/18178405/mpeg-dash-and-fragmented-mp4
now I get it, its like hosting a bunch of mp4 files glued ass to
mouth, and mplayer insists on parsing every single one of them before
decoding anything.

libavformat does. (Also I write libavcodec, but that was wrong. It's a
libavformat issue.)

These mp4s can most likely be read just fine without stepping through
the whole file by reading some (yet unsupported) index elements.

Post by Rasz
I found a bug, or at least very undesirable result of the way mplayer
handles streams like that currently.
I tried VLC, it will also parse every single fragment, but VLC is
"clever" about it, it only fetches header data, 8MB out of 56MB file,
before it starts playing ("clever" in quotation marks because despite
fetching only essential data its doing it very slowly).
Mplayer on the other hand first fetches _whole_ file to parse
individual fragments .. and then it fetches that very same data again
while playing it, result is 112MB downloaded to watch 56MB stream.
It seems libavcodec is not at fault here, VLC uses it and acts differently?

It might just be a difference in buffering.

Post by Rasz
I would still love to know how YT players are able to seamlessly start
playing without preloading anything :( Even better if there was a way
of incorporating it into mplayer, or at the very least copy VLC
"clever" method of only fetching essential (not that essential if YT
flash player can live without it) metadata.