Invalid Ogg file: Stream Undecodable
Open, LowPublic
Actions

Assigned To

None

Authored By

	Daniel_Mietchen
	Feb 25 2014, 10:40 AM

Description

An example is
https://commons.wikimedia.org/wiki/File:Highly-Sensitive-Quantitative-Imaging-for-Monitoring-Single-Cancer-Cell-Growth-Kinetics-and-Drug-pone.0089000.s004.ogv (archived at http://www.webcitation.org/6NeGLJfNt ), which displays fine for me if played directly via
https://upload.wikimedia.org/wikipedia/commons/9/99/Highly-Sensitive-Quantitative-Imaging-for-Monitoring-Single-Cancer-Cell-Growth-Kinetics-and-Drug-pone.0089000.s004.ogv
or - after download - in VLC.

The error seems to affect a few other files on Mediawiki sites, as per
https://encrypted.google.com/search?q=%22%28Invalid+Ogg+file%3A+Stream+Undecodable%29%22&btnG=Search&oe=utf-8 .

https://commons.wikimedia.org/wiki/Commons:Deletion_requests/File:Gmw.ogg
suggests that the issue may be related to width mismatches.

Note: There can be MANY causes for this error, the error is only a symptom. But the overall meaning is that parsing the file encountered one or more tracks that it is not expecting or cannot read. This can happen to video and/or audio files.

One common cause of this is because the file is broken, partial or in some other way not entirely compliant. It might be possible to remix the file with ffmpeg and reupload it to fix this, but sometimes its can be unsalvageable.
Another common cause seems to be coverart in a secondary track. Coverart is not handled currently (Does anyone have the technical specification of ogg coverart ?) and throws this error.

Version: unspecified
Severity: normal

Details

Reference: bz61900

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T44725 Multimedia file format support (tracking)
		Open		None	T63900 Invalid Ogg file: Stream Undecodable

Event Timeline

• bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:54 AM

• bzimport added a project: TimedMediaHandler.

• bzimport set Reference to bz61900.

Daniel_Mietchen created this task.Feb 25 2014, 10:40 AM

When I run avprobe on the file it outputs:

[theora @ 0x9e59540] 7 bits left in packet 82

which might be related.

I uploaded a new version of [[File:Highly-Sensitive-Quantitative-Imaging-for-Monitoring-Single-Cancer-Cell-Growth-Kinetics-and-Drug-pone.0089000.s004.ogv]] which was just the same file re-encoded in order so that people can use the file on commons. This bug should stay open as we should fix whatever the underlying issue is in File_ogg.

Same for https://commons.wikimedia.org/wiki/File:Irene_Rosenfeld_-_Adon_Olam.ogg

(In reply to Yuri Astrakhan from comment #3)

Same for
https://commons.wikimedia.org/wiki/File:Irene_Rosenfeld_-_Adon_Olam.ogg

That one has a bunch of warnings for avprobe. Maybe something wrong with the file:

bawolff@Bawolff-L:~$ avprobe Irene_Rosenfeld_-_Adon_Olam.ogg
avprobe version v10_alpha1-151-gaab40bb, Copyright (c) 2007-2014 the Libav developers

built on Jan 16 2014 23:34:44 with gcc 4.7 (Debian 4.7.2-5)

[flac @ 0x9df74e0] invalid sync code
[flac @ 0x9df74e0] invalid frame header
[flac @ 0x9df74e0] decode_frame() failed
Input #0, ogg, from 'Irene_Rosenfeld_-_Adon_Olam.ogg':

Duration: 00:00:10.49, start: 0.000000, bitrate: 21037 kb/s
  Stream #0.0: Audio: flac, 44100 Hz, stereo, s16

avprobe output

This file is from a friend of mine, it was encoded using VLC's wav to flac conversion. I will ask her to re-encode using some other conversion utility. The file still plays fine in Chrome. I wonder if we should have better error info though, otherwise most people wouldn't even notice it. Thanks for checking it!

Hmm, of course the other hand is utilities like avprobe can guess the size and codec (flac), and all we are using File_OGG (the part that's erroring) for is to do that.

So even if the file is slightly not to the standard or something, perhaps we should still be doing better.

(In reply to Yuri Astrakhan from comment #5)

This file is from a friend of mine, it was encoded using VLC's wav to flac
conversion. I will ask her to re-encode using some other conversion utility.

To save your friend some trouble, I just tried to use avconv to reconvert it back to ogg/flac. Since flac is lossless, that shouldn't have any quality difference.

But then I hit bug 60554... :s

Anyways, wrong length still displayed, but at least the file is playable

VLC 2.1.4 only plays the first couple seconds of the file, I think it's just broken...

Urgh that was about https://commons.wikimedia.org/wiki/File:Treadmill-Exercise-Induces-Neutrophil-Recruitment-into-Muscle-Tissue-in-a-Reactive-Oxygen-Species-pone.0096464.s002.ogv which isn't actually mentioned on this bug. Not sure whether I was given the right link to look at?

@brion, yes, this seems to have been the wrong file for this bug. Some files indeed affected by the bug are
https://en.wikipedia.org/wiki/File:Scullion-the_fruitsmelling_shop.ogg
and
https://en.wikipedia.org/wiki/File:Pink_floyd_pigs.ogg .

(In reply to Daniel Mietchen from comment #10)

@brion, yes, this seems to have been the wrong file for this bug. Some files
indeed affected by the bug are
https://en.wikipedia.org/wiki/File:Scullion-the_fruitsmelling_shop.ogg
and
https://en.wikipedia.org/wiki/File:Pink_floyd_pigs.ogg .

Ok, those two appear to contain Motion-JPEG video streams for cover images:

$ avprobe Scullion-the_fruitsmelling_shop.ogg
avprobe version 10, Copyright (c) 2007-2014 the Libav developers

built on Apr 12 2014 15:55:31 with Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)

[ogg @ 0x7fd67a002200] max_analyze_duration reached
Input #0, ogg, from 'Scullion-the_fruitsmelling_shop.ogg':

Duration: 00:00:30.04, start: 0.000000, bitrate: 48 kb/s
  Stream #0.0: Audio: vorbis, 44100 Hz, stereo, fltp, 48 kb/s
  Metadata:
    ALBUM           : Scullion
    ARTIST          : Scullion
    DATE            : 1979
    TITLE           : The Fruit Smelling Shop
    track           : 3
  Stream #0.1: Video: mjpeg, yuvj420p, 200x200 [PAR 96:96 DAR 1:1], 90k tbn
  Metadata:
    comment         : Cover (front)
    title           :

avprobe output

$ avprobe Pink_floyd_pigs.ogg
avprobe version 10, Copyright (c) 2007-2014 the Libav developers

built on Apr 12 2014 15:55:31 with Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn)

[ogg @ 0x7ff849002200] max_analyze_duration reached
Input #0, ogg, from 'Pink_floyd_pigs.ogg':

Duration: 00:00:30.03, start: 0.000000, bitrate: 75 kb/s
  Stream #0.0: Audio: vorbis, 32000 Hz, stereo, fltp, 70 kb/s
  Metadata:
    ALBUM           : Animals
    ARTIST          : Pink Floyd
    DATE            : 1977
    GENRE           : Classic Prog
    TITLE           : Pigs (Three Different Ones)
    track           : 3
    album_artist    : Pink Floyd
    COMPOSER        : Roger Waters
  Stream #0.1: Video: mjpeg, yuvj420p, 200x200 [PAR 96:96 DAR 1:1], 90k tbn
  Metadata:
    comment         : Cover (front)
    title           :

avprobe output

It's possible that's throwing us off.

Is this just a catch-all for encoding errors in files on Commons, or should we leave this open for a real technical issue with TMH?

• Gilles added a project: Multimedia.Nov 24 2014, 3:16 PM

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:36 PM

Restricted Application added subscribers: Matanya, Aklapper. · View Herald TranscriptSep 4 2015, 6:36 PM

Jdforrester-WMF removed a project: Multimedia.Sep 21 2015, 4:02 PM

zhuyifei1999 subscribed.Apr 11 2016, 1:05 AM

MarkTraceur unsubscribed.Jan 27 2017, 10:11 PM

An example is https://commons.wikimedia.org/wiki/File:Highly-Sensitive-Quantitative-Imaging-for-Monitoring-Single-Cancer-Cell-Growth-Kinetics-and-Drug-pone.0089000.s004.ogv

That does not seem to be an example anymore. However, https://en.wikisource.org/wiki/File:LibriVox_-_exoblivione_lovecraft_jp.ogg seems to be.

Aklapper merged a task: T288896: Some Ogg files show "Invalid Ogg file: Stream Undecodable".Aug 20 2021, 9:09 AM

Aklapper added a subscriber: AlexisJazz.

In T288896#7297155, @Aklapper wrote:

My computer can decode the files, so why can't MediaWiki?

Likely because your computer has different software installed than the Wikimedia servers? :)

Can you install VLC on the Wikimedia servers? :)

So I took a quick look at specifically: https://commons.wikimedia.org/wiki/File:Klaviersonate_Nr._31_As-Dur_op._110_-_I._Moderato_cantabile,_molto_espressivo.ogg from T288896: Some Ogg files show "Invalid Ogg file: Stream Undecodable"

"Stream Undecodable" is only returned by very few points

Here it is returned because after the parsing of the comments header, the framing bit/byte is not 1.

I suspect the length of the METADATA_BLOCK_PICTURE is incorrect for this image. It says the length of this METADATA_BLOCK_PICTURE header is 5b9d0100 (so 105819 bytes). If you process that you end up ending with "6c 75 5a 6b 49 41 74 34". After that should be a framing bit/byte (according to our implementation). But there isn't and honestly, it looks like the data continues after that. In addition, I think this is raw picture data, because I see null bytes in the middle of that picture data, while picture data is supposed to be base64 encoded I think (as these values are supposed to be strings). Didn't want to look deeper into it than that.

So I went through all Ogg files on Wikimedia Commons for which processing has failed. I downloaded them and run oggz validate on them. Some of them (424) failed validation. But many (1082) have passed validation and those are most curious why they failed processing. I think to make this issue more actionable, we should focus on those which are passing oggz validate but are failing here. Why and how could we improve that.

Many of these files can probably be fixed by processing them manually in some way, but each will have to be diagnosed manually.

Note that oggz validate only validates the pure muxing/framing, not the (audio/video) coding validity. For instance https://commons.wikimedia.org/wiki/File:60to0Hzin60s.ogg is clearly a broken ogg/vorbis file which doesn't play back in VLC media player. The audio buffer constantly drops (showing this in the vlc log as well). This indicates that the timestamps for the samples are incorrect (something that VLC is strict about due to its streaming heritage).

It seems that most of these files do actually have transcoded derivatives (again, the timestamping seems suspicious, because that specifically is something that ffmpeg can correct/ignore). Probably passing the files with derivatives through ffmpeg with passthrough encoding and timestamp realigning or something and then reuploading is the best strategy for those files.

OK, it looks like much better text is to run:

ffmpeg -i <filename> -f null -err_detect +crccheck+bitstream+buffer+explode+careful+compliant+aggressive -v error -xerror -

This way it looks like truly bad files are detected (which should probably be deleted), while those which just have some non-standard encodings are passed. The latter can be then remuxed with:

ffmpeg -i input.ogg  -map 0 -c copy output.ogg

Those files should then be re-uploaded.

Interesting, I found a case where the file passes oggz validation and ffmpeg does not complain. I remuxed it with ffmpeg and re-uploaded it. But Mediawiki still complains that it cannot decode it: https://commons.wikimedia.org/wiki/File:2018-10-10_kreisende_stare.ogg

This worked very well for many files. But sadly there is a bug in ffmpeg so files which include attachments (e.g., a cover image) cannot be easily remuxed: https://trac.ffmpeg.org/ticket/4591

This fails:

ffmpeg -i KR369.ogg -map 0 -c copy output.ogg

Input #0, ogg, from 'KR369.ogg':
  Duration: 01:32:53.80, start: 0.000000, bitrate: 164 kb/s
    Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 152 kb/s
    Metadata:
      ALBUM           : Küchenradio.org
      ENCODED-BY      : auphonic.com
      ARTIST          : Philip Banse
      TITLE           : KR369 Hotel Berlin
      PUBLISHER       : Küchenstud.io
      URL             : http://kuechenstud.io/
      DATE            : 2014
      GENRE           : Podcast
      RIGHTS-DATE     : 2014
      RIGHTS          : 2014 CC BY SA
      LICENSE         : http://creativecommons.org/licenses/by-sa/3.0/de/
      RIGHTS-URI      : http://creativecommons.org/licenses/by-sa/3.0/de/
      ENCODED_BY      : auphonic.com
      comment         : DocPhil, Cindy, Frau Katja und Onkel Andi zu Besuch bei Susanne DeOcampo-Herrmann, der Chefköchin im Hotel Berlin. Wir waren lange nicht mehr zusammen und hatten wie gewohnt alle auf einmal viele Fragen zur: Küchenplanung, Hotelgeschichte, Filetier-Tech
    Stream #0:1: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 500x500 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn, 90k tbc (attached pic)
    Metadata:
      comment         : Cover (front)
[ogg @ 0x55e1657f0e80] Unsupported codec id in stream 1
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
    Last message repeated 1 times

https://commons.wikimedia.org/wiki/File:KR369.ogg

OK. I made a pass over all application/ogg files. The fmpeg -err_detect command I mentioned above detected some badly broken files which I reported for deletion and they got deleted.

Other files I remuxed using ffmpeg (command above) and reuploaded them. That fixed issue for some files. But for many it didn't change a thing. I have no idea why.

And there are files which cannot be remuxed with ffmpeg because they include cover (https://trac.ffmpeg.org/ticket/4448 and https://trac.ffmpeg.org/ticket/4591). I am not sure if cover is the reason why Mediawiki fails to process them in the first place. But I got an idea that it would be nice to extract cover and use it as a thumbnail, so I made T301438.

This is what I am able to fix. If somebody knows what to do more, please continue.

List of files not fixed by ffmpeg:

List of files ffmpeg cannot remux:

Oh, and I wanted to do the same for mp3 files, but I could not because I do not have autopatrol flag which seems to be required for uploading (fixed) mp3 files.

• brion moved this task from To sort to In progress on the TimedMediaHandler board.Feb 18 2022, 6:32 PM

Aklapper merged a task: T334673: Invalid Ogg file: Stream Undecodable.Apr 13 2023, 2:59 PM

Aklapper added a subscriber: • I.

So far, there are still files with this error. What is the situation? Hurry up and solve it

• I awarded a token.Apr 20 2023, 11:39 AM

This comment was removed by • I.

TheDJ updated the task description. (Show Details)Apr 20 2023, 11:57 AM

In T63900#8795879, @I wrote:

So far, there are still files with this error. What is the situation?

@I: For every and any task, the situation is described by the description and comments in that task. No need to ever ask. Thanks.

Hurry up and solve it

@I: Please do not tell other people what they are supposed to do. I hope your line was a bad machine translation.

Invalid Ogg file: Stream UndecodableOpen, LowPublicActions

Description

Details

Related ObjectsSearch...

Event Timeline

avprobe output

avprobe output

Invalid Ogg file: Stream Undecodable
Open, LowPublic
Actions

Related Objects
Search...