In the wikitext editor, you can put an image inside another image's caption. You can also do that in the visual editor. So far so good.
If you try to double nest images in the wikitext editor (for example, [[ File:Example.jpg | thumb | [[ File:Example.jpg | thumb | [[ File:Example.jpg | thumb ]] ]] ]]) things will break:
However, you can do the same thing in the visual editor and it appears to work fine:
It's only once you save that you realize you've produced broken wikitext. Ideally, we'd somehow prevent the user from doing something like this in the first place.