Page MenuHomePhabricator

Mailman mailing list archiver truncates if a line begins with "From"
Closed, ResolvedPublic

Description

http://lists.wikimedia.org/pipermail/wikitech-l/2010-September/049543.html is unexpectedly truncated. The full version of the message is available at http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/49879/focus=49886

Perhaps the wikitech-l archive needs to be rebuilt or something? The full message definitely went through to the mail server; it appears that the only place it's not showing up properly is in that particular pipermail archive link.


Version: unspecified
Severity: normal

Details

Reference
bz25231

Related Objects

StatusSubtypeAssignedTask
Resolved MZMcBride
ResolvedNone
ResolvedNone
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedRobH
ResolvedDzahn
ResolvedRobH
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
Resolved JohnLewis
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
DuplicateDzahn
ResolvedDzahn
ResolvedDzahn
DuplicateDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
InvalidDzahn
ResolvedDzahn
ResolvedDzahn
DeclinedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
Resolved JohnLewis
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 11:12 PM
bzimport set Reference to bz25231.
bzimport added a subscriber: Unknown Object (MLST).

I think this is related to

https://bugs.launchpad.net/mailman/+bug/266068 ("From" at line start fools archiver, loses/splits message body)

Another one: http://lists.wikimedia.org/pipermail/foundation-l/2011-February/063760.html

Updating the bug summary. I looked at the launchpad.net link (and from it the Debian bug tracker link), but I don't see the issue. Everyone seems to call this "upstream."

  • Bug 27043 has been marked as a duplicate of this bug. ***

http://mail.python.org/pipermail/mailman-users/2010-November/070550.html seems to indicate that a mailman upgrade (to at least 2.1.13) would fix this issue. Anyone know which version is currently deployed?

Then, from looking at http://packages.ubuntu.com/search?keywords=mailman, I'm guessing lists runs Ubuntu Hardy. If you want the mailman package upgraded, I'd recommend asking them for the next LTS release, Lucid: http://releases.ubuntu.com/lucid/

Adding upstream tag + fixing as INVALID since we can't help this, and it seems to have been already in the latest version.

I'd say ask someone (or file a bug) to update mailman and/or to "Lucid"

(In reply to comment #8)

Adding upstream tag + fixing as INVALID since we can't help this, and it seems
to have been already in the latest version.

I'd say ask someone (or file a bug) to update mailman and/or to "Lucid"

What? Who's running the latest version?

Re-opening this, as the problem isn't solved (and it certainly isn't an "invalid" bug). The bug for updating mailman is listed as a dependency of this bug already (bug 22708).

Another possibly related (unescaped characters?) bug: bug 22973.

After mailman has been updated, I guess rebuilding the archives would fix the bug?

(In reply to comment #11)

After mailman has been updated, I guess rebuilding the archives would fix the
bug?

After mailman has been updated, this bug needs to be tested again to see if the behavior still exists. If the truncation has stopped, this bug can be resolved. If the truncation is still happening, this bug needs further investigation and would remain open/unresolved.

Rebuilding the archives (to fix this or other issues) would be a separate bug.

The examples seem to be working now (Guessing a rebuild happened during the move?) but can't confirm against gname since it appears to be down.

(In reply to comment #13)

The examples seem to be working now (Guessing a rebuild happened during the
move?) but can't confirm against gname since it appears to be down.

Which examples seem to be working?

It seems pipermail is now prefixing lines that begin with "From" with a ">", because it's still 1983: http://lists.wikimedia.org/pipermail/wikitech-l/2012-September/063361.html.

This seems to be fixed in mailman 2.1.16, see https://lists.openstreetmap.org/pipermail/talk/2015-March/072415.html where a space was added before the "From". I'm still seeing the bug (i.e. fake quoting by prefixing >) in our archives (mailman 2.1.13). Both have pipermail 0.09.

This should be fixed now since we are now on mailman 2.1.18. Wanna confirm?

I'm not sure how to replicate, for instance https://lists.wikimedia.org/pipermail/wiki-research-l/2015-February/004150.html was not broken in the first place.

Of course regenerating the archives is not a good idea ;) ; I guess the best bet is just to send an exact duplicate of the test case to any list.

@MZMcBride any comments now after the upgrade?

@MZMcBride any comments now after the upgrade?

I haven't sent a paragraph starting with "From" to a Wikimedia mailing list since the upgrade. Do we have a test list?

MZMcBride set Security to None.

Given that the truncation is no longer happening and that we can always re-open this task or file a new one if "From" goofiness continues to happen, I'll mark this task as resolved.

we can always re-open this task or file a new one if "From" goofiness continues to happen, I'll mark this task as resolved.

full ack, thank you:)