Page MenuHomePhabricator

Respect X-Forwarded-For only from trustworthy sources
Closed, DeclinedPublic

Description

We unconditionally respect the X-Forwarded-For header that gets fed into
kraken's machineries. Regardless of whether the client IP is a trusted one,
or it is not a trusted one. This distorts our reports/graphs.

Instead, we should only respect the X-Forwarded-For header for the client IPs
in $wgSquidServersNoPurge in wmf-config/squid.php of operations/mediawiki-config.


Version: unspecified
Severity: normal

Details

Reference
bz54783

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:27 AM
bzimport set Reference to bz54783.
bzimport added a subscriber: Unknown Object (MLST).

Although Ops only seem to trust $wgSquidServersNoPurge in
wmf-config/squid.php of operations/mediawiki-config (private Email with
Faidon, and afterwards analytics-internal), the Wikipedia Zero team also
seem to trust the X-Forwarded-For header also from Opera proxies (private
emails that lead up to:

https://raw.github.com/wikimedia/metrics/5fb67552555c32e4cd4b08b6c4d4ec264b07351f/pageviews/zero/pageview_zero.png

).

We currently trust XFF from all SSH and OPERA IPs listed at http://meta.wikimedia.org/wiki/Zero:-OPERA

SSH is not yet handled by the Zero since all partners use DPI for whitelisting, effectively ignoring HTTPS traffic

(In reply to comment #3)

We currently trust XFF from all SSH [...]

I do not know SSH in this context. What does it stand for?

(I don't understand what SSH means here either)

squid.php is what MediaWiki considers as trusted as an XFF source (e.g. what would appear on IP edits). Ops doesn't have such whitelists -- apart from the very unusual & special Zero detection, we don't care about the values of XFF, so far.

I think Analytics should move into a direction that makes sense for you from an analytics perspective (e.g. you even be able to tell us what other large proxies exist out there, purely by analyzing the request stream :) and we might find a way to converge such info in the future.

nice find in the old tickets! See also: T120121

We should move all proxies out of the zerowiki and into metawiki. This will allow a much more transparent management of the proxy ips, and won't as easily get stale.

We should move all proxies out of the zerowiki and into metawiki. This will allow a much more transparent management of the proxy ips, and won't as easily get stale.

T89838 ! :)

So I've linked above that we have a separate task already about moving Zero's trusted proxy lists to metawiki, for more-transparent / community management, which is what Varnish trusts for external XFF-setting proxies in the general case now (Zero or not) - T89838

We also have a separate task about improving Varnish's XFF code in the general case (not being fooled by corner cases, etc) - T120121

Separately, there's the issue of MediaWiki's own separate handling of XFF in the general MediaWiki case, and specifically how it handles things here in our WMF installation... is this ticket about either of those things now (which are certainly complex things we could do better at, but is there specific work for teams in mind here)? Or is this basically now an off-topic ticket going nowhere?

Or is this basically now an off-topic ticket going nowhere?

My money's on this, closing. In Analytics we just use the Client IP thing set by Varnish from XFF