Page MenuHomePhabricator

Correctly answer to OPTIONS method (Squids appear to reject CORS OPTIONS query before it ever gets to the API)
Closed, ResolvedPublic

Description

Steps to reproduce:

  1. Go to https://de.wikipedia.org
  2. Execute the following code:

var xhr = new XMLHttpRequest();
xhr.upload.addEventListener('progress', function () {}, false);
xhr.withCredentials = true;
xhr.open('GET', 'https://commons.wikimedia.org/w/api.php?action=tokens&type=edit&origin=https://de.wikipedia.org&format=json', true);
xhr.onreadystatechange = function() {

console.log(xhr.responseText);

}
xhr.send();

This sends a request using the OPTIONS method with header
Access-Control-Request-Method: GET
This is correct according to the specs as I just learned:
http://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html -> set force preflight flag
http://dvcs.w3.org/hg/cors/raw-file/tip/Overview.html#resource-preflight-requests

The server answers with

HTTP/1.1 403 Forbidden

with a header
X-Squid-Error: ERR_ACCESS_DENIED 0


Version: wmf-deployment
Severity: minor
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=44921

Details

Reference
bz41731

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 1:09 AM
bzimport set Reference to bz41731.

drogus wrote:

I try to make such request against wiktionary with similar results. Easy way to reproduce is to use curl:

curl -v -X OPTIONS "http://en.wiktionary.org/w/api.php?action=opensearch&search=a&format=json?origin=http:%2F%2Fen.wikipedia.org" -H "Origin: http://en.wikipedia.org"

I checked also other origins with no luck.

The code in comment #0 does cross script request so unlikely to ever succeed.

Works when using the syntax suggest in gerrit 9624

$.ajax( {
'url': 'https://commons.wikimedia.org/w/api.php',
'data': {

		'action': 'tokens',
		'type': 'edit',
		'format': 'json',
		'origin': 'https://de.wikipedia.org'

},
'xhrFields': {

		'withCredentials': true

},
'success': function( data ) {

		console.log( data );

}
} );

No idea, what the different to your code is. (I do not know how ajax() is implemented and if that makes a different).

(In reply to comment #2)

The code in comment #0 does cross script request so unlikely to ever succeed.

It is a cross script request, yes, but with the line
xhr.withCredentials = true;
it will succeed as commons.wikimedia.org accepts CORS requests from de.wikipedia.org.

(In reply to comment #3)

No idea, what the different to your code is. (I do not know how ajax() is
implemented and if that makes a different).

There is no real difference, just a bit more jQuery-overhead. But with jQuery you can't add event listeners to xhr.upload (well, you can, if you really want to use uggly hacks).

Low priority: Workaround in comment 3, plus Squid will get superseded by Varnish.

I'm not very familiar with CORS, but isn't OPTIONS requests only needed if you're using a method other than GET, HEAD, or POST or setting a caching header?

I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia (However, I have never read the spec, and just skimmed right now for where the word OPTIONS is used, so I may misunderstand).


(In reply to comment #5)

Low priority: Workaround in comment 3, plus Squid will get superseded by
Varnish.

The upload varnishes give a 403 for options, so the transition to varnish probably doesn't matter.

drogus wrote:

I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia

CORS is needed if you want to do any XHR request from the different domain from the browser.

So it depends if you treat using API directly from the browser a valid use case, I would argue that it would be nice to have it working.

(In reply to comment #7)

I don't think OPTIONS are needed for any valid use case for CORS on Wikimedia

CORS is needed if you want to do any XHR request from the different domain
from
the browser.

So it depends if you treat using API directly from the browser a valid use
case, I would argue that it would be nice to have it working.

I mean options should not be needed if you're only doing GET, HEAD or POST with CORS, even if from the browser. So you shouldn't need OPTIONS from the browser (I think. Not a cors expert)

jgonera wrote:

Bawolff, OPTIONS is needed if you want to bind to the upload progress event on POST AJAX requests. We want to use it in MobileFrontend to show a progress bar for image uploads.

See https://bugzilla.wikimedia.org/show_bug.cgi?id=44921 for details.

Is there any potential downside in just letting Squid accept OPTIONS and pass it through?

Indeed, our Squid config blocks OPTIONS requests. This is definitely deliberate, with OPTIONS being explicitly singled out and seems to be there for many years.
I'll ask around to find out the rationale behind it -- please do as well, Brion might remember :) We need to find out soon anyway, since we're the Varnish switch is imminent.

acl options method OPTIONS
[...]
<? if ( $upload ): ?>

Deny HTTP methods other than GET/HEAD

http_access deny !gethead
<? else: ?>

Deny HTTP OPTIONS method requests

http_access deny options
<? endif ?>

What is the use case for this BTW? I know this is for CORS preflight requests, but what specifically? (from which domain to which, what it's going to be used for etc.)

(In reply to comment #11)

What is the use case for this BTW? I know this is for CORS preflight
requests,
but what specifically? (from which domain to which, what it's going to be
used
for etc.)

Uploading a file to Commons while the user is on some other domain (*.wikipedia.org or whatever), and showing upload progress messages to the user.

I want to use it in [[de:Benutzer:Schnark/js/screenshot.js]], a script to generate screenshots and uploading them directly, to show a progress bar while the screenshot is uploaded (this already works if you upload locally).
According to comment 9, Juliusz Gonera wants to do the same in MobileFrontend.

If you don't bind to the upload progress event, no OPTIONS request is needed, but then you can't show progress bars to users.

(In reply to comment #11)

Indeed, our Squid config blocks OPTIONS requests. This is definitely
deliberate, with OPTIONS being explicitly singled out and seems to be there
for
many years.
I'll ask around to find out the rationale behind it -- please do as well,
Brion
might remember :) We need to find out soon anyway, since we're the Varnish
switch is imminent.

If i recall correctly, we put that in to block all the WebDAV requests hitting the Squids from also busting through to the Apaches, for no need.

I suppose we can open up the OPTIONS method for the API (only).

I can confirm that mobile web want to do exactly the same as Michael M. Thanks for looking into this guys.

(In reply to comment #13)

(In reply to comment #11)

Indeed, our Squid config blocks OPTIONS requests. This is definitely
deliberate, with OPTIONS being explicitly singled out and seems to be there
for
many years.
I'll ask around to find out the rationale behind it -- please do as well,
Brion
might remember :) We need to find out soon anyway, since we're the Varnish
switch is imminent.

If i recall correctly, we put that in to block all the WebDAV requests
hitting
the Squids from also busting through to the Apaches, for no need.

I suppose we can open up the OPTIONS method for the API (only).

I've just deployed the following change in the Squid config:

-http_access deny options
+http_access deny options !api_php

So Squid should accept OPTIONS now, for ^/w/api.php requests only.

I'll work on a corresponding change for Varnish as well.

Varnish now accepts the OPTIONS method as well, as long as a corresponding Origin request header is present for CORS.

Thanks so much for doing this Mark. I can't seem to get it working though...

I tried this on enwiki with the following steps:

mw.config.set( 'wgMFAjaxUploadProgressSupport', true )

  • Attempted to upload an image

The ajax request failed:

OPTIONS https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814 Origin https://en.m.wikipedia.org is not allowed by Access-Control-Allow-Origin. load.php?debug=false&lang=en&modules=jquery%2Cmediawiki%2CSpinner%7Cjquery.triggerQueueCallback%2Cl…:128
XMLHttpRequest cannot load https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814. Origin https://en.m.wikipedia.org is not allowed by Access-Control-Allow-Origin.

This is the request url info:

Request URL:https://commons.m.wikimedia.org/w/api.php?useformat=mobile&r=0.5664375186897814
Request Method:OPTIONS
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Access-Control-Request-Headers:accept, content-type
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
Host:commons.m.wikimedia.org
Origin:https://en.m.wikipedia.org
Pragma:no-cache
Referer:https://en.m.wikipedia.org/wiki/Special:Uploads
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36

Am I doing something wrong?

(In reply to comment #17)

Am I doing something wrong?

I can confirm that my code from comment 0 now does work (and it does work with POST instead of GET, too). But I can confirm that my script has problems, too, and I'm not sure whether I resolved them now or not (it might be just my browser cache).

Try to add the origin parameter in the URL even for POST requests, instead of the body. This seems to work.

It does look like the origin has to be in the url not the body. Confirmed.

(In reply to comment #19)

It does look like the origin has to be in the url not the body. Confirmed.

To quote the documentation:

origin - When accessing the API using a cross-domain AJAX request (CORS), set this to the originating domain.

This must be included in any pre-flight request, and therefore must be part of the request URI (not the POST body).

...