[09:11] < bd808> Sir_Designer_: Do you have any guess about how long you had been working on the form before you hit submit when you got the missing request token error submitting for a Scholarship? We see that error in the logs several times per day and my only guess so far is that PHP has garbage collected the server side session due to inactivity while the user is working on their answers.
[09:12] < bd808> I haven't had any reports of people not being able to submit on retry so I haven't worked too hard to reproduce the problem. It is annoying though.
[09:14] <Sir_Designer_> bd808 20 minutes with onr back as i hit return inadvertently and it chided me that i did not finish
[09:14] <Sir_Designer_> gotta run.
[09:14] < bd808> Thanks for the data. That matches my guess about session garbage collection.
[09:14] < bd808> I'll open a bug for next year :)

The guess about session cleanup is just that... a guess.

If that is the problem it would be possible to add a javascript heartbeat callback that would keep the server side session alive while people were parked on the application form. Pinging back to the server every 5 minutes or so should be enough to keep the session alive. The client side script could even be really smart and only make the pings if there had been mouse/keyboard activity since the last ping went out.

An alternate solution might be to fire a quick ajax request on form submit that validated that the XSRF token was still good and if not acquired a new one and wrote it into the form before letting the browser continue to submit.

@NiharikaKohli has started a patch for this at https://gerrit.wikimedia.org/r/#/c/179471/. After talking it over a bit on the patch and irc I think she's going to try and make a solution that is a combination of the two solutions that I proposed above. Namely to periodically ping the server with an ajax call and fetch a new xsrf token either each time or in the event that the server side session has timed out.

Change 179471 had a related patch set uploaded (by BryanDavis):
Ping server to keep the session alive

https://gerrit.wikimedia.org/r/179471

Patch-For-Review

Change 179471 merged by jenkins-bot:
Ping server to keep the session alive

https://gerrit.wikimedia.org/r/179471

2015-01-19 logs have multiple occurrences of the "Missing or invalid CSRF token" message.

12 of the logged errors had a null CSRF token posted; 13 contained tokens that did not match the expected value.

I am having the same problem today and it is not clear how to complete the form without getting this error

In T63269#1031448, @Jarekt wrote:

I am having the same problem today and it is not clear how to complete the form without getting this error

This horrible process *should* work:

• Copy your long form answers to a text buffer somewhere (gedit, notepad, vim, whatever)
• Do a hard reload of the submission form to ensure you get a new CSRF token in your session
• Fill the form out in <5 minutes
• Submit

This is horrible user experience. We tried to make a bandaid for the underlying problem but it obviously did not stick. Next year we should move sessions into redis or memcached which should eliminate the semi-random expiration problem.

So we know for sure that the bug isn't fixed? Do we have a reliable way to reproduce and test?

In T63269#1837110, @Nemo_bis wrote:

So we know for sure that the bug isn't fixed?

Yes.

Do we have a reliable way to reproduce and test?

No. I've been able to reproduce this randomly a couple of times, but no reliable way yet, I'd say.

Maybe this is a dumb question, but why does the scholarships application app need a CSRF check? It doesn't look like there's any account or authentication system involved in applying (from looking at https://wikimania-scholarships.wmflabs.org/apply).

In T63269#2010158, @kaldari wrote:

Maybe this is a dumb question, but why does the scholarships application app need a CSRF check? It doesn't look like there's any account or authentication system involved in applying (from looking at https://wikimania-scholarships.wmflabs.org/apply).

Its not a dumb question. Fundamentally it has a CSRF check because any constructive action accessed via HTTP should be forced to use POST and also be reasonably guarded against cross-site request forgery attacks in my opinion. All of the constructive forms in the application follow these rules. It would be possible to remove the CSRF protection token validation from the application form itself, but that adds configuration complexity to the application and introduces a possibility of a a future attack vector when and if the act of applying for a scholarship expands the attack surface to include activities which you would consider to be worth protecting.

I'm pretty sure the root of the bug is crappy PHP session state handling. We rely on the PHP runtime entirely for this and Ubuntu hasn't historically done a great job of managing the purging of on disk PHP session state files. Moving sessions to memcached/redis would probably make it all go away.