Page MenuHomePhabricator

browser test "Given I am logged in" doesn't detect failure, must wait and check for success
Closed, ResolvedPublic

Description

The last Flow IE10 runs against beta labs failed "Create new topic logged in.Add new Flow topic".

The reported failure in https://wmf.ci.cloudbees.com/view/flow/job/Flow-en.wikipedia.beta.wmflabs.org-internet-explorer-10/157/testReport/%28root%29/Create%20new%20topic%20logged%20in/Add_new_Flow_topic/ is something about

And the talk and contrib links are not visible

But if you watch the screencast https://saucelabs.com/jobs/d4e55826d29841aea42661ff8acdbea7 , things have gone wrong long before this: the user is NOT logged in! The test has "Given I am logged in" and the test does fill in the login form, but the Flow page still shows "Log in" in the top right instead of "Selenium user", and the newly created topic is authored by an anonymous IP.

The mediawiki_selenium gem's login_with() function must actually check the page displayed after submission (usually the Main_Page) for a successful login and clearly report if login failed. Currently it makes no checks to see if login succeeded.

As to how it failed, there doesn't seem to be any delay or waiting after visit(LoginPage).login_with(...) , so maybe the browser test jumped to visit FlowPage before the login page has gone through form submission, login to CentralAuth, redirect to login.wikimedia.org, setting loginwiki token cookie, redirect back to some success page, etc. that give the browser the info to assert that it is logged in on the next page it visits. I just did a beta labs login and it took 13 seconds, but the Sauce Labs movie jumped from the login form straight to the Flow page in less than a second.


Version: unspecified
Severity: major

Details

Reference
bz61584

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:50 AM
bzimport set Reference to bz61584.

Agreed. I noticed this had started happening recently for IE, and I added a check that the login completes to the shared login method with this commit: https://gerrit.wikimedia.org/r/#/c/114096

I believe that Flow is using the shared login. However, we need to update the gem to pick up this change.

Change 114755 had a related patch set uploaded by Zfilipin:
Updated to the latest released version of mediawiki_selenium Ruby gem

https://gerrit.wikimedia.org/r/114755

Change 114755 merged by Cmcmahon:
Updated to the latest released version of mediawiki_selenium Ruby gem

https://gerrit.wikimedia.org/r/114755

Should be resolved with the latest mediawiki_selenium gem