Page MenuHomePhabricator
Create Task
Maniphest T58057

Tests should not rely on the existence of specific pages (be wiki agnostic)
Closed, ResolvedPublic
Actions

Description

On MobileFrontend there is a newly added test that expects the Barack Obama article to be setup in a certain way.

It fails on mine as my Barack Obama doesn't have an external links directory.
Ideally I would like to see QA tests setup pages e.g. QA_TestPage1, QA_TestPage2 (or maybe a QA namespace might make sense here) for the purpose of the test or at least be customisable to point to different pages..

Feature: Validate External Links

Scenario: External Links resolve                       # features/external_links.feature:3
  Given I am on the Barack_Obama article               # features/step_definitions/external_links_steps.rb:1
  When I expand External Links Section                 # features/step_definitions/external_links_steps.rb:5
    timed out after 5 seconds, waiting for {:id=>"External_links", :tag_name=>"span"} to become present (Watir::Wait::TimeoutError)
    ./features/step_definitions/external_links_steps.rb:6:in `/^I expand External Links Section$/'
    features/external_links.feature:5:in `When I expand External Links Section'
  And I click on the White House official website link # features/step_definitions/external_links_steps.rb:9
  Then I receive White House official website page     # features/step_definitions/external_links_steps.rb:13

Version: wmf-deployment
Severity: normal

Details

Reference
bz56057

Related Objects
Search...

Event Timeline

• bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:27 AM
• bzimport added a project: Quality-Assurance.
• bzimport set Reference to bz56057.
• bzimport added a subscriber: Unknown Object (MLST).
Jdlrobson created this task.Oct 23 2013, 6:04 PM
• Cmcmahon added a comment.Oct 23 2013, 8:05 PM

This is by design.

Browser tests in the public repo are intended to use the beta cluster, test2wiki, and/or production as targets, and we are in a position to manage the contents of those test environments. We put this in place from the very beginning to a) avoid the "works on my machine" problems b) keep the setup/teardown overhead to a minimum and c) run the tests in a complex environment that approximates production as closely as possible.

We have a few tests that create new pages where it makes sense to test creating new pages. There is no reasonable mechanism by which to delete pages in Mediawiki.

So, if you want a suite of tests to run against a local Mediawiki environment, you could do a number of things:

  • maintain a branch of tests that do not depend on particular test data being in place
  • at runtime, swap out the existing foo_page.rb files for similar files that point to the URL of your choice, override the browser.url variable, or otherwise alter the existing value for e.g. Barack_Obama
  • maintain a minimal set of test data in your local env for the tests to pass without alteration
zeljkofilipin added a comment.Oct 24 2013, 11:15 AM

I agree that each test should make sure everything it needs exists and create users, pages, files (...) if they already do not exist.

If deleting pages is hard, that is a separate problem that should be fixed.

zeljkofilipin added a comment.Jan 28 2014, 12:55 PM

Duplicate of bug #60347?

gerritbot added a comment.Apr 9 2014, 9:19 PM

Change 125008 had a related patch set uploaded by Jdlrobson:
QA: WIP Make the test environment agnostic of existing pages

https://gerrit.wikimedia.org/r/125008

gerritbot added a comment.Apr 14 2014, 8:13 PM

Change 125857 had a related patch set uploaded by Jdlrobson:
QA: Make the test environment agnostic of existing pages #2

https://gerrit.wikimedia.org/r/125857

gerritbot added a comment.Apr 14 2014, 8:13 PM

Change 125859 had a related patch set uploaded by Jdlrobson:
QA: Remove dependency on Duel Masters article

https://gerrit.wikimedia.org/r/125859

gerritbot added a comment.Apr 14 2014, 8:13 PM

Change 125861 had a related patch set uploaded by Jdlrobson:
QA: Remove dependency on "Foo bar" article

https://gerrit.wikimedia.org/r/125861

gerritbot added a comment.Apr 14 2014, 8:13 PM

Change 125864 had a related patch set uploaded by Jdlrobson:
QA: Remove dependency on Barack Obama article

https://gerrit.wikimedia.org/r/125864

gerritbot added a comment.Apr 15 2014, 5:49 PM

Change 125008 merged by jenkins-bot:
QA: Remove dependency on San Francisco article

https://gerrit.wikimedia.org/r/125008

gerritbot added a comment.Apr 15 2014, 7:34 PM

Change 125857 merged by jenkins-bot:
QA: Make the test environment agnostic of existing pages #2

https://gerrit.wikimedia.org/r/125857

gerritbot added a comment.Apr 15 2014, 7:35 PM

Change 125859 merged by jenkins-bot:
QA: Remove dependency on Duel Masters article

https://gerrit.wikimedia.org/r/125859

gerritbot added a comment.Apr 15 2014, 9:40 PM

Change 125861 merged by jenkins-bot:
QA: Remove dependency on "Foo bar" article

https://gerrit.wikimedia.org/r/125861

Jdlrobson added a comment.May 7 2014, 5:40 PM

Can someone review https://gerrit.wikimedia.org/r/125864 - getting it merged will resolve this bug as it will mean MobileFrontend has no page specific code.

gerritbot added a comment.May 7 2014, 5:54 PM

Change 125864 merged by jenkins-bot:
QA: Remove dependency on Barack Obama article

https://gerrit.wikimedia.org/r/125864

Jdlrobson added a comment.May 7 2014, 7:23 PM

Win.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL