Page MenuHomePhabricator

mediawiki-vagrant virtual machine should be ready for running Selenium tests
Closed, ResolvedPublic

Description

mediawiki-vagrant sets up a virtual machine ready for mediawiki development:

http://www.mediawiki.org/wiki/Vagrant

It would be great if it also installed Ruby, Firefox and checked out browsertests repository, so the developers could run Selenium tests:

https://gerrit.wikimedia.org/r/#/admin/projects/qa/browsertests


Version: unspecified
Severity: normal

Details

Reference
bz46920

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 1:15 AM
bzimport set Reference to bz46920.
bzimport added a subscriber: Unknown Object (MLST).

(In reply to comment #0)

It would be great if it also installed Ruby, Firefox and checked out
browsertests repository, so the developers could run Selenium tests:

Ruby 1.8.7 is already available -- do you need a newer version?

Also, I'd rather not make Firefox available on the guest VM. This requires installing X11 and managing its configuration too. Is it sufficient to have Firefox running on the host?

If Selenium needs additional ports opened, that could be done too.

Yes, we need at least Ruby 1.9.

http://www.ruby-lang.org/en/downloads/ says it is available in Debian/Ubuntu via "sudo apt-get install ruby1.9.1"

Is there a way to control Firefox on the host machine from the virtual machine? Selenium tests need to drive _a_ browser. Firefox driver is the simplest to install, that is why I have asked for Firefox.

If you want to keep mediawiki-vagrant VM small and clean, could we create a separate VM just for development of Selenium tests?

(In reply to comment #2)

Yes, we need at least Ruby 1.9.

http://www.ruby-lang.org/en/downloads/ says it is available in Debian/Ubuntu
via "sudo apt-get install ruby1.9.1"

Yes, I can confirm that it is available on Ubuntu Precise, which is what the VM is running.

All package installation and configuration is done via Puppet and organized into modules, which encapsulate all the configurations and packages needed for fitting the machine for some particular purpose. You can see the current set of modules in puppet/modules.

Because Selenium tests require more than just ruby, it makes sense to add an additional module, 'selenium', which will contain everything required for using the VM for browser tests.

We don't need to worry just yet about whether the module should be loaded by default or whether the user should explicit enable selenium test support. (I'm in favor of making it part of the default, but I'm also worried about protracting the initial setup even further, when it already takes quite a while.) The work of puppetizing the various dependencies will be the same regardless.

Is there a way to control Firefox on the host machine from the virtual
machine?
Selenium tests need to drive _a_ browser. Firefox driver is the simplest to
install, that is why I have asked for Firefox.

Yes, the guest VM can access the host as though the two computers were two independent machines connected by a network link.

If you want to keep mediawiki-vagrant VM small and clean, could we create a
separate VM just for development of Selenium tests?

Puppet and Vagrant both provide abstractions for assigning to machines optional roles. It'd be nice if you could merge any Puppet work you do back to the MediaWiki-Vagrant repository. I think it'd be worthwhile to develop some common interface for provisioning specialized MediaWiki instances. Labs does this quite successfully.

https://gerrit.wikimedia.org/r/61729 (Gerrit Change I18825437e1f0ad005b1e4f6db850968603573b01) | change APPROVED and MERGED [by Ori.livneh]

Works quite well now. A patch adding a friendly and helpful README to the module root would be most welcome :)

Related URL: https://gerrit.wikimedia.org/r/62148 (Gerrit Change I07d12300de3c31c667f2b6ca274b7c737acd4dd5)

Something is wrong:

vagrant@mediawiki-vagrant:/srv/browsertests$ bundle exec cucumber features/login.feature
Using the default profile...
FFF-----------------------

(::) failed steps (::)

Unable to pick a platform for the provided browser (RuntimeError)
./features/step_definitions/login_steps.rb:2:in `/^I am at Log in page$/'
features/login.feature:5:in `Given I am at Log in page'

Failing Scenarios:
cucumber features/login.feature:7 # Scenario: Go to Log in page

6 scenarios (1 failed, 5 skipped)
24 steps (1 failed, 23 skipped)
1m2.917s

I am not sure what I am doing wrong. I have the latest versions of virtualbox and vagrant on mac os x 10.8.3.

"vagrant up" outputs a few warnings, but mediawiki at http://127.0.0.1:8080/ seems to work fine.

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'precise-cloud'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 80 => 8080 (adapter 1)
[default] Running any VM customizations...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] The guest additions on this VM do not match the installed version of
VirtualBox! In most cases this is fine, but in rare cases it can
cause things such as shared folders to not work properly. If you see
shared folder errors, please update the guest additions within the
virtual machine and reload your VM.

Guest Additions Version: 4.1.12
VirtualBox Version: 4.2
[default] Setting hostname...
[default] Configuring and enabling network interfaces...
[default] Mounting shared folders...
[default] -- /vagrant
[default] -- /tmp/vagrant-puppet/manifests
[default] -- /tmp/vagrant-puppet/modules-0
[default] Running provisioner: puppet...
Running Puppet with site.pp...
stdin: is not a tty
warning: Could not retrieve fact fqdn
warning: Host is missing hostname and/or domain: mediawiki-vagrant
info: Applying configuration version '1368535903'
notice: /Stage[first]/Apt/Exec[add wikimedia apt key]/returns: executed successfully
notice: /Stage[first]/Apt/File[/etc/apt/sources.list.d/wikimedia.list]/ensure: defined content as '{md5}23e6b3219c190324ccfb207bd0b14829'
info: /Stage[first]/Apt/File[/etc/apt/sources.list.d/wikimedia.list]: Scheduling refresh of Exec[apt-get update]
notice: /Stage[first]/Apt/Apt::Ppa[git-core/ppa]/Exec[add-apt-repository --yes ppa:git-core/ppa]/returns: executed successfully
info: /Stage[first]/Apt/Apt::Ppa[git-core/ppa]/Exec[add-apt-repository --yes ppa:git-core/ppa]: Scheduling refresh of Exec[apt-get update]
notice: /Stage[first]/Apt/Exec[apt-get update]: Triggered 'refresh' from 2 events
notice: /Stage[main]Package[python-pip]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Browsertests/Package[ruby-bundler]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Browsertests/Package[ruby1.9.1-full]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Browsertests/Package[firefox]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Memcached/Package[memcached]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Misc/File[/etc/update-motd.d/51-cloudguest]/ensure: removed
notice: /Stage[main]/Misc/Package[ack-grep]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Misc/Package[toilet]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Misc/Package[htop]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Misc/File[/etc/profile.d/color.sh]/ensure: defined content as '{md5}c5f113f55b64c1c0afaf71e0a3ac31af'
notice: /Stage[main]/Misc/File[/home/vagrant/.bash_aliases]/ensure: defined content as '{md5}e9dbc4ef75a87f849b657f01d329caa4'
notice: /Stage[main]/Mediawiki::Php/Package[php5-mysql]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php5-gd]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php5-memcached]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php-apc]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php5-mcrypt]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php5-xdebug]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php5]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Php/Package[php-pear]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mysql/Package[mysql-server]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mysql/Exec[set mysql password]/returns: executed successfully
notice: /Stage[main]/Misc/File[/etc/update-motd.d/10-help-text]/ensure: removed
notice: /Stage[main]/Browsertests/File[/etc/profile.d/mediawiki-url.sh]/ensure: defined content as '{md5}5aaa6d68169b6c352562e0ccd9b386a9'
notice: /Stage[main]/Mediawiki/Exec[set mediawiki install path]/returns: executed successfully
notice: /Stage[main]/Misc/File[/etc/update-motd.d/60-mediawiki-vagrant]/ensure: defined content as '{md5}25e3ee14dbc19b357707fac39c43b4a6'
info: /Stage[main]/Misc/File[/etc/update-motd.d/60-mediawiki-vagrant]: Scheduling refresh of Exec[update motd]
notice: /Stage[main]/Misc/Exec[update motd]: Triggered 'refresh' from 1 events
notice: /Stage[main]/Apache/Package[libapache2-mod-php5]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Browsertests/Exec[use ruby 1.9.1]/returns: executed successfully
notice: /Stage[main]/Apache/Package[apache2]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Apache/File[/var/www/favicon.ico]/ensure: defined content as '{md5}b1d335c7c87c3d9b0693db6a5dc69d7b'
notice: /Stage[main]/Mediawiki::Apache/Apache::Site[default]/Exec[disable default]/returns: executed successfully
info: /Stage[main]/Mediawiki::Apache/Apache::Site[default]/Exec[disable default]: Scheduling refresh of Service[apache2]
notice: /Stage[main]/Apache/File[/etc/apache2/conf.d/disable-sendfile]/ensure: defined content as '{md5}41d3afffd151814a6f66956b47ef8a7a'
info: /Stage[main]/Apache/File[/etc/apache2/conf.d/disable-sendfile]: Scheduling refresh of Service[apache2]
notice: /Stage[main]/Mysql/File[/home/vagrant/.my.cnf]/ensure: defined content as '{md5}8326476b52ff635907d8d73794e512b7'
notice: /Stage[main]
Exec[update profile]/returns: executed successfully
notice: /Stage[main]/Misc/File[/etc/update-motd.d/50-landscape-sysinfo]/ensure: removed
notice: /Stage[main]/Mediawiki::Apache/Apache::Site[devwiki]/File[/etc/apache2/sites-available/devwiki]/ensure: defined content as '{md5}f4ffdd1e14390b20d4023c5fba65ff21'
notice: /Stage[main]/Mediawiki::Apache/Apache::Site[devwiki]/Exec[enable devwiki]/returns: executed successfully
info: /Stage[main]/Mediawiki::Apache/Apache::Site[devwiki]/Exec[enable devwiki]: Scheduling refresh of Service[apache2]
notice: /Stage[main]/Apache/File[/etc/apache2/ports.conf]/content: content changed '{md5}38b31d20326f3640a8dfbe1ff5d1c4ad' to '{md5}a5e8260a8200b74fa79bd2faad7ffa89'
info: /Stage[main]/Apache/File[/etc/apache2/ports.conf]: Scheduling refresh of Service[apache2]
notice: /Stage[main]/Mediawiki/File[mediawiki-vagrant logo]/ensure: defined content as '{md5}f9afb7d19e39247820bc1e81883a96f7'
notice: /Stage[main]/Misc::Virtualbox/File[/bin/update-guest-additions]/ensure: defined content as '{md5}f89508064a40d69a402f6acaef0a516f'
notice: /Stage[main]/Misc::Virtualbox/File[/etc/profile.d/check-guest-additions.sh]/ensure: defined content as '{md5}22a10967dd853052217aea06a788e7c2'
notice: /Stage[main]/Misc::Virtualbox/File[/etc/virtualbox-version]/ensure: created
notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/content: content changed '{md5}58c9e04b29e08c2e9b3094794d3ebd0e' to '{md5}85d9091ce522552c620bbd32a6fb849f'
info: /Stage[main]/Memcached/File[/etc/memcached.conf]: Scheduling refresh of Service[memcached]
notice: /Stage[main]/Memcached/Service[memcached]: Triggered 'refresh' from 1 events
notice: /Stage[main]/Mediawiki::Apache/Apache::Mod[rewrite]/Exec[a2enmod rewrite]/returns: executed successfully
info: /Stage[main]/Mediawiki::Apache/Apache::Mod[rewrite]/Exec[a2enmod rewrite]: Scheduling refresh of Service[apache2]
notice: /Stage[main]/Apache/Service[apache2]: Triggered 'refresh' from 5 events
notice: /Stage[main]/Mediawiki::Phpsh/File[/etc/phpsh]/ensure: created
notice: /Stage[main]/Mediawiki::Phpsh/File[/etc/profile.d/phpsh.sh]/ensure: defined content as '{md5}fd71176fa4430a42d54b59c65f8794cd'
notice: /Stage[main]/Mediawiki::Phpsh/Package[exuberant-ctags]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Mediawiki::Phpsh/Package[phpsh]/ensure: created
notice: /Stage[main]/Mediawiki::Phpsh/File[/etc/phpsh/rc.php]/content: content changed '{md5}4be8174040e048f012fcee91c62f19f7' to '{md5}eb167331f1c9094d205906ebd129a770'
notice: /Stage[main]/Mediawiki::Php/Package[php5-intl]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Git/Package[git]/ensure: ensure changed 'purged' to 'latest'
notice: /Stage[main]/Browsertests/Git::Clone[qa/browsertests]/Exec[git clone qa/browsertests]/returns: Cloning into '/srv/browsertests'...
notice: /Stage[main]/Browsertests/Git::Clone[qa/browsertests]/Exec[git clone qa/browsertests]/returns: executed successfully
notice: /Stage[main]/Browsertests/File[/srv/browsertests/config/secret.yml]/ensure: defined content as '{md5}cabd87cb5a69581a57c3595793f6c77f'
notice: /Stage[main]/Browsertests/Exec[bundle install]/returns: executed successfully
notice: /Stage[main]/Mediawiki/Exec[check settings]/returns: executed successfully
info: /Stage[main]/Mediawiki/Exec[check settings]: Scheduling refresh of Exec[mediawiki setup]
notice: /Stage[main]/Mediawiki/Exec[mediawiki setup]/returns: executed successfully
notice: /Stage[main]/Mediawiki/Exec[mediawiki setup]: Triggered 'refresh' from 1 events
notice: /Stage[main]/Mediawiki/Exec[configure phpunit]/returns: executed successfully
notice: /Stage[main]/Mediawiki/Exec[require extra settings]/returns: executed successfully
notice: /Stage[main]/Browsertests/Mediawiki::User[Selenium_user]/Exec[mediawiki user Selenium_user]/returns: executed successfully
notice: /Stage[main]/Git/Package[git-review]/ensure: ensure changed 'purged' to 'latest'
info: Creating state file /var/lib/puppet/state/state.yaml
notice: Finished catalog run in 793.83 seconds

If I uncomment "include role::browsertests" in puppet/manifests/site.pp and run "vagrant provision" I get a couple of warnings.

$ vagrant provision
[default] Running provisioner: puppet...
Running Puppet with site.pp...
warning: Could not retrieve fact fqdn
warning: Host is missing hostname and/or domain: mediawiki-vagrant
info: Applying configuration version '1368536937'
notice: Finished catalog run in 2.65 seconds

I am a bit confused about puppet/manifests/browsertests/README saying I should edit puppet/manifests/extras.pp. (The file does not exist.) Is that correct?

When I ssh into vm, I get this:

$ vagrant ssh -- -X
Welcome to Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-39-generic x86_64)

(...)

  • MediaWiki is located at /vagrant/mediawiki.
  • To interactively evaluate PHP and MediaWiki code, run 'phpsh'.
  • For a database shell, run 'mysql'.
  • For help, visit https://www.mediawiki.org or MediaWiki-General on chat.freenode.net.

    System load: 0.06 Processes: 84 Usage of /: 4.4% of 39.36GB Users logged in: 0 Memory usage: 46% IP address for eth0: 10.0.2.15 Swap usage: 0% IP address for eth1: 10.11.12.13
  • System restart required ***

Your VirtualBox guest additions are out-of-date. Please run 'update-guest-additions'.


WARNING! Your environment specifies an invalid locale.
This can affect your user experience significantly, including the
ability to manage packages. You may install the locales by running:

sudo apt-get install language-pack-UTF-8
  or
sudo locale-gen UTF-8

To see all available language packs, run:

apt-cache search "^language-pack-[a-z][a-z]$"

To disable this message for all users, run:

sudo touch /var/lib/cloud/instance/locale-check.skip

When I run the tests, I get this:

vagrant@mediawiki-vagrant:~$ cd /srv/browsertests
vagrant@mediawiki-vagrant:/srv/browsertests$ bundle exec cucumber features/login.feature
You are using an old or stdlib version of json gem
Please upgrade to the recent version by adding this to your Gemfile:

gem 'json', '~> 1.7.7'

Using the default profile...
FFF-----------------------

(::) failed steps (::)

Unable to pick a platform for the provided browser (RuntimeError)
./features/step_definitions/login_steps.rb:2:in `/^I am at Log in page$/'
features/login.feature:5:in `Given I am at Log in page'

Failing Scenarios:
cucumber features/login.feature:7 # Scenario: Go to Log in page

6 scenarios (1 failed, 5 skipped)
24 steps (1 failed, 23 skipped)
1m1.817s

I have to go now, I will try upgrading VirtualBox guest additions and installing language-pack-UTF-8 later today. Maybe that will help.

Related URL: https://gerrit.wikimedia.org/r/63724 (Gerrit Change Ie5a7c2cb04ec21ba4cfcc8c6a4b23cf0d0db3792)