Page MenuHomePhabricator
Create Task
Maniphest T63285

mediawiki-vagrant reports an error when provisioning "browsertests" role
Closed, ResolvedPublic
Actions

Description

Author: jhall

Description:
Reported on the #wikimedia-qa IRC channel, and I was able to repro locally. When a user provisions the "browsertests" role for mediawiki-vagrant, the obsolete mediawiki-selenium is deployed, rather than the newer (and expected) mediawiki_selenium Rubygem (note the underscore).

Repro steps:

  1. Start with a fresh mediawiki-vagrant install (i.e. git clone the project).
  2. cd to new "vagrant" sub-directory.
  3. vagrant up
  4. vagrant enable-role browsertests
  5. vagrant provision (this step produces the error message shown below)

As near as I can tell, the error shown below does not actually prevent the "browsertests" role from working as expected; I am still able to run browser tests from the /srv/browsertests/tests/browser directory on the VM.

Error message:

notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile
err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85

Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=50993

Details

Reference
bz61285

Event Timeline

• bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:05 AM
• bzimport added a project: MediaWiki-Vagrant.
• bzimport set Reference to bz61285.
• bzimport created this task.Feb 12 2014, 9:47 PM
• bzimport added a comment.Feb 12 2014, 9:52 PM

jhall wrote:

Complete messaging from the "vagrant provision" command is as follows:

Jeffrey-Hall:vagrant jeffreyhall$ vagrant enable-role browsertests
Ok. Run 'vagrant provision' to apply your changes.
Jeffrey-Hall:vagrant jeffreyhall$ vagrant provision
[default] Running provisioner: puppet...
Running Puppet with site.pp...
info: Applying configuration version '1392241885.c958ae3'
info: mount[files]: allowing mediawiki-vagrant access
notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile
err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85
notice: Finished catalog run in 3.76 seconds
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

FACTER_fqdn='mediawiki-vagrant.dev' FACTER_forwarded_port='8080' FACTER_shared_apt_cache='/vagrant/apt-cache/' FACTER_provider_name='virtualbox' FACTER_provider_version='4.3.6' puppet apply --templatedir /vagrant/puppet/templates --verbose --config_version /vagrant/puppet/extra/config-version --fileserverconfig /vagrant/puppet/extra/fileserver.conf --logdest /vagrant/logs/puppet/puppet.c958ae375.log --logdest console --modulepath '/tmp/vagrant-puppet-1/modules-0' --manifestdir /tmp/vagrant-puppet-1/manifests --detailed-exitcodes /tmp/vagrant-puppet-1/manifests/site.pp || [ $? -eq 2 ]

Stdout from the command:

info: Applying configuration version '1392241885.c958ae3'
info: mount[files]: allowing mediawiki-vagrant access
notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: Could not locate Gemfile
err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 10 instead of one of [0] at /tmp/vagrant-puppet-1/modules-0/browsertests/manifests/init.pp:85
notice: Finished catalog run in 3.76 seconds

gerritbot added a comment.Feb 18 2014, 5:29 PM

Change 113988 had a related patch set uploaded by EBernhardson:
Repoint bundle install cwd

https://gerrit.wikimedia.org/r/113988

gerritbot added a comment.Feb 18 2014, 5:30 PM

Change 113988 merged by jenkins-bot:
Repoint bundle install cwd

https://gerrit.wikimedia.org/r/113988

Jdforrester-WMF added a comment.Mar 11 2014, 10:30 PM

Now getting:

jdforrester@local:~/Documents/git/vagrant$ vagrant enable-role browsertests && vagrant provision
Ok. Run 'vagrant provision' to apply your changes.
[default] Running provisioner: puppet...
Running Puppet with site.pp...
info: Applying configuration version '1394576869.c7f1990'
info: mount[files]: allowing mediawiki-vagrant access
notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: /usr/lib/ruby/1.9.1/fileutils.rb:247:in `mkdir': Permission denied - /home/vagrant/.gem/ruby (Errno::EACCES)
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:247:in `fu_mkdir'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:221:in `block (2 levels) in mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `reverse_each'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `block in mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `each'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/pathname.rb:514:in `mkpath'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:15:in `run'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:8:in `install'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/cli.rb:222:in `install'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/task.rb:22:in `run'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor.rb:246:in `dispatch'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/base.rb:389:in `start'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/bin/bundle:13:in `<main>'
err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 1 instead of one of [0] at /tmp/vagrant-puppet/modules-0/browsertests/manifests/init.pp:85
notice: Finished catalog run in 4.81 seconds
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

FACTER_fqdn='mediawiki-vagrant.dev' FACTER_forwarded_port='8080' FACTER_shared_apt_cache='/vagrant/apt-cache/' FACTER_provider_name='virtualbox' FACTER_provider_version='4.2.18' puppet apply --templatedir /vagrant/puppet/templates --verbose --config_version /vagrant/puppet/extra/config-version --fileserverconfig /vagrant/puppet/extra/fileserver.conf --logdest /vagrant/logs/puppet/puppet.c7f199012.log --logdest console --modulepath '/etc/puppet/modules:/tmp/vagrant-puppet/modules-0' --manifestdir /tmp/vagrant-puppet/manifests --detailed-exitcodes /tmp/vagrant-puppet/manifests/site.pp || [ $? -eq 2 ]

Stdout from the command:

info: Applying configuration version '1394576869.c7f1990'
info: mount[files]: allowing mediawiki-vagrant access
notice: /Stage[main]/Browsertests/Env::Var[MEDIAWIKI_URL]/Env::Profile[set_mediawiki_url]/File[/etc/profile.d/puppet-managed/set_mediawiki_url.sh]/ensure: created
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: /usr/lib/ruby/1.9.1/fileutils.rb:247:in `mkdir': Permission denied - /home/vagrant/.gem/ruby (Errno::EACCES)
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:247:in `fu_mkdir'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:221:in `block (2 levels) in mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `reverse_each'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:219:in `block in mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `each'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/fileutils.rb:205:in `mkdir_p'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/1.9.1/pathname.rb:514:in `mkpath'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:15:in `run'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/installer.rb:8:in `install'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/cli.rb:222:in `install'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/task.rb:22:in `run'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor.rb:246:in `dispatch'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/lib/ruby/vendor_ruby/bundler/vendor/thor/base.rb:389:in `start'
notice: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: from /usr/bin/bundle:13:in `<main>'
err: /Stage[main]/Browsertests/Exec[install browsertests bundle]/returns: change from notrun to 0 failed: bundle install --path /home/vagrant/.gem returned 1 instead of one of [0] at /tmp/vagrant-puppet/modules-0/browsertests/manifests/init.pp:85
notice: Finished catalog run in 4.81 seconds

Stderr from the command:

jdforrester@local:~/Documents/git/vagrant$

ori added a comment.Apr 16 2014, 6:40 AM
  • Bug 63990 has been marked as a duplicate of this bug. ***
gerritbot added a comment.Apr 16 2014, 7:07 AM

Change 126207 had a related patch set uploaded by Ori.livneh:
Make sure ~vagrant/.gem is owned by vagrant:vagrant

https://gerrit.wikimedia.org/r/126207

ori added a comment.Apr 16 2014, 7:11 AM

James or Stefan, could you verify that the patch fixes things?

gerritbot added a comment.Apr 16 2014, 3:50 PM

Change 126207 merged by jenkins-bot:
Make sure ~vagrant/.gem is owned by vagrant:vagrant

https://gerrit.wikimedia.org/r/126207

• dduvall added a comment.Jun 3 2014, 6:17 PM

I believe I'm running into the same issue, even with proper ownership and permissions on ~vagrant/.gem. A manual execution of bundle install reveals a failed compilation of the ffi gem.

At first I thought this might be a missing dependency but the issue persists after installing libffi—and looking into the gem source it appears to be bundled with its own libffi so that's definitely not it. I'll do some more digging.

Output of bundle install --path /home/vagrant/.gem in /srv/browsertests/tests/browser:

Fetching source index for https://rubygems.org/
Using builder (3.2.2)
Installing ffi (1.9.3) with native extensions /usr/lib/ruby/1.9.1/rubygems/installer.rb:552:in `rescue in block in build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

/usr/bin/ruby1.9.1 extconf.rb

Gem files will remain installed in /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.3 for inspection.
Results logged to /home/vagrant/.gem/ruby/1.9.1/gems/ffi-1.9.3/ext/ffi_c/gem_make.out
(... useless rubygems backtrace)

• dduvall added a comment.Jun 3 2014, 11:08 PM

The failure I was having appears to have been an issue with the older packaged version of Bundler that is installed by puppet/modules/browsertests/manifests/init.pp. After removing it (apt-get remove --purge ruby-bundler) and installing the latest RubyGems version, I can bundle install without a problem.

A short-term workaround to this issue would be to make sure the Puppet manifest installs Bundler using the "gem" provider. However, there is a separate issue at the moment that makes this workaround a bit messy: The GEM_HOME environment variable is being set relative to the user's home directly (by our script in /etc/profile.d/puppet-managed/gem_home.sh) meaning that gems installed as root (which is what Puppet will do for a package resource) are not accessible to other users. In this case, the bundler gem installed via a Puppet package will not be accessible to the user trying to execute bundle install (the vagrant user).

A broader solution (and maybe a more correct one in my opinion) would be to better isolate ruby dependencies via something like RVM or rbenv, where multiple ruby binaries and gems can be managed independently of system ones (see bug 50993) or even independently for users or specific applications. This will allow for more up-to-date versions of ruby and gems to be installed for specific roles without affecting others.

I'll go down the second road (referencing bug 50993 as it seems more directly relevant) and, if I run into any major blockers, I'll come back and implement the short-term workaround.

Aklapper added a comment.Aug 17 2014, 11:26 AM

[No patches left here - resetting bug report status]

bd808 added a comment.Aug 17 2014, 11:48 PM

The browsertests role was removed entirely in I6a02788a42a5dcd58dc03ce845fedfe791da3bcb. If there are still issues with the various roles that enable browser tests for specific extensions a new bug should be filed.

bd808 moved this task from Backlog to Done on the MediaWiki-Vagrant board.Feb 26 2015, 12:54 AM
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 7:01 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL