I am super excited about beta labs. It would be awesome if we could get beta labs also mimicking the production setup for mobile/MobileFrontend so we can start doing more accurate pre-deployment testing outside of production.
Version: unspecified
Severity: normal
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T42605 Supporting MobileFrontend on beta labs | |||
| Resolved | hashar | T46041 [OPS] adapt varnish mobile puppet class on beta |
I have done a lot of the work on beta with the assistance of the operations team for anything related to puppet. I have zero knowledge about the mobile architecture and currently focusing on Jenkins for most of October.
I guess you will want to team up a developer with mobile frontend knowledge with someone from op and have them setup the instances in the deployment-prep labs project (that is beta). I will be glad to offer some guidance as to startup the project sanely, but cant really dedicate any amount of time on that during october.
Max pointed out on IRC the other day that even getting MobileFrontend enabled on beta labs (without the infrastructural components to mirror production) would be better than nothing and would, I imagine, be pretty straightforward to start with.
As for getting the infrastructure to mirror production, I believe Patrick Reilly and Asher Feldman have the most intimate knowledge of things are set up in prod.
In order to make Beta Labs scale from a staffing perspective, we're moving away from a model where you ask Antoine to set something up, and he does all of the magical black box stuff to make it happen. At this stage, Antoine is playing the role for Beta that Mark Bergsma plays for production. Deploying to Beta is more and more looking like the same process that we use to deploy to production. There are definitely differences, but Antoine can guide you on those issues.
Let us know who needs access and training, and Antoine can work with that person to set it up.
sumanah wrote:
Summary: Michelle ought to look at this issue to decide how/whether to push forward in the next Mobile iteration.
QA's highest priority in terms of new things on the beta cluster is MobileFrontend. The blocker is that infrastructure is not in place on beta the way it is on prod.
First we have to enable the extension & enbable subdomains -- basically, everything we had to do for Wikivoyage. That's Extension:MobileFrontend, the m. subdomain, and so on. This is already well-documented on wikitech.wikimedia.org -- I assume that's https://wikitech.wikimedia.org/view/MobileFrontend/Deployment and so on.
Then we also have to ensure that we replicate the cache layer. Mobile requests go to the squids first (which have device detection), and if we want to serve them the mobile-optimized site then the requests go to the mobile varnish cluster which has MF caching. Only if the page isn't in varnish does it hit the Apaches. We need that full stack to feel confident that the environment is production-like and thus worth using for testing. The puppet scripts for the varnish & squid bits are already in place (I confirmed this today via a conversation with Daniel Zahn & Leslie Carr) per http://doc.wikimedia.org/puppetsource/manifests/site.pp . So it's feasible for us to use those puppet scripts to replicate the squid layer and the varnish layer in beta.
The relevant WMF folks who'd work on this are Michelle (mobile QA) and possibly Arthur and Max (domain knowledge of MF/varnish/squid setup). The next mobile sprint is the first or second week of January. So it's up to Michelle whether this is high-priority enough to make this a story she'd add to that backlog.
This seems like a good investment to me. This is a complex enough project that a) it will likely serve as a model for future enhancement to beta labs and b) will likely put infrastructure in place that later enhancements to labs will use.
There is one challenge that I know of relating to squid configuration that we need to figure out. Squid ACLs (which are used for detecting a mobile device and routing the request to varnish) live in a git repository on Fenari. They are not in a public repo, so I'm not sure what is the best way to manage this.
I propose that we schedule some exploratory time to finish answering unanswered questions and make sure we have the resources we need lined up during our upcoming sprint (18 Dec - 2 Jan), and then possibly slate actually getting MobileFrontend working in beta labs for our following sprint (2 Jan - 14 Jan).
What's the status of this bug? Is Mobile any closer to being able to do reliable testing on betalabs?
mgrover wrote:
Hey Greg, we're almost done. There's some code that Antoine is waiting for a code review on and we have a few configuration glitches but other than that we're pretty close http://en.m.wikipedia.beta.wmflabs.org/ is the site.
mgrover wrote:
MobileFrontend is now configured (http://en.m.wikipedia.beta.wmflabs.org)
The items we would like to finish in Phase II