Page MenuHomePhabricator
Create Task
Maniphest T56393

Add Sikuli to the machines that run browser tests
Closed, DeclinedPublic
Actions

Description

Sikuli ( http://www.sikuli.org ) is a tool for testing desktop interactions, such as mouse clicks and keyboard typing.

This tool is needed for testing typing in different languages, because the usual WebDriver tests are not able to generate proper keyboard events that completely emulate typing in advanced input methods for Japanese, Chinese and Indic scripts. We have many bugs in this area in the VisualEditor (e.g. Bug 52716), so we need proper regression testing.

This week I worked with Željko in the WMF office and we built a prototype test that uses Sikuli to test typing in Japanese:
https://gerrit.wikimedia.org/r/#/c/83966/

This works on Željko's laptop. The next step is to get it working on the virtual clients that run our continuous integration tests.

Thank you!

Version: wmf-deployment
Severity: enhancement

Details

Reference
bz54393

Related Objects

Event Timeline

• bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:05 AM
• bzimport added projects: Quality-Assurance, I18n.
• bzimport set Reference to bz54393.
• bzimport added a subscriber: Unknown Object (MLST).
Amire80 created this task.Sep 20 2013, 6:26 PM
hashar added a comment.Sep 20 2013, 8:30 PM

Moving from CI to QA

gerritbot added a comment.Dec 17 2013, 1:55 PM

Change 83966 had a related patch set uploaded by Zfilipin:
[WIP] [Browser test] Testing typing in VisualEditor

https://gerrit.wikimedia.org/r/83966

hashar added a comment.Dec 17 2013, 2:09 PM

Good news: sikuli has been packaged for Debian/Ubuntu. There is a package libsikuli-script-java which seems to provide the needed material:

http://packages.ubuntu.com/precise/all/libsikuli-script-java/filelist

I have no clue how to launch the tests though. Maybe we just have to use /usr/bin/sikuli ?

gerritbot added a comment.Dec 17 2013, 2:14 PM

Change 102141 had a related patch set uploaded by Hashar:
contint: install libsikuli-script-java for browser tests

https://gerrit.wikimedia.org/r/102141

hashar added a comment.Dec 17 2013, 2:21 PM

I have deployed the package libsikuli-script-java on the integration-selenium-driver.pmtpa.wmflabs instance. The script is available as /usr/bin/sikuli

Could you craft a simple change that would attempt to run a single browser test using sikuli ?

gerritbot added a comment.Dec 17 2013, 2:27 PM

Change 102141 merged by ArielGlenn:
contint: install libsikuli-script-java for browser tests

https://gerrit.wikimedia.org/r/102141

Matanya added a comment.Jan 26 2014, 1:36 PM

Amir, this patch was merged, can this be closed?

Amire80 added a comment.Jan 26 2014, 1:42 PM

No, it is not really active yet.

zeljkofilipin added a comment.Apr 3 2014, 3:05 PM

After further investigation with Amir today, looks like Sikuli does not work with headless machines: "The system running Sikuli scripts or any apps using SikuliX features must have a real screen connected. So called headless systems are not supported (Java restriction)."

1: http://www.sikulix.com/quickstart.html

zeljkofilipin added a comment.Jun 28 2014, 8:53 PM

Resolving. See comment #9 for details. Please reopen if you think Sikuli is the way to go.

gerritbot added a comment.Jul 29 2014, 12:49 PM

Change 83966 abandoned by Zfilipin:
[WIP] [BrowserTest] Testing typing in VisualEditor

Reason:
Sikuli is not the way to go. We will investigate another tool.

https://gerrit.wikimedia.org/r/83966

gerritbot added a comment.Oct 7 2014, 2:32 PM

Change 165204 had a related patch set uploaded by Zfilipin:
contint: Sikuli is no longer used anywhere

https://gerrit.wikimedia.org/r/165204

gerritbot added a comment.Oct 10 2014, 7:30 PM

Change 165204 merged by Dzahn:
contint: Sikuli is no longer used anywhere

https://gerrit.wikimedia.org/r/165204

• Mattflaschen-WMF subscribed.Mar 4 2015, 4:16 AM

Did anyone investigate using it headless with Xvfb? People have gotten this working with Linux and Jenkins (see for example https://answers.launchpad.net/sikuli/+question/248704#comment-1 ). Their setup differs, but it shows a fully automated headless installation can work.

• Mattflaschen-WMF mentioned this in T90884: Investigate using the sikuli-like Applitools framework for visual testing.Mar 4 2015, 4:18 AM
zeljkofilipin added a comment.Mar 4 2015, 8:57 AM

@Mattflaschen the last time I have checked sikuli web page, it explicitly said running headless is not supported. Maybe something changed in the meantime.

• Mattflaschen-WMF added a comment.Mar 6 2015, 5:04 AM
In T56393#1087507, @zeljkofilipin wrote:

@Mattflaschen the last time I have checked sikuli web page, it explicitly said running headless is not supported. Maybe something changed in the meantime.

I think it depends on how you define "headless". Sometimes it means "without anything related to Xorg whatsoever", but oftentimes it just means "without a normal X server installation and associated drivers" or "without a physical monitor/physical video card".

https://answers.launchpad.net/sikuli/+question/248704#comment-1 is quite clear that he got it working on Amazon Web Services (presumably Elastic Compute Cloud) and Jenkins. This is pretty similar to our setup, except we use OpenStack instead of AWS (of course there are other differences, but there doesn't seem to be any unsolvable issue).

xvfb (Jenkins plugin) is an established tool for this, so the Launchpad comment makes sense.

• Mattflaschen-WMF added a comment.Mar 6 2015, 5:07 AM

You can also find other people discussing using Sikuli with xvfb, e.g. https://github.com/jatalahd/SikuliRobotLibrary/wiki/Hints-and-Tips

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL