We should find a Jenkins plugin that should allow us to customize e-mail notification messages. The default messages are not really useful.

jhall wrote:

I used this plugin in a previous job and found it to be very flexible (among other things, it provides for customized e-mail subject strings):

https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

This is how a typical Jenkins report looks like (see attached file).

Created attachment 13911
typical Jenkins report

Attached:

If you do not have an e-mail client, this is how an e-mail notification for a failed Jenkins job looks like:

From: jenkins-no-reply@cloudbees.com
To:
Date: Tue, 26 Nov 2013 19:02:57 +0100
Subject: Build failed in Jenkins: VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox #172

See https://wmf.ci.cloudbees.com/job/VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox/172/changes

Changes:

[roan.kattouw] Share code between conflict resolution and the edit source switch

[roan.kattouw] Don't build out save dialog when initializing editor

[roan.kattouw] Use InputWidget facilities in byte limit counter

[krinklemail] Remove the toolbar tracker

Started by user Chris Mcmahon
Building remotely on s-e4476767 in workspace https://wmf.ci.cloudbees.com/job/VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox/ws/
Cloning the remote Git repository
Cloning repository https://gerrit.wikimedia.org/r/mediawiki/extensions/VisualEditor
git --version
git version 1.7.11.7
Checking out Revision 07464e20a535fe1efdbd7f8e862ff8f7b3825722 (origin/master)
[VisualEditor-en.wikipedia.beta.wmflabs.org-linux-firefox] $ /bin/sh -xe /tmp/hudson495896231200628031.sh
+ export MEDIAWIKI_USER=Selenium_user
+ MEDIAWIKI_USER=Selenium_user
+ export MEDIAWIKI_PASSWORD_VARIABLE=MEDIAWIKI_PASSWORD_SELENIUM_USER_WMFLABS_ORG
+ MEDIAWIKI_PASSWORD_VARIABLE=MEDIAWIKI_PASSWORD_SELENIUM_USER_WMFLABS_ORG
+ export BROWSER_LABEL=firefox
+ BROWSER_LABEL=firefox
+ export MEDIAWIKI_URL=http://en.wikipedia.beta.wmflabs.org/wiki/
+ MEDIAWIKI_URL=http://en.wikipedia.beta.wmflabs.org/wiki/
+ curl -s -o use-ruby https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/ruby/use-ruby
+ RUBY_VERSION=2.0.0-p247
+ source ./use-ruby
++ [[ -z 2.0.0-p247 ]]
++ ruby187='1.8.7-*'
++ [[ 2.0.0-p247 == 1.8.7-* ]]
++ BASE=/scratch/jenkins/addons
++ mkdir -p /scratch/jenkins/addons
++ '[' '!' -f /scratch/jenkins/addons/use.sh ']'
++ wget -q -P /scratch/jenkins/addons https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/lib/use.sh
++ . /scratch/jenkins/addons/use.sh
++ mkdir -p /scratch/jenkins/addons/ruby
+++ addon_architecture
+++ uname -m
++ ruby_name=ruby-2.0.0-p247-x86_64
++ '[' '!' -f /scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64.tar.bz2 ']'
+++ addon_platform
++++ head -n 1
++++ cat /etc/issue
+++ issue='Fedora release 17 (Beefy Miracle)'
+++ fedora_regex='Fedora release 17*'
+++ [[ Fedora release 17 (Beefy Miracle) == Fedora release 17* ]]
+++ echo fc17
+++ return
++ wget -q -P /scratch/jenkins/addons/ruby https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/ruby/fc17/ruby-2.0.0-p247-x86_64.tar.bz2
++ '[' '!' -f /scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin/ruby ']'
++ tar xjf /scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64.tar.bz2 -C /scratch/jenkins/addons/ruby
++ export RUBY_HOME=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64
++ RUBY_HOME=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64
++ export PATH=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin:/opt/jdk/latest/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/opt/maven/latest/bin:/opt/ant/latest/bin:/opt/android/android-sdk-linux/tools:/opt/gradle/latest/bin:/home/jenkins/bin:/opt/jdk/jdk.latest/bin
++ PATH=/scratch/jenkins/addons/ruby/ruby-2.0.0-p247-x86_64/bin:/opt/jdk/latest/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/opt/maven/latest/bin:/opt/ant/latest/bin:/opt/android/android-sdk-linux/tools:/opt/gradle/latest/bin:/home/jenkins/bin:/opt/jdk/jdk.latest/bin
++ unset GEM_PATH
++ unset GEM_HOME
++ unset BUNDLE_PATH
+ gem install bundler --no-ri --no-rdoc
Successfully installed bundler-1.3.5
1 gem installed
+ '[' -d modules/ve-mw/test/browser/ ']'
+ cd modules/ve-mw/test/browser/
+ bundle install
Fetching gem metadata from https://rubygems.org/..........
Fetching gem metadata from https://rubygems.org/..
Installing builder (3.2.2)
Installing ffi (1.9.3)
Installing childprocess (0.3.9)
Installing diff-lcs (1.2.4)
Installing multi_json (1.8.2)
Installing gherkin (2.12.2)
Installing multi_test (0.0.2)
Installing cucumber (1.3.8)
Installing i18n (0.6.5)
Installing faker (1.2.0)
Installing yml_reader (0.2)
Installing data_magic (0.16.1)
Installing json (1.8.1)
Installing net-http-persistent (2.9)
Installing page_navigation (0.9)
Installing rubyzip (1.0.0)
Installing websocket (1.0.7)
Installing selenium-webdriver (2.37.0)
Installing watir-webdriver (0.6.4)
Installing page-object (0.9.3)
Installing mime-types (2.0)
Installing rest-client (1.6.7)
Installing rspec-expectations (2.14.3)
Installing syntax (1.0.0)
Installing mediawiki-selenium (0.1.12)
Using bundler (1.3.5)
Your bundle is complete!
Use bundle show [gemname] to see where a bundled gem is installed.
+ bundle exec cucumber --verbose --profile ci --tags @en.wikipedia.beta.wmflabs.org
Code:

• features/support/env.rb
• features/support/hooks.rb
• features/support/modules/url_module.rb
• features/support/pages/cursor_test_page.rb
• features/support/pages/login_page.rb
• features/support/pages/visual_editor_page.rb
• features/step_definitions/bullets_steps.rb
• features/step_definitions/common_steps.rb
• features/step_definitions/cursor_steps.rb
• features/step_definitions/general_markup_steps.rb
• features/step_definitions/headings_steps.rb
• features/step_definitions/links_steps.rb
• features/step_definitions/references_steps.rb
• features/step_definitions/shared_steps.rb
• features/step_definitions/transclusion_steps.rb

Features:

• features/bullets.feature
• features/edit_utf8_logged_in.feature
• features/general_markup.feature
• features/headings.feature
• features/indent_outdent.feature
• features/links.feature
• features/references.feature
• features/transclusion.feature

Parsing feature files took 0m0.111s

Build step 'Execute shell' marked build as failure
Recording test results

From what I understand, the people would like the e-mail notification to look similar to this page:

https://wmf.ci.cloudbees.com/view/r-ve/job/VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome/164/

(In reply to comment #6)

From what I understand, the people would like the e-mail notification to look
similar to this page:

https://wmf.ci.cloudbees.com/view/r-ve/job/VisualEditor-en.wikipedia.beta.
wmflabs.org-linux-chrome/164/

Yeah, that would be *much* more useful. It would also be nice to still get the recent changes that were checked in like in the current mail, eg:

Changes:

[roan.kattouw] Share code between conflict resolution and the edit source
switch

[roan.kattouw] Don't build out save dialog when initializing editor

[roan.kattouw] Use InputWidget facilities in byte limit counter

[krinklemail] Remove the toolbar tracker

And if they could provide links to changesets gerrit, that would be even better :)

It would also be good if there was a direct link provided to the build in jenkins.

Hey, I'm GEOFBOT from GCI! I think I'll try out this task, but I have no clue how to start out. Where should I start out?

Hi Geoffrey and welcome!

For future reference, if you get stuck you can ask questions here, at #wikimedia-qa freenode IRC channel or at QA mailing list[1].

To answer your question, start with either installing Jenkins[2] on your local machine or create a free account at a hosted Jenkins service, like Cloudbees[3].

When you have a test Jenkins instance, create a few jobs and start sending e-mail notifications to your e-mail.

Then install the plugin[4] Jeff has suggested and try changing the e-mail notifications so they look more like this report[5].

If you get stuck, or if you have any questions, let us know.

1: https://lists.wikimedia.org/mailman/listinfo/qa
2: http://jenkins-ci.org/
3: http://www.cloudbees.com/
4: https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
5: https://wmf.ci.cloudbees.com/view/r-ve/job/VisualEditor-en.wikipedia.beta.wmflabs.org-linux-chrome/164/

Cloudbees URL: https://geofbot.ci.cloudbees.com/job/test/

Geoffrey, this might be the problematic part: "Have your Jenkins administrator place the script inside JENKINS_HOME\email-templates." (from plugin documentation[1]).

You should contact cloudbees support and ask how to do it.

1: https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

Progress!

I managed to extract the first line of console output to find who started the job. It took a lot of guesswork (I have never coded in Groovy before, especially not within a HTML :P)

Geoffrey, feel free to attach the e-mail to this bug, or to send it to me, so I could take a look.

Progress on email-ext customization

attachment progress1.html ignored as obsolete

First version of jenkins template ready for testing

1. To use this template, place it under $JENKINS_HOME/email-templates/
2. make sure email-ext plugin is installed
3. Under the job config for post-build actions, remove "email notification" and add "editable email notification"
4. Change default content type to html
5. Change body to ${SCRIPT, template="wikimedia-jenkins.template-1"}
6. Optional - change the subject using the token references; here is the one i used: ${PROJECT_NAME} ${BUILD_NUMBER} - ${BUILD_STATUS}
7. Set the recipients, etc.
8. Open Advanced Settings... and there you can change when an email notification is triggered.

attachment wikimedia-jenkins-1.template ignored as obsolete

Geoffrey, can you attach an example e-mail, so we can see if the template does what we need?

Example html 2

I don't have tests set up on my jenkins, so unfortunately I couldn't test that aspect of the template

I replaced all of the LAN image references with a cloudbees link so they would work, but the links to the build will be broken.

attachment progress2.html ignored as obsolete

Template v1.1

Better version with some bugs ironed out

Would be nice if someone tested out the "tests" part of it because I don't have tests set up for my local jenkins build.

attachment wikimedia-jenkins.template ignored as obsolete

Example html 3

The example html for template 1.1

attachment progress3.html ignored as obsolete

Geoffrey,

I did not want to mess up existing job (test), so I have created another one (test2). Short documentation on how to set up a job to run tests is available[1]. I have cloned browsertests[2] repository and the job runs tests from login.feature[3] file. I will attach e-mail that was sent after the job failed.

1: https://github.com/wikimedia/mediawiki-selenium/blob/master/docs/job.md
2: https://github.com/wikimedia/qa-browsertests
3: https://github.com/wikimedia/qa-browsertests/blob/master/features/login.feature

Created attachment 13979
e-mail notification with failed tests

Attached:

I forgot to say, now you can test e-mail notifications using your jobs. If you have any questions, let me know.

Geoffrey,

see attached file on how an e-mail notification after failed test should look like. It is generated by this test:

https://geofbot.ci.cloudbees.com/job/test2/lastCompletedBuild/testReport/

Created attachment 13990
how e-mail notification after failed test should look like

Attached:

(In reply to comment #24)

Created attachment 13990 [details]
how e-mail notification after failed test should look like

Do you want the entire table there in the email? Right now I have the notifications working with everything and it looks pretty close to the jenkins build page.

Attached:

Geoffrey, the last e-mail that I got from your local Jenkins looked good to me! Please upload the template here so I can test it with our Jenkins and if it works I will close the task.

If you would like to continue working on the template (making it look more like Jenkins web page) I will create another task.

If you would like to work on something else, we have plenty of tasks.

Final email template (for now)

Attached:

(In reply to comment #26)

If you would like to continue working on the template (making it look more
like
Jenkins web page) I will create another task.

Sure, I would definitely like to finish up this template.

(In reply to comment #15)

Created attachment 13967 [details]
First version of jenkins template ready for testing

1. To use this template, place it under $JENKINS_HOME/email-templates/
2. make sure email-ext plugin is installed
3. Under the job config for post-build actions, remove "email notification"

and
add "editable email notification"

4. Change default content type to html
5. Change body to ${SCRIPT, template="wikimedia-jenkins.template-1"}
6. Optional - change the subject using the token references; here is the one

i
used: ${PROJECT_NAME} ${BUILD_NUMBER} - ${BUILD_STATUS}

7. Set the recipients, etc.
8. Open Advanced Settings... and there you can change when an email

notification is triggered.

If you want the subject to be constant for one build, make the subject just ${PROJECT_NAME}

attachment wikimedia-jenkins-1.template ignored as obsolete

Asked cloudbees to upload the template:

https://cloudbees.zendesk.com/requests/14940

As far as I am concerned, this bug is resolved. If you need more/less information in Jenkins e-mail notification, leave a comment. See attached e-mail for "before" and "after".

Created attachment 14047
before

Attached:

Created attachment 14048
after

Attached:

Change 103114 had a related patch set uploaded by Sn1per:
Add jenkins email template for better email notifications

https://gerrit.wikimedia.org/r/103114

Change 103114 merged by Cmcmahon:
Add jenkins email template for better email notifications

https://gerrit.wikimedia.org/r/103114

Change 106513 had a related patch set uploaded by Zfilipin:
Use new e-mail template

https://gerrit.wikimedia.org/r/106513

Change 106513 merged by Cmcmahon:
Use new e-mail template

https://gerrit.wikimedia.org/r/106513

I have another way to resolve.

I can generate an HTML file and send it as an email body.

Changing email body programmatically is possible now.

You can go to Manage Jenkins > Configure System and set Default Content as
           ${FILE,path="email-body.html"}​​​​​​​

           under Extended E-mail Notification area.


Now, you just have to generate email-body.html file programmatically directly under your workspace root.

Also, make sure that default content type is set to HTML.

src: https://softwaretestingboard.com/qna/2205/modify-email-subject-jenkins-using-automated-script-written