They used to show up in gdash when you ticked the "Show Code Deploys" checkbox.
adding "&target=drawAsInfinite(deploy.any)" to the graphite urls doesn't work :/
Version: wmf-deployment
Severity: normal
See Also:
https://rt.wikimedia.org/Ticket/Display.html?id=6970
https://bugzilla.wikimedia.org/show_bug.cgi?id=66174
Change 119339 had a related patch set uploaded by BryanDavis:
Fix MW_STATSD_PORT to point to correct listener
After these patches land and we get some data in graphite again I think we'll need to look at the gdash configuration and update the metric names that it uses to identify deployments as well. deploy2graphite and scap send different metrics to graphite.
See https://gerrit.wikimedia.org/r/#/c/111409/ for the change from carbon to statsd that should have been accompanied by a change to the gdash configuration and port number as well.
When we figure out what all the new deploy metrics are they should be added to templates/gdash/deploy_addon.erb in oeprations/puppet.git to fix the marks added.
There is some additional problem with the current gdash configuration.
When the "Show Code Deploys" checkbox is active, something is causing the generated graphite URLs to contain an extraordinary number of superfluous ampersands. In one URL I just examined there are 4188 extra ampersands inserted between the deployment metric stanzas and the remainder of the graph description.
When these ampersands are removed from the graphite URL the graph renders (albeit with no deploy markers).
For what it's worth, I was seeing graphite urls like that (tons of &s) on Friday the 14th.
The configuration changes now have data being recorded in graphite for scap runs again, but there are three remaining issues:
Assigning to Ori in the hope that he can find some time to look into the txstatsd behavior and the missing metrics. Once those issues are fixed it should be pretty easy to correct the gdash configuration.
I think the assigning missed but I am doing it now.
Dear ori,
A ticket just for you?
Best Wishes,
Chase
:)
Change 208085 had a related patch set uploaded (by Filippo Giunchedi):
gdash: adjust deploy metrics
Somebody needs to take a look at the statsd events that scap tries to send. They really haven't ever worked properly and it seems quite likely that the packets were implemented incorrectly for txstatsd and possibly for the new replacement as well.
likely you don't need to increment and time the same metric, so https://phabricator.wikimedia.org/diffusion/MSCA/browse/master/scap/log.py;ef15380f0ffc839e64956a4c974ca280f4b660db$349 can be removed and likewise https://phabricator.wikimedia.org/diffusion/MSCA/browse/master/scap/main.py;ef15380f0ffc839e64956a4c974ca280f4b660db$265
other than that it seems fine on the surface, the deploy. metrics are working
Change 209462 had a related patch set uploaded (by Filippo Giunchedi):
gdash: fix deploy addon urls
so two additional issues identified, one fixed in https://gerrit.wikimedia.org/r/209462 the other was due to conversion from timer to simple counter to extended counter, the end result of which was metrics named the same which end up esisting as a .wsp file (simple counter) and a directory (extended counter or timer) with the former masking the latter from showing up. note that a similar root cause is probably what's causing T98380