Bristol Performance Sprint Post-Mortem¶
Functional benchmarking plus buildbot
I’ll start with apologies for the tardiness of this post. Predictably, things got crazy as soon as I landed back home and between catchup and holidays, it’s been tough. Thanks to Helge Tesdal for shaming me into posting finally. Fortunately, I had so much more fun at the sprint than over my holidays, the memory is still pretty fresh. :)
In fact, I couldn’t put it down. Despite running on only a few hours sleep, a massively overloaded brain, and AirCanada’s genuinely respectable selection of on-demand movies, I kept working on the buildbot integration. But I’m getting ahead of myself.
Since my last post on the performance sprint, Andrew continued to alternate between poking at the write concurrency issues and doing functional benchmark comparisons for some of the work being done at the sprint to measure impact. I really want to resume working with Andrew on the write concurrency issue. In particular I want to document what of the various performance optimizations, experimental or otherwise, have any significant impact on the performance. I had a lot of fun working with Andrew so maybe we can do some remote sprinting.
Meanwhile, I know Tom was working on the documentation in the buildout and refining the “candidate” story. The “candidate” stuff is a buildout configuration file that is intended to make it easy for developers to do comparison benchmarks to measure the impact their work has on the baseline Plone performance. I’m very excited about this work as a part of making functional benchmarking accessible and it’s definitely something we’ll want to take advantage of as we poke at the write concurrency issues.
For my part, I further refined the funkload and zope.testing integration. I also added a zc.recipe.testrunner clone to support adding the funkload test runner to a buildout. My real focus on the last day and on the flight home, however, was trying to work towards buildbot integration. I started by adding a build-diffs console script that uses “fl-build-report –diff” to generates differential reports comparing the most recent benchmark report against the previous report and against any available reports from a day, a week, a month and a year ago. The goal is to use this script in a buildbot setup, then to check the diffs generated to raise an error or a warning when performance appears to be negatively impacted by a change. Buildbot could then also be a way to access the longer term (day, week, month, year) diffs for a more long-term view of performance changes.
This next step is the actual buildbot integration. All the funkload support is there and the buildout works as needed. Getting buildbot to detect status from the diffs and to provide links to the diff reports will require some support from a buildbot step. I began researching buildbot steps on the flights home, but haven’t started work yet. Matt has a server ready and waiting when we finally do have a working buildbot setup so I’d really like to get this done.
Hopefully it won’t be as long until I get to poke at the write concurrency issue and finishing the buildbot integration as it was until this post. :)
Updated on 07 January 2009
Imported from Plone on Mar 15, 2021. The date for this update is the last modified date in Plone.
Comments
comments powered by Disqus