https://www.rpatterson.net/Ross Patterson's Blog - Posts tagged buildout2023-08-22T15:29:28.024899+00:00ABloghttps://www.rpatterson.net/blog/sea-sprint-2012-report-out/Sea Sprint 2012 Report-Out2012-09-26T00:00:00+00:00Ross Patterson<section id="sea-sprint-2012-report-out">
<blockquote>
<div><p>A Python snob’s adventures in JS testing land.</p>
</div></blockquote>
<p>I was surprised and excited to receive an invite to the first Sea
Sprint from <a class="reference external" href="http://plone.org/author/cbc">Chris Calloway</a>. When I read that it was also being put
on by Andrew Leeb and Ian Anderson of <a class="reference external" href="http://andersonleeb.com/">Anderson Leeb, Inc.</a> I knew I
wanted to go. I’d had the chance to hang out with them at the <a class="reference external" href="http://weblion.psu.edu/symposium">Plone
Symposium East 2012</a> and the <a class="reference external" href="http://www.coactivate.org/projects/pre-pse12-strategicesque-sprintacular/project-home">sprints before</a> and <a class="reference external" href="http://weblion.psu.edu/symposium/schedule/sprints">after</a> and
enjoyed their company a lot. When I heard I could get my airfare
sponsored, I knew I absolutely had to go. Not to geek out, but this
was the first sprint travel sponsorship and I’m pretty excited about
that.</p>
<section id="location-and-hosting">
<h2>Location and Hosting</h2>
<p>This was one of the better organized sprints I’ve been to. Flights
were coordinated to arrive and depart within 1-2 hours of each other
making rides around arrival and departure fast and easy. We got to
<a class="reference external" href="http://en.wikipedia.org/wiki/Oak_Island,_North_Carolina">Oak Island</a>, a beautiful and very long barrier island not far from
the border with South Carolina, Friday afternoon and had some time to
go to the beach and go for a quick swim.</p>
<p>I’d never been to the atlantic coast on either side nor had I been to
the Carolinas before and have always wanted to. Living on the north
Pacific for so long, I’ve never swam in warm ocean water before, very
interesting. The two beach houses were <em>right on the beach</em> with deck
walks extending out to the dunes and a covered outside area. The beds
were comfy and there were great views all the way around. It was
off-season but the temperatures were still warm to temperate so there
were no crowds and great weather.</p>
<p><a class="reference external" href="http://plone.org/author/cbc">Chris Calloway</a> is simply an <em>amazing cook</em>. He made breakfast,
lunch, and dinner for 14 people almost entirely by himself both
Saturday and Sunday. As if the scope of that weren’t enough, these
meals were top-notch. Seriously, the food really has to be called out
here. Thanks so much Chris!</p>
</section>
<section id="the-sprint">
<h2>The Sprint</h2>
<p>After a delicious dinner at a local Thai restaurant sponsored by Dylan
Jay, <a class="reference external" href="http://glicksoftware.com/">David Glick</a> did an excellent job facilitating the planning of the
sprint work. This was the most efficient sprint planning I’ve been
to, great job David! Before we went to bed Friday night, we all knew
what we were working on the next day.</p>
<p>My biggest hope for my participation in this sprint was to get better
at <a class="reference external" href="http://en.wikipedia.org/wiki/JavaScript">JavaScript</a>/<a class="reference external" href="http://en.wikipedia.org/wiki/ECMAScript">ECMASCript</a> while still contributing towards the
goals of the sprint. I’ve been confounded by JS much more than I’d
like and have never really been able to develop the proficiency I’d
like with it. Since I learn best by writing tests with a good
<a class="reference external" href="http://docs.python.org/library/unittest.html">testing framework</a>, I decided to work on writing JS unit tests for
the bits of Deco and the toolbar that are JS heavy:</p>
<ul class="simple">
<li><p><a class="reference external" href="http://pypi.python.org/pypi/plone.app.tiles">plone.app.tiles</a></p></li>
<li><p><a class="reference external" href="http://pypi.python.org/pypi/plone.app.deco">plone.app.deco</a></p></li>
<li><p><a class="reference external" href="http://pypi.python.org/pypi/plone.app.toolbar">plone.app.toolbar</a></p></li>
</ul>
<p>I started working with <a class="reference external" href="http://blog.fourdigits.nl/robgietema/robgietema">Rob Gietema</a>, someone whose mad JS skills
I’ve always admired, and <a class="reference external" href="http://claytron.com/blog">Clayton Parker</a>, who is much in the same
boat as I when it comes to JS skills. Rob introduced us to
<a class="reference external" href="http://busterjs.org/">Buster.JS</a>, a new cross-browser JS unit testing framework that
everyone seems to prefer to use. Clayton began writing tests for
<a class="reference external" href="http://pypi.python.org/pypi/plone.app.tiles">plone.app.tiles</a> JS and I began looking into ways to integrate the
Buster.JS tests into our general testing story.</p>
<p>In Plone land, our <a class="reference external" href="http://www.buildout.org">buildouts</a> do a pretty good job of serving two
testing purposes:</p>
<dl class="simple">
<dt>Developers</dt><dd><p>Deployed locally and give individual developers a <a class="reference external" href="http://pypi.python.org/pypi/zope.testrunner">test runner</a>
they can use to run some or all of the tests before they commit
changes.</p>
</dd>
<dt>Continuous Integration</dt><dd><p>Deployed on our <a class="reference external" href="http://jenkins-ci.org/">CI server</a> to run all tests and catch regressions
and interactions.</p>
</dd>
</dl>
<p>I set my sights on providing something that served the same two purposes
for JS unit tests.</p>
<p>When we started, running the <a class="reference external" href="http://busterjs.org/">Buster.JS</a> tests was a fairly manual
process and done in a very different way than running our other tests.
One had to:</p>
<ol class="arabic simple">
<li><p>change directories</p></li>
<li><p>fire up the <a class="reference external" href="http://busterjs.org/docs/server-cli/">buster-server</a></p></li>
<li><p>capture a <a class="reference external" href="http://busterjs.org/docs/capture-server/">browser slave</a></p></li>
<li><p>run <a class="reference external" href="http://busterjs.org/docs/test/runner/">buster-test</a></p></li>
</ol>
<p>I started working on automating as much of this as possible and
integrating it with our buildout test runners if possible.</p>
<p>I didn’t make much progress on the first day, Saturday. In the
morning, I was mostly learning how things were done and understanding
the architecture. After lunch, I had a pretty good idea of what I
wanted to do, so I began trying to understand the <a class="reference external" href="http://busterjs.org/">Buster.JS</a>
internals enough to add what we needed. That turned out to be a
pretty deep rabbit hole for me at my JS skill level so in the evening,
I abandoned that effort and started writing a <a class="reference external" href="https://github.com/plone/buildout.deco/blob/24cddf529507cf0d1e198783dd8b8c25b6ab3fb7/buster-alltests.in">simple Python script</a>
that would do the all the steps for you and finally shutdown the
browser slave and <code class="docutils literal notranslate"><span class="pre">buster-server</span></code>. At least I was getting
somewhere.</p>
<p>With that experience under my belt, I knew that what we needed was:</p>
<ul class="simple">
<li><p><a class="reference external" href="http://pypi.python.org/pypi/zope.testrunner#test-runner">test discovery</a></p></li>
<li><p><a class="reference external" href="http://pypi.python.org/pypi/zope.testrunner#layers">shared test fixture</a> set up and tear down</p></li>
<li><p>control of <a class="reference external" href="http://pypi.python.org/pypi/zope.testrunner#test-selection">which tests</a> are run</p></li>
</ul>
<p>Look familiar? These are all the services provided by
<a class="reference external" href="http://pypi.python.org/pypi/zope.testrunner">zope.testrunner</a> that we make so much use of in our buildouts.</p>
<p>There was one more issue in that capturing browser slaves by just
running <code class="docutils literal notranslate"><span class="pre">firefox</span></code> meant that <code class="docutils literal notranslate"><span class="pre">buster-test</span></code> runs could potentially
be polluted by any extensions, themes, or other add-ons in your
Firefox profile. The same would be in similar senses for other
browsers. This is also a familiar issue, which is precisely why
<a class="reference external" href="http://seleniumhq.org/docs/03_webdriver.html">selenium.webdriver</a> does a lot to invoke the browsers it’s driving
in an isolated way.</p>
<p>So on Sunday I set about integrating the running of Buster.JS tests
with <code class="docutils literal notranslate"><span class="pre">zope.testrunner</span></code> and to capture browser slaves using
<code class="docutils literal notranslate"><span class="pre">selenium.webdriver</span></code>. I did this in a new package,
<a class="reference external" href="https://github.com/plone/buster-selenium#buster-selenium">buster-selenium</a>. By Sunday night, I had it integrated with the
testrunner in <a class="reference external" href="https://github.com/plone/buildout.deco">buildout.deco</a> such that Buster.JS tests would be run
alongside the Python <a class="reference external" href="http://docs.python.org/library/unittest.html">unittest</a> tests. By default it would use
<code class="docutils literal notranslate"><span class="pre">selenium.webdriver</span></code> to capture a Firefox slave and use it’s JS
engine to run the tests. I also added a part to run Buster.JS unit
tests and coverage reports and <a class="reference external" href="http://busterjs.org/docs/test/reporters">output XML</a> that could be integrated
into our Jenkins CI. Finally, I added support for capturing browser
slaves using <a class="reference external" href="http://selenium-grid.seleniumhq.org/">Selenium Grid</a>. That was enough to <a class="reference external" href="http://pypi.python.org/pypi/buster-selenium">release
buster-selenium</a>.</p>
<p>There is more that can be done:</p>
<ul class="simple">
<li><p>use <a class="reference external" href="https://github.com/reebalazs/buster-qunit">buster-qunit</a> run older QUnit tests</p></li>
<li><p>better <a class="reference external" href="http://docs.python.org/library/unittest.html">unittest</a> integration, consturct one <a class="reference external" href="http://docs.python.org/library/unittest.html#unittest.TestCase">unittest.TestCase</a> for
each Buster.JS test case</p></li>
</ul>
<p>Let me know if you think those would be valuable. Hearing what the
interest is will make it more likely that I’ll get around to it.</p>
</section>
<section id="conclusion">
<h2>Conclusion</h2>
<p>This was a really great sprint, we got a lot done and I feel like I
was able to make a worthy contribution. I’m really glad that the
organizers are thinking seriously about doing this every year.</p>
<div class="note update admonition">
<p class="admonition-title">Updated on 26 September 2012</p>
<p>Imported from Plone on Mar 15, 2021. The date for this update is the last
modified date in Plone.</p>
</div>
</section>
</section>
A Python snob’s adventures in JS testing land.2012-09-26T00:00:00+00:00https://www.rpatterson.net/blog/pycon-2012-sprints/PyCon 2012 Sprints2012-03-16T00:00:00+00:00Ross Patterson<section id="pycon-2012-sprints">
<blockquote>
<div><p>Sprinting after PyCon and my first time working with and contributing to Pyramid.</p>
</div></blockquote>
<section id="sprinting-on-pyramid">
<h2>Sprinting on Pyramid</h2>
<p>Really my conference review should be taken worth a grain of salt,
because I never go for the talks, it’s all about <a class="reference external" href="https://us.pycon.org/2012/community/sprints/projects/">the sprints</a>. Having so
thoroughly enjoyed sprinting since I came out of my shell and joined
the <a class="reference external" href="http://plone.org">Plone</a> community in earnest, it’s always the
only thing I’m really focused on. What can I say? In this wonderful
age where, through open-source communities and the recognition of the
ambiguous stuff that helps them thrive, we all recognize that things
like docs and diversity are as important as the quality of code or the
power of features, I’m still a code monkey at heart.</p>
<p>Since there were no Plone sprints that I heard of or saw on the
boards, I took the chance to sprint on <a class="reference external" href="http://www.pylonsproject.org/projects/pyramid/about">Pyramid</a>, part of the
<a class="reference external" href="http://www.pylonsproject.org/">Pylons</a> project and love child of
<a class="reference external" href="http://repoze.org/">repoze</a>. I’ve been wanting to learn Pyramid
for quite some time now since I noticed how many of my favorite <a class="reference external" href="http://zope.org/">Zope</a> people have moved to Pyramid. So I went to the
Pyramid room, sat down and asked what someone with plenty of zope
experience could do.</p>
<p>The <a class="reference external" href="https://github.com/Pylons/pyramid/wiki/Sprint-Ideas">Pyramid sprinters</a> were a great
group, very welcoming and friendly. They remind me a lot of the
things I most love about the Plone community. It also gave me the
chance to work closely with <a class="reference external" href="http://plope.com/">Chris McDonough</a>.
<a class="reference external" href="https://twitter.com/#!/chrismcdonough">Chris</a> is very smart, builds
great software, and is just very kind-hearted.</p>
<p>Here’s some of the stuff I worked on:</p>
<ul>
<li><p><a class="reference external" href="https://github.com/Pylons/pyramid_debugtoolbar/pull/57">add IP range/network restriction</a> to
<a class="reference external" href="http://readthedocs.org/docs/pyramid_debugtoolbar/en/latest/">pyramid_debugtoolbar</a></p>
<p>I used this simple feature task to cut my teeth on the Pyramid
development environment and tool-chain. It was the first time in a
long time I worked outside a <a class="reference external" href="http://www.buildout.org/">buildout</a>
environment and it made for some interesting comparisons. They keep
things really simple relying only on <a class="reference external" href="http://packages.python.org/distribute/">distribute’s setuptools</a> and <a class="reference external" href="http://www.virtualenv.org/en/latest/index.html">virtualenv</a>.</p>
<p>I have to say, I still prefer working with buildout. Buildout’s
caching is much better than <a class="reference external" href="http://packages.python.org/distribute/easy_install.html">easy_install’s</a> and
virtualenv so updating the environment after changes is smoother and
faster. For testing against multiple <a class="reference external" href="http://www.python.org/">Python</a> versions and implementations, such as
<a class="reference external" href="http://www.jython.org/">Jython</a> and <a class="reference external" href="http://pypy.org/">PyPy</a>,
Pyramid uses <a class="reference external" href="http://readthedocs.org/docs/tox/en/latest/">tox</a>
which is pretty slick and very useful. I’d love to see something
like this for deploying the same buildout with different versions of
Python.</p>
</li>
<li><p><a class="reference external" href="https://github.com/repoze/repoze.sendmail/pull/2">support Python 3</a> in
<a class="reference external" href="http://pypi.python.org/pypi/repoze.sendmail/">repoze.sendmail</a></p>
<p>This was my first time tackling Python 2 and 3 compatibility
issues. I have to say it’s just not as hard as I feared.</p>
</li>
<li><aside class="system-message">
<p class="system-message-title">System Message: INFO/1 (<span class="docutils literal">/builds/rpatterson/ross-pattersons-site/blog/pycon-2012-sprints/index.rst</span>, line 15); <em><a href="#id1">backlink</a></em></p>
<p>Duplicate explicit target name: “repoze.sendmail”.</p>
</aside>
<p><a class="reference external" href="https://github.com/repoze/repoze.sendmail/pull/3">improve and centralize email message serializing</a> in
<a class="reference external" href="http://pypi.python.org/pypi/repoze.sendmail/">repoze.sendmail</a></p>
<p>The one place where there was a lot of Python 2 and 3 compatibility
pain was in the encoding of email messages. In this case, as in
others I suspect, the problem was that encoding was being done all
over the place in a not-very-well-architected way and the real
solution was to clean things up and put in more clearly defined
boundaries and interfaces.</p>
<p>That task would have been <em>impossible</em> without <a class="reference external" href="http://pyvideo.org/video/684/the-email-package-past-present-and-future">R. David Murray</a>.
I pestered <a class="reference external" href="http://www.bitdance.com/blog/">him</a> endlessly and <a class="reference external" href="https://twitter.com/#!/rdavidmurray">he</a> helped me learn all the
deep, dark corners of the email protocols and the Python <a class="reference external" href="http://docs.python.org/library/email.html">email</a> package, both of which
have considerable quirks which some would call bugs. At the end of
it, I had a clear sense of all the quirks of encoding emails and all
the things that the <cite>email</cite> package doesn’t already do. I baked
those into <a class="reference external" href="https://github.com/repoze/repoze.sendmail/blob/master/repoze/sendmail/encoding.py">repoze.sendmail.encoding</a>
and added that support as a part sending mail through
repoze.sendmail.</p>
<p>Once that was in place, finishing Python 2 and 3 compatibility was a
snap.</p>
</li>
<li><p><a class="reference external" href="https://github.com/Pylons/pyramid_mailer/pull/12">add Python 3 support</a> to
<a class="reference external" href="http://readthedocs.org/docs/pyramid_mailer/en/latest/">pyramid_mailer</a></p>
<p>Again, once <cite>repoze.sendmail.encoding</cite> was in-place, I just ripped
all the email encoding support out of <cite>pyramid_mailer</cite> and Python 2
and 3 compatibility was a cinch.</p>
</li>
<li><aside class="system-message">
<p class="system-message-title">System Message: INFO/1 (<span class="docutils literal">/builds/rpatterson/ross-pattersons-site/blog/pycon-2012-sprints/index.rst</span>, line 15); <em><a href="#id2">backlink</a></em></p>
<p>Duplicate explicit target name: “pyramid_mailer”.</p>
</aside>
<p><a class="reference external" href="https://github.com/Pylons/pyramid_mailer/issues/14">fix Cc/Bcc witout To handling</a> in
<a class="reference external" href="http://readthedocs.org/docs/pyramid_mailer/en/latest/">pyramid_mailer</a></p>
<p>While I had my hands in <cite>pyramid_mailer</cite>, I fixed a bug reported by
<a class="reference external" href="http://michael.merickel.org/">Michael Merickel</a>.</p>
</li>
<li><p><a class="reference external" href="https://github.com/Pylons/pyramid/issues/465">add support for multiple request parameters</a> when registering
<a class="reference external" href="http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/api/config.html?highlight=request_param#pyramid.config.Configurator.add_route">Pyramid views</a></p>
<p>I’m not even sure what this stuff does, but Pyramid has such great
test coverage, I was able to get started. I’m still waiting for
feeback to see what I’m not yet understanding.</p>
</li>
<li><p>updating <a class="reference external" href="http://docs.python.org/library/warnings.html#warnings.catch_warnings">warnings</a>
handling in <a class="reference external" href="https://github.com/Pylons/pyramid/pull/467">Pyramid tests</a></p>
<p>Just a simple test fixture update to cleanup output.</p>
</li>
</ul>
<div class="note update admonition">
<p class="admonition-title">Updated on 16 March 2012</p>
<p>Imported from Plone on Mar 15, 2021. The date for this update is the last
modified date in Plone.</p>
</div>
</section>
</section>
Sprinting after PyCon and my first time working with and contributing to Pyramid.2012-03-15T00:00:00+00:00