Our new deployment cycle

A number of years ago, Chaos suggested that we deploy Button Men site updates on a regular deployment cycle. Personally, I had never deployed according to a regular deployment cycle before, not at uni or at work, so I didn't know why such a cycle would be useful, but I was more than happy to accept from Chaos's experience at her work that we would see benefits.

For a long time, we had a deployment cycle that looked like this:
  • the second and third weeks of the month were for accepting any pull requests that had passed code review and unit testing
  • the last week of the month was for testing on a test server to attempt to find bugs in the code originating from recently accepted pull requests
  • the first week of the month was a merge freeze, designed to allow us to focus on fixing bugs caused by the deployment to the live site
This cycle worked quite well, I think, but there was always a large amount of work for the testers to be done in the one week before deployment, and very little active testing going on in the rest of the cycle. That meant that the week before every deployment felt like a scramble, both for testers to find bugs and for developers to fix bugs. It also meant that we had trouble deploying when there was extensive testing required to test a large change to the site. One example of this was Turbo, where the testing required many months---this held up our ability to easily deploy other site updates, and made us see that we needed a different way of doing business.

After some discussions, Chaos and I changed the way that deployments occur, with a view to allowing multiple changes to the site to be tested independently of each other. It now looks like this:
  • pull requests will be considered and tested at any time during the deployment cycle, since each pull request has a corresponding virtual instance running that version of the code
  • the second and third weeks of the month are for accepting any pull requests that have passed code review, unit testing, and manual testing on a virtual instance
  • the last week of the month is for testing the version of the code that contains all accepted pull requests on our staging instance, and the intention is to look for bugs caused by interactions between different pull requests, as well as those missed during individual pull request testing
  • the first week of the month is still a merge freeze for fixing bugs caused by deployment to the live site
Subjectively, the load on testers is more evenly distributed than it used to be, and we're picking up bugs earlier, which means that devs have more time to fix them before our nominal deployment deadline at the beginning of the month. Also, it's far easier to deal with many different types of pull requests simultaneously, since those pull requests requiring much testing don't delay the acceptance and testing of more straightforward pull requests.

I think that the new deployment cycle works at reducing stress levels across the board, and that's got to be a good thing.

Comments

Post a Comment

Popular posts from this blog

The thorny interaction between konstant and fire

Those who have been on the site for a while will know that there are quite a few buttons that are not yet playable. Mostly, this is because a die or button skill hasn't yet been implemented. But sometimes, the reasons are more subtle. Two of the US States buttons, Arizona [k(7) g(9) (12) F(15) (X)] and Hawaii [n(4) m(5) k(8) F(13) d(Y)] have been disabled for the entirety of the time that most of the other US States buttons have been playable, since January 2016. And on a first glance, it would appear that these buttons should have been enabled for play, since all the component die skills were already implemented at the time. The story really starts in September 2015, when devious and irilyth found a konstant bug while testing the button Hawaii on the dev testing site. It turned out to be caused by the underlying code not correctly determining whether an attack involving konstant requires fire support. I submitted a fix for this, and as usual, Chaos ran her automated testing bot o...
Read more

Phases of a turn

One of the first times I really thought about game states in a game was when I was learning Magic: the Gathering. There, it can be really important to know which phase of the turn is currently occurring, so much so that sometimes, you need to explicitly say things like "I'm beginning my upkeep phase" or "I'm about to end my main phase", so that opponents were given the opportunity to react during that phase. When I started working on Button Men, I thought that things would be much simpler, and that turn order wouldn't be nearly as important. Looking at the status of class BMGameState in July 2013, many of the game states were already present: startGame applyHandicaps chooseAuxiliaryDice loadDiceIntoButtons addAvailableDiceToGame specifyDice determineInitiative startRound startTurn endTurn endRound endGame Interestingly, we haven't yet used the game state set aside for allowing a handicapped game, it's just there as a placeholde...
Read more

Why PHP? A brief history of the resurrection of Button Men Online

For anyone who knows anything about web development, it may seem odd that we are using PHP for our back-end, especially since we made the decision in 2012, when there were many other feasible languages available. Here is the story of how we started out, and how we incidentally ended up being stuck with PHP. 12 Nov 2012 I broached the subject of resurrecting the Button Men Online site on the  Buttonmen Site Outage News  group, after nine months of downtime of the old site. One of the things that I left open was the choice of programming languages. 13 Nov 2012 After getting a bit of interest, I posted a suggestion on November 13 that we go with Python and PostgreSQL. Brooks chimes in that he is a Zend certified PHP developer, and suggests that Python frameworks may not be as robust as the Apache backend, and that the Apache backend is more widely supported on cloud architecture. He also mentions Pagodabox, which he uses for his PHP projects. Chaos posts that she knows Pyt...
Read more