We took a good look at the way we developed software in 2009 and made the decision to implement fundamental changes to our process to improve product quality and focus in 2010. In 2009, we operated under a waterfall model - Design --> Build --> Test --> Deploy --> Operate. Some of us liked to think we were following a "Waterfall/Agile Hybrid", and this thinking was brought about from the fact that we'd combined elements of Agile into our "Build" phase that had carried over from a failed attempt to transition to Scrum in 2008. The only elements I could think of that carried over were burn down charts and daily standup meetings. But nobody liked the process as it produced a lot of waste (context is inevitably lost when work items are transitioned through the phases) and the business wasn't getting the product it wanted.
Enter Scrum in 2010.
We enlisted the help of an external company, Rallydev, to show us the way; to enlighten us on how to do Scrum and how we might build a better product. My aim, more than anything, was to convert the masses. I wanted them to become believers in the process, because if they believe they'll work to iron out the kinks during execution. But before Rallydev came to help us, we had to convince the executives that Agile was the way to go. I'd heard horror stories of Agile transitions not working because of a lack of executive buy in. So I brought in a buddy of mine and his CEO to talk to executive management within my division. My buddy, and his CEO, had successfully transitioned their company from waterfall to Agile, and they were exuberant about the outcome. After a great conversation my executives were convinced, and I bought 11 copies of Lean Software Development and had all of the senior leaders (including myself) read it. Soon we were all using phrases like "fail fast" and "conceptual integrity" and were eager to see how lean software development would impact our teams.
I put a business case together to invest in training, and in mid-January Rallydev came to educate us on the principles of Scrum. I wrote about that a few weeks ago. I flew in all of my North American Development Managers and invited all of the General Managers and future Scrum Masters to the RallyDev sessions. They gave us a great general overview of scrum and we spent one day walking through a Sprint planning session, which was key. My one complaint is that I was looking for something more perscriptive on how to do Scrum with off-shore resources. They gave us a few suggestions, but we were left to figure it out on our own, and I've detailed how we do it below. My team left the Rallydev session believing in Agile, as far as I could tell, and we were primed to make the switch to Scrum.
Since January, we've deployed Scrum to 5 of 7 teams, with 2 of those teams based in China. I came to the conclusion that one of the best ways to foster collaboration is to maximize the amount of hours team members are in contact with each other by aligning teams geographically. We created the 2 China-based teams with resources that had previously been embedded with North American teams and had never worked together (even though they sat 15 feet away from each other in the office in China). We also had to find Scrum Masters for these teams, and it was recommended to me by our China office coordinator that I use our China-based QA leads for the position, which I promptly did.
So, in China, we're running with 2 semi-autonomous Scrum teams who liase with a Product Owner in North America on a daily basis. They follow the exact same process that we do here in North America. And this frees us up from day-to-day management of off-shore resources, as they're managing themselves.
During my research, the most helpful thing I read was not Lean Software Development, which is more philosophical in nature, but Scrum and XP from the Trenches - an ebook that spells out exactly how one team does Scrum from a tactical point of view. Fittingly, I'd like to provide information on how it is that we do Scrum here at Active.com.
But before I begin I'd like to set the stage:
- We have 7 development teams, 5 teams in North America, 2 teams in China.
- Every North American team has at least one resource that works remotely, from home.
- We have 9 resources that do QA. 2 of them are team leads based in North America. The rest are based in China.
Team Structure
Integrating Team Members
Team members must sit together. Period. Close collaboration is a key feature of Scrum, and it's amazing how well a thin, 6 foot high cube wall can subdue collaborative efforts. We made everybody move so that they were sitting together after Scrum training sessions. Remote team members are required to be on a Skype video call when there's only one remote member on the team, or a Skype call and tokbox video chat when teams that have more than 1 remote member (Skype only supports 1 to 1 video chatting). We tried a lot of video conferencing software, and tokbox, thus far, is the best for the money (i.e free). The verdict's still out, though, as teams that have the Skype/tokbox combo find the combo cumberson. I'd love to have the funds to pay for a Polycom V700 for each of my teams, but I'd also love to cruise around on a flying carpet. I've gotta live in reality. Here are a few photos of the video feature in play with my teams.
Our smallest team consists of a Scrum Master in San Diego, a Development Manager in Georgia and a Developer in North Carolina. I made them try the video setup for 1 sprint, and they hated it. Agile wants teams to have some control over their destiny, and since the team decided they wanted to discontinue video chat in their retrospective, I decided not to press the issue. But the other teams love it, and each voted to continue with video chat. Remote team members have also commented that they feel more like their part of the team as they're involved with small talk and can get a feel for what's going on in the office.
My next step with remote workers, however, is to build an Embodied Social Proxy, and we're already assembling a prototype.
QA
Developers can test, but Testers cannot develop. As such, we don't have QA personnel permanently assigned to Scrum teams. During each spring, QA brokers determine where best to utilize their QA resources. For example, if team A is developing important, high-revenue features that require a lot of specialized attention, the QA lead will assign the bulk of his/her resources to that team leaving the others to do their own QA. Or the QA lead might choose to re-assign his/her resources each day. The point is that it's up to the QA broker to determine where to apply QA effort. Their responsibility is to attend all Scrum meetings and Sprint Planning sessions for their respective products and orchestrate their teams (which, again, are entirely based in China) on a daily basis through a Skype/tokbox call.
The diagram above shows the alignment of one QA team to 3 Scrum teams.
Sprints
Planning
Most of the team operate in 3 week timeboxes (Sprints). The first day of each Sprint is always the Sprint Planning meeting. This is when the Product Owner (the guy with the vision and who's on the hook for P&L) determines which user stories the team will work on. Each team has its own Sprint Planning meeting, and in some cases Product Owners attend multiple planning meetings in a week as they have multiple teams. The Product Owner comes to this meeting with a prioritized "backlog" of user stories. And user stories are always in the following format:
As a [who] I want to [what] so that [why], for example
We've significantly reduced the waste we otherwise incurred through over-documenting requirements by using user stories. One of the big challenges for us was to get Product Owners to construct a backlog of user stories that met this format. The idea is that they're independent, negotiable, valuable to the end user and estimable. The key here is the negotiable part, where the user story is just vague enough that the Scrum team can negotiate scope to get the user story to fit into the Sprint. We use the term "barely sufficient" a lot, meaning let's do the least amount of work required to produce a shippable feature that we can learn from, and it's an art to learn how to negotiate correctly.
Sprint planning meetings follow the same format for all teams, that that format is:
- Determine Delivery Team Capacity – each member of the delivery team (scrum master, QA, developers, product owner) comes to the meeting knowing what days off they’ll need during the next iteration. The goal is to determine how many hours each resource can dedicate to the sprint. Note, “focus factor” is then applied to the team’s capacity. For example, if a team of 4 is available for 480 hours over a 3 week period, a focus factor of 50% brings capacity down to 240 hours. I highly recommend we start with a focus factor of 50% for each team.
- Team Estimation (see attachment) – The Product Owner explains his top priority user stories to the delivery team. The team then collectively decides whether each story feels S, M, L, XL, or XXL. Relative sizing stops once the team feels like enough work is represented to fill the time box. The delivery team then selects a user story with a size of 2 or 3 story points to use as a baseline. If the team has already done a sprint, they can use a previously-sized user story. If this is the first sprint, the team must use the poker planning technique on one of the user stories in the “S” category from the Relative Sizing step. The idea is to size all of the user stories that were relatively sized. The sizing unit is “story points” and is one of 1,2,3,5,8,13, 20, 40, 100.
- User Story Decomposition – The delivery team then dives into each story and determines what tasks need to be completed before the story is “done”. Each task is given an estimate in hours and someone from the delivery team volunteers to take them on. As each task is determined and sized, the task owner’s capacity (from step 1) is reduced. When a given person’s capacity is 0 this is a signal that he is no longer in a position to commit to additional work. The team can then commit to the sprint plan as it stands or other team members can sign up for the remaining tasks until they too reach capacity.
- Re-Sizing – Based on what was learned during User Story Decomposition, the team may feel that its initial story point estimates are now inaccurate. The team, through poker sizing, will re-estimate the size of each user story in the iteration using poker planning techniques.
- Commit to the Sprint Plan -- After the team has gone through each of the stories (sizing, task breakdown, estimate tasks, determine task owners), ask for a "Fist of Five" (vote for consensus) on whether the team can commit to delivering the items within the sprint. If all members of the delivery team show a 3 or higher, the sprint is ok to proceed. If not, the team needs to work through the sticking points until a positive consensus is reached.
- Backlog Sizing – If time permits, the team should run through the Team Estimation step for the next high-priority user stories in the backlog. The idea is to have baseline sizes (story points) for items expected to be worked on during the proceeding 2 sprints.
These meetings typically take between 4 and 8 hours. We end up writing all of our user stories and tasks on notecards, and they get placed on a board on a wall near the Scrum team:
We also create a virtual board on a paid website called Scrumy, which I highly recommend.
This happens even with our China-based teams. Our Sprint planning meetings happen over a 4 hour Skype/tokbox meeting with the Scrum team and Product Owner. It starts around 4pm PST and 8am in China. We've started off with a reduced focus factor of 25% for our China-based teams (focus factor is the percentage of time that each person dedicates toward the sprint. If a person works 8 hours a day, 2 of those hours are dedicated to the Sprint each day) to accomodate inefficiencies with communcation back with the Product Owner in North America due to the time difference, and because the process is relatively new.
The China-based teams also have a task board on a wall with sticky notes.
Scrum Meetings
Each team has a 15 minute scrum meeting each day (usually in the morning). The China-based teams have it at 9am their time (5pm PST) so that our Product Owners can attend. We keep it short, each team member talks about what they did yesterday, what they'll do today and if they have any roadblocks. Follow-on conversation is left to the "16th minute", and people have the option of leaving for that conversation. At the end of every Scrum meeting, the Scrum Master updates the burndown chart and moves tasks along the task board.
Demo and Retrospective
The second to last day of the Sprint is dedicated to the demo and retrospective. This is a 2 to 4 hour meeting where the Scrum team demos the user stories they've developed to stakeholders and answers any questions they might have. A lot of good conversation usually comes out of this. After the demo, we kick into a discussion about the Agile process. This discussion covers:
- What worked well - things we should keep on doing
- What should change - things we should stop doing
- New ideas - things we should start doing
- Acknowledgements - kudos to people on the team
The retrospective is super interesting to me, as it allows me to observe how others are taking to Agile. This is a very iterative endeavor for us, and we're always "inspecting and adapting" as the Rallydev trainers put it.
Conclusion
We're still new at this, and we haven't deployed any significant enhancements to Production under this new Agile process, yet. But as far as process in and of itself goes, the teams and the Product Owners love it. Teams have bonded strongly together and they've become closer to the business by working directly with Product Owners. And the business always knows what it's getting through involvement and demos.
More to come on this topic as time progresses.
Sportspower
Facebook
MySpace
YouTube
Twitter