In light of the recent news about American Airline's maintenance problems, I sat down and talked with my Dad about aircraft maintenance.  You see, my Dad was a maintenance Crew Chief for Pan American Airlines for nearly 30 years and I wanted to understand how things got so bad for American.  My Dad explained the inner workings of Pan Am's maintenance program to me.  The program dated back to late 1960's and early 1970's and was remarkably agile-sounding.  Here's the program in a nutshell.

Aircraft are regularly called in for routine servicing.  The servicing was usually scheduled for about 3 days.  A QA specialist (or several QA specialists) examined the aircraft and used a set of cards to write up work orders for each item that required servicing.  The QA specialist then sorted the cards by theme (interior, engines, avionics, etc.), estimated how many items could be completed in 3 days based on historical data, and slotted them on a task board.  The crew chief for each "theme" area would pick up the cards for his team.  The crew chief and his team would triage the work items and prioritize them.  Then, the team members would select the work items they would work on and provided the crew chief with time estimates to complete each of the tasks they selected.  This allowed the mechanics doing the work to actually estimate the duration of their tasks.  The team would then commit to completing the tasks within the 3 day service period.  Each day, they'd check in with each other to make sure things were on track.  During the service period, the crew chief worked on some maintenance tasks, but he also made sure that everyone on his team had what they needed to get their job done.  At the completion of the service period, the QA specialist would come back and inspect each maintenance item and either accept or reject them (and FAA inspectors checked things as well). Team members were also encouraged to think about how they did their job and suggested new or better ways to do things (in fact, they received monetary incentives for coming up with innovative ideas).

Now, I don't know about you, but this sounds very much like Scrum to me.  If you think about it, we have a highly complex piece of equipment with multiple integrated systems.  We have a mission critical maintenance program.  We have high priority items that need to be completed...you know, done done...to keep the aircraft safe and airworthy (and others that are lower piority like a reading lamp that isn't working).  You can't mess up here.  So how do you do it effectively and efficiently: Scrum.  Here's how I see it:

We have a time boxed iteration (the 3-day service period).  We have a product owner (the QA specialist), a ScrumMaster (the Crew Chief) and a team (the mechanics).  We have a prioritized backlog (the task board and task cards) and team members selecting their tasks, providing task estimates, and committing to a team goal of having the aircraft ready to roll in 3 days.  We have a daily stand up (checking in to make sure they're on track).  We have a servant leader ensuring his team has what it needs to meet their goal and we have a product owner review at the end of an iteration.  And most importantly, we have retrospection and continuous improvement.  Seems like Scrum to me. 

So, maybe Scrum is just a repackaging of things we already knew that worked well.  It's just being applied to software development now instead of aircraft.  Toyota picked up on much of this and Taichi Ohno implemented lean manufacturing there in the early 90's with a lot of similar tactics.  Maybe we shouldn't be too proud of ourselves for discovering something new...we really just rediscovered something old that worked and there's nothing wrong with that.  So, let's file this one under "Nothing New Under the Sun".  Or better yet, let's file it under "My Dad Was a ScrumMaster", he just didn't know it.

With our move to our new company came a new way of working for our team.  We used to be a tight knot team, all working the same long-term 3+ year enterprise project.  Now we're working on equally engaging projects, but they are shorter term.  Our "team" has been split into smaller teams of 3, 2 and even 1 person per project.  We've also been working on distributed teams with our other offices around the country as well.  As such, our velocities are for individuals, small teams and disjointed teams.  It looks like this will be the modus operandi for the foreseeable future as well: Mix-n-match teams to put the right people on the right jobs for our clients.  So, the question that has been occurring to me lately is, how do I get reliable velocities to adequately estimate bids for future work with our mix-n-match teams? 

One idea I'm toying with is trying to gauge the velocity of individual team members using Rally.  For example, I can calculate Dave's velocity as 12 points per sprint, Mike does 10 points per sprint, and Jeff does 11 points per sprint.  If we have a project that requires Dave, Mike and Jeff together, are their velocities additive?  Can I assume that this team would have a combined velocity of 33 points per sprint?  This begs the question, is the whole greater than the sum of it's parts? 

In the next few months, we're going to test this theory.  First we'll get a good handle on individual velocities. Then, we'll start tracking the mix-n-matched teams to see if individual velocities are additive or something else (better or worse).  This approach is going to require that we establish a common basis for estimating.  We'll have to come up with a baseline for our story points that everyone can agree upon.  I'm not entirely convinced this is possible, but I think it is worth investigating.  If you've dealt with a situation like before, I'd love to hear your thoughts on how to tackle a complex issue like this one.

A great question that I've heard several times now is "How do you award bonuses or implement employee incentive programs in an agile organization?".  As we've been rolling out our enterprise agile strategy at DTS, we've been wrestling with this question too.  Since agile practices don't encourage heroism or the individual developer but instead focus on team accomplishments, we thought that bonuses needed to be dished out on a team-wide basis.  We've considered project based bonuses for the entire project team and we've also considered iteration based bonuses for the project team.  In the project based model, we've been considering providing bonuses if our project teams complete their work ahead of schedule, under budget, and with 100% client acceptance.  The structure of the bonus would be something like 1% of the total contract value to be divided amongst the project team as they see fit.

Our other option is to do iteration based bonuses.  This idea came to me from my friend Bruce Scott who works at a Silicon Valley company called ParAccel.  Bruce ran a few "experiments" with agile teams and iteration based bonuses.  Before I just launch into the results of Bruce's experiments you should know who Bruce is. 

Bruce has more than 25 years of experience building successful and profitable IT enterprises.  He has been responsible for product management and engineering, research, services and support at SenSage. As VP of Engineering, he improved the product strategy and engineering process, leading to strategic partnerships with EMC, IBM, HP and Tokyo Electron. He also founded and served as CEO of PointBase, a Java-based embedded database provider. Building the company from the ground up, his licensing savvy and strategic partnerships led to its successful sale to DataMirror.  Back in 1984, Bruce co-founded Gupta Corp. As VP of Database and Connectivity R&D, he developed its first database product, SQLBase, the first database server to run on a PC LAN and support the client-server architecture. Prior to Gupta, Bruce was one of four co-founders of Oracle Corp., serving as the principle engineer and architect of Oracle database's first three versions. Hundreds of thousands of Oracle users know Bruce by the product's default username and password "Scott/Tiger." So, Bruce has credibility if know what I mean.

Bruce's experiment was simple.  In the middle of an iteration that appeared stagnant or not very likely to conclude successfully, he announced that if his development team burned down the iteration to zero by then end of the iteration, each team member would receive a $1000 bonus.  Here's what the burndown chart looked like for that iteration:

What looked to be a failing iteration was rescued by providing an incentive to burn down the iteration successfully.  The burndown rate following the bonus announcement is staggering. 

The second experiment Bruce ran was to announce a $1000 bonus at the start of the next iteration.  Here's what the burndown chart looked like for that iteration:

According to Bruce, the team stayed below the ideal burndown line until 1/30 where they added another significant story. The story wasn’t completed and was removed at the end of the sprint. This was a 31 point sprint, which was the highest this team had ever done. The sprint velocity before this was 23.5!

So, judging from Bruce's experiments, I'd say iteration based bonuses look like pretty good incentives to keep agile teams going.  And, when it comes to project based bonuses, here's Bruce's take on the subject:

On the topic of bonuses per sprint or per project, I had the identical discussion with my CEO. I convinced him to do it per sprint for the following reasons:

1) A project bonus is un-Agile like. What if the scope of the project changes? This is easier and more realistic to manage on a sprint than a project basis.

2) There is a backpackers saying in regards to minimizing the weight of a backpack that says “watch the ounces and the pounds will take care of themselves.”

3) Per sprint bonuses provide more immediate rewards and have a more likely impact on velocity than a long term reward.

I'm not sure which we'll finally implement at DTS, but I'm leaning heavily toward iteration based bonuses based on Bruce's results.  If you have any opinions on the topic, or have any metrics you'd like share based on bonuses or incentives, I'd love to hear them.

Let's face it, no matter where you work, fires flare up from time to time.  Some are serious and need to be addressed, some are just smoldering and can be put off for a short time, and some are just people yelling fire when there is no fire.  Some organizations are perpetually in fire drill mode and can't break the habit.  So, if you're an agile organization and your agile teams are committed to completing their iteration tasks, how do you fight these fires without interrupting your agile teams?  Even if you allow a buffer of time within your iteration planning for handling unexpected requests, you still run the risk of distracting your development teams with potentially harmful context switching. 

An interesting solution may be to assemble an agile fire fighting team.  Essentially, the team would be composed of developers from different project teams.  The fire fighting team would be assembled for two-week iterations (synchronized with your project iterations) and team members would rotate in and out of the team.  One team member would be the fire captain.  The fire captain is responsible for handling fire drill requests, triaging them, prioritizing them, and working with the fire fighting team to task them out. During their rotation on the fire fighting team, team members respond to urgent requests and work on them the same way they would any other task in an iteration, committing to address them by the conclusion of the iteration.  What if there are no fires to fight?  Well, don't get too excited...it's not off to the beach for the fire fighting team.  If there are no fires, team members can address defects that may exist on their current project backlogs, they can work on documentation tasks (I knew you'd love that), or they can use the time as part of their hackathon innovation allowance. 

My personal preference is that organizations work as hard as they can to move beyond the fire drill mode.  I think it is harmful to your development teams, reduces overall quality and value you are delivering to your customers, and ultimately it will impact you organization in a negative way. That said, if you can't wean yourself off the fire drill mentality, consider putting together a fire fighting team.

I've heard people say that scrum teams don't have a heart.  We plow through backlogs with our heads down.  We finish a project.  We start another and plow through the next backlog.  Story, plan, task, sprint, demo, retrospect, repeat.  Story, plan, task, sprint, demo, retrospect, repeat.  Where's the heart?  When do scrum teams look beyond these super focused iteration based tasks and think about innovation.  I agree with this assessment.  I think that agile and scrum teams can get stuck in this rut.  Well, we've all heard about the Google 20%...20% of time spent working on innovation.  I like that, but can most organizations really afford 20% of our time to do free thinking?  Probably not, we're not all making billions of dollars like Google.  But that doesn't mean we can't do something about this dilemma.  Personally, I like something like a hackfest or hackathon.  Maybe after 6 weeks of focused development, your team gets a week to do some focused play.  A one week iteration...tell us what you're going to work on...be innovative, do something cool and commit to it.  At the end of the week, same old same old...do a review.  Everyone DEMONSTRATES their cool idea.  Not a diagram, not a slideshow...demo something you can show us. 

I think that unless you're completely dominant in your market space, you need to be doing something like this to keep you innovative and on the competitive edge.  It's the old innovation curves again.  You want to always be anticipating or defining what the next thing is.  If you don't, you slide down the laggard side of the innovation curve and risk becoming irrelevant.  If you slide down that curve, it's very difficult to get back up to that next innovation curve.  So, give your teams time to innovate...it's good for your organization's future, it's good for your customers, and it's good for the professional growth and health of your development teams.

Something our team has wrestled with over the course of our agile adoption is how architecture and framework development fit into our Scrum process.  We strive to deliver an increment of functionality to our client at the conclusion of each sprint.  However, in many of our early sprints, we have to do architecture and framework development.  The things we develop on these types of architecture/functionality tasks are not really useable by our clients and we really can't demo them either.  Brian Noyle recently wrote a post on his blog asking a similar question.

So, we've decided that the best of course of action is to strike a good balance between architecture/framework and functionality development.  In earlier sprints, we've accepted that architecture/framework will be a big part of what we're working on.  But we've also become cognizant of the fact that we need to show something to the client in terms of functionality.  So, we usually comb through the backlog and pick the highest prioritized backlog item that we can complete early in the first sprints, while still doing the architecture/framework tasks we need to get done.  In this way, we ensure that the client sees forward progress that they can touch and understand right out of the gate, and we build the architecture and framework to support future development efforts on the project.  As we progress through our development, architecture/framework tasks become fewer and our delivery of functionality increases steadily.  In graphic form, it looks something like this:

As I've been contemplating moving agile throughout our entire organization here at Data Transfer Solutions, I've been considering the usefulness and effectiveness of an Iteration Zero.  Many agile teams use what's known as Iteration Zero to put the necessary systems in place to enable the delivery of value to the customer.  It's essentially the getting started iteration.  It takes place before any development begins.  I think Peter Schuh described Iteration Zero very well in his book Integrating Agile in the Real World.  Peter says:

"An iteration zero does not deliver any functionality to the customer. Instead the project team focuses on the the simple processes that will be required for the adoption and use of most agile practices. From a management point of view iteration zero may include:

• - Initial list of features identified and prioritized.
• - Project planning mechanism identified and agreed upon.
• - Identification of and agreement upon a team customer, essential stakeholders, and business users and the nature of iterative planning process, such as the time of planning meetings and the length of iterations."

I personally think that the use of an Iteration Zero is very pragmatic and I think that Peter's idea of what Iteration Zero looks like is very realistic.  I believe that in the real world of software development, customers aren't really ready to jump right in and start on a sprint from day one.  Many of them don't entirely understand the Scrum "process".  Most of them don't truly understand their requirements.  I think an Iteration Zero can be useful in educating the customer (and coming to an agreement with them) about the agile planning process.  I also think it can be used to develop the initial prioritized backlog for the project.  Now, I'm not advocating heavy up front requirements gathering here, but you do need some time to create effective user stories to start working from. From a management point of view, check out Agile Support's post on Iteration Zero as it exists within their Agile Contract Engagement Roadmap as seen below:

Aside from the project management side of Iteration Zero, I think there can be a good case for the development team to engage in Iteration Zero as well.  I think that as we switch between projects, we also tend to switch technologies or development platforms.  To do so, development teams may need time to stand up a new database server because the new customer uses Oracle and the last customer needed SQL Server.  Hopefully you have both running so you don't need to worry about things like this, but you get the point.  Sometimes the development team needs some time to get things set up to support the project before they start delivering potentially shippable product increments each sprint.  For a quick idea of what another agile team has done during Iteration Zero, check out Energized Work's post on their Iteration Zero.

I'd like to know your thoughts on Iteration Zero.  Does your team use an Iteration Zero?  If so, what kinds of tasks do they typically include?  And most importantly, do you find them useful and effective in delivering value to your customers?

Dave Bouwman and I had lunch with some friends from another GIS company yesterday.  They wanted to talk to us about our experiences with agile and Scrum.  They're very interested in implementing Scrum as their "methodology" of choice for their development teams.  But, they kept alluding to the fact that they needed to come up with an "official" date to get started.  The conversation continued and I finally asked my burning question "So, what's holding you guys back from getting started?"

The answer was that they needed more time to get their backlog together and have it be adequately defined.  I thought about that for a while and made a few suggestions that I think any teams can use to quickly get started using Scrum.  First and foremost, I think teams shouldn't worry about making their Scrum start "official" or sanctioned or even extremely well defined.  Many very successful Scrum teams started completely under the radar by implementing some simple elements of Scrum that made them more effective and efficient.  I also think teams shouldn't worry about getting it "right" the first time.  Remember the Scrum mantra "Inspect and Adapt"?  If you start off and don't get it right, inspect, adapt, and keep moving forward.

OK, so what to do about your first backlog, right?  If you really want to get started quickly, assemble enough of a backlog for maybe 2 sprints.  Get your Team started on those backlog items, while the ScrumMaster and Product Owner hash out the details for a more comprehensive backlog.  If you'd like to do more planning and feel better about your backlog, build a backlog for a single release.  Let your Team get started on that backlog while again, the ScrumMaster and Product Owner build out more release backlog items for future releases.  Finally, if you really want to assemble a detailed backlog before your Team starts working, consider a Sprint Zero.  Make Sprint Zero a one-week Sprint where the ScrumMaster, the Product Owner, and the Team work on assembling a detailed backlog.  You can also use Sprint Zero to put anything else in place that you think you need to support your Scrum implementation.

When you're assembling your first backlogs, keep in mind that you don't need to put excruciating amounts of detail into your user stories that are farther down the line.  The level of detail in your user stories should be proportionate to how close you are to actually working on the story in question.  The further in the future the user story is the less detail it should contain.  Remember that one of the benefits of Scrum is the reduction of time wasted on requirements that may not ever be implemented.  It is more likely that a user story that is one year out will not be implemented than one that is just two weeks away.

So, what are you waiting for?  Pick a project, big or small, get some quick backlogs together, and start scrumming today.  It's never too soon. 

One of the key tenets of Scrum is the team's absolute, uninterrupted focus on the current sprint tasks.  One of the key responsibilities of the ScrumMaster is to ensure this focus and to fend off unnecessary interruptions.  It sounds easier than it is.  As a ScrumMaster, I have had the distinction of experiencing first hand the difficulty of defending our team's focus during a sprint. 

Case in point: Our team was working on a very focused and successful sprint when we were approached by a former colleague.  He wanted us to begin working on an internal development project immediately and stressed that it was of ultimate importance to the organization.  We would need to cancel our current sprint to accommodate his request.  After a bit of probing about the "importance" of this project, it became very clear that this wasn't as high of a priority as he thought it was.  In addition, I had previous experiences with said colleague and knew that his tendency was to focus too much on relatively unimportant tasks. 

After some deliberation, I made the decision as ScrumMaster to tell him that we would not be canceling our current sprint to work on his request.  Instead, we would prioritize his request in our backlog and get to it in our next sprint.  I assured him that the two developers that would work on his request would be 100% committed to his tasks during the two-week sprint.  His tasks would receive the same uninterrupted focus as our present tasks were enjoying.  After explaining this to him, he acquiesced and accepted a delayed start for his tasks.

We did indeed work on and complete his tasks during our next two-week iteration.  In fact, when we were done with our sprint, he was surprised to find that all of his functional requests had been completed (of course, it was no surprise to our committed team).  At the conclusion of our sprint we found out that his team was not ready to implement our new functionality.  Neither the servers we needed nor the necessary software infrastructure was available to deploy the application.  Nearly 3 months later, his non-agile team was finally ready to deploy our functionality.  So, his high priority request really turned out to be very low priority in the end.

The moral of this little story is that ScrumMasters need to know when to say "No."  Sometimes, there may be a very valid reason to cancel your sprint to address very high priority requests.  But, ScrumMasters need to use a lot of discretion and ensure the validity of these "high" priorities.  It is extremely important not to interrupt your teams for just anything.  In this case, had we interrupted our Sprint to fulfill this request, we would have shifted our focus from a highly productive Sprint to address a low priority need.  In the end, we did the right thing by saying "No."  We completed our sprint very successfully and satisfied our paying client and our team remained focused and effective. 

We’ve all been there before, The Useless Meeting.  They come in many shapes and forms but share one common element:  the unproductive waste of time.  Nothing can be more damaging to an agile team than being interrupted from their valuable work to attend one of these sucking voids. 

Recently, our Scrum Team had an otherwise productive Sprint interrupted by just such a meeting. We were halfway through our Sprint when we received a Friday afternoon email about a mandatory all-hands offsite meeting of our entire group.  Our V.P. had decided that the next Wednesday, we would all gather in Denver to discuss business strategy for the upcoming year.  Yes…our development team was ecstatic to attend, knowing full well it was going to be chock-full of endless PowerPoints with lots of charts and numbers. 

We decided to raise our voices a bit and protest.  “It’s interrupting our Sprint” was the battle cry.  No luck, that didn’t resonate.  So I tried a different tack.  “If you’re going to interrupt our work make it productive.  If you’re not going to cancel the meeting, make it interesting.”  Our V.P. perked up.  He couldn’t fathom the fact that our development team wasn’t interested in charts, projections, revenues, etc.  He immediately visited our office to find out why we were so upset.  We gave a few reasons and suggested that the group offsite meeting should be about the great things his teams were doing.  Make it a motivational meeting.  Get the teams excited about their work.  He left our office agreeing and made a change to the agenda.  He even asked us to give a talk about the power of agile and our success with Scrum.  We still didn’t want to interrupt our Sprint, but at least we thought this wouldn’t be a complete waste of time.  We worked very hard for the next few days preparing a killer presentation about Scrum and how it could benefit the whole organization, not just our team.

We arrived in Denver at noon.  We were sixth on the “revised” agenda.  The offsite meeting started late.  The first speaker was our HR rep.  Not very inspirational, but she kept it short and sweet.  Next up was our V.P.  He talked for about an hour and a half about things that he thought were very important (he was slated for only 30 minutes on the agenda).  Lots of charts, numbers, and broad, sweeping statements.  Next up was one of our senior product managers.  Lots more charts, lots of numbers and all completely meaningless.  In fact (and I must share this classic Dilbert moment with you) at one point, someone asked where his numbers had come from.  His answer was literally (and I quote) “Ummm…from a matrix…in my head”.  After an hour of this babble (slated for 30 minutes on the agenda) he finally sat down.  The R&D pitch was next.  It was a little confusing as our R&D guy had started his job about 1 week prior to the meeting.  He ran over time as well.  At 3:50 PM in a meeting scheduled to end at 4:00, our CEO strolls in and delivers the most lackluster presentation of the day which ends promptly at 4:20.  The meeting is called to an end and we never got to deliver our Scrum presentation.  The team was incredibly disappointed and couldn’t believe we spent a whole day in a useless meeting.

When we sat down for our Sprint Retrospective, I asked how disruptive this offsite meeting was to the Team.  Aside from the obvious waste of valuable time, the Team agreed that the meeting was indeed disruptive to their work.  Some of our developers were deep into solving some complex problems and had to divert their attention to attend the offsite meeting.  Our lead architect and I were also distracted as we had to entertain our V.P. for the entire time he visited our office, and we had to work on preparing our presentation (which of course we never got to give).  The day following the offsite was difficult as well as most of the Team struggled to get back into the flow of their work.  We were tight on time and some of the developers put in time on the weekend just to finish their coding for the Sprint.  The morale of the office was down as well.  Team members couldn’t believe that so much time was wasted with such flippancy.  In the end, we ran out of time in our Sprint and weren’t able to adequately test what was developed.  We had sacrificed quality for the sake of a useless meeting!

The takeaway message here is really for those who manage Scrum teams and it is very simple: Don’t disturb your Scrum Team mid-Sprint unless it is absolutely essential.  The work of the Sprint is very important and it is providing value to your customers.  If you are going to disturb your Scrum teams, you had better have a very good reason to do so.  If at all possible, schedule your disruption well in advance so your Teams can plan their Sprint tasks around your activity.  From the story above, it should be evident that unproductive interruptions can have a negative impact on performance, quality, and morale. 

The message for ScrumMasters is equally important: If managers cannot justify the importance of these time-wasting activities, don’t interrupt your Scrum Team during a Sprint.  This is easier said than done.  Our managers can exert undue pressure on us to mandate participation in these activities.  The best thing to do is to make them very aware of the consequences of their actions.  In fact, if they insist on disturbing the Team, it is probably beneficial to demonstrate the impact of their actions post-Sprint with hard facts so they have tangible evidence of their impact on quality and performance.  As a last resort, you may want to give them a gift-wrapped copy of Scott Adams’ excellent book Always Postpone Meetings with Time-Wasting Morons.

In every Sprint we do, we try very hard to produce the highest quality software possible.   However, we’re all human and we still produce software with defects. On one of our current projects, we had reached a point at which our Product Owner wanted to demo the software we had developed so far to his statewide GIS coordinators. He set the priority for our next Sprint to be to fix any defects in our defect list so he could have a smooth demo. We agreed, but also made sure we included a few small pieces of functionality that we could develop to meet our goal of producing some potentially shippable product increment during each Sprint.

 Since we made our move to Rally Software, managing our defect list has become much easier. And moving defects from our defect list to our Sprint backlog was even easier. After we had filled our Sprint backlog with all of the bugs that we estimated we could fix in our two week  Sprint, we created a set of tasks for each defect. The minimum requirement for each defect was that we would verify the defect, fix the defect, test the fix, and have an in-house Product Owner proxy verify the fix.

We were quickly moving through our Sprint backlog and it looked like we’d finish very far ahead of our ideal burndown for the Sprint. We decided that one of our developers would take a look at any new, incoming bug reports from the customer, triage the defects, and create new tasks in the Sprint on an ad hoc basis. At first, I thought this might not be the best idea, and it sort of goes against Scrum’s “rule” of not adding new work during a Sprint. However, given our burndown rate and our success at bashing the other bugs at that point in the Sprint, we thought we should really try to fix as much as we could for our customer’s demo. We told our customer that the Friday at the midway point of the current Sprint would be the cutoff for defects that we could address in the current Sprint. We suddenly received a flurry of defects before the deadline, but nothing that was too hard to handle.

Our developer did a great job triaging the defects as they arrived. We decided that anything that was an enhancement would be placed in the project backlog to be prioritized at a later date by the Product Owner. High priority bugs, as defined by the Product Owner, were placed in the Sprint backlog. We triaged, fixed, tested, and verified nearly every bug. At the end of the Sprint, we successfully fixed all of the high priority bugs in our defect list.

One of the unexpected benefits of this bug bashing Sprint was the knowledge transfer we experienced between developers. Because the tasks were so short in duration, our developers were picking up lots of tasks quickly, and most had nothing to do with the functionality they had developed. This in turn created a need for more communication between our developers than usual. This gave them a much better and deeper understanding of our entire codebase.

At our Sprint retrospective, the development team expressed a high level of satisfaction with the Sprint. They were very happy to have the opportunity to understand each other’s work on a deeper level. The team also decided that instead of fixing defects in drips and drabs, they preferred fixing them in one dedicated bug bashing Sprint before each release. They thought the benefit of the knowledge transfer, and the ability to focus on just defect fixes was significant enough to do more bug bashes in the future. This was a huge indicator for me that our team was really grasping the “inspect and adapt” mantra of Scrum. They had thought about what made this Sprint so productive and decided that it was a practice worth keeping. The bug bash Sprint was a smash for our team. Maybe it can be for yours as well.

Try this: Go to Google and search on GIS Project Management.  You'll get back about 2,490,000 results (give or take a few).  Now start clicking on them and see how long it takes you to find a result that remotely resembles an Agile approach to GIS Project Management.  I got about 140 results in before I gave up.  Notice, I didn't search on GIS application development or some similar search term.  I specifically wanted to know how people view all types of GIS Project Management.  There was a reason I tried this search (and no, it wasn't due to sheer boredom at work) . 

The other night, I was at a get together of some regional ArcGIS developers.  They asked about Scrum and we started talking about the requirement that every iteration must produce some potentially shippable product increment.  Being GIS folks, someone asked how you would apply that "rule" to non-development GIS projects.  It made me sit back and think for a bit.  So, out of curiosity, and to find out if anyone out there used an Agile approach to these types of projects, I searched Google using several appropriate search phrases.  To my astonishment, I couldn't find a single reference to anyone using Agile methods for non-development GIS Projects.  In fact, most led me to very rigid, stepwise, waterfall approaches to GIS project management.

Before I got heavily involved in application development and enterprise GIS architecture, I was a GIS analyst and project manager on lots of non-development GIS projects.  How would I have applied Scrum to those projects?  I thought about that for a while.  First I thought, "Maybe it doesn't work outside of our little software development fiefdom?”  I went to sleep that night without a clear answer in my head.  But the next day, I thought some more and decided that Agile practices, and Scrum, could be used to manage non-development GIS projects without any alteration.  For one thing, Scrum is not really a set of "rules", but more a set of guidelines.  Ken Schwaber in his book The Enterprise and Scrum states that projects are "too complex for a single set of rules to suffice in all circumstances".  My second thought was that the phrase "potentially shippable product increment" was confusing to some.  To me, what it boils down to is that at the end of an iteration, a Team must produce something of value as defined by the Product Owner.

Applying my "new" definition, I started to think about how this would apply to non-development GIS projects.  What's valuable to a GIS Product Owner?  Data, definitely data.  Analyses, map products, metadata (always of questionable value), models, and probably others I'm not thinking of.  Now, in the development world, to deliver a potentially shippable increment (or "something of value") every iteration, we decompose user stories into their simplest forms and provide targeted functionality to address that story.  In non-development projects, we can do the same thing. 

Let's consider an example involving a mutli-layer statewide data request from your Product Owner.  We don't have to deliver all of the data in a single iteration.  Have your Product Owner prioritize the data layers.  Maybe the hydrologic data layer is a higher priority than the land-use layer.  If the hydrologic layer will take longer to deliver than one iteration, break it into smaller prioritized pieces.  Maybe your Product Owner is really only interested in second order streams.  Produce all of the second order otream data during your next iteration.  If you cannot complete that data layer in a single iteration, break it down again.  Maybe your Product Owner is currently interested in second order streams in Kitsap County.  You probably get the point by now.  It works for data production and will work for analyses, map products, models...and yes, even the dreaded metadata!  Work can always be decomposed and re-prioritized by you and your Product Owner. 

The point is to constantly provide your Product Owner with something of value that s/he can use at the end of every iteration.  In our statewide data example above, maybe your Product Owner can only view second order streams in Kitsap County this week, but s/he can at least do that.  The next iteration may include second order streams in King County, and so on.  This can even be a benefit to your Team.  You may find that you do not have to deliver a hydrologic data set for the entire state of Washington.  As you deliver valuable increments of data, your Product Owner may decide that the Puget Sound area was really all that was needed now that s/he has been using the data you already delivered.  You're done with that request, on to the next "something of value"!

So, keep delivering.  Keep getting feedback.  Keep adapting your process to deliver the most value possible to your Product Owner.  It doesn't matter if it's a GIS application, analysis, map, model, or metadata...you'll find your Team delivering higher quality, higher value GIS products faster and more efficiently using Agile practices.

Shortly after we read Ken Schwaber's book Agile Project Management with Scrum, we put together a presentation to our management team to convince them that we could make Scrum work.  To our surprise, they gave us the high-five and now we actually had to do it.  We could have tried out Scrum on some small projects just to test the waters.  But at the time, we were working on a complex multi-year enterprise GIS development project for a large state agency.  The client is very communicative and very conscientious about the scope and value of the project.  We decided that this was the right kind of project and client to engage with our new found knowledge of Scrum.  Before we proceeded, we gave another presentation about Scrum to the client.  During the presentation we outlined that this process would require a much higher degree of client-contractor interaction than usual and wasn't quite as easy as it all sounded.  The client agreed that Scrum looked very good and agreed to become a committed Product Owner.  That sealed the deal.  We had our management's approval, we had a willing and able client, and we had a team of developers who were committed to making Scrum work on this project.

One challenge we had with this project was that there was a heavy architecture and framework component that we were building on top of the ArcGIS ArcObjects model.  The architecture and framework components were not easy to demonstrate to a client and we didn't know how we would demo them at the end of our first Sprint.  We made a decision that we would work on our framework for an internal client at our company for the first few Sprints.  The internal client was a subject matter expert who knew the client's line of business very well.  So, we thought we'd "shield" the client from having to deal with complex framework issues.  

Before our first Sprint, we needed to come up with a Project Backlog to work from.  The project was originally scoped and scheduled by another project manager using a traditional waterfall approach.  As such, we had a Microsoft Project Gantt chart that gave the broad strokes of the project schedule.  We also had a functional requirements document that had very high level use cases for the project.  The use cases were written in very technical language and didn't really communicate what the goal of each use case was.  But, without any better starting point, we used these two sources to build our first Project Backlog.  Because the project is divided into very neat modules based on specific business practices within the client organization, we actually built several "Release Backlogs" based on these modules.  The first "Release Backlog" we would tackle would be the Framework Backlog.

We had decided to do a two-week Sprint for our first Sprint.  It would be enough time to see how this Scrum thing worked, but short enough to pull back if it began falling apart.  We had our first Sprint planning meeting and timeboxed it to four hours.  It was very enlightening for the entire Team.  Our lead GIS software architect, Dave Bouwman, detailed the entire architecture and framework for the rest of the Team.  For some of them, this was the first time they had heard any of the details.  Dave answered all of their questions until the Team was satisfied that they understood the work ahead.  We then prioritized the Framework Backlog ourselves and tried our best to estimate the number of hours each large chunk of work would take.  It was very difficult to estimate because the use cases and work items in the backlog encompassed so much work. 

Next, we began the process of selecting our Sprint 1 Backlog items.  As we selected a use case from our Framework Backlog, we placed it into our Sprint Backlog.  We then fleshed out all of the tasks that we needed to accomplish to complete the use case or work item.  We decided that tasks should be no longer than 4 hours in duration, so we needed to do quite a bit of decomposition to get to that point.  As we defined the tasks, we also decided who on the Team would actually be responsible for that particular task.  We kept track of how much time each Team member was committing to for the Sprint as they selected their tasks.  Once someone reached 85% utilization (our organization's magic utilization number) for the two week Sprint, they couldn't select any more tasks.  We did it!  We were ready to roll on our first Sprint...

The first Sprint actually went surprisingly well.  We were off and developing our Framework.  We had our Daily Scrums and felt very good about this new level of project communication and synchronization.  We used an Excel spreadsheet that we designed to track the progress of our Sprint.  The Team's sprint burndown curve was looking great and we were heading for a successful first Sprint.  However, at the end of the Sprint, we realized that we had over committed ourselves slightly.  There were a few items that were not going to be finished.  Being the Sprint rookies that we were, we chalked it up to bad estimating and moved forward.  We ended the first Sprint with about 16 hours of unfinished work. 

After the Sprint ended, we held a Sprint Review.  We timeboxed it to two hours.  Everyone demonstrated their work to our internal client and we were all very happy with how things progressed out of the gate.  We also did a Sprint Retrospective and timeboxed it to two hours as well.  To get the most out of the Retrospective, we used several techniques from a book called Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen.  First we developed a set of Working Rules.  That is, a set of rules that would help us have more productive meetings.  Next we did a Timeline activity.  On our white board, we set up a calendar of the days in Sprint 1.  Each Team member used large Post-It notes to add significant events that occurred during the Sprint.  The events could be technical, organizational, or team related.  After everyone posted their events, we sat back, discussed each event and analyzed what we had all shared with each other.  We tried to determine at which points during the Sprint the Team was feeling surges of high energy and when they were feeling low.  This helped us pick out positive and negative things that occured during the Sprint.  Based on this review, the Team had concluded the following about our Sprint::

1. We didn't have enough information to effectively build the components we needed.  Not enough design work was done to prepare for the Sprint.
2. Our lead architect (who had all of the design knowledge) was being constantly pulled away from his Sprint tasks to respond to non-project requests from elsewhere in the organization (he's in very high demand!).
3. We had some non-project infrastructure issues we needed to address in order for the team to work together more efficiently.

So here we were at the end of Sprint 1, feeling good, but knowing we had definite room for improvement and learning.

My Reflections on Sprint 1

Looking back at our first Sprint, there are many things I notice now that we could have done differently. 

The first thing that strikes me now is that we probably should not have focused on only framework components.  After convincing the client how great Scrum would be, we said goodbye for a few weeks and worked on things the client couldn't see.  At the end of the sprint, we didn't deliver a single piece of business functionality...and thus we broke the first rule of Scrum.  After listening to Ken Schwaber's Google Tech Talk on Scrum and attending Mike Cohn's Certified ScrumMaster course, I realized that we should have at least found one small increment of functionality that we could have pulled all the way through a layer of our framework during our first Sprint.  It would have kept the client engaged and helped us prove that our framework was working.  While it's alright to be doing heavy architecture during the opening Sprints, always keep in mind that it is more important to deliver business functionality.  As the Sprints progress, architecture and framework work will steadily decrease as the focus on business functionality increases.  If they are plotted on a graph, with amount of work and time on each axis, architecture/framework and business functionality would look something like this:        

Another observation on our first Sprint was our difficulty in estimating the size and complexity of the high level use cases.  I think our main difficulty was in the fact the use cases were written too technically and too abstract and didn't emphasize the user story.  We have since been writing our Project Backlog items as User Stories.  User Stories let us clearly identify the requirement in terms of the user's expectations, needs, and conditions of satisfaction.  They are structured as follows:  "As a [user role or type], I want to [do something useful], so that [some reason why]".  We have also since discovered that estimating with hours for large tasks is not effective.  We have begun using Story Points to estimate our Project Backlog items, but that is a topic another blog entry altogether.

A common trap we fell into was the belief that we had to do design work before our Sprint work could be tackled.  We also did no documentation or testing during our first Sprint.  We were really not "done" with any of the components we built in Sprint 1.  All of the functionality developed during a Sprint should be designed, coded, tested, and documented during that Sprint.  We are actively working to make this our definition of a complete increment of business functionality for our future Sprints. 

What did seem to work well for us was the two-week Sprint length.  The Team has agreed to keep going at this Sprint length for the duration of the project.  It keeps us very engaged with the client and gives us a very clear focus and end-point to our work.

We did decide as a Team that we were spending far too much time on the Sprint Review and Sprint Retrospective and have reduced the timeboxes to 1 hour for the Review, 30 minutes for regular sprint Retrospectives, and 2 hours for release sprint Retrospectives.