Try this: Go to Google and search on GIS Project Management.  You'll get back about 2,490,000 results (give or take a few).  Now start clicking on them and see how long it takes you to find a result that remotely resembles an Agile approach to GIS Project Management.  I got about 140 results in before I gave up.  Notice, I didn't search on GIS application development or some similar search term.  I specifically wanted to know how people view all types of GIS Project Management.  There was a reason I tried this search (and no, it wasn't due to sheer boredom at work) . 

The other night, I was at a get together of some regional ArcGIS developers.  They asked about Scrum and we started talking about the requirement that every iteration must produce some potentially shippable product increment.  Being GIS folks, someone asked how you would apply that "rule" to non-development GIS projects.  It made me sit back and think for a bit.  So, out of curiosity, and to find out if anyone out there used an Agile approach to these types of projects, I searched Google using several appropriate search phrases.  To my astonishment, I couldn't find a single reference to anyone using Agile methods for non-development GIS Projects.  In fact, most led me to very rigid, stepwise, waterfall approaches to GIS project management.

Before I got heavily involved in application development and enterprise GIS architecture, I was a GIS analyst and project manager on lots of non-development GIS projects.  How would I have applied Scrum to those projects?  I thought about that for a while.  First I thought, "Maybe it doesn't work outside of our little software development fiefdom?”  I went to sleep that night without a clear answer in my head.  But the next day, I thought some more and decided that Agile practices, and Scrum, could be used to manage non-development GIS projects without any alteration.  For one thing, Scrum is not really a set of "rules", but more a set of guidelines.  Ken Schwaber in his book The Enterprise and Scrum states that projects are "too complex for a single set of rules to suffice in all circumstances".  My second thought was that the phrase "potentially shippable product increment" was confusing to some.  To me, what it boils down to is that at the end of an iteration, a Team must produce something of value as defined by the Product Owner.

Applying my "new" definition, I started to think about how this would apply to non-development GIS projects.  What's valuable to a GIS Product Owner?  Data, definitely data.  Analyses, map products, metadata (always of questionable value), models, and probably others I'm not thinking of.  Now, in the development world, to deliver a potentially shippable increment (or "something of value") every iteration, we decompose user stories into their simplest forms and provide targeted functionality to address that story.  In non-development projects, we can do the same thing. 

Let's consider an example involving a mutli-layer statewide data request from your Product Owner.  We don't have to deliver all of the data in a single iteration.  Have your Product Owner prioritize the data layers.  Maybe the hydrologic data layer is a higher priority than the land-use layer.  If the hydrologic layer will take longer to deliver than one iteration, break it into smaller prioritized pieces.  Maybe your Product Owner is really only interested in second order streams.  Produce all of the second order otream data during your next iteration.  If you cannot complete that data layer in a single iteration, break it down again.  Maybe your Product Owner is currently interested in second order streams in Kitsap County.  You probably get the point by now.  It works for data production and will work for analyses, map products, models...and yes, even the dreaded metadata!  Work can always be decomposed and re-prioritized by you and your Product Owner. 

The point is to constantly provide your Product Owner with something of value that s/he can use at the end of every iteration.  In our statewide data example above, maybe your Product Owner can only view second order streams in Kitsap County this week, but s/he can at least do that.  The next iteration may include second order streams in King County, and so on.  This can even be a benefit to your Team.  You may find that you do not have to deliver a hydrologic data set for the entire state of Washington.  As you deliver valuable increments of data, your Product Owner may decide that the Puget Sound area was really all that was needed now that s/he has been using the data you already delivered.  You're done with that request, on to the next "something of value"!

So, keep delivering.  Keep getting feedback.  Keep adapting your process to deliver the most value possible to your Product Owner.  It doesn't matter if it's a GIS application, analysis, map, model, or metadata...you'll find your Team delivering higher quality, higher value GIS products faster and more efficiently using Agile practices.

I'm not a huge believer in tools for tools sake.  And, I'm really not a big believer in big tools for project management, especially when you’re going Agile.  However, since we started scrumming, we have been using spreadsheets to manage our product, project, release, and sprint backlogs.  As our projects grew and our team embraced Scrum, our spreadsheets became increasingly complex.  We were using Excel to not only create backlogs, but also to create "executive" dashboards, developer dashboards, team utilization reports, actual time keeping for tasks, etc.  The spreadsheets were turning into a relational database of sorts...and they were getting unwieldy.  While the team was really enjoying what we had created in Excel, the ScrumMaster (that's me), was working behind the scenes to "manage" the spreadsheets.  It was taking a few hours every Sprint just to update the spreadsheets, create the correct linkages, generate the reports, etc.  In addition, as we tried to link multiple projects across Excel workbooks for utilization reports, timekeeping, and other assorted TPS type reports, things grew even more complex.

So, we thought, "We're a group of developers, let's turn this into an application."  I sketched out a database schema for the back-end and started putting together estimates for building our own custom solution.  At the same time, I began evaluating COTS solutions.  I looked at several vendors' products.  Some were desktop applications and others were web apps.  When I finished my evaluation, it looked like our costs to design, develop, and implement our own solution were easily going to be in the six-digit range.  Now, we work for a consulting firm and internal R&D/dev projects don't usually get funded at that level.  So, we turned our sights back to the COTS products and did a more rigorous evaluation.  We narrowed the field down to two solutions and trialed them both for a month.  At the end of the trial, it was clear that Rally Software had the solution that fit our team the best.

Rally is a web based solution that is completely hosted on Rally's servers.  This meant ease of maintenance and no worries about up-time, updates, database maintenance, etc.  Overall, we concluded that Rally hit about 95% of the functionality we were looking for as well.  Rally offers three different flavors of its solution: Rally Community, Rally Program, and Rally Enterprise. 

• Rally Community is free for up to 10 users and supports single project, single release teams.  It offers basic Agile project management and integrated defect management as well. 
• Rally Program supports Agile project management across multiple teams, multiple programs and projects, and multiple releases.  It has options for integrated defect management and test management, as well as an available web API.  It runs $39/month for the core package, and an additional $8/month for each option ($10 for the API). 
• Rally Enterprise is a full blown project management solution which has everything the Program solution has, plus all of the options and premium support.  It runs $65/month. 

We selected the Rally Program solution.  We also purchased the defect management option, and we have a trial license for the Web API.  We found the Rally sales team to be very helpful in getting started up.  They're support has been good so far and they even gave us a 1 hour into to Rally for the entire team. 

We just started using the application this week on one of our biggest projects.  We found it very easy to import our existing user stories and backlog items from Excel into Rally using their CSV import templates.  Once we had our backlogs populated, it was simple to create Projects, Releases, and Iterations within the program.  Moving a backlog item into a Release or Sprint is as easy as dragging and dropping it from the backlog into the Release or Sprint backlog.  Creating Tasks within a Sprint backlog item is also extremely easy.  We were able to assign Tasks to team members and instantly see how each task affected individual utilization for the Sprint.  In addition to creating backlog items, etc., we also imported our existing defect lists from our SharePoint site using Rally's defect CSV templates.  Again, totally easy operation. 

In the short time we've been using Rally, the entire team has been completely impressed with the functionality and great performance.  We can't believe how fast the app works.  You won't even realize it's a web app after a few minutes of using it.  Our seasoned developer team has stated that they think Rally is one of the best web apps they've ever seen, and that's saying a lot.  The Team has been jumping right in and taking advantage of everything Rally has to offer.  And I have been taking it a little easier now that I don't have to spend countless hours maintaining our old spreadsheets!

You've probably heard this conversation before (or even been a participant):

Developer 1: I'm done with the Foo functionality?

Developer 2: Is it tested and documented too?

                                Developer 1: I said it was done, not done done!

A question that arises often for many Scrum teams is "When is some particular functionality done?"  A primary rule of Scrum is that at the end of every iteration, or Sprint, the Team must build a potentially shippable product increment.  That is Scrum's definition of "done".  So, what exactly does a potentially shippable product increment mean?

The first thing a Team needs to understand is that potentially shippable does not necessarily mean shippable.  What it means is that if a Product Owner decides that they want to immediately implement the functionality developed during a Sprint, they could.  So, what must be accomplished to consider specific functionality able to be implemented immediately?  If you ask ten different teams, you will probably get ten different answers.  Some may consider designed and coded software potentially shippable.  Others may insist on testing and user acceptance.  In short, there are numerous ways to define "done".  Check out the diagram below which illustrates the range of what "done" can consist of.

A Team can feasibly decide that "done" means everything from Analysis through Live.  The goal of any Team should be to extend the range of "done" as far as possible.  Our Team currently defines "done" to mean analyzed, designed, coded, tested, and documented.  However, we're still struggling with what exactly documented means.  Some of our Team believes that developer documentation is enough.  Others have document their functionality in our project Wiki.  I would personally like to see User Documentation.  I think that if something is to be potentially shippable, that means that end users have some sort of documentation, even it's just in help files. 

So, the next question is, how well does the functionality have to perform to be considered potentially shippable.  I don't believe that functionality has to be completely cohesive to be shippable.  Mike Cohn gives a good example of this: "We can develop the Print Preview functionality and consider it potentially shippable, even if we can't print yet."  That means that we have developed a piece of complete functionality that does what it is supposed to do and does it well.  It lets the user preview what they want to print.  As for the actual printing, we may tackle that in the next Sprint.  But if you want to do print previews, we've got that for you now, it's "done".  You can implement it today and starting print previewing to your heart's content.

My best advice to other Scrum Teams is to clearly define what "done" means for your Team.  Take some time as a team to think about it seriously, write it down, and post it in your Team room if you need to.  Currently, our Team places individual task cards on our Sprint Task Board to make sure we are not just designing or coding our functionality, but also testing and documenting it.  I am confident that eventually, we'll get to the point where everyone on the Team recognizes our definition of "done" and no longer needs those prompts.  It will just be part of our project Zen.

Finally, clearly communicate your definition of "done" to your Product Owners, and strive to constantly meet that definition.  It's important that your Product Owners can clearly see what has been "done" in each of your Sprint Reviews.  If they view functionality that is truly "done", they'll be able to provide better feedback for the Team.  And, if they ask for a particular piece of functionality to be implemented tomorrow, you're Team will be able to confidently deploy that functionality.  It's what a good Product Owner expects, and what your Team should be able to deliver.  

So, the next time you ask one of your Team members if they're done with the Foo functionality, you should be very clear about what they mean when the tell you it's done...done!

In just about every article I’ve posted so far, I’ve most likely mentioned the book Agile Project Management with Scrum by Ken Schwaber.  You're probably wondering why I keep referring to it.  No, I don't get $1.00 each time I cite the book (although, that would be nice).  So I figured, it’s probably about I time I wrote a quick post about this book.

Who should read this book?  Anyone interested in learning the basics of Scrum.  No need to be a project manager type.  This book is for everyone.  The managers, the developers, the DBA’s, the analysts, the clients…anyone who works on any type of project can benefit from this book.

What’s inside?  Ken starts off by explaining the basics of Scrum.  In the first 14 pages, he covers the science of Scrum and provides a great overview of the entire process.  If you read nothing else, read these 14 pages.  The remainder of the book goes into detail about the various Scrum roles, planning a Scrum project, project reporting, and scaling projects with Scrum.  Most of the book is filled with case studies that clearly illustrate the concepts Ken is relating about Scrum.  The book concludes with 6 pages of the essential rules of Scrum.

Why it’s on my bookshelf?  This is the book that made it all click for me.  In my eyes, this is the bible of Scrum.  All of the basics are covered in a very easy to understand way.  As a ScrumMaster, any time I feel our Scrum process is slipping a little, I open the book, read a few pages, reflect on our Team’s current process, and use the guidance from this book to help make the necessary course corrections to get back on track.  The book is filled with numerous case studies that provide great insight into situations that real world ScrumMasters, Scrum Teams, Product Owners, and their organizations frequently run into.  I have probably read this book at least six times now as we have started on our Team’s Scrum journey.  As our Scrum process matures and I reflect on the case studies in this book, I can easily recognize things that our Team, our Product Owner, our organization, and I (the ScrumMaster) have done, both good and bad.  I have gained a lot of confidence by coming to realize that we're not the only Team who has experienced the ups and downs that Ken described in the case studies.  Overall, this is a great read for anyone who is interested in producing high quality, valuable products faster.  If you’re considering using Scrum on your projects and you haven’t read Agile Project Management with Scrum, what are you waiting for...run out now and get this book!!!  (Or just order it from Amazon)...

In Scrum, you can be one of two things: a Pig or a Chicken.  The names come from Ken Schwaber's book in which he describes an old joke.  There are a few versions of the joke, but I like this one the best:

Sometimes Snap Shots bring you the information you need, without your having to leave the site, while other times it lets you "look ahead," before deciding if you want to follow a link or not.

Should you decide this is not for you, just click the Options icon in the upper right corner of the Snap Shot and opt-out.

Success is a funny thing.  When we started Scrum, I think we got the go ahead because management didn't think it would work.  They cut us some slack to try something new, but we'd be back to waterfall before we knew it.  Now that we're using Scrum and producing quality, valuable software on time and under budget, everyone wants to know how we did it.  They also want to start marketing it and selling it to potential clients.  But they don't really want to sell Scrum.  They just want to sell the results.  Part of the problem is that although we have worked very hard to explain the basic practices of Scrum and Agile processes, most of the folks trying to sell our services (and our "process") still don't get it. 

I recently attended a short-list interview for a large county government GIS contract.  We were in the top three vendors selected to compete for the contract.  Dave Bouwman (our lead architect) and I were asked to provide input on our software development practices.  We had a few slides about our development environment, a few on our relevant project work, and one slide on Scrum.  The Scrum slide had no text, just a diagram illustrating the Scrum process that we borrowed from Mike Cohn at Mountain Goat Software (we like being lean and like to talk more about processes than try to put them on great big slide presentations).  When I got to the pre-interview walk through of the team's presentation, the Scrum slide was conspicuously absent from the main presentation.  In its place was a very impressive looking program and software development workflow diagram.  Lots of boxes with labels like initial concept discovery, statement of work, draft work order, kickoff, work execution, testing, acceptance, and closeout.  They were all lined up with nice little arrows connecting them in the traditional waterfall fashion.  At punctuated, pre-determined points along the waterfall, there were other boxes representing where we would have contact with the client.  The only contact prescribed for the client during the work execution was for program financial status and schedule status. 

I asked where in this melee of boxes on the flowchart our Scrum process fit.  The answer I got was "Oh, in the work execution box of course'.  Wow!!!  I couldn't believe what I had heard.  The whole thought process was that Scrum was just a little development process that worked inside the larger project management framework...just another little cascade in the larger waterfall.  This program would have multiple projects running at one time, all working in a waterfall framework, with only the work execution "boxes" utilizing Scrum to develop software.  For all of our hard work, we still hadn't convinced the management team that Scrum was an entire project management process.  We had been talking about Scrum as an overarching process for everything for months now.  We had been emphasizing not only client communication, but very active client participation in the Scrum process, not just project financial/schedule status reports.

Fortunately for us, we had a set of backup slides with our Scrum process diagrammed out.  The interview panel asked how we would manage our projects and I was able to show the slide and give them the Scrum elevator speech.  They were interested in how Scrum would provide them valuable, useable software in a relatively short period of time.  They asked several follow up questions and were genuinely interested in the process.  Our senior manager on the interview team backed our Scrum play and added some good insight into how our GIS development team was effectively using Scrum on other large scale projects as well.  However, he took the default position of saying that whatever suited the client, be it Scrum or a more traditional waterfall approach, we would provide it.  He wasn't being a jerk.  He was just trying to sell our services and I understand that.

The point of all this is that it is very difficult to break the almost institutionalized acceptance of the waterfall approach to project management.  Agile methodologies and Scrum in particular are scary to most people.  Some mistake its relative simplicity for inadequacy in terms of project reporting and project management.  Some underestimate the value of the iterative approach and the inherent increase in client communication and satisfaction that it provides.  Most are just pre-conditioned to believe that if we don't understand everything up front, we can't run a successful project.  So, I'm not upset with our management team.  I know why our Scrum slide turned into a little box on the larger waterfall slide. They believe Scrum is truly just a software development methodology that fits neatly into one of their waterfall boxes.  However, Scrum is a powerful project management process that works for many types of projects.  I know of people in other industries using Scrum for anything but software development.  I met a woman at a training class a few weeks ago who is using Scrum on real world construction projects for large hospitals.  I've even heard of people using Scrum to plan their weddings.

So, keep selling Scrum to your management teams.  Keep educating them.  Keep correcting them when they get it wrong.  But do it in a manner that helps them to understand and accept Scrum over time.  It’s our job as practitioners of Scrum to help others understand the process as more than just a software development methodology.  We need to become Scrum evangelists of sorts, bringing the real message of the efficiency and value of Scrum to the non-believers. 

On our flight home after the short-list interview, I referred our manager to Ken Schwaber's book Agile Project Management with Scrum.  I also handed him a whitepaper called A CIO's Playbook for Implementing Scrum by Rebecca Traeger (available from the Scrum Alliance website).  He's coming around slowly.  Before we took off, he commented that he see's the value in Scrum, "It's basic common sense.  How could it not work?"  In my head I thought, "It's basic common sense, how could you still not get it?"  Instead, I just smiled and said "Yup, you're right!" and kept my hope up that he'd actually read the whitepaper and the book someday. 

You can now get Chris Spagnuolo's GeoScrum! delivered directly to your e-mail inbox.  As soon as new posts are available, they'll be immediately e-mailed to you.  To subscribe to my new e-mail delivery service, just click on the navigation button to the right that says Get GeoScrum! e-mail delivery.

Shortly after we read Ken Schwaber's book Agile Project Management with Scrum, we put together a presentation to our management team to convince them that we could make Scrum work.  To our surprise, they gave us the high-five and now we actually had to do it.  We could have tried out Scrum on some small projects just to test the waters.  But at the time, we were working on a complex multi-year enterprise GIS development project for a large state agency.  The client is very communicative and very conscientious about the scope and value of the project.  We decided that this was the right kind of project and client to engage with our new found knowledge of Scrum.  Before we proceeded, we gave another presentation about Scrum to the client.  During the presentation we outlined that this process would require a much higher degree of client-contractor interaction than usual and wasn't quite as easy as it all sounded.  The client agreed that Scrum looked very good and agreed to become a committed Product Owner.  That sealed the deal.  We had our management's approval, we had a willing and able client, and we had a team of developers who were committed to making Scrum work on this project.

One challenge we had with this project was that there was a heavy architecture and framework component that we were building on top of the ArcGIS ArcObjects model.  The architecture and framework components were not easy to demonstrate to a client and we didn't know how we would demo them at the end of our first Sprint.  We made a decision that we would work on our framework for an internal client at our company for the first few Sprints.  The internal client was a subject matter expert who knew the client's line of business very well.  So, we thought we'd "shield" the client from having to deal with complex framework issues.  

Before our first Sprint, we needed to come up with a Project Backlog to work from.  The project was originally scoped and scheduled by another project manager using a traditional waterfall approach.  As such, we had a Microsoft Project Gantt chart that gave the broad strokes of the project schedule.  We also had a functional requirements document that had very high level use cases for the project.  The use cases were written in very technical language and didn't really communicate what the goal of each use case was.  But, without any better starting point, we used these two sources to build our first Project Backlog.  Because the project is divided into very neat modules based on specific business practices within the client organization, we actually built several "Release Backlogs" based on these modules.  The first "Release Backlog" we would tackle would be the Framework Backlog.

We had decided to do a two-week Sprint for our first Sprint.  It would be enough time to see how this Scrum thing worked, but short enough to pull back if it began falling apart.  We had our first Sprint planning meeting and timeboxed it to four hours.  It was very enlightening for the entire Team.  Our lead GIS software architect, Dave Bouwman, detailed the entire architecture and framework for the rest of the Team.  For some of them, this was the first time they had heard any of the details.  Dave answered all of their questions until the Team was satisfied that they understood the work ahead.  We then prioritized the Framework Backlog ourselves and tried our best to estimate the number of hours each large chunk of work would take.  It was very difficult to estimate because the use cases and work items in the backlog encompassed so much work. 

Next, we began the process of selecting our Sprint 1 Backlog items.  As we selected a use case from our Framework Backlog, we placed it into our Sprint Backlog.  We then fleshed out all of the tasks that we needed to accomplish to complete the use case or work item.  We decided that tasks should be no longer than 4 hours in duration, so we needed to do quite a bit of decomposition to get to that point.  As we defined the tasks, we also decided who on the Team would actually be responsible for that particular task.  We kept track of how much time each Team member was committing to for the Sprint as they selected their tasks.  Once someone reached 85% utilization (our organization's magic utilization number) for the two week Sprint, they couldn't select any more tasks.  We did it!  We were ready to roll on our first Sprint...

The first Sprint actually went surprisingly well.  We were off and developing our Framework.  We had our Daily Scrums and felt very good about this new level of project communication and synchronization.  We used an Excel spreadsheet that we designed to track the progress of our Sprint.  The Team's sprint burndown curve was looking great and we were heading for a successful first Sprint.  However, at the end of the Sprint, we realized that we had over committed ourselves slightly.  There were a few items that were not going to be finished.  Being the Sprint rookies that we were, we chalked it up to bad estimating and moved forward.  We ended the first Sprint with about 16 hours of unfinished work. 

After the Sprint ended, we held a Sprint Review.  We timeboxed it to two hours.  Everyone demonstrated their work to our internal client and we were all very happy with how things progressed out of the gate.  We also did a Sprint Retrospective and timeboxed it to two hours as well.  To get the most out of the Retrospective, we used several techniques from a book called Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen.  First we developed a set of Working Rules.  That is, a set of rules that would help us have more productive meetings.  Next we did a Timeline activity.  On our white board, we set up a calendar of the days in Sprint 1.  Each Team member used large Post-It notes to add significant events that occurred during the Sprint.  The events could be technical, organizational, or team related.  After everyone posted their events, we sat back, discussed each event and analyzed what we had all shared with each other.  We tried to determine at which points during the Sprint the Team was feeling surges of high energy and when they were feeling low.  This helped us pick out positive and negative things that occured during the Sprint.  Based on this review, the Team had concluded the following about our Sprint::

1. We didn't have enough information to effectively build the components we needed.  Not enough design work was done to prepare for the Sprint.
2. Our lead architect (who had all of the design knowledge) was being constantly pulled away from his Sprint tasks to respond to non-project requests from elsewhere in the organization (he's in very high demand!).
3. We had some non-project infrastructure issues we needed to address in order for the team to work together more efficiently.

So here we were at the end of Sprint 1, feeling good, but knowing we had definite room for improvement and learning.

My Reflections on Sprint 1

Looking back at our first Sprint, there are many things I notice now that we could have done differently. 

The first thing that strikes me now is that we probably should not have focused on only framework components.  After convincing the client how great Scrum would be, we said goodbye for a few weeks and worked on things the client couldn't see.  At the end of the sprint, we didn't deliver a single piece of business functionality...and thus we broke the first rule of Scrum.  After listening to Ken Schwaber's Google Tech Talk on Scrum and attending Mike Cohn's Certified ScrumMaster course, I realized that we should have at least found one small increment of functionality that we could have pulled all the way through a layer of our framework during our first Sprint.  It would have kept the client engaged and helped us prove that our framework was working.  While it's alright to be doing heavy architecture during the opening Sprints, always keep in mind that it is more important to deliver business functionality.  As the Sprints progress, architecture and framework work will steadily decrease as the focus on business functionality increases.  If they are plotted on a graph, with amount of work and time on each axis, architecture/framework and business functionality would look something like this:        

Another observation on our first Sprint was our difficulty in estimating the size and complexity of the high level use cases.  I think our main difficulty was in the fact the use cases were written too technically and too abstract and didn't emphasize the user story.  We have since been writing our Project Backlog items as User Stories.  User Stories let us clearly identify the requirement in terms of the user's expectations, needs, and conditions of satisfaction.  They are structured as follows:  "As a [user role or type], I want to [do something useful], so that [some reason why]".  We have also since discovered that estimating with hours for large tasks is not effective.  We have begun using Story Points to estimate our Project Backlog items, but that is a topic another blog entry altogether.

A common trap we fell into was the belief that we had to do design work before our Sprint work could be tackled.  We also did no documentation or testing during our first Sprint.  We were really not "done" with any of the components we built in Sprint 1.  All of the functionality developed during a Sprint should be designed, coded, tested, and documented during that Sprint.  We are actively working to make this our definition of a complete increment of business functionality for our future Sprints. 

What did seem to work well for us was the two-week Sprint length.  The Team has agreed to keep going at this Sprint length for the duration of the project.  It keeps us very engaged with the client and gives us a very clear focus and end-point to our work.

We did decide as a Team that we were spending far too much time on the Sprint Review and Sprint Retrospective and have reduced the timeboxes to 1 hour for the Review, 30 minutes for regular sprint Retrospectives, and 2 hours for release sprint Retrospectives.

Before I begin blogging about our Scrum journey, it is probably worthwhile for me to provide an overview of what exactly Scrum is.  I'll provide a concise description here and elaborate on specific Scrum components in future posts.  Scrum is an Agile process which was originally developed by Ken Schwaber and Jeff Sutherland in the early 1990's.  Essentially Scrum is a lightweight, iterative process for managing complex projects.  Scrum emphasizes complete project visibility, self-organizing teams, and constant inspection and adaptation throughout the life of a project. 

The Players in Scrum

Scrum has 3 basic roles: the ScrumMaster, the Product Owner, and the Team.  The ScrumMaster is responsible for the Scrum implementation and is the driving force behind the Scrum process. The ScrumMaster essentially replaces the traditional project manager.  However, the ScrumMaster is not the team leader.  Instead, the ScrumMaster's primary responsibility is to remove any impediments that stand in the way of the Team reaching its goal of producing quality products.  In effect, the ScrumMaster is a servant leader. 

The Product Owner is responsible for managing and prioritizing the work that the Team needs to do, known as the Product or Project Backlog.  The Product Owner also secures funding for the development of the product. 

The Team is a cross-functional team, usually consisting of no more than 3-8 team members.  Mike Cohn of Mountain Goat Software describes the size of a Scrum the best as "The two pizza team.  If you need more than two pizzas to feed the team, it's too big!"  The Team is responsible for developing the product and for selecting items from the Project Backlog that they can turn into potentially shippable functionality at the end of every iteration.

The Mechanics of Scrum

Scrum is an iterative process.  It is based on constant cycles of inspection and adaptation.  Scrum breaks complex work into short iterations called Sprints.  A Sprint is generally 2 to 4 weeks in duration.  Before a Sprint, the Product Owner reviews the current Product Backlog of requirements (we call them User Stories) and prioritizes them to ensure that the items that are most valuable are at the top of the Project Backlog. 

Before a Sprint begins, the Product Owner, the Team, and the ScrumMaster gather at a Sprint Planning Meeting to discuss the current Product Backlog.  Our Sprint Planning meetings are generally timeboxed to 4 hours.  The Team asks all of the questions they need to in order to fully understand the Project Backlog items.  More time is spent defining the higher priority items and less time on the lower priority items. When the Team is satisfied that it understands the Backlog items, they select the highest priority items that they believe they can complete by the end of their Sprint.  The Team estimates the size and duration of the items they select to work on during the upcoming Sprint.  This selection of work that the Team collectively commits to is called the Sprint Backlog.

During the Sprint, The Team is left to work on the Sprint Backlog items with minimal interruption.  It is the responsibility of the ScrumMaster to ensure that the team is not disturbed or interrupted from their work.  During the Sprint, no work may be added to the Sprint Backlog by the Product Owner.  The goal of every Sprint is to produce a potentially shippable product increment.  This means that during each Sprint the Team must produce high quality, tested, complete functionality.  The Team's progress during a Sprint is tracked on a Sprint Burndown Chart.  The Sprint Burndown Chart indicates the amount of work remaining in the current Sprint.  The Team's overall progress on a project is tracked on a Product (or Project) Burndown Chart.  This indicates how much work is remaining on the entire project.

During the Sprint, the Team and the ScrumMaster meet on a daily basis at a meeting called the Daily Scrum (or just the Scrum).  The Scrum is a short, time-boxed meeting of no more than 15 minutes.  The Scrum is meant to synchronize the work of the Team and provide transparency into the development of the product on a daily basis.  During the Scrum, Team members answer three basic questions:

1. What did you work on yesterday?
2. What are you working on today?
3. Are there any impediments to getting your work done?

At the completion of the Sprint, the Team, the ScrumMaster and the Product Owner gather at a timeboxed meeting called the Sprint Review.  Our Sprint reviews are timeboxed to 1 hour.  At the Sprint Review, the Team demonstrates the functionality they developed during the Sprint.  No PowerPoints...no prototypes...no "This would work if..."  The Team must demonstrate working software that has been coded, tested, and documented. 

After the Sprint Review, the Team and the ScrumMaster gather without the Product Owner for a Sprint Retrospective.  The Sprint Retrospective is a timeboxed meeting (ours are timeboxed to 30 minutes for regular Sprints and 2 hours for release Sprints) at which the Team determines what has been working for them, what hasn't, and how they can make their Sprints more effective in the future. 

At the completion of the Sprint Retrospective, the whole cycle begins again with a Sprint Planning Meeting.  To visualize the entire Scrum process, check out this re-posting of Mike Cohn's graphic below (I think it's the clearest representation of the process):

Like most people in GIS and software development, I have an extensive book collection that I rely upon for ideas and information about varous topics.  From time to time, I'll post information about books I'm currently using in the Bookshelf Category.  I'll provide a brief summary of the book and sometimes a short review.  Hope you find some of these books helpful.

I have been working in the Geographic Information Systems (GIS) field for over ten years.  If it involves GIS, I've probably done it...everything from field data collection to large scale enterprise GIS deployments.  I've worked with teams of varying sizes and technical ability in a wide variety of roles, from GIS technician to project manager.  Over the years, I accumulated a wealth of knowledge from the teams I've worked with.  This includes both technical knowledge as well as insights into team dynamics.  The one constant throughout my career, however, has been a general frustration with traditional project management methodologies as they applied to GIS and software development. 

My first exposure to project management was at the hands of the Project Management Institute (PMI).  I learned about the benefits of project management standards and dove headfirst into PMI's Project Management Body of Knowledge (PMBOK) and believed that this was the cookbook for project management.  I learned many valuable lessons from the PMI methodology that were very useful.  However, much of what was in the PMBOK didn't seem to apply to my GIS and software development projects.  The process didn't fit, no matter how hard we tried to make it.  Somehow, it just wasn't flexible enough to accommodate the dynamic world of GIS development projects.  Projects were constantly in turmoil.  Sometimes, we'd deliver products that didn't entirely fulfill the customer's needs.  Other times, we'd satisfy the customer at the cost of budget over runs.  Most times, our projects would just come in behind schedule.  But we followed PMBOK...what could have possibly went wrong?

I decided at that point that maybe I was just doing something wrong.  I talked with several project management mentors who all espoused the virtues of the waterfall approach to managing large complex projects.  "Lots of up front planning", "Do in-depth requirements analysis before you get started", "Get a good plan in place and you won't go wrong", "Document everything in detail and the team will know what to do down the line", "Use the processes and tools we've developed.  Plug the numbers into the tool and you'll be good to go".  These were the types of answers I would commonly hear from my mentors.  Sounded good to me.  I'd go back and plan, plan, plan until nothing was left to chance...I mitigated every risk.  We put together great requirements documents that we culled from interviews with clients.  We had lots of impressive binders with all sorts of documents and plans.  We'd impose a good structure on the team, and delegate tasks according to the plan.  We walked away from the client confident that this time we'd nail it. 

Six months later, the projects were again woefully over budget and behind schedule.  Things were taking longer than we expected.  We were over committed.  We hadn't produced anything of value, let alone released anything to the customer.   We hadn't seen the client in over half a year.  The only communication was the company standard monthly status report.  We had to renegotiate our contract with our customers.  It turned into an adversarial relationship.  By the time we delivered our final products, technologies had changed, the customer's requirements had evolved and we had missed the mark.  Although the projects were technically impressive, they failed to achieve any great measure of success in terms of client satisfaction.  In addition, some had lost money for our company.  I looked again to my mentors and upper management on ways to fix these apparent problems managing these GIS development projects.  The organization that I worked for was inflexible.  "We manage by the waterfall model, it's proven and it works", "We have a process you have to follow", "Modify the contract to get an extension" was the direction I got.  I'm drowning in this....maybe I'm just not cut out to be a project manager.  The doubts crept in. 

Then one day, I happened upon a book that I received for attending a local GIS workshop.  The book was Agile Development with Iconix Process.  It sat on my bookshelf for a few months.  Then I opened the book.  I couldn't believe what I was reading.  Adaptive planning...embracing change...evolving requirements!  How in the world do you manage a moving target like this?  But I kept reading.  It described several different Agile methodologies including Extreme Programming, Test-driven Development, Crystal, Feature-driven Development, and several others.  At the end of this list were two small paragraphs on the Scrum development process.  Little did I know at the time that I had just read two paragraphs that would eventually alter my professional life as a project manager.

Over the next 2 years, I switched jobs a few times looking for the right organization.  Throughout those two years, I investigated Agile methodologies.  I found the Agile Manifesto online through the Agile Alliance website.  I loved it.  It was more than just a project management methodology...it was a philosophy.  It struck a chord with me.  It was the answer I had been seeking for the past 7 years.  If you've never read the manifesto, here it is in its superb simplicity:

"We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

- Individuals and interactions over processes and tools

- Working software over comprehensive documentation

- Customer collaboration over contract negotiation

- Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more."

For the next year, I slowly began using Agile principles with our small local team of analysts and developers.  It was under the corporate radar and nobody really knew it was even happening.  I started the slow movement away from traditional project management.  I'd let our team decide what we were doing for the next week.  We stopped trying to document everything.  We talked to the client on a regular basis and shifted our project work to fit with their needs.  We still had to report to the organization as if we were following the plan.  But it was working.  We were delivering things to the client quickly and efficiently.  And more importantly, we were delivering valuable analyses and software to our customers.  Our new found Agile approach was validated when we won an award from ESRI for Best Analytical Presentation at the 2006 ESRI International Users Conference.  It was on a project that we had employed Agile principles on.

In the spring of 2007, I joined a new organization as a project/office manager.  I was given the freedom to shape the office however I needed to in order to get it working better.  It was there that I met Dave Bouwman, our Senior GIS Software Architect.  Dave and I shared the same vision of a truly agile team of GIS developers. We knew agile worked.  We had heard the stories.  We had both tasted a bit of agile success.  We picked up copies of Ken Schwaber's book Agile Project Management with Scrum.  As we read it, we became very excited about the prospect of using Scrum on our GIS development projects.  We didn't hesitate at all.  We convinced our management team that this was worth trying.  We got the green light...and we dove right in!  This blog is the story of our Scrum journey...