The Cooper Syndrome

Thursday, March 20, 2008 11:19:23 AM (Mountain Daylight Time, UTC-06:00)

image Yesterday's keynote speaker at the ESRI Developers Summit was Alan Cooper.  Cooper is the author of The Inmates are Running the Asylum and About Face.  His talk was entitled Post Industrial Management.  Mainly, he discussed how to get non-technical people to have success and achieve their goals with software products.   He did make some very interesting and valid points regarding the management and executive view of software development.  In particular, he emphasized that organizational structure and technologies that worked in the past are failing today in the post-industrial bit-centric world.  I completely agree on this point.  Most of what passes for management today is based on industrial management, but the industrial age is over.  We need a new wave of executives to find the right organizational cultures and management tactics to support the new post-industrial knowledge workers.

He also hit on one of the topics that I really think is an issue in software management today and that is the difference between effectiveness and efficiency in software development.  I have written about this before and feel very passionate about it, and it's worth mentioning again.  Cooper provided a quote from Peter Drucker that I think really summarizes this argument very well:

Effectiveness is more important that efficiency.  Business can decline and even fail at the same time that process reform is dramatically improving efficiency by saving money for the company.

I would agree and would argue along with Cooper that ignoring effectiveness to focus on cost reduction is a primary cause of software construction failures.

Cooper also made some great analogies to describe the way software is developed.  He doesn't see software development as an industrial activity.  He believes it's more like a pre-industrial craft in that things are made one at a time, they're not formulaic, and they are complicated and nuanced.  But, he also see's it as a post-industrial craft with more pieces, built by disparate teams, with abstracted notions and massive interaction between the parts.  He described the post-industrial craft of software development as existing in a brittle environment with invisible or intangible products, and working in a world of rapidly evolving tools.  I agree with most of that, except I'd like to believe that the products we produce aren't invisible to our end users.

I really liked the points discussed above.  However, I found the remainder of his talk to be a little off-base and very biased to a waterfall or modified waterfall approach to software development.  In addition to promoting a waterfall approach, he made several misstatements about how agile teams work and plan.  His first comment was about planning.  He advocated that to successfully manage software projects, you must have detailed written plans.  He also claimed that contrary to engineers' claims, requirements don't change.  He went on to slam agile practices by saying that agile practitioners say that "We shouldn't plan because things change so rapidly".  Anyone who has managed a software project using agile practices knows this is a huge waterfallacy (a fallacy about agile spread around by waterfall advocates).  Agile teams do plan at many different levels.  In fact, studies show that over the life of a project, agile teams do more planning that traditional development teams...they just do it iteratively.

Where Cooper really began to diverge from an agile approach to software development was in his description of what he called The Software Construction Blueprint.  Here are the required elements of Cooper's "blueprint":

  1. A narrative description of user personas and scenarios
  2. A behavioral overview in narrative form
  3. Detailed form and behavior specifications document
  4. Detailed code examples showing how software will work
  5. Detailed schedule and test plan

According to Cooper, by definition, blueprints are buildable and biddable.  If you build the blueprint described above, Cooper wholeheartedly believes that you can give a fixed price bid that you are confident in.  Now, I don't know about you, but I've worked on many projects that produced blueprints or whatever you want to call it, but it was heavy up-front requirements analyses.  They were not successful!  On top of that, our customers are not paying us for all of these narratives and specifications...they're paying us for working software.  Now, according to Cooper, an interaction designer builds this entire blueprint and distills everything anyone needs to know about building the software and hands off the design to a production team that actually builds the software.  Because the interaction designer does his job and divine's every requirement exactly, there should be no questions or problems negotiating the software development minefield on the part of the production team

I have several issues with this line of thinking.  First and foremost, I believe this sets up serious silos within the development organization.  It says that the production team has no input on design and design has no real input on production beyond initial design.  I really believe that teams shouldn't have different titles and divided responsibilities.  The team is the team, period. Break down these silos and strive for more cross-functional teams.  It builds a better understanding of the various project roles and makes for more effective teams. 

A side issue I have to throw out there is the fact that Cooper is an interaction designer and really came across as selling his services in the keynote (something I found rather distasteful).  Afterward, someone asked Cooper if he or his organization ever builds what they've designed.  He answered "No, we don't".  I won't elaborate here, but I'm pretty sure you're thinking what I'm thinking.

The bigger issue with his line of thinking is that all requirements can be defined up front and that they don't change over time.  We've all worked software projects before and we all know this isn't true.  Customers essentially have some idea of what they want, but as the software is developed and demonstrated, they begin to understand what they really want.  In agile, we can address those new ideas or changes without any worry that our big upfront design and planning would have been wasted.  But, after asking Mr. Cooper about customer involvement in the production phase of his development model, I completely understood why he believes that requirements don't change.  My question was simple, "Where in the production process do you share completed functionality with the customer and how do you manage the change that is inherent when a client begins looking at what has been developed?"  Aside from the relative hostility with which he answered this question, it was evident that in his model, there is no communication or collaboration with the customer in the production phase.  Apparently, it is possible for an interaction designer to not only define every last requirement up front, but they also have the innate capability to anticipate every change the client would make and addresses them in the blueprint.  I guess it would stand to reason that if you ignore your customers and not allow their voice to be heard in the production phase, you'd think requirements don't change too!

What really perturbed me was that Cooper said he doesn't believe that the customer even knows what they want, and that the interaction designer knows better than anyone what they need.  I take real umbrage with this assertion.  Essentially, his opinion was ignore your customers, you know what they need.  This to me is a real problem in the software industry.  This kind of thinking is exactly what leads to the 65% of features in most software that are rarely or never used (Check out the Standish Group's CHAOS Report).  Having someone of Cooper's stature stand up and advocate this is not going to do anything good for the software development industry.  And, considering the already low rate of agile adoption in the GIS development world, the last thing that we needed to hear in a keynote session were these toxic, old-school notions of software development.  I've been searching for a term to describe this way of thinking and thanks to the keynote speech at the Dev Summit this week, I think I have one now...The Cooper Syndrome.  Maybe at next year's Dev Summit, Dave Bouwman, myself, or someone else can give the agile counterpoint to the Cooper Syndrome.

If you want some more insight on the keynote, check out Brian Certain's blog post on the keynote.

Friday, March 21, 2008 8:12:23 PM (Mountain Daylight Time, UTC-06:00)
This post is spot on. I attended this keynote session and didn't understand why Mr. Cooper gave this presentation. The practices he advances do not result in satisfied clients. And I agree with Chris' assessment that he became fairly hostile toward the audience in the question and answer portion of the talk. If anyone from ESRI reads this blog, please take note of my displeasure with the speaker selection for your keynote. Maybe Chris is correct and someone should give the counter argument to Mr. Cooper's at the next Developers Summit.
R. Fillion
Saturday, March 22, 2008 1:39:41 PM (Mountain Daylight Time, UTC-06:00)
Well if anyone is reading this post at ESRI, know that I completely disagree with these statements and think Alan Cooper was an excellent choice.

Personally, I think the keynote was insightful and spot on. But, I also believe that there are many interpretations to what he said, and that this is just one of the interpretations. Of course in my opinion its wrong, but that's just my opinion.

There are fundamental flaws in every development processes, and if there wasn't we'd all be using the same one, right? I think a realist can't say this one is better than that one, period...especially in the ESRI world where you have small shops using the product and then large corporations doing the same. Although Mr. Cooper did point out that there are fundamental flaws in agile development it doesn't mean he was promoting waterfall or iterative. Just reiterating that he said Agile is a self fulfilling prophecy...if you go in saying you have no plan, and then things change, well that's because you had no plan to start with.

Yes, blueprints are made at the beginning of a project, but with a blueprint the foundation is the most important, so you could still have some sort of agile-ness developement on top of that afterwards.

I work for a quite large company (5K+ employees) and work on government contracts. Requirements have to be flushed out up front because the company is bidding against other larger companies for the contract. How realistic is it to say "I think I can do this job in X amount of man-hours (don't site Mythical Man Month quote, please, I've read it) but we really don't have a plan, or requirements, so things may change." There's just no way to do agile development in this case. Maybe for little one or two man shops where you know your customers on a first name basis, but not a large company like ESRI or my own.

I also agree that user's know what they want to do, but they dont know how to implement it? Should they write the code for you too? No! Users should define functionality but not implementation, which is what Mr. Cooper was expressing. I think Mr. Cooper's metaphor with the grape turning into wine was well put, meaning the user know what they want, but it takes someone skilled to translate it into something beautiful.

Hey but on a good note, this is a great decision and thanks for letting me post! I'm going to come back here for more GIS info.
Monday, March 24, 2008 12:49:23 PM (Mountain Daylight Time, UTC-06:00)
I'd have to agree with Chris on this one. Alan Cooper was a complete tool especially during the Q&A. His methodology is not going to help software development move forward any time soon.
Bill Fischer
Wednesday, March 26, 2008 10:52:06 AM (Mountain Daylight Time, UTC-06:00)
Blueprints are biddable...Ah, the beloved construction metaphor... :)

If you ever looked "Kings of Construction" or a similar series about big construction projects, you know that they have four things in common with the typical software project: They are complex, they are innovative, and they are invariably over time and over budget...
Ilja Preuß
Wednesday, March 26, 2008 9:17:57 PM (Mountain Daylight Time, UTC-06:00)
I agree with most of your article, except for this part:

"What really perturbed me was that Cooper said he doesn't believe that the customer even knows what they want [...]. This kind of thinking is exactly what leads to the 65% of features in most software that are rarely or never used"

In my experience, this is incorrect. It is our *customers* who are demanding those 65% of features of which *we* know that they are never going to be used! This is what the YAGNI principle (You Ain't Gonna Need It) is all about.

Here's some inspiration:
http://positivesharing.com/2008/03/top-5-reasons-why-the-customer-is-always-right-is-wrong/
Jurgen Appelo
Wednesday, March 26, 2008 10:20:46 PM (Mountain Daylight Time, UTC-06:00)
Jurgen,you are very correct. What I should have stated is that the customers should be able to identified what they "want" but prioritize what they "need". Many times, we really help our clients prioritize their needs and other times, their needs only become clear once we start delivering increments of functionality to them. I think this is where we can eliminate a lot of waste. either way though, communications with your customer is essential if you want them to feel like they've received a good ROI.
Thursday, March 27, 2008 8:05:48 AM (Mountain Daylight Time, UTC-06:00)
I strongly support Agile in the GIS industry, and having a keynote presenter (at the Dev Summit!) dismissing agile as a second-rate development methodology, is plain ridiculous. Requirements change, it is human nature. Spending all of the design time upfront building a “blueprint” is risking. Customers will see something somewhere along the implementation phase and suddenly the design to build “X” turns to building “Y”. Being Agile is the best way to handle most of these cases…

Excellent post!
Shane H.
Sunday, March 30, 2008 11:36:29 PM (Mountain Daylight Time, UTC-06:00)
I attended the ESRI Developer Summit and came away from Alan Cooper’s presentation with a different perspective than Chris. Alan isn’t hostile to agile techniques. He promoted agile techniques for Design Engineers to develop prototypes and waterfall techniques for Production Engineers to deliver products. Moreover, Alan will deliver the closing keynote at the Agile 2008 conference in Toronto this coming August. www.agile2008.org I doubt that they asked him to be their closing keynote speaker if he truly dismissed agile as a second-rate development methodology.

As a result of Alan's talk, I've already started using agile techniques with design engineers and realized why our production engineers find agile techniques so unsettling. BTW, the design engineers and production engineers don't agree with those different job titles. However, I know who they are because of their past behavior with regard to requirements definition and the tasks that they naturally gravitate to.
Jay D.
Comments are closed.