Defining and measuring the success of an Agile methodology



Alistar Cockburn describes in this post  the success of a methodology with the formula:

Methodology Success = Project delivered + Staff would do it again

In software projects people is a fundamental variable, so here with methodology is not intended say RUP or FDD or say XP or Scrum or Kanban. Instead is intended that

A methodology is the conventions the team agrees to and actually follow


When the team achieves a methodology success as defined above, it is possible to infer that the team and also the method they used, RUP or FDD or one of the others,  helped the project success.
In case of failure, it is hard to say if it was because the method was not helpful. Indeed there are many factors that affect the project. These factors include the team together with the tools and technology used and the available resources. Also the context matter, the environment within the team operates (i.e. the company culture, structure, rules, ...). The project itself could be impossible to achieve.

So from a failure is not possible to infer information on the method itself. Instead

 when there is a statistically relevant number of teams successes with a specific method

we can infer that the method, say RUP or FDD or say XP or Scrum or Kanban, helps project success.


Let me expand on the meaning of  Project delivered + Staff would do it again  based on my personal subjective experience and on reflections I made reading this book [1]:

It means that team accomplished the project goal and fulfilled team members needs

Is useful to note that the project goal includes fulfilling the needs of the users, the stakeholders and of the core group of the company [2].
Is also useful to take into account that a project has multiple outcomes that should be considered too:

the project could have
  • increased the team-work effectiveness
  • evolved the tools and technology used for accomplishing projects
  • elaborated useful information and taught new things and generated new knowledge
  • resolved conflicts influenced or attained consensus between teams and departments within the company
or the opposite, the project could have
  • created more technical debt
  • increased the distrust of the customers
  • increased conflict and misunderstanding between company departments

Is useful to note when considering 'Staff would do it again'  that this evaluation to be meaningful requires that the Staff does not show evident dysfunctional dynamics because that would affect his judgment. Extreme examples from real life are people addicted that don't want to change an harmful habit or people that intentionally ends functioning beneficial relationship.


A final and interesting note, the comment from Alistar in in the final comment:



Really astonishing how few projects pass those two tests



About the first test, project delivered, this data is available [3][4]. And about the state of Agile development this data is available [5].



[1] Small groups as complex systems; H. Arrow, J.E. McGrath J.L Berdahl; Sage Publications; 2000
[2] Core group theory, Art Kleiner
[3] http://www.ambysoft.com/surveys/success2011.html
[4] http://blog.mountaingoatsoftware.com/agile-succeeds-three-times-more-often-than-waterfall
[5] http://www.versionone.com/state_of_agile_development_survey/11/


Tags :   |  |  |  |

Print | posted @ Monday, February 27, 2012 6:28 PM

Comments have been closed on this topic.