While there has been a bit of research detailling the advantages of agile software development not that much systematic work has been done to find the issues arising from using the agile paradigm. So, what are the issues with agile software development? In “A Comparison of Issues and Advantages in Agile and Incremental Development between State of the Art and an Industrial Case” authors Petersen and Wohlin list a number of issues and advantages related with Agile software development.

Issues with Agile Software Development

Petersen and Wohlin gives three lists of issues. I think they will be helpful to many professionals so I reproduce them here in my own words:

Issues which were already known from previous research

1. Challenges with continous testing
2. Increased release frequency leads to increased maintenance
3. More management overhead due to the need to coordinate more teams
4. Detailled dependencies are discovered late due to the lack of big-up-front design
   

   General issues found in the article

5. Long requirements engineering due to complex process
6. Requirements priority is hard to create and maintain
7. Waiting times in the process, especially design waiting for requirements
8. Reduction of test coverage due to lack of resources and independent testing
9. Increased configuration management effort

Finally, they list a number of context specific issues which I will not mention here.

Comments

Issue 1: Continuous testing for Agile

I have had plenty of experience in working in projects where setting up continous testing has been difficult to say the least. There are so many things that have to work in order for continuous testing to work. Just to mention a few:

• There must be a working automated build and packaging script.
• There must be a working environment in which the aforementioned script is running regularily.
• Developers must work on a common branch or at least commit or merge to the common branch frequently.
• It must be possible to deploy the software to a test environment automatically. Deployment cannot include manual steps and frequent changes which break the automated testing.
• Test interfaces must be relatively stable so that the automated test cases do not break all the time.
• Suites of automated test cases must be available.
• The automated test cases must provide meaningful information about the system under test.
• The development team must establish a sound culture around continuous testing so that the information is used and acted upon.

Issue 2: Multiple Configurations and Agile

Personally, I have not encountered that many problems arising from this issue. I guess that most people handle this by simply requireing customers to use the latest version of the software if they want to get support. In Chrome, for instance, the latest version of the software is downloaded and installed in the background and will be used the next time you start the browser.

Issue 3: More Management Overhead

Again, I have never seen this is a major problem in practice. Most methods require coordination and the overhead rises rapidly as more teams are involved. A software system must not be too simplified if it is going to serve its purpose.

Issue 4: Dependencies Discovered Late

The idea that dependencies would be discovered late due to the lack of big design up front is a bit foreign to me. In my experience any development method will suffer from this problem. The key is to maintain a modular architecture and to always be willing to replan and refactor. Successful design work requires frequently moving between different abstraction levels so that what is desireable can be reconciled with what is possible.

Issue 6: Hard to Maintain Priorities and Agile

In my experience creating and maintaining priority lists for agile software development requirements lists is very hard and seldom successful. On the surface, it sounds very easy to maintain a list of priorities and then simply do the most important thing first. Logically, if we do the most important things we are always working on the right thing and nothing can go wrong… I have a few issues with this:

• There is no single oracle to turn to, to obtain the priorities of requirements. Sometimes stakeholders will have widely varying needs and expectations.
• Maintaining a long list of requirements where the items with the lowest priority will be implemented some time in the far future is discouraging to both developers and stakeholders.
• Our limited cognition and inability to foresee the future makes it hard to determine the real priority of things.

My preferred approach is to try to organize requirements into minimal meaningful releases or potentially shippable increments. In that way only a rough priority is needed for each and the decision comes down to – do we do this in this release or the following release?

Issue 8: Reduction of Test Coverage and Agile

Since all testers are absorbed into feature teams they will be working hard with the rest of the team to meet the requirements selected for the current iteration. Who will cover the rest of the testing needs? I have seen three or four approaches to solving this issue.

Next Steps

I really hope that Petersen and Wohlin or some other research group will follow up by quantifying the risk levels associated with each of these issues. What is the probability that each of these issues will occur in a specific setting? What level of consequence will the issue lead to? What do people generally do to handle these risks? Does it work?

During the next few days I will expand on a few of the issues above in separate posts.