Agile software development: Curse or blessing?
Agile software development: Curse or blessing?
Throughout the last 19 years i have learned to know a huge variety of ways to deliver digital projects. From waterfall to hyper agile, i've seen it all in more than 30 ambitious projects by now. Today i wanna shed some light on myths about agile, since it has become a huge buzzword, with a special focus on working with digital agencies.
First of all i think that it is important to understand that all agile methods are at first very theoretical and never fit 100% onto any project. Nevertheless they inherit some very relevant principles that ensure the success of projects. So the key is to pick the most important principles and define your own way of agile working that fits your project in terms of size, manpower, knowledge and other factors.
In the following article we will go through many of these principles and methods to provide a clear view on the myths of agile working. Let's start with probably the most relevant question: scrum or kanban?
While in their most extreme expression scrum is mostly focusing on slicing a project into manageable chunks (versions) and working on them in sprints while the rest of the tasks (or stories) remains in a backlog, Kanban is rather focusing on having an overview over all to-dos in one consolidated board.
The pros for Scrum are a very clear view on the next steps and what is most important for the next release as well as a nearly exact planning of resources per sprint cycle. The biggest disadvantages are additional effort that comes through the specialty of sprints and the loss of overview in late stages of the project when approaching the deadline. There are several meetings that are relevant to maintain this project structure: Refinement, estimations, sprint planning, review, retrospective.
For Kanban the pros are a better overview of the open tasks and the omission of the additional meetings, while the cons are an additional effort to align everyone on next steps and due to missing estimations the lack of the overall duration for tasks. The project might slip away regarding the time-to-market.
The question of scrum of kanban can not be answered generally. Now that we have an overview of the pros and cons, we will have to take a look at some key criteria:
a) Project size: While scrum seems to be a bit overengineered for a 100k project with a few people, the kanban style might get problems in organizing a 5m project with huge manpower. So we can say the bigger the project, the more likely it is to use scrum. We will get to a discussion on where to draw the line later in this article.
b) Agency or Inhouse-IT: Agencies will usually use scrum. Why? They need to efficiently plan their resources. A developer needs exactly 40 hours of work per week. To know how much 40 hours is in tasks, the estimations are mandatory for most agencies. If you have your own inhouse IT you might rather skip all of the overhead and focus on developing itself.
c) Dev team knowledge: While a very experienced frontend developer that has for example programmed more than 20 different PDPs (product detail pages) will exactly know what to do and therefor creates less effort in refinement of the tasks, a junior will still need more help to succeed to deliver his tasks in time. So we can say it is more likely that experienced developers can work with kanban and are rather capable of organizing themselves partly, than a junior developer the clearly needs to know what to do and what's next.
So at this point we could say: Small inhouse projects with experienced devs are kanban and big agency projects with average dev know-how are scrum? Kind of, but that would just describe the status quo of the industry and there is, in my opinion, a lot to improve.
The huge overhead agencies create with stiff scrum methods is an enormous driver of project costs. Most agencies miss out on the opportunity to raise the development time within projects. Usually 30-35% of time in projects is none-development, sometimes it goes up to an unacceptable amount. The higher the percentage of none-development time in projects is, the less efficient is the agency.
So now i would like to raise the question: When you had your own inhouse IT, let's say three developers and one product owner, would you work strict to scrum with refinement, estimations, sprint planning, review, retrospective or would you rather trust your PO that he/she will keep everyone busy and happy, put them in one room so they can ask questions anytime and skip all unneccessary meetings? I guess i know the answer. And isn't this a clear contradiction to what agile software development and especially scrum originally stands for?!
But why is everyone working like this with agencies? Because most agencies enforce this structure and there are no alternatives yet. I think this is a major opportunity for agencies to point out two key USPs everyone should look for when evaluating potential agency partners: The efficiency and the experience of the dev team. Agencies who will be able to prove this or even fix the percentage of development in a contract will differentiate themselves from rather inefficient players in the long run.
So after we have defined our goal, which is more efficiency, how do we get there? Let's have a look at the improvement potential:
Discovery: A delivery method and workflow need to be adressed in a discovery already.
The delivery method needs to fit to the project size: It doesn't make sense to work in sprints with all the overhead when you have 20 tickets on the board.
The staffing needs to fit the project size: It doesnt make sense to staff a backend and frontend lead with three juniors, when it would be more efficient for the client if two dev leads would do all on their own.
User stories: It is shocking how much time is invested to slice for example a PDP in an unbelievable high amount of bloated user stories and acceptance criteria. Sometimes i have the feeling that it would be more efficient to give the layout to one dev in one task and say: "Get back to me when you're done and we will check on improvement potential". Sure, this is exaggerated, but again: The less experienced a dev is, the more detailed you will have to describe every little component of a task.
Project managers: They are the key players for project success. If they feel too well in scrum structures and love to push responsibilities to others instead of living the project, you will miss out on the one of the biggest improvement potentials. A good project manager knows about everything, maintains the overview and keeps the project efficient. That also involves unpleasant tasks like saying to your boss that you can not invoice a dev with 100% for his/her lack of experience.
Overheads: Refinement meetings with more than ten people and all backend and frontend devs where you discuss three tasks in one hour are ineffecient. The technical refinement or tasks or stories should be done by one dev only. And this should be the dev lead or solution architect that keeps the technical overview for the project.
Testing: This is also a downside of scrum. Since you have to close the sprint on a regular basis, you can not do the client side testing on the same board. There needs to be an additional workflow or board for the acceptance of the tasks or stories. We should also think of the redundance of internal testing and client side testing (on top of technical code review). I wonder if it wasn't a good approach to directly test with clients as early as possible, but agencies usually hide devs from clients because they think it's not efficient or due to langauge barriers (mostly for offshoring companies). So agencies rather touch a ticket three times than having 5min call with the client.
Hybrid methods: Why is no one working with kanban boards with backlogs by the way? Usually a lot of the tickets are shifted from sprint 1 to sprint 2 and so on. Why then not have a kanban board as a kind of ongoing sprint with a backlog and organizing everything dynamically?
Deployment: In most sprint cycles there is one big deployment at the end where the dev lead consolidates every feature from multiple devs in one big deployment. For ongoing projects that are live everyone is hyping the buzzword continous deployment where you deploy every feature as soon as it is finished. Why not work kanban and deliver and test feature by feature and not sprint by sprint?!
Ticket Ping Pong: Person a) writes the use story, person b) adds the acceptance criteria, group c) makes a refinement meeting, person d) develops on the ticket, person e) makes a code review, person f) makes an internal QA, person f) puts back the ticket to d) for improvement, person d) brings it back to Codereview for e), e) forwards it to f) for another QA and then e) merges into current branch and deploys, g) does a last check before it goes to the client, h) client checks on the ticket and plays it back for a bug -> either this or a new ticket will start over again - is this efficient? No.
Invoicing transparency: The invoicing should be as transparent as possible: Per person, task and time. Potential ineffiencies, due to agency mistakes or overheads due to inexperienced staff should be deducted and visible. If a project manager tries to invoice every minute in a project and delivers poor quality the client will lose trust and generate lots of unneccessary discussions. The agency might be on the legal side here, when the invoicing is on time and material basis, but they miss out on the chance of equalizing obvious inefficiencies.
So what are the conclusions to raise efficiency?
a) skip theory, drill down processes to efficiency
b) a project structure must serve the project success not the internal agency preferences
c) less people with more experience
d) less meetings, reduce overheads
d) instead: more responsibility per person
e) continous deployment and continous testing can be quite powerful
f) gain more trust through transparency
Still, there is a lot to improve for digital agencies - we will have to keep an eye on the evolvement of agile processes in software development.