For some years, agile frameworks have been gaining ground to traditional software. These methods take the functional software, the work of the whole team with the client and allow changes in the requirements throughout the entire process.
But how can we evaluate the work of an agile team? In this entry we tell you how we do it in Intelligenia .
What framework do you use?
First of all is to know what agile software development framework you are using. We in intelligenia are implementing Kanban because of its flexibility, the existence of a large number of maintenance projects and the ease with which it is implemented.
We know that most development projects use Scrum, but we have wanted to be different and risk using a different method, we are so adventurous!
Kanban is a pull system in which only new tasks enter when the ones that were in development have been completed. In addition, these tasks live on a board in which each column symbolizes a particular state, allowing a constant view of the status of the project at any time.
A task board based on the Trello web tool of one of our projects in which the technical team works hand in hand with the commercial team.
If you do not sound Kanban, do not worry, there are many sources of information on this methodology. We are following the book Kanban: Successful evolutionary change for your technology business by David J. Anderson . This is the best source to implement this agile system completely and not stay in the anecdotes of using a task board, but go further and look for continuous improvement or kaizen .
Let us begin by describing the various evaluation processes.
We know that most development projects use Scrum, but we have wanted to be different and risk using a different method, we are so adventurous!
Kanban is a pull system in which only new tasks enter when the ones that were in development have been completed. In addition, these tasks live on a board in which each column symbolizes a particular state, allowing a constant view of the status of the project at any time.
A task board based on the Trello web tool of one of our projects in which the technical team works hand in hand with the commercial team.
If you do not sound Kanban, do not worry, there are many sources of information on this methodology. We are following the book Kanban: Successful evolutionary change for your technology business by David J. Anderson . This is the best source to implement this agile system completely and not stay in the anecdotes of using a task board, but go further and look for continuous improvement or kaizen .
Let us begin by describing the various evaluation processes.
How deep is the framework implemented?
Before starting to account and measure quantitatively, it is recommended to evaluate the depth of implementation of the framework.
There is a questionnaire developed by the coach agile Christophe Achouiantz and that shows in the entry Depth of Kanban - A good coaching tool of his blog.
The questionnaire is based on evaluating 7 dimensions through questions that are answered with a yes or no. Each dimension is marked on a radar type chart and we can visually see the depth of Kanban for our team.
Questionnaire adapted to our language:
This questionnaire has to be answered by the team members of each of the projects.
Achouiantz speaks of limiting the level according to the answers, so that if, for example, in Display, the answer for question 4 is "no", question 5 and following are not answered. This restriction, while it may be interesting, seemed to us that it was going to demotivate the team more than the account, so we have not applied it. Maybe when we do the evaluation again in a year we will be more demanding.
In our case, we have teams in which there are several developers, so they have had to answer the questionnaire among all, since what we are trying to see is the depth of the use of Kanban by work team in a concrete project.
Let's talk about some of our projects, which we will call (for privacy reasons) project A:
Project A is the project that Kanban's best implementation has had, thanks in large measure, to a very collaborative client and involved in this way of working.
There is a questionnaire developed by the coach agile Christophe Achouiantz and that shows in the entry Depth of Kanban - A good coaching tool of his blog.
The questionnaire is based on evaluating 7 dimensions through questions that are answered with a yes or no. Each dimension is marked on a radar type chart and we can visually see the depth of Kanban for our team.
Questionnaire adapted to our language:
This questionnaire has to be answered by the team members of each of the projects.
Achouiantz speaks of limiting the level according to the answers, so that if, for example, in Display, the answer for question 4 is "no", question 5 and following are not answered. This restriction, while it may be interesting, seemed to us that it was going to demotivate the team more than the account, so we have not applied it. Maybe when we do the evaluation again in a year we will be more demanding.
In our case, we have teams in which there are several developers, so they have had to answer the questionnaire among all, since what we are trying to see is the depth of the use of Kanban by work team in a concrete project.
Let's talk about some of our projects, which we will call (for privacy reasons) project A:
Project A is the project that Kanban's best implementation has had, thanks in large measure, to a very collaborative client and involved in this way of working.
This project has a large code base and has many technological challenges, which makes it difficult to improve development processes (see Improve dimension). In contrast, both the client and the development team have great discipline and great collaboration between them, which is seen in the Dimensions, Visualize and Limit WIP dimensions .
As we have just implemented Kanban, we still do not have the protocols and workflows defined and even though we do not have monthly meetings, the client-team feedback of development is constant. All this is reflected in the graph.
As we have just implemented Kanban, we still do not have the protocols and workflows defined and even though we do not have monthly meetings, the client-team feedback of development is constant. All this is reflected in the graph.
Kanban Metrics
1. Lead
The lead is the time from when the client requests functionality until it is implemented in the system.
If we encounter a deviation from the very large lead times, we may have the following problems:
- Sizes of very different tasks.
- Delivery cadence very irregular.
- Bottlenecks in some states.
- Too much concurrence of tasks (or projects).
The development time will help us to know if the problem is developmental or organizational.
2. Cycle
Cycle is the time from when a task enters the development team until it is included in production.
We do not only use this measure, but we have defined several workflows to measure waiting times:
- Developmental.
- Deployment in the system in production.
So we can see what part of the process is causing the bottleneck.
3. Number of task setbacks
This metric I have not found in the bibliography (I guess I have not searched extensively), but I think it can be interesting when it comes to detecting quality problems in the process.
It can be measured by project, by state that originates the setbacks or even by developer. The idea is to see if you are developing software that does not meet the requirements that were requested, either because it is incomplete or because it is wrong.
Also, the software may be correct, but the code does not have enough quality and in the code review performed by a colleague, return the task to a previous state.
In general, the states of revision are the ones that generate the most setbacks, as seen in this graph.
It can be measured by project, by state that originates the setbacks or even by developer. The idea is to see if you are developing software that does not meet the requirements that were requested, either because it is incomplete or because it is wrong.
Also, the software may be correct, but the code does not have enough quality and in the code review performed by a colleague, return the task to a previous state.
In general, the states of revision are the ones that generate the most setbacks, as seen in this graph.
4. Speeds and estimates of development times
For each task we should have an estimate of its development time and record its effective development time when finished.
First because knowing the hours of pure development allows us to see if we have a good performance, but we do not have to obsess. We know cases of engineers and developers who, in less hours, perform more work than others, so we should not take this measure as something absolute.
At the beginning of the implementation of the process there may be many tasks that take time out of development, as shown in this graph.
Second, because we can examine the average development time per task (and its deviation) to see if we are creating tasks of different classes or all of a similar size.
Finally, the difference between estimated and effective times allows us to know how well we make the estimates. If there is a big difference, that implies that we either lack experience, or we are not doing the division in tasks correctly, having tasks described in an ambiguous or very unspecific way.
Are there more measurements?
Most development teams estimate with story points in Scrum , but in Kanban we have not applied that estimate for several reasons:
- Inexperience of the team: the team was formed just a few months ago.
- Lack of knowledge of some projects: in some maintenance projects 100% of their functionality was not known.
- Difficulty translating story points to actual development hours for each project because of the idiosyncrasy of each project (some projects are maintenance projects, others are development projects and some have different priority levels).
- Of course, you can create classes of tasks, so that there are critical incidents, improvements, tasks of improvement intangible, etc. Each type of task would have its own measurements, which may even allow for achieving service level agreements with customers.
Since we are now implementing Kanban, we have not yet reached this end.
How often do the measurements?
David J. Anderson tells us in several experiences in his manual that the improvement process took 15 months. We have made the initial measurements 3 months after beginning the implantation process. Obviously the results can be improved, because we are still in the beginning.
What is clear is that making a change is something complex and requires your time, so more than a speed race, implementing an improvement in the work processes of an organization, is a race of resistance.
What is clear is that making a change is something complex and requires your time, so more than a speed race, implementing an improvement in the work processes of an organization, is a race of resistance.
Corollary: do not obsess about the metrics, worry about the culture of your team
I share Anderson's view that before beginning any Kanban implementation process, the development process needs to be improved . For this, we will have to see if the equipment generates clean and quality code. A high technical debt makes it impossible to implement this process. If there is not a mentality of responsibility and demand with the work of oneself, no matter the framework that you implant, you will not have good results.The team has to be clear that they have to be always improving the way they work. A developer who is not training and trying to do things better and better, stays behind.
Kanban can help developers be more disciplined, the team has more performance and those interested know what is being done at any time, but without a culture of continuous improvement , or kaizen , Kanban is useless.
And you, what agile framework do you use?
And for that frame, how do you measure the performance of your agile team?
No comments:
Post a Comment