Agile development has been on the agenda of software development teams for many years. As more teams struggle with remote collaboration and faster delivery expectations, here’s what I’ve learned actually delivers results.
Sprint based development helped product teams adopt an iterative approach to software development and established a pretty much organic approach to growing a product. This article is about my experience in managing Agile development teams. My focus is being Agile to deliver high priority projects fast and to target dates.
The Objectives and The Process
In implementing an Agile Transformation across a company, you need clear objectives. The main ones I came across are:
- Tracking developer utilization. When the company views software development as a production capability, measuring resource utilization over time becomes important. Measuring task load distribution and task burn rate consistency over time become important. This metric is important if the task backlog is large, everything in backlog is important, and needs to be delivered.
- Improving feature and major release development speed. In a customer focused company, prioritization of task backlog becomes important. Identifying key deliverables, giving them high priority, and driving planning accordingly speeds up development, sometimes with cost of lower efficiency. This metric is important if customer satisfaction is highly correlated with the latency in delivering desired features.
- More transparent planning and management. Building a plan and delivering according to the plan is always a challenge. When a project is delivered, software engineers and engineering managers value the ability to look back to the project delivery over time and see how the plan and delivery has changed over time. Transparency also gives teams a way to review plans in the middle of a project and project forward based on experiences from similar project.
I have a preference for improving planning for faster development of high priority features, as opposed to focusing on developer utilization. My experience has been that moving to Sprint based planning with 2 week sprints significantly improved critical project delivery times, by at least 50% for all such projects. This is because working with Sprints improves communication, and improves project visibility. When a project moves faster, that also improves customer engagement. This reduces the delay between tasks, people are responsive, wait times are gone, project timeline shrinks.
Priority driven planning for big development projects requires a planning approach that goes beyond Sprint based planning, it needs to cover multiple Sprints. This is a big topic on its own and needs a separate discussion.The main difference between utilization based management and feature development speed based management becomes clear when development organization is large and has many teams. In the utilization-based approach you need to delegate the objectives to the manager hierarchy and every manager will try to maximize their utilization metric every Sprint. This is much easier to manage hierarchically, but high priority long term objectives for the whole organization get blurry when you cascade it down. On the other hand in prioritized approach management needs to build a plan for each high priority project for the whole organization. The overall plan is agile and needs to be reviewed every month or so. The plans and its revisions are cascaded down. At every management level, plans provide the guideline for Sprints planning.
The Internal Process for Sprints
For Sprint based execution, I implement the following steps in every Sprint.
- Sprint planning. At the beginning of every Sprint, usually the first meeting in the morning, we have a planning meeting where we set the target task list for every person in the team. I use a few best practices in these meetings. First is capacity planning for individuals. Some people may have commitments that are beyond our Sprint plan, and for these we may need to allocate a certain percentage of their time. It is best to capture all software development commitments as tasks as long as their delivery can be tracked. Commitments beyond software development may include vacation time, time committed to other organizations like presales engagements, HR projects, etc.
- Sprint closure. This happens at the end of day of the last day of the Sprint. In the closure meeting every person goes over their tasks starting with what they completed, and if delivery of a task happened different from how it was planned we may go into details. We look at the planned but not completed tasks if they exist and discuss how we can improve planning for the next Sprint to improve the situation.
- Sprint retrospective. This meeting takes place after Sprint closure and we have an open community discussion of what worked well in the Sprint and what can be improved. Creating an action list and assigning responsibilities to individuals can be part of this process to be reviewed at next retrospective. My observation is that having a transparent planning and execution process helps drive employee satisfaction, and the periodic retrospective enables better support among the team members.
- Mid-week Sprint review. In implementing a Sprint, one of the things I did was have a weekly meeting, usually in the middle of the week to look at where we are in the plan, and how are we expecting to finish the week and Sprint. This is the meeting where we also ensure task statuses are updated by individuals to reflect the progress in the Sprint.
- Daily standup. The daily standup meetings I run are truly simple meetings where the objective is cross group communication on daily work and also identification of issues that block or slow down individuals. What I don’t do is map the daily discussion to the Sprint plan or tasks during this meeting. The mid-week review takes care of that task. This ensures the daily standup is short every day and people pay attention to each other rather than reviewing their own Sprint plan.
Types of Tasks
Tasks are basis for all our development plans. Task backlog everything we want to do, the Sprint plan is a subset of tasks that we plan to deliver in the current Sprint. Sprint tasks have two basic properties, they should be mapped to a single individual, and they should have a defined deliverable which may be implicit.
In my experience the Sprint tasks are different from tasks a Software Development (SD) organization uses to track the work that needs to be done, and it is best to decouple these two task repositories from day one. Of course there will frequently be a relation between Software tasks and Sprint tasks, but this relation is usually not one-to-one. A Sprint task may have a direct mapping to an SD task. Frequently, SD tasks have higher granularity than Sprint tasks, so an SD task may map to multiple Sprint tasks. There are also Sprint tasks that may not be a Software Development task, like tasks for HR department or Sales department.
- Development Tasks. These are usually the bulk of the tasks. Each task identifies a specific deliverable, frequently a piece of code, and once the delivery is done the task is complete.
- Technical Design Tasks. Sometimes engineers are given a task where it is not clear how this can be implemented. In such cases, we create a separate task for the technical design, and the implementation task then can be performed after design is complete.
- Conceptual Design Tasks. Some of the work engineering teams drive may not be supported by a Requirements Analysis (RA) team, sometimes this is an RA capacity issue, or sometimes the task scope is purely technical so it is beyond RA capabilities. In such cases we use tasks whose purpose is to define the scope. Once this task is complete, then the team takes the deliverable and can define a project with appropriate task. When a separate SD system exists, the new tasks needs to be create there also.
- Test Support. When delivery of a software release is done, the release goes to Quality Assurance (Q/A) team for comprehensive testing to ensure compliance with the specifications. In a delivery schedule prioritized organization, the speed with which developers respond to bug reports from Q/A is important. We plan developer capacity in a Sprint for this effort, and in addition to the capacity as the issues are reported we create task in the Sprint mid-Sprint for tracking.
- Adhoc request. It is highly desirable to prevent mid-Sprint adhoc requests to impact Sprint plan, but how much this happens can not be controlled by the developemt team. I ensure such requests are accompanied by a ticket in the SD system as well as an associate ticket in the Sprint.
- Critical Bug Fixes. For me and every team I manage, these are tasks with highest priority, and are difficult to plan. Such tasks are usually raised by the support team and the most important role of management is establishing a transparent process in classifying a bug as Critical. We don’t want every bug to be defined Critical, on the other hand we want every truly Critical bug to be identified as such.
Tooling for Onsite and Remote Teams
Throughout the years and different companies, I had the chance to experiment with different tools and ways of running a Sprint. What I talk about here is what I found most productive for a fast development environment. I used a number of different tools for tracking task in Software Development scope, most popular being Jira and I also tried Microsoft toolset that was well integrated with Visual Studio.
For both tools, my view is that SD tasks are different from tasks assigned to developers and trying to make them the same is not worth the effort. SD tools are great environment for the integrated environment of design, development, Q/A, and support. On the other hand creating tasks, deleting tasks, merging tasks, splitting tasks take much longer. In these tools it is also complicated to take snaphots and do time based analysis.
I will separate the tools I use to run Sprints into two.
- Tools to run a daily standup
- Tools to plan and run Sprints
Daily Standup
If the team is fully onsite, then it is possible to use whiteboards to run the processes. For onsite daily standup the tool I used most was a whiteboard with the calendar view for the Sprint, usually for two weeks. For every team a separate whiteboard and space is needed. Every team member gets a color and during the daily standup we take notes on the board. At the end of a Sprint we take the picture of the board for archive and clean it up for the next Sprint.
For a remote team I tried many online tools, but at the end I chose a simple solution. The best way I found is using a spreadsheet to take the notes. It is very fast, searchable, you can keep the history, analyze the data, and you can create timesheets for special projects or customers as needed. The sheet has 5 columns for the workdays of the week and every team member has its own row. I then replicate the whole 5 column team rectangle for every week extending the spreadsheet downwards. Every week has data headers and also Sprint number as needed. I recommend that remote daily standups to be done by video conferencing, and video to be mandatory. This way the team starts the day listening to each other with clear focus.
Sprint Plan
I have tried many things for Sprint planning over the years. I tried to use whiteboard, both with a pen and with sticky notes. In both cases, writing the tasks down was cumbersome, especially editing wording or task structuring becomes difficult. I also tried keeping the task list online and printing each task on a label which we then put on a sticky note. The effort was still to much.
At the end working with an online tool worked the best, both for onsite teams and remote teams. I used two different tools that I am happy with, my favorite being the Trello. The tool works very fast where the whole team works on the board concurrently during the Sprint planning and reviews. It has limited reporting capabilities, and data import and export is limited, however it is fast and captures what we want to do in working Agile. That is being able plan execute transparently and fast.
The second tool I used is Jira. Companies seems prefer Jira use if the SD tasks are managed by Jira, however I don’t consider this important. In every organization I managed the development teams had to create separate tickets for Sprints and Sprint backlog. You may want to link SD tickets to related Sprint tickets but this is cumbersome. The method I use frequently is not direct linking but putting in the ID of SD task in the related Sprint tasks.
One thing I want to mention here is that I do not allow the use of Sprint tasks as a placeholder for SD tasks. All development tasks need to have a related SD task visible across the company and all documentation and resources needed for development should be accessible from SD task. Sprint tasks can not contain documentation, resources or comments that may be needed after the Sprint closes.
Conclusion
I tried to share what Agile Development and Sprint planning means for me when I want to improve the delivery speed of high priority projects. Sprint process requires time plan and execute, and what I talked about here is the result of trying to minimize the overhead. One thing I did not mention in the article is use of Scrum as an Agile process. While I adopted many elements of Scrum process, I prefer not to use Scrum. Scrum introduces some roles and related overhead that has not been a good fit in all the places I managed the engineering teams. If you have teams that work on only one or two projects it may be easier to use. In my case I always had a mix of product development work, support tickets, special requests from key customers, and special requests from Sales teams. When the backlog tasks become dynamic, I found bypassing the management structure of Scrum and passing the responsibility to the teams more productive.
This brings me to the important topic of project planning that I briefly mentioned in this article. Being Agile in development does not always mean the external world is agile. I always had big planned releases, and big key projects that span many departments and many Sprints. Furthermore these releases and key projects usually are not static; a lot of changes come up during the development, and sometimes while the delivery is being tested by the business teams. One needs an Agile project management approach that works well with Sprint based planning. This turns the Agility of the development process into outcomes the customers and the company expect. This is a lengthy topic and I will talk on this in a separate post.