Agile is a combination of two processes that are designed to develop software. The two methodologies that form Agile are : Scrum and Kanban.
Key to understanding agile is understanding the objective tree. Here is an example objective tree for a company like amazon.com which aims to build an e-commerce website.
There are many people who argue about the number of levels in the tree and how they should be called. But the key insight here is that :
a) Objectives form a tree
b) When child objectives are done, the parent objective is automatically done
There are two methodologies within Agile : scrum and kanban.
Objectives are broken into user stories. A prioritized list of user stories is called a backlog. Product management owns the backlog.
Objectives are achieved by working on them in 2 weeks sprints. Team pulls the maximum number of stories that can be completed into the sprint backlog and starts working on them when the sprint starts. Sprint backlog is owned by the team.
Every day of the sprint there is a 15 minute standup meeting where team members describe 3 things :
- What they worked on yesterday
- What they are planning to work on today
- Blockers ( if any )
This is an opportunity for micro-course-correction. If a story was done, team members can move to help with another story. If there is something that blocks a team member, management can move to unblock them. If a story might spillover, it can be called out.
At the end of the sprint there is a sprint retrospective to re-evaluate and course correct at a higher level.
Every N sprints ( for a finite value of N ) there is 1 refactor sprint. No new features are delivered in this sprint. Architectural runway is prepared for incoming stories and code is made more maintainable.
Why is the process like this ?
Reason for product management owning the backlog is to communicate clearly what the current priority is. This can be challenging in a changing environment. For example, management may ask the team to support an application on Windows 2012 platform and mark it high priority since largest customer base is on that platform. Afterwards it could turn out that highest paying customers are on RHEL 7.1. Management can communicate this information to engineering by adding a new story to support RHEL 7.1 and deprioritizing the story for Windows 2012 support.
Team owns the backlog because the team is responsible to ensure that the features ( stories ) committed at the beginning of sprint is done by end of sprint. This is why it’s the team that need to pull the stories into the sprint ( and not the manager ) so they feel a sense of ownership.
Standups give an idea about the pulse of the sprint : whether the team is on track, if it needs adjustment etc. Since software development is a collaborative process, it can get blocked on other people. For example : UI team can get blocked on UX mockups, developers can get blocked on APIs that were promised by other team members, one team member may be blocked for code review from another etc. Daily meetings help with dealing with these problems as they arise. It also enables the team to help others, for example, if someone airs a database problem, someone else on the team may have encountered and solved the problem before. Standup meetings should be short and sweet. If they are too long, since they occur everyday, they become a productivity drag.
The sprint retros are an opportunity to improve things at a higher level. For example, developers may notice that the CI system could run a larger portion of automated tests, reducing QA burden. Developers may point out that the code is getting too complex in one area and request refactor stories.
1. If a story is in the backlog, it may get done. If a story is not in the backlog, it will never get done.
Why is this expectation there ? Sometimes product management may request a feature. Engineering does not know if that feature was important, whether it was a priority etc. Moreover, there could be multiple managers requesting different priorities for stories fuelling confusion. This expectation sets up management to close out the relative priority of objectives before people set out to achieve them.
There is no such thing as an implicit requirement. Management owns the product backlog.
2. Target spillover for any sprint is zero
Why is this expectation there ? For hard software engineering problems it is required that the team is a highly committed one. An uncommitted team member will usually bring down the whole team since the job is collaborative in nature. Target spillover of zero ensures that the team remains focussed on the problem at hand.
3. No activities are planned mid-sprint
Why is this expectation there ? This is to help the team remain focussed so that the cost of context switching is lowered. It is called a “Sprint”, not a “Trot” or “Gentle Walk”. Management is expected to defend the team from external influences. Course corrections are possible at the end of sprint. Therefore, it is important to keep the sprints as short as possible to get a meaningful amount of work done. The length of your sprint should be the minimum amount of time it takes you to respond to external influences. Usually, it is 2 weeks.
4. QA should raise a ticket if there is a problem.
Why is this expectation there ? It is possible that QA notices a problem and communicates it verbally or through email. Now people may go on vacation etc cutting the team’s ability to deliver.
There is no such thing as an implicit ticket. Raise it in issue management software.
5. No story is done until :
a) All code is reviewed
b) There are no open tickets
c) Documentation is complete
Why is this expectation there ? Code reviews are important because it lets someone else get familiar with that code base allowing them to take over in the event of the author’s absence. It is important to close all open tickets before calling a story done since otherwise your development process simply turns feature requests to bugs. Documentation should be developed lock step with code since that is when it is easiest to write it.
6. Do not test from feature branches
Why is this expectation there ? Someone could merge something else into the development branch making the feature tested in feature branches to be unviable in development branch. Development branch is what ultimately gets shipped to users so that is where things should be tested.
7. Stories are of the form “As a [user] I want to do [task] so that I can [accomplish goal]”. If they are not in that form, it can be safely deleted.
Why is this expectation there ? Often stories may be created with no clarity on what the next step is. Often developers may misunderstand the requirement and deliver a different feature requiring multiple unnecessary round trips and feedback loops to complete the feature. If the story is not in the prescribed form, it can be deleted and considered non-existent since it motivates people to state the objectives of the story explicitly.
A few points
- In Scrum the keywords are focus and course correction.
- Manager – employee relationship is co-operative. The team does everything to make the manager look good ( by delivering high quality features that delights users ) and managers does everything to setup the team and employees for success : removing impediments, giving them tools and access they need to meet the objectives.
- Dev – QA relationship is quasi-adversarial. QA tries to defend the customer demanding issues be fixed within reason. Dev team appreciates QA for finding the bugs they have missed that could have caused them to deliver an inferior product.
- Agile requires self motivated team. An under-motivated team may over inflate estimates or under commit into a sprint.
- There is no such thing as half-agile. Agile gives management a certain amount of power and engineers certain amount of power. This is to improve efficiency by reducing friction and miscommunication. If management says : ‘I will accept these parts of the process that gives me power and ignore these parts that require me to relinquish power’ this process will fail. It requires humility on the part of management and engineering.
This is usually for support. After product is developed, support is trained on resolving bugs that get into production. Support can’t wait 2 weeks to resolve issues. When tickets come into the queue people take them on and try to get through them as fast as possible.
Scrum is more proactive and Kanban is more reactive. Kanban is for situations where a 2 week turnaround time is unacceptable.
Agile came about as a response to the Waterfall model of software development. It helps solve the most important problems with waterfall such as quicker course correction, improving focus and reducing miscommunication.
In waterfall, the software developed in following phases ( one after other ) :
- Gathering requirements
- Designing solution
- Implementing solution
- Testing solution
- A mistake in early phase cannot be detected until last phase.
- There is little focus as work drags on for years.
- People working on each step only communicates with people in previous step increasing communication errors.