Discovery Phase In Software Development - Get It Right And Win Big!
It makes sense to have a plan before you start a software development project. To explain why we'll pose a metaphor.
There's not a single licensed, experienced builder who would be willing to build you a house without a blueprint or set of detailed plans. There are too many problems that could come up along the way:
- What if you change your mind after you see their choices?
- What happens if their understanding of your vision is different from yours?
- How do they know when they're "done"?
- How do they know how much they should spend on the project?
- How can they ensure that all the building activities happen in the right order?
Attempting a big project without a plan carries a lot of unnecessary risks. This is as true for someone building a house as it is for a business building new software. The difference is that building a house carries the same challenges as it has for hundreds of years.
Software development is a unique beast. Requirements between one custom software development project are rarely the same as another. For big projects, it's almost impossible to know everything upfront. Things can change halfway through a project and require rapid adjustment.
Given the challenges that software development poses, it makes sense to plan as carefully as possible. The period of planning that happens at the start of a project is referred to as the "Discovery phase".
Our team have mastered the discovery phase and can help you work through your own discovery. We run a process called Product Roadmapping where we run a short series of 2-hour meetings to understand everything that will get your product a MVP. Drop us a line to see if we can help with your upcoming project. At the very least you'll get some free experience-backed advice.
In this article, we're going to try to provide some insights into the discovery phase in software development from an agency that has gone through it dozens of times over the last few years.
Understanding the discovery process in software development
We'll start at the top with what the discovery phase is, why it's important and what's included, and throw in some insights to keep the article exciting :).
What is the Discovery phase and how does it work?
The Discovery phase was probably invented by someone who tried to build software for a client without taking the time to deeply understand what they were building.
The Discovery phase is a period at the start of the project where the development team becomes intimate with the problem they're solving and plans the best solution.
It's usually conducted as a series of meetings where lots of questions are asked. The goal for the development team is to understand the business and why they're willing to pay for software to be developed. Here are the kinds of things we ask our clients during this process:
- How will this software impact your bottom line?
- When do you expect to see a profit from this project?
- What will life look like for your business after the software is live?
- What are the weaknesses and strengths of your current approach or system?
- How quickly do you need the software to be live for the project to be a success?
- Who will use the system?
- What is the most important thing the system should do for each user?
We have a template of around 120 questions which we can draw from during these meetings to get a rounded picture of what the business's goals are and what we're building.
A well-executed discovery phase will be broken into a few stages, each task with gathering details on something that will help steer the project. We typically break our discovery phase into four parts:
- Goals
- Flows
- Features
- Roadmap
Main stages of the discovery phase
If you read a traditional project management book, they'll probably have a different answer to this question but for our team, the four stages of discovery are Goals, Flows, Features and Roadmap.
Goals
The Goals stage of the discovery process is focused on understanding what the business hopes to get out of this project. They've decided that custom software is going to result in a net financial gain sometime in the future. Our job is to get to the bottom of what their vision of the future looks like.
Without understanding what the business is expecting to get from the project, it's very difficult to make it happen.
There's usually more than one goal behind a software development project. Often it relates to improving efficiency for the client and ultimately reducing costs or increasing revenue, but there can be a number of smaller goals along the way:
- Fix processes that the team have become used to working around
- Outdo the competition
- Integrate systems the avoid unnecessary manual effort
Flows
Flows are the sequences users follow as they attempt to do something in a system. For example, changing your Facebook profile picture would look like this:
- Log in
- Open profile
- Select profile image
- Choose to change profile image
- Browse gallery for the right image or upload a new image
- Edit image
- Upload
- View profile page to review new picture
This flow can be mapped out visually to show the features a user needs to interact with to achieve a goal.
Capturing all the major flows the system's users will expect to undertake allows us a macro-view of all the functionality the systems should include.
Features
With all of the flows captured, the next step of the Discovery phase is to figure out what features are required to let the user achieve their outcome.
Taking our Facebook profile example, there are a few features needed:
- A photo gallery to allow the user to browse their images
- Uploading and document storage
- Editing and image adjustment
Not to mention the broader feature sets like user profiles, the database to store everything, navigation to move around the app, etc.
The best way to capture these requirements is through user stories and acceptance criteria. User stories follow a specific syntax and aim to make the feature easy to understand for developers and the business. Acceptance criteria are essentially the things that need to happen for the feature to be considered complete.
When our team go through this exercise, we'll put our heads together and figure out roughly how many hours of effort each story will require to build out in totality from prototyping all the way through to finished code. This exercise helps with the final step of the process.
Roadmap
The final step is the Roadmap. Trying to squeeze every feature into the first build is a recipe for failure. The goal of most products is to get to market as quickly as possible to start receiving feedback so they can evolve. In order to release a useful first release, the Roadmap creates a plan for which features will be included and which will be left until later.
This step needs to be a discussion between the business and the product team, where each makes their case for the features that are the most valuable to include in the first release.
Since each feature has an estimate assigned to it, once there's consensus on what's included, all the estimates can be tallied to create a project schedule and expected budget.
At this point the Discovery phase is essentially complete and the project can start. The development team have answers to all of the questions they needed to understand to build the right product. The business has confidence the project will do what they want, and understand the costs and timeframes involved.
Why is Discovery useful?
Discovery aligns the business and development team in a way that removes much of the risk involved in software development. There are an infinite number of things that can go wrong during a complex application build, and taking the time to try and think through everything often solves problems before they're even born.
In a lot of projects, it's also the first time the business and extended development team get the chance to work together. Once the design starts, it's too late to ask some of the obvious questions that could make the development team look unprofessional.
What activities are included in Discovery?
Here are the activities you can expect to undertake during the discovery period:
- Meetings, meetings, meetings!
- Defining goals
- Capturing user flows (often on a whiteboard)
- Recording user stories
- Defining features
- Figuring out which features will make up the MVP and future releases
Depending on how big the project is, sometimes the development team will build a set of wireframes or create a prototype or low-fidelity design to visually explore the requirements.
Who needs to be involved?
At a minimum, the "product owner", or person responsible for communicating the business's wants and needs should be available to chat with the project manager from the developer's team.
It might seem like the more the merrier for discovery meetings, but it's better to have a small focused group of individuals who can make the decisions quickly. We find the best balance when these people are involved:
Development team
- Project manager
- Lead developer
- Designer
Business team
- Product owner or project manager
- Someone technical
four or five people is usually a good balance. There's an argument for adding backend engineers, quality analysts, and more people from the business team, but we find it makes the process longer and less effective.
Project discovery phase deliverables
At the end of the discovery phase, these are the deliverables that should be produced:
- Product roadmap, or some kind of master document
- Set of user flows
- Suite of user stories and acceptance criteria
- Release plan that details which features will be included in the MVP, and which will wait until future releases
- A recommendation of technology stack and implementation approach
- (Optional) A quote or breakdown of costs
There's not really a one-size-fits all to the deliverables that come out of a discovery phase. In some situations, the project financials have been finalised and there's no need for an estimate. At a minimum, there needs to be some kind of master document that captures all the user flows, features, and the release plan that details what the software's rollout will look like.
How much does it cost to start the discovery phase?
Our team typically charge around $3,000 for a discovery phase with a new client, which is refundable if the client decides to work with us.
Since the process is separated from the main project, and we know we're getting paid, there's no need to rush through the discovery phase. This is a problem for a lot of companies that build the discovery phase into their fixed price. Clients that don't understand the value of a discovery phase in software development will often push back on a long, expensive period of planning. As a result, it's underpriced and cut short.
How long does a typical discovery phase take?
The short discovery phase usually takes about two weeks. A long discovery phase can take up to two months.
As a rough guide, this is where the time goes:
- 2 hour meetings (10%)
- Writing user stories and acceptance criteria (50%)
- Estimating effort (15%)
- Planning technology stack (5%)
- Planning the release map (5%)
- Writing the documentation (15%)
It might seem like meetings are the main activity of the discovery is meetings as the development team and business try to get on the same page. Realistically, most of the time is spent translating notes from those meetings into user flows and user stories. For every feature, there will be a handful of stories and for every story, there should be a handful of acceptance criteria.
These stories are the details that drive the project and tell the dev team exactly what to do, so there's no shortcut around the lengthy process.
Next steps after discovery
Once you've finished up the discovery phase, it's on to building the software. By this point, the development team have all the answers they need to start designing and developing.
Most of the time, we'll jump into a set of wireframes and a prototype to validate our understanding of the requirements through a low-commitment solution. We've written a detailed guide on prototyping so we won't go too deep on the topic!
Tips and Tricks for the Discovery Phase
Here are some tricks and tips we've picked up from going through dozens of discovery phases.
Tools to use
Since our team are completely remote, we need special tools for coordinating with clients. This is the stack we've come to depend on:
- Figjam
- Zoom
- Jira
- Planning Poker extension
Figjam is a great tool to start off with. It's honestly the best collaborative whiteboard tool for talking through ideas. Clients can join Figjma sessions for free, everyone can participate by drawing and adding sticky notes, and everything happens in real-time!
Zoom is our go-to meeting tool, but we're partial to Google Meet too. Even though meeting online can be a bit more difficult for communication, the benefit is that it's super easy to record the call and go over the details later.
Jira is our project management tool of choice. Honestly, it's an ugly old beast to use, even for experienced users, but it's built specifically for development teams and it provides a lot of useful functionality.
Since we create estimates for each of the user stories during the discovery phase, we like to use a tool called Planning Poker to facilitate the process. The idea behind Planning Poker is that everyone on the team gets a set of cards they can assign to a story to indicate how much effort is required. The final estimate won't be locked in until everyone on the team agrees on a single estimate.
Behaviour Driven Development Acceptance Criteria
We've mentioned user stories and acceptance criteria a few times since they're the best mechanism for capturing requirements. Our favourite approach for capturing user stories is behaviour driven development (BDD).
The idea behind BDD is to think through all the scenarios around a feature and then capture them using the Gerkin or Cucumber format like this
A user requirement looks like this:
As an administrator
When I try to run a revenue report
It will run succesfully because I have the right credentials
In the requirement above, the feature is a revenue report that can only be run by an administrator. The other scenarios that might happen are:
- A non-administrator tries to access the report and it fails
- A non-authenticated admin tries to access the report directly and can't because they're not logged in
- An administrator tries to run the report without entering any filters or values
This is a pretty basic example, so there aren't too many different scenarios that would realistically happen. For more complex features that developers may struggle to understand, having a batch of examples can be a massive help!
The only downside to BDD is it takes a long time to capture the requirements upfront. But it's worth it when the feature is built with all the edge cases in mind.
Wrapping up
That's it for our overview of the discovery phase in software development. If we haven't answered your question, or there's something you'd like to know, leave a question below and one of our team will get back to you in a few business days.
