Why do requirements fail in Testing?

As a Test Manager, I feel let down when requirements fail in testing due to being missed in the solution design. This not only adds to added expenditure for the test team, but the solution also has to be reworked to accommodate the missing requirement, thereby warranting additional re-test.

So….is it the developer’s fault that the solution has not accommodated the requirements correctly?

Or is it the Business Analyst’s fault that requirements were not documented/explained properly?

In testing, I follow a certain rule of thumb:

Testing is about ensuring that the solution is working as per the business requirements specified by the customer.

So it is not the responsibility of the Testing team to gather requirements. Rather, we validate the solution and identify whether what was delivered by the development team is fit for purpose and meets the client needs or not.

How can an organization ensure that requirements are gathered correctly and BA does not miss many requirements?

Let us start with an example. Since retail sale is projected to be up by 50%, business wants a shopping cart that can address the high demand and allow them to ship the purchased item via several wholesale trader at different parts of the world.

So a BA is assigned to the project. BA sets up a face-to-face meeting with the customer/business sponsor. The customer explains in broad terms that they are anticipating a rise in sales and their current system which uses VB.NET (old system) has not been scaled properly. The customer also wants to use the latest technology for the look and feel. They would like to replace the existing application with a cutting edge solution that can enhance their business presence in the market. They would also like a mobile app to be delivered as part of the solution.

If the BA starts writing the Business Requirements Document (BRD) with what little knowledge he has of the system after this initial face-to-face meeting, then this task is likely to fail.

The BA should first identify all the stakeholders as different parties may have different priorities. He should hold face-to-face meeting with all the stakeholders and document all their requirements and priorities. Note that at this early stage, the business will not have a clear idea about how the application should look. They only know that it should offer cutting edge solution and should have a much more superior look and feel than their current system.

Once BA has gathered all high level requirements from all the stakeholders, he should hold a workshop with all of them to go over what was discussed with individual members. This workshop allows all stakeholders to exchange feedback and review the requirements from other stakeholders. This workshop MUST result in some concrete action items for the BA. A follow-up email should be sent to all stakeholders to share the meeting minutes and action items. BA should start storing all email correspondence and documents into a central repository.

Following this workshop, BA should take a bit of pause and review all the requirements collected so far. Software development is impacted when requirements are not properly defined or is left to the discretion/understanding of the development team and could be significantly different to what the business wanted in the first place. It is therefore the responsibility of the BA to bridge the gap between the uncertainty and clearly defined requirements. The BA must establish who will be the users accessing the system, what are their needs, what regulations and policies will govern the user access and to what detail they have been captured in the document. Here I am assuming that the organization already has an approved Business Definition Template (also known as a Business Requirements Document).

The BA should accept the fact that “He WILL NOT get all the requirements correct or completely documented”. So to minimize the number of missing requirements, he should start with small steps. Have a checklist of things to do and start ticking them one by one. Document the newly added requirements and share it with business sponsor or SME and ask for feedback. Have the shortest possible feedback loop. This means that after having the discussion about the requirement, do not accommodate a long period before seeking business feedback. Hence make the loop small. Frequently review the requirements and update the document if there are discrepancies. Go back to business for further clarification or feedback. Adopt a hybrid Agile methodology where you meet the business almost daily to document requirements as well as seek feedback and thus ensure that the requirements exactly matches that of the business.

One thing we have not mentioned so far is the skill sets of the BA. The BA should possess the following:

  • Ability to think like the business; why the business is asking for what they are and what benefits they will get including ROI (Return on Investment)
  • Be a strong communicator and have people management skills. For a large project, the BA will find himself talking to a large number of stakeholders with different expectations. It is the responsibility of the BA to articulate the requirements to the group and manage all their expectations. A BA with poor communication skills will fail in that regard.
  • Must have a strong technical background and must be aware of current industry standards.
  • Must be able to change requirements with changing expectations from the business even when the build development is in progress
  • Must have the ability to formulate solutions for the business even when requirements are light in nature. Should be able to provide several options for stakeholders to consider. Once one of the options is selected as the preferred option, then update the document with detail definition.

Having gathered the requirements so far, BA should draw some flow charts and other process diagrams with screen mockups and seek feedback from all the stakeholders. I have found such paper based drawing to be of immense value before formally documenting the requirements in a Business Definition Document. When translating the requirements into a visual application, the BA should use the following practical tools:

  • User stories – Define who the actor/user is, what he is trying to do and what benefit he will get from that requirement.
  • Use cases – This is where you provide details of the user stories. Use simple terms and no technical terms. Explain as if the user is describing the process.
  • Prototypes – After gathering user stories/use cases, draw some UI sketches on paper. Paper based sketches are the best, can be thrown away without any cost, and allows quick turnaround time for receiving feedback from the business/customer. Remember, a picture tells a thousand stories. It is much easier for you draw the mock-up on paper and for client to view it to get a visual understanding of how the UI will look. Also allows for a better understanding of user stories and workflows.
  • Workflow diagrams – Use state diagrams to consider all different states for user interaction. This allows a visual way to assess code coverage and ensures that developer doesn’t forget to code for specific state transitions.
  • Wireframes – Use tools such as Microsoft Visio to draw wireframes of the application screens. Again the idea is to get quick and simple feedbacks from the client. Stay away from designing wireframes in IDE tools such as Visual Studio. It is mainly because your focus should be the quick development of the application screen and not start coding. You may get carried out by that process when you use IDE development tools.
  • Mockups – Only generate mockups after your prototypes and wireframes have received sufficient feedback from the customer. Use IDE tool that will be used for development. You can start creating a small demo mockup for customer feedback. This is the stage where you can start using the IDE. Using IDE earlier in the requirements gathering process ought to be avoided.
  • Swim lanes – Use swim lanes to explain processes and their timelines.
  • Database models – Generate database diagram for primary/foreign key relationship
  • Business Rules – List all the business rules for the application, as defined by all the stakeholders

Once the above steps are completed, the BA would be in a position to document all of this is in the BRD as a first draft and send the document to all stakeholders for feedback. Provide a schedule during which all feedback are to be sent. Incorporate all feedback in the document and then send for final review/feedback. Same as before, set a schedule for feedback. Once all feedback are documented, send the BRD for approval.

So as you can  see, if the above processes are followed by the BA, we can eliminate the possibility of a missing requirement.

There have been too many instances in my experience when developer interpreted the requirements in one way, while the BA meant something else or may not have even considered that scenario. When test team starts doing static tests to verify the solution mapping against documented requirements and considers all different combinations, that is when missing requirements bubble up to the surface, and the finger-pointing starts.

I have come across a project in recent days where the BA has completely ignored to seek feedback from the business stakeholders. As such, test team has raised a number of defects which require clarification from the business even though the build has already been completed. So the failue to consult the business has resulted in the project being de-scoped from a release in order to consult the business about all the outstading defects and re-jig the requirements document. This should have been avoided. Not only does it show a lack of understanding of the processes, it also gives IT a bad name and depicts us as too expensive.

If all the technical teams are to collaborate on a project, then we need everyone to step up and work as a team.

  • The BA should do their level best to capture and document the customer needs/requirements. Should use different form of practical tools to document the requirements and seek continuous feedback. Should also consult development and test team before finalising the BRD.
  • The developer should understand what the requirements are intended to do. Before applying his own logic or interpretation, should have detailed discussion with the BA. Please note that once the BRD has been baselined and approved, BA is there to assist the technical teams into shaping the customer requirements into a proper solution.
  • Test team should work closely with BA and developer to ascertain if the requirements are working as per the business rules or not. If there is a difference in opinion between BA and developer on how a feature should behave, tester can work as a mediator, if required. Tester should also have empathy for the customer and try to think in the customer’s shoes.

The success of a project depends on all its participants working closely to deliver a solution that benefits the customer/business. So if the project succeeds or fails on account of missing requirements that could have been easily fixed by following some basic principles, then it would indeed be a disappointing outcome for the entire organization!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.