Testing the “Negative” in Software

26 Feb 2021

Complete Product Test Plans

Having a complete test plan is important when testing any product with software. Functionalities, features, and user interfaces need to be thoroughly tested to verify satisfactory performance before a product can be released to the public. Items and documents such as wireframes, entity-relationship diagrams, and developer notes are great tools to use when developing a test plan for main functionality and features, but to have a more robust and complete plan, "negative" test cases also must be included.

Negative test cases are important to ensure that any mishandling, invalid, or interruptive behavior performed to an application or device does not produce unexpected behavior. Some important negative test categories include "interrupt" and "edge" test cases. "Interrupt" test cases include actions that may disrupt main functionality.

An example of an interrupt test for almost any mobile application can be to receive a phone call while the application is launched to the foreground. "Edge" test cases usually refer to the minimum or maximum testing of situations. For example, a cloud-based storage application that only allows you to upload one to five pictures at a time would include and "edge" test case where the user will try to upload five pictures at a time or try to perform an upload when no pictures are selected.

Tests like these are necessary to perform to feel fully confident when releasing an application or product. Creating these tests may seem simple but there are many different factors to think about to efficiently test negative scenarios.

Here are a few examples; interrupt tests like receiving a phone call or text messages while using the application is important and is a basic negative test that should always be run, but some products may require other negative tests that can be easily overlooked. Entering emojis in text fields when entering data is another example of an easily overlooked negative test case as emojis can have negative effects when being used in changing device names or entering device passwords.

For a more in-depth example, imagine you have a cloud-based storage application, and my Wi-Fi network is dropped, how will that effect an upload in progress? Will the upload continue, pause, or cancel? Will the user be notified of this change in network and give the option on how to proceed? These are all important questions that need to be answered especially when thinking about the end user.

Using the same cloud-based application as an example, imagine a scenario where a user is performing an upload over Wi-Fi of photos that exceed 2GB and after uploading 100MB, the user's Wi-Fi network connection is dropped forcing the application to complete the remaining 1.9GB upload over mobile network. If the upload is not paused or does not notify the user of the upload continuing over mobile data, the user may become frustrated that the application for using mobile data for the large upload, especially if that user is already low on mobile data. This frustration could cause the user to write a negative review for the application causing the rating of the application to go down.

By including these negative test cases, managers and developers can feel more confident with releasing their products. Having a team in house to develop these test cases and scenarios as the product is developed is important. To ensure the best coverage, companies can partner with third party test firms to help with test plan development or even perform their own sets to help ensure all aspects of functionality have been reviewed and verified. In-house testers can sometimes develop "blinders" the more they test the product, it is always a good idea to bring in a fresh point of view.

Negative test cases are always an important part in testing and can be easily overlooked. By being aware of how these negative scenarios can impact a product through this testing, companies can work to further improve the overall end user experience. When it comes to testing, we never want to forget to look at the positives that the "negatives" can bring.