What is Software Testing – Definition and Explanation

When you test, either by writing code or performing manual testing, you do so because you want to ensure that your software works as intended. This unit focuses on writing code that tests various parts of your application code. The test code won’t ship with the application code. Think of test code as scaffolding that supports the software development process throughout your program’s lifetime.

Let’s start by defining testing with some examples, before we dive into the concepts.

The testing process

Testing involves a series of test methods that run against your product code, and return a pass or fail result. The result is based on certain criteria that is asserted. Test results appear in a list of red “x” marks to represent failing tests, and green “✔” marks to represent passing tests. These symbols make it easy to assess what functionality is working or not at a glance.

test

Test method definition

Tests are very much like regular methods. They’re organized within test classes, have method signatures, and accept parameters. Tests reference and call your product code (another way of saying your non-test code), and compare how it behaves to an expected outcome.

[TestMethod]
public void AddTest()
{
    // Arrange
    var calculator = new Calculator();

    // Act
    var actual = calculator.Add(1, 1);

    // Assert
    Assert.AreEqual(2, actual);
}

In the preceding code, the product code is the Calculator class and its Add() method. The comparison happens when calling Assert.AreEqual(2, actual). This pattern is something you find in most, if not all, test methods. Here’s how this test would appear in Visual Studiotest-intro-testmethod-909

One difference between tests and product code is they don’t run as part of your app’s normal function. So instead of using F5 (or the large green run button at the top of your development environment) to run as you would your app, you choose which tests to run. You can make this choice through Visual Studio Test Explorer or other helpful editor tools. Tests are there for support but because they aren’t shipped with the app, they run independently like their own app. test-intro-run-tests-collage-118

 

How can tests help prevent regressions in functionality?

Remember the phone number bug from the scenario mentioned in the introductory unit? As soon as new code was added to accept international phone numbers, the function for adding domestic phone numbers broke! A test using domestic phone numbers as input might have caught this behavior change sooner. It would have tested not only the new functionality for international numbers, but also the old functionality for domestic phone numbers.

As we implement new code, it might not always occur to us what old scenarios could be affected by new changes. The phone numbers are a simple example, but imagine apps with hundreds of different input formats. Tests make it easy for all variations of the old behavior to be checked with a simple test run.

 test-intro-phone-example-89

 

Why we test

Testing can be a powerful tool to improve the quality, architecture, and overall health of a code base. In this unit, we discuss some of the ways that testing can have a positive impact in the software industry.

Validate code changes and quality

Any successful software tends to grow in functionality and behavior. Sometimes it can grow beyond what the creators ever imagined to support more scenarios and customer requests. As functionality grows, it can become more difficult for a single developer to remember all the functionality it contains or keep track of how to preserve that functionality.

Tests not only keep track of the different capabilities of a program, but can also continue to check that the old functionality didn’t break as new code is added. A fundamental purpose of testing is validating that code changes don’t break existing functionality, and ensuring that newly added code will continue to work as expected even with future changes.

test-quality-cycle-90

Tests can help catch bugs before you even check in your code changes. It’s a best practice to run tests before major steps in your development cycle. This final check allows you to improve your code quality so your users don’t have to report that something broke in your latest update. It’s much nicer to catch bugs while you’re still developing the code, rather than when you’re in the middle of deploying your app or when the app is in production with customers using it! Catching bugs early in the development cycle saves time and money.

Industry examples

A good example of the power of testing can be found in the C# compiler itself! Millions of developers write code in C# every day. Have you ever wondered how tools can keep adding language features and releasing new language versions without breaking existing code? Tests provide a final set of hundreds of thousands of checks to make sure previous scenarios still work. There’s no way one person, or even one team could remember all those scenarios.

It’s especially true when you consider that the C# compiler is open source, and accepts contributions from a world-wide community of developers. The C# compiler team can accept changes from the community with confidence, in part because of the checks that testing provides. Testing is helpful for individuals and at the team level, allowing you to scale your product and your team as they grow.

Effects on architecture

Testing can also force some architecture improvement. You can’t test small parts of your code if it’s structured as one gigantic method. Tests can help you break up all the functions of your code into more modular components. Tests can reduce repetition, improve stability, and even make your code easier to read and navigate.

For example, if you’re using the same logic in many places throughout your app and then realize there’s a problem with it, you’ll have to find and update all the places you used it. Instead, if you use a method, you only need to update it once, because even though that method is called many times throughout your app, you only wrote the logic once in the body of the method.

Testing helps you pause and consider if you’re repeating the same logic multiple times and could use a method instead. It provides developers a chance to restructure code for the best reusability and stability going forward.

Code coverage and code health

Code coverage is a metric that indicates how much of an app’s production code is covered by tests. It indicates if tests actually exercise all the product code, including branching logic and method overloads. Code coverage can give a basic idea of what areas need more testing. Visual Studio even has tools that can highlight what lines are covered by tests, and what lines aren’t, in your editor.

test-why-code-coverage

Tracking the code coverage percentage over time can give you an idea if the new code being added to your repository has tests. If new code isn’t being tested, it might be an indication that the repository is building up technical debt. While code coverage is useful, it’s not an ultimate indication of repo health. It should only be one of many factors used to assess the health of a repo.

Different code coverage engines can calculate coverage differently, and many programs can appear to have low coverage, despite being well tested. For example, heavily testing certain methods that have high use, and ignoring other methods, might actually be the right thing to do for a particular repo. We wouldn’t encourage every repo to try to achieve 100 percent code coverage, because that isn’t a practical investment for many businesses.

We’ve also featured the Best Web Hosting Services of 2023.

There’s much debate about whether or not there’s a universal code coverage percentage that repositories should aspire to. For now, we believe the best guidance is to judge on a case-by-case basis, and not arbitrarily hold your team to a number without a deeper discussion.

Leave a Reply

Your email address will not be published.