Testing

Unit tests, integration tests, code coverage, etc.

  • Testing null, empty, blank

    In many cases, when you have a method that accepts a mandatory string parameter, you want to verify that the parameter contains a value that isn’t null, nor empty, not blank (i.e. consisting solely of whitespace characters). While writing the check is easy, testing it can be a bit annoying.

  • Flaky tests

    A flaky test is a test that can flip from success to failure without any code change. Such failures can be annoying and difficult to diagnose. In this post, I’ll focus on browser tests, where flaky tests can happen more often.

  • Code coverage on integration tests?

    Should you collect and measure code coverage on integration tests or only unit tests? In this post I’ll share some thoughts on this topic.

  • AssertJ cheatsheet

    AssertJ is a an assertions library for unit tests in Java that is well worth considering. Here are some examples.

  • Exclude class from JaCoCo coverage

    JaCoCo’s exclude configuration works with classes, so the .class extension is relevant in specifying the path.

  • Code coverage in .NET Core and SonarCloud

    In this post, I’m creating a code coverage report for a .NET Core project. I’m also using SonarCloud to analyze the project’s quality.

  • Adding code coverage with nyc

    In this post, I’ll add code coverage to the build pipeline and configure TeamCity to break the build if the code coverage drops.

  • Adding mocha unit tests

    In this post, I’ll add unit tests to the example application that I’ve been fiddling around with in the recent posts.

  • Adding WebdriverIO tests

    In this post, I’ll add some automated browser tests using PhantomJS and WebdriverIO.

  • Smoke test for a Docker image

    According to Wikipedia, a smoke test is a preliminary test that reveals simple failures severe enough to (for example) reject a prospective software release. The process of smoke testing aims to determine whether the application is so badly broken as to make further immediate testing unnecessary. If we consider our dockerized blog-helm web application, a possible smoke test can be: can we pull the image from the registry? If we run the image, does the container stay alive or does it crash immediately? In this post, I’ll implement this in an extra build configuration in TeamCity with a generic bash script doing the actual work.

  • Mocking AWS with localstack

    We use AWS at work and I’ve been learning more and more about it. AWS offers so many services it’s even difficult to remember all of them. It’s quite impressive how many things a developer could build upon in order to deliver a scalable solution. The phrase “standing on the shoulders of giants” is quite fitting.

  • TDD with .NET Core and Visual Studio Code

    In this post we’ll see how to create a .NET Core solution with two projects with using nothing but the command line. Once that is setup, we’ll do a bit of TDD using Visual Studio Code.

  • Code Coverage with JaCoCo

    Code Coverage is a useful set of metrics that show you how much of your code you’re impacting during testing. It doesn’t say much about the quality of your tests (you can read more in the old post What is code coverage?), but a 30% coverage is definitely worse than 90%. Let’s see how we can use JaCoCo to see our code coverage in the Java world. We’ll check a few options to use it, such as using it manually, using it within a CI, breaking the build with it, etc. The assumption is we’re working with a Maven project.

  • Functional Testing - WebDriverIO Integrated Test Runner

    In the previous series of posts, we had a look at functional testing with WebDriverIO and mocha. We explored the page object pattern and saw the challenges of asynchronous programming with promises. We even saw some ways of mitigating those challenges. However, there is another approach to the same problems. Instead of running our tests with mocha, we can run them using the integrated test runner of WebDriverIO. From version 4 onwards, I’ll quote from their site, “the integrated test runner allows you to write asynchronous commands in a synchronous way so that you don’t need to care about how to propagate a Promise to avoid racing conditions”. If you’re starting a new project, or if you don’t have many tests to migrate, this is definitely an interesting approach. This combines a synchronous-like approach without using anything extra.

  • Functional Testing - Hiding Async

    In the previous post, we explored the Page Object pattern and rewrote our tests to use this technique. Sometimes, it can be that the tests appear to be a bit verbose due to the usage of promises. Additionally, promises and asynchronous programming in general can be somewhat confusing to developers. Let’s see some ways of making the tests shorter and easier to read.

  • Functional Testing - Page Object Pattern

    So far, we’ve seen how to write functional tests using the WebDriverIO API directly. Let’s see what problems we may encounter with that and how the page object pattern comes to rescue. Our example functional tests work against Google’s homepage. One test verifies that the search text box exists and it’s visible. Another test types something into that search text box and performs the search. In order to identify the text box, we use the same selectorinput[name=q]. The problem starts here and it’s not unique to functional tests: it’s copy pasting around magic strings.

  • Functional Testing - Dealing with failed tests

    In the previous post, we wrote some functional tests and we encountered our first failing test. Let’s see how we can go about investigating these failures, both on a developer machine and during continuous integration.

  • Functional Testing - Selectors

    So far, we have essentially only one test, a test that verifies the title of the browser. Let’s try to make a more interesting test. We’ll need to interact with elements on the page and simulate the user’s journey.

  • Functional Testing - First steps towards a framework

    Before we go any further, let’s take a break and add a bit of a structure to our functional tests. This will allow us to reduce the code we have to write for each test. We’ll also start thinking about how to organize the tests into folders and files.

  • Functional Testing - Reducing Code Duplication

    Let’s continue our functional testing examples by adding a few more tests. This will demonstrate why we need to start thinking about the structure of our tests and why we should be developing a framework that will allow us to write less code. Let’s add one more test. We already have a test that verifies Google’s homepage title. We’ll do the same for Yahoo’s homepage. By the way, maybe it goes without saying, but normally you would be verifying your own site, during development and CI.

  • Functional Testing Hello World

    Let’s have a look at a first example of writing and running a functional test. This is going to be a very basic hello world example, but still it gives an opportunity of looking at the bare minimum usage of WebDriverIO and a first taste of asynchronous programming with promises.

  • What is functional testing?

    In the previous series of posts, we’ve explored the basics of unit testing and the principles around it. When developing websites, there is another important type of testing: functional testing (also known as browser testing).

  • Using sinon mocks

    In the previous posts we had a look at sinon spies and stubs. There is one more technique we can use in order to orchestrate our test dependencies: mocks.

  • Using sinon stubs

    In the previous post we had a look at sinon spies. With spies, we are able to determine if a specific function was called or not. Usually the dependencies between units are more interesting, they involve units co-operating, exchanging data and so on. Spies do not suffice. Let’s have a look at another technique, using stubs.

  • Using sinon spies

    In the previous post, we implemented a new feature for our calculator: it makes a bell sound when you divide by zero. The bell is a simple function that the calculator calls and it is provided as a constructor dependency. We wrote a unit test for this as well, but the code for that is a bit verbose. Let’s see how we can use a mocking library like sinon to reduce and standardize the testing code.

  • The division by zero bell - Dependencies in unit tests

    We left our calculator in the previous post in a decent state, being able to do the four basic mathematical operations. In the special case of division by zero, we want the calculator to make a noise like a bell. Let’s see what we can do about this.

  • What is code coverage?

    Last time we had a look at test driven development and our calculator learned how to do multiplication. In this post, we’ll add division and talk a bit about code coverage and unit test quality.

  • What is Test Driven Development?

    Let’s continue building up our calculator with more mathematic operations. So far we have addition and subtraction, so multiplication comes up next.  In this post, we’ll have a look at test driven development. Even though the examples are a bit trivial, I hope they’ll outline the important points.

  • A closer look to a basic unit test

    In the previous post we started writing a basic Calculator class and added the first unit test. Let’s have a closer look at that unit test and extend our calculator with more features.

  • What is a unit test and why should I care?

    So, let’s start with some basics. What is a unit test? A unit test is a piece of code that validates the expected behavior of a unit in isolation. I guess the next question is, what is a unit? A unit is the smallest piece of code in a given programming language. Typically that is a function or a class method. Let’s see it with an example.

  • Code coverage for open source .NET with AppVeyor and Coveralls

    Code coverage is a useful metric of the quality of your code. It shows how much code is being covered by unit tests. It doesn’t necessarily mean that the unit tests are well written, but no metric can probably tell you that. However, aiming for a specific code coverage, let’s say 70%, is a good practice, because failing to meet the goal might mean somebody didn’t write enough unit tests.

  • Chai as promised with Chai string

    Here’s a small tip that puzzled us at work the other day for a while. How do you use chai-as-promised together with chai-string?

  • Adventures with automated browser tests in JavaScript

    This is a long post, be advised! It goes through the adventures we have had at work with automated browser tests in JavaScript. It has been a journey full of challenges and knowledge build up, a journey that still goes on!

This site uses third party cookies from Google Analytics and Google AdSense Accept and close popup