Black-box and white-box testing

As you might remember, if you read a previous blog post of mine about tips for software testers, I mentioned two elementary terms used in software testing – white box and black box testing. In this article, I will explain what they are and what are their positive and negative aspects.

What do white box and black box testing mean?

Black box testing is when you have no idea how the software you are testing was programmed. You might not even know exactly how the app should work but it is your duty to squeeze out every single functionality or bug you can come up with.

This kind of testing is often done by testers who have little to no programming knowledge. It’s actually a good thing because they have the perspective of a usual end user. Black box testing is considered high-level testing, which means that its main goal is to test functionalities from the behavioral point of view.

White box testing, also known as clear box testing, happens when you have insight into the code and/or general knowledge about the architecture of the software in question. It’s counted among low-level testing and focuses mainly on integration and unit testing.

White box testing requires programming knowledge or at least a deep understanding of the code that implements a specific feature. In the first case – a tester builds the code for e.g. unit tests. In the second – a tester uses tools dedicated to testing backend functionalities, e.g. Postman. Of course, clear box testing isn’t only used in the testing of backend functionalities. You can build automates that will test frontend functionalities but generally clear box testing focuses on testing the code itself.

What are the pros and cons of black-box testing?

As I mentioned earlier, in black-box testing you have no idea how the functionalities have been implemented. You can get very creative with the ways you use certain parts of the software you are testing and go with your intuition. This way you can find bugs developers haven’t thought of. 

Manual tests are one of the ways to implement black-box testing. You can either click through the app yourself or use an automation package. I like to use Selenium. What benefits does automated testing bring? Let’s say you want to register 100 customers. Why risk hand cramps and watery eyes from filling in the same registration form for two hours? I’ve done it too many times in the past. Now Selenium does it for me once I write a code it can execute many times in a row. Minus the watery eyes and hand cramps of course. 

Black-box testing adds the user perspective but beware of misinterpretations

The most obvious advantage I see in black-box testing is detecting bugs and glitches in the software by using and abusing it as some end users would. What’s the most appropriate time to run black-box tests? One can do it during development but even so, one can NEVER forget about it right before putting the software to production.

The biggest con to black-box testing (which is not really a con if you also test software with white-box techniques) is the fact that you can sometimes lose a lot of time testing something that was not intended to exist in the first place. Imagine a situation when a certain functionality had a different purpose than you interpreted. Until you ask a developer or product owner, you might not even realize that your interpretation was in fact wrong. 

Being in the dark (I can’t get enough of these cheesy puns) can be good and bad at the same time. That’s why you have to ask questions, often ahead of time to make sure that you understand what you’re testing. 

The good, the bad, and the ugly about clear-box (white-box) testing

In clear-box testing you have it all laid out for you – you know the language the code has been written in, you know what it has to do, what are the conditions of the cases implemented in the software, etc. This gives you a nice base to work with. 

If you have a little bit of experience with similar software – you can easily make at least one of two scenarios that should always be checked when testing the code in question. 

White-box testing scenario

A good example is http request testing. Let’s say you have a function that sends PUT request and takes one parameter “client_id”. Part of your code might look like this:

import requests

import json

def test_should_send_put_request(self)

url = ‘http://myexample123.com’

client_uuid = ‘1d2f3g4h’

request = requests.put(url, data=json.dumps(client_uuid))

expected_response_code = 200

response_code = request.status_code

self.assertEqual(expected_response_code, response_code)

You’ve just tested whether the url in question correctly receives PUT request with the given client_uuid. That’s one of the first scenarios when testing http requests – checking if they work with correct data. 

Next, you have to check what happens if you paste incorrect data, and so on.  To check a lot of combinations of client_uuid you can use Data Driven Tests from ddt library in Python. I’ll talk more about them in my next blog post.

After checking generic scenarios, you can get more creative and start adding a human element to them. To do that, think of the way this request could be sent by a user or, even better, multiple users. There are a lot of tools that can help you with that. I’ll write about them in one of my upcoming articles.

Code coverage vs. user perspective

The biggest pro to clear box testing is that you can make multiple combinations of cases, covering your code with high percentage of tests. And by that you can assure your clients that the code is optimized, and ready for further maintenance and development. This is essential when it comes to building your brand – development of code that will work for years.

But let’s take a step back. What if you checked only the code, and not actually how it is used by people who don’t know anything about the application you are developing? It is easy to lose perspective when being “inside” all the time. That’s the biggest con to clear box testing.

A combination of both techniques works best

That’s why if you are a tester using both techniques – clear and black box testing – you have to keep yourself aware and always think as an end-user would.

If you prefer graphical presentations I’ve prepared a table that sums the pros and cons of clear and black box testing.