Unit Testing – Introduction

“Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.” – Michael C. Feathers

Do you want to banish the evil spirit of terror? Or maybe you just want to be sure that you are producing a code without errors? For whatever reasons you are here I will give you a few more to stay with me and catch up with test framework in Magento 2.

Tests, tests! As-always-not-sufficient-budget is killing me even without those nightmarish tests, I do not want to write them. Ah, maybe I do but I wont be able to delivery the final product on time.

This is our everyday horror, the eternal fight of temptation to write a better code covered with tests versus finite budget and the final deadline. Guess who will win?

Should we really write a tests?

Yes, we should. Even more – it is a must!

Tests reduce the amount of bugs in a new code and also give us strong fundamentals when working with legacy code. Tests improve code quality, improve architecture, simplify refactoring process, save us from ourselves and other developers and finally – please, sit down, deep breath.. – reduce the final time of building the app. As a result of all those factors we feel better, more comfortable, we banish the fear of unpredictable changes. Therefore we are able to deliver better and more bulletproof applications. The green light is our salvation; we desire it. We love it. We crave for it.

It is hard to start, after all we need to learn something new and master particular tools. But hey, this is our daily fuel – challenges!

Apart from language context what is the most basic concept of tests?

Assertion is very fundamental. Basically we want to write a code which will assert that the code we already wrote yield expected results with a given input and predictable environment setup. We want to know if results of some operations meet our expectations. We want to assert that the result is what it should be. Whatever is the flow of the program, no matter if the test is complex or not, finally if the assertion holds true then we are safe.

In case of UT we want to test only the particular part of the code, generally some public interfaces of the class. We want to know if our algorithm works as we want to in abstraction of the environment. That is why we create mocks and stubs (I will write about those two bastards in the next article) instead of real classes just to set up minimal and known environment. We want to isolate our algorithms and treat them like a black boxes. Feed them with data and check with assertion that the result is correct.

Tests in Magento 2

We have a few levels of available tests for Magento2. You can check all of them by looking into the dev/tests dir. As you can see there are the following types:

  • Static
  • Unit
  • Integration
  • Functional
  • Api-functional
  • Js
Static Test

Static tests are responsible for checking that the code follow the best practices and standards. Here we have a few testsuites and each cover different area:

  1. PHP Coding Standard Verification. This suite basically run Code Sniffer, Mess Detector, Copy Paste Detector and checks the code against PSR2. Nevertheless Magento 2 developers do not follow the PSR2 rules.
  2. Code Integrity Tests check modules dependencies and composer validity.
  3. Xss Unsafe Output Test basically checks the output. It is possible to mark code for example with /* @noEscape */ comment to force the green and this is very common practice – unfortunately.
  4. And the last two are Less an JavaScript code analysis.
Unit Test

In Magento2 we use PHP Unit to test units of code in isolation. I will talk about this more deeply later but for now it is worth to mention that Magento2 gives ready to use configuration.

Integration Test

Integration tests also use PHP Unit but here we want to test if parts of the system works together without any problems. We check the integrity of different interfaces and therefore we sacrifice the isolation for the sake of ability to check the consistency of system at higher level. Beside of that there is a mini framework dedicated to build integration tests in Magento. You can find it in dev/tests/integration/framework/Magento/TestFramework. There is a lot of knowledge here I we will come back to this topic during some testing kata sessions.

How to run tests in general?

I do not assume which IDE do you use but I want to assume that you can handle few CLI command. That’s enough for a start. Also tests run in PHP Storm are slower, especially it makes a difference for integration test. To run simplest integration test in PHP Storm it takes about two minutes when I can run the same test using terminal in about 20s. That makes a difference.

Go to the root of Magento, get some candies and prepare yourself.

Easiest way to run any type of test is simply to run

Available types are: all, unit, integration, integration-all, static, static-all, integrity, legacy. This command will simply run all testsuites in a given test scope. This solution is best for CI but an overkill for common tests. It consumes too much resources and takes too much time to be efficient at any rate.

Configuration first.

Each type of tests has its own autoloader, bootstrap and sample xml configuration file – phpunit.xml.dist. The best practice is to copy this file and rename it to phpunit.xml. This is an example of configuration from integration tests:

Let’s analyze it.

The main <phpunit> tag, besides of defining xsi schema also contains some configuration options. You can check all of possible option in the PHP Unit documentation. As default colors are enabled but you might want to enable also the verbose output. To do that simply put verbose attribute into the tag:

<testsuites> define the scopes of tests. Inside of this tag are located <testuite>. Attribute name obviously sets the name of the testsuite and is used later to run particular test. Inside <testsuite> we can define which tests files we wish to use. Here are possible definitions:

Final result might look like this:

Suffix is the suffix of files inside the directory. Good practice is to name tests according to the following pattern <class_under_test>Test.php.

In <php> we can set ini settings, const and global variables.

The last interesting thing is <logging>. PHP Unit is able to generate coverage report based on all performed test. It is possible to export result data to csv, xml or – this is my personal best friend – html.

These are the basic possibilities that will help you out in most cases. Now we are ready to rumble.

How to run Unit Tests?

Go to <root>/dev/test/unit and run following command

Configuration will be automatically loaded from local xml file. If you want to run test from different location then pass the path to phpunit.xml in parameter “-c|–configuration“.

In case you want to execute particular test which is not defined in testsuites then simply use the path to file as a command’s argument

However, unlike for integration tests I recommend to use IDE like PHP Storm to execute your tests.

How to run Integration Tests?

Here things getting a little bit more complicated. The general strategy of running test remain the same however we have to abandon the environment isolation. To achieve this task we have to configure access to additional database. We need this to bootstrap Magento2 and save data. Obviously, saving tests data in main database is a very bad idea.

Rename the following file dev/tests/integration/etc/config-global.php.dist to dev/tests/integration/etc/config-global.php

As you can see, besides the admin account configuration, it is also required to configure database credentials. Example configuration might look like this:

Also we have to add some additional configuration in phpunit.xml. As in the former case the file is located in tests directory dev/tests/integration/phpunit.xml.dist Rename it again.

The main differences are in <php> tag. Each const is well commented and there is not much to add. The most important thing here is TESTS_CLEANUP option. If set to true then whole application will bootstrap from scratches including truncation of all tables. If set to false then app will start from last known state. Nevertheless, we can still run some test in isolation by adding appropriate comment. By isolation here I mean that all changes will be rollbacked immediately after the test’s execution. But this is a different story…

Also, if cleanup is disabled then dev/tests/integration/tmp will be created. Here are stored sandbox data like etc/config.php, etc/env.php, pub/static, var and so on. It means that we can perform cleaning current Magento’s test instance simply by removing appropriate sandbox directory.

Now, it’s time to run test. Actually, there is nothing new here. The only difference is that tests should be run from dev/tests/integration – to be more precise, tests should be run from directory with config file.

Try to run one of Magento’s default testsuite:

For integration test you must load up with a lot of patience since it takes relatively lot of time to execute even simplest one.

For the next few article we will finally write some real tests. I think that it will be the best if we could cover some most repetitive tasks like testing some basics algorithms with UT and more importantly, write some integration tests.

Maybe I will do a small screencast from string calculator kata. We shall see.