Author Archives: Christian Stüdemann

Mocking the ObjectManager: Pitfall ahead!

If you mock up the ObjectManager, be aware of a very annoying pitfall, that might drive you nuts as long as you are not aware of it (and maybe even, if you already know about it)…

The following code works as expected:

/**
 * @test
 */
public function myControllerTestWorksFine() {
    $mockObjectManager = $this->getMock(Tx_Extbase_Object_Manager, array(), array(), '', FALSE);
    $this->myController->injectObjectManager($mockObjectManager);
    $mockObjectManager->expects($this->at(0))
        ->method('get')
        ->with(SomeClassToConstruct)
        ->will($this->returnValue($someMockedupClassToReturn));
    $this->myController->indexAction();
}

The (abbreviated) code will simply build a mockup for the ObjectManager, inject this mock into the controller, add an expectation together with a ‘with’ and ‘will’ statement and call the controller’s index action. Fine. The controller will at some point call the ObjectManager to get an instance of ‘SomeClassToConstruct’ and will get the defined result.

Now let’s change the order of the inject- and expect-statements:

/**
 * @test
 */
public function myControllerTestWorksFine() {
    $mockObjectManager = $this->getMock(Tx_Extbase_Object_Manager, array(), array(), '', FALSE);
    $mockObjectManager->expects($this->at(0))
        ->method('get')
        ->with(SomeClassToConstruct)
        ->will($this->returnValue($someMockedupClassToReturn));
    $this->myController->injectObjectManager($mockObjectManager);
    $this->myController->indexAction();
}

This won’t work like expected and will lead to a very hard to find error. Suddenly the parameters sent to the ObjectManager’s get-method don’t match the expected once any more. The reason: There is a hidden get-call added, you cannot see. To get this work again, you have to change ‘$this->at(0)’ to a ‘$this->at(1)’, because of the additional (unknown but counted) get-statement. This is ugly.

To avoid situations like these (and searching for the error / the solution for hours), use this coding guideline: Whenever you inject a mock object into another class, do it directly after generating the mock without any other statement inbetween. ($this->getMock directly followed by the inject-Statement)

Remember the bloody Cache!

The Situation:

You change annotations in your classes (e.g. the @validate annotations for your controller or for your entity objects). Of course you implemented some test cases before to check, if the validator is used and does it’s work. But… The bloody validator is not called, the annotation has exactly NO effect.

The Reason:

Typo3 caches the class information (incl. the annotations) internally in order to not parse all the class structures on each and every request (i.e. to save time…). As long as this cache is not refreshed, your annotation changes will have no effect at all.

The Solution:

Bildschirmfoto vom 2015-01-10 21:23:26

  1. You can uninstall and re-install your extension on each and every development step. This is pretty annoying, but cleares the cache reliably.
  2. If you just overwrite your php-files inside the Typo3 ext folder (see also I am lazy (The Server Architecture)), you have to clear the cache manually at least when changing class information like annotations.

The Problem:

You don’t have to do this very often, because most of the time you just change the internal implementation and not the class structure. Thus you don’t think of clearing the cache and just skip this step. This blog post is just here to remember you on that topic. Whenever something is not working at all, try to clear the cache and retest first. This might solver your problem immediatly.

Test the Test

Just a recommendation and a “best practise” coming from my current daily work on Test First Development. I wanted to introduce a new ValueObject, thus I started to write the Unit Test for it. It looked like this in my IDE (Eclipse; pseudo code used):

Bildschirmfoto vom 2014-12-17 21:43:52This is the source code:

class MyTest {
    public function checkDefaultValues() {
        $myValueObject = $this->createValueObject();
        $this->assertInstanceOf(Tx_Extbase_DomainObject_AbstractValueObject, $myValueObject, 'ValueObject does not implement AbstractValueObject');
        $this->assertABC(/*...*/);
    }

    /**
     * @test
     */
    public function checkSetAndGetPropertyA() {
        // ...
    }

    /**
     * @test
     */
    public function checkSetAndGetPropertyB() {
        // ...
    }
}

After i wrote the test like quoted above I tended to implement the ValueObject without any prior test run, because I was just very sure about what’s to be done and it was quite abvious that the test will fail very fast without any testable class in place. Fortunately I controled my feelings and had a break. I just started the test and did the implementation step by step based on the upcoming error messages. I was astonished! Why? Read on…

The very first test run failed, because no ValueObject class was available at all. That’s no rocket sience. So I did the first very, very basic implementation of “ValueObject”. It looked like this:

class ValueObject {
}

Simple. Important: I did not extend the ValueObject from the base class Tx_Extbase_DomainObject_AbstractValueObject on purpose to find out, if the “assertInstanceOf” of my test (highlighted in line #4 of the test case implementation) would fire an assertion failure. Surprise: It didn’t!

Why? Because “checkDefaultValues” was never executed at all. And that is, because I forgot to add the “@test” annotation for this test. I wasn’t aware of this bug, because the annotations were minimised in my IDE view (have a look at the screenshot on top of this post).

If I would have implemented the ValueObject in the usual way using a copy&paste of another value object or by adding the extension in the very first implementation, I wouldn’t have seen this flaw in the test.

Outcome: “Test First Development” means also: “Implementation driven by test defects.” This will test your tests and will let you have better tests and better software. The only thing on your side: You have to be disciplined… Right after this experience I won’t do it differently any more. 😀

ValueObjects stored as properties in other ValueObjects

As I mentioned in Testing and Coding a ValueObject it is necessary to clone an object, whenever you store it to or retrieve it from a ValueObject in order to make the ValueObject immutable. Just to add one fundamental topic to this general policy: Whenever you store a ValueObject (InnerVO) as a property inside another ValueObject (VOContainer), you don’t have to do this cloning process at all.

class InnerVO extends Tx_Extbase_DomainObject_AbstractValueObject {
    // ...
}

class VOContainer extends Tx_Extbase_DomainObject_AbstractValueObject {
    /**
     * @var InnerVO
     */
    protected $innerVO;

    /**
     * @param InnerVO $innerVO
     * @return void
     */
    public function __construct($innerVO) {
        $this->innerVO = $innerVO;
    }

    /**
     * Read the innerVO
     * @return InnerVO The innerVO attribute
     */
    public function getInnerVO() {
        return $this->innerVO;
    }
}

Why not cloning?

As long as you follow the formerly discussed policy to make each and every ValueObject immutable, you don’t have to clone it at all while throwing it all over your sourcecode, because it will never change. This is one of the major advantages of making a ValueObject immutable and ensure it’s immutability by forbidding each and every setter method.

The other way around: Whenever you want to change the value of the innerVO object (see above) stored inside VOContainer, you have to create a new innerVO object holding the new values. And because VOContainer is a ValueObject and therefor it’s immutable, you have to build a new VOContainer as well, because the VOContainer’s innerVO cannot be overwritten. There is no setter method and VOContainer must be immutable as well.

locallang with Typo3 4.5

Location of the locallang file

Some offical documentation does not refer to the locallang solution for Typo3 4.5 any more. In other word’s: It’s a little tricky nowadays to find out about how this works, but don’t panic, it’s simple: Locallang (the storage of translated terms for the module and the frontend) must be located at Resources/Private/Language/locallang.xml to be recognised by the extbase framework. This is simple, if you know it. 😉

Format of the locallang.xml

Secondly you have to concider the locallang.xml format to make things work. As long as the official documentation is available, you’ll be happy with it and it’s “Elements” link at the end of the page.

Be aware: In the moment you change from 4.5 to a higher version (4.6+), all this is void, because the localisation feature changes. If you want to work with 4.5 (LTS) as an intermediate step to a 4.6 or 6.2 version, you shouldn’t spend too much effort on the complete file structure of the locallang.xml, because it will be overwritten by the new framework very fast.

To define just as much as minimally necessary, your locallang.xml file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<T3locallang>
    <meta type="array">
        <type>module</type>
        <description>Language labels for the CS Events plugin</description>
        <labelContext type="array"/>
    </meta>
    <data type="array">
        <languageKey index="default" type="array">
            <label index="CsEvents.appointmentListEmpty">Sorry, currently no events are available.</label>
        </languageKey>
        <languageKey index="de" type="array">
            <label index="CsEvents.appointmentListEmpty">Zur Zeit sind leider keine Termine verfügbar.</label>
        </languageKey>
    </data>
</T3locallang>

As you can see, I define the <meta> and <data> elements only and I totally ignore the <orig_hash> and <orig_text> elements here. Those elements are used for distributed translations. You don’t need them, if you don’t want anybody to translate your’s extension’s quotations. I strongly recommend to not spend too much effort on this, but to proceed to Version 6.2 (LTS) as soon as possible. I’ll do that, thus I won’t add any content for additional translations here.

Running Typo3 4.5.xx Selenium testcases nowadays

If you want to (or have to) use Typo3 4.5.xx today, you will run in certain trouble trying to get the Selenium tests working, because the related phpunit extension requires an older version of Selenium, which itself requires some older browser versions to run. Here are the details and the solution:

  • Typo3 4.5.xx (actually tested with .32 and .37) comes with phpunit extension 3.7.22
  • phpunit extension 3.7.22 requires Selenium Server 2.32.0 and is not compatible to later versions like 2.43.1
  • Selenium Server 2.32.0 requires Firefox 20.0.1 and is not compatible to later versions like 33.0

Solution:

  • For the Selenium server it’s quite simple: Download the jar of version 2.32.0 from the official repository and run it using java. If you want to run two different Selenium servers on the same machine in parallel, you just have to set up a different port for the second one to listen to (use the -port nnnn option to do so).
  • For the Browser it’s a little more tricky, because (I assume) you want to use the latest browser version for your personal web surfing (and for your Typo3 development and testing environment) while you have to use the former version 20.0.1 for the Selenium WebDriver. I downloaded 20.0.1 from the official repository and installed it to a directory, that’s not part of my PATH declaration (in my case /opt/firefox-20.0.1/).
  • After installation you have to change your test script to make it tell the Selenium WebDriver to use the local Firefox installation. Have a look at the following code sample:
protected function setUp() {
    parent::setUp();
    $this->setDesiredCapabilities(array('firefox_binary' => '/opt/firefox-20.0.1/firefox'));
    $this->setBrowser('firefox');
    $this->setBrowserUrl('http://your.url/');
    // other stuff to set up
}

In line 3 you find the definition for the WebDriver that is telling, which Firefox binary to use. Please note that this refers to the binary directly and not to the directory only.

I wrote my own Selenium Base Test class to do stuff like this in order to not copy and paste this part for each and every test class. I strongly recommend to do so, if you want to do the basic setUp and tearDown jobs centralised.

Big challenge: Selenium causes the ObjectManager to fail

It might happen (in my case it did) that running Selenium tests will cause your ObjectManager to fail. After (!) the first Selenium test run, the ObjectManager or some other instance that uses the $GLOBALS[‘TYPO3_DB’] connection might fail like this:

Bildschirmfoto vom 2014-10-20 21:43:31I simply wrapped my ObjectManager call into a try-catch-block to get some more details about the issue. This is the stacktrace (you might face something similar):

#0 [internal function]: PHPUnit_Util_ErrorHandler::handleError(2, 'mysql_real_esca...', '/opt/typo3_src-...', 774, Array)
#1 [...]t3lib/class.t3lib_db.php(774): mysql_real_escape_string('Tx_Extbase_MVC_...', 0)
#2 [...]t3lib/cache/backend/class.t3lib_cache_backend_dbbackend.php(205): t3lib_DB->fullQuoteStr('Tx_Extbase_MVC_...', 'tx_extbase_cach...')
#3 [...]t3lib/cache/frontend/class.t3lib_cache_frontend_abstractfrontend.php(103): t3lib_cache_backend_DbBackend->has('Tx_Extbase_MVC_...')
#4 [...]typo3/sysext/extbase/Classes/Object/Container/ClassInfoCache.php(59): t3lib_cache_frontend_AbstractFrontend->has('Tx_Extbase_MVC_...')
#5 [...]typo3/sysext/extbase/Classes/Object/Container/Container.php(289): Tx_Extbase_Object_Container_ClassInfoCache->has('Tx_Extbase_MVC_...')
#6 [...]typo3/sysext/extbase/Classes/Object/Container/Container.php(127): Tx_Extbase_Object_Container_Container->getClassInfo('Tx_Extbase_MVC_...')
#7 [...]typo3/sysext/extbase/Classes/Object/Container/Container.php(95): Tx_Extbase_Object_Container_Container->getInstanceInternal('Tx_Extbase_MVC_...', Array)
#8 [...]typo3/sysext/extbase/Classes/Object/ObjectManager.php(83): Tx_Extbase_Object_Container_Container->getInstance('Tx_Extbase_MVC_...', Array)
#9 [...]typo3/sysext/extbase/Classes/MVC/Controller/AbstractController.php(167): Tx_Extbase_Object_ObjectManager->create('Tx_Extbase_MVC_...')
#10 [...]typo3conf/ext/csevents/Tests/Unit/Controller/AppointmentControllerTest.php(51): Tx_Extbase_MVC_Controller_AbstractController->injectObjectManager(Object(Tx_Extbase_Object_ObjectManager))
#11 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestCase.php(828): Tx_Csevents_Tests_Unit_Controller_AppointmentControllerTest->setUp()
#12 [...]typo3/sysext/extbase/Tests/Unit/BaseTestCase.php(62): PHPUnit_Framework_TestCase->runBare()
#13 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestResult.php(648): Tx_Extbase_Tests_Unit_BaseTestCase->runBare()
#14 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestCase.php(776): PHPUnit_Framework_TestResult->run(Object(Tx_Csevents_Tests_Unit_Controller_AppointmentControllerTest))
#15 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestSuite.php(775): PHPUnit_Framework_TestCase->run(Object(PHPUnit_Framework_TestResult))
#16 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestSuite.php(745): PHPUnit_Framework_TestSuite->runTest(Object(Tx_Csevents_Tests_Unit_Controller_AppointmentControllerTest), Object(PHPUnit_Framework_TestResult))
#17 [...]typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit/PHPUnit/Framework/TestSuite.php(705): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult), false, Array, Array, false)
#18 [...]typo3conf/ext/phpunit/Classes/BackEnd/Module.php(831): PHPUnit_Framework_TestSuite->run(Object(PHPUnit_Framework_TestResult))
#19 [...]typo3conf/ext/phpunit/Classes/BackEnd/Module.php(593): Tx_Phpunit_BackEnd_Module->runAllTests(Object(PHPUnit_Framework_TestSuite), Object(PHPUnit_Framework_TestResult))
#20 [...]typo3conf/ext/phpunit/Classes/BackEnd/Module.php(288): Tx_Phpunit_BackEnd_Module->renderRunningTest()
#21 [...]typo3conf/ext/phpunit/Classes/BackEnd/Module.php(206): Tx_Phpunit_BackEnd_Module->renderRunTests()
#22 [...]typo3conf/ext/phpunit/Classes/BackEnd/index.php(86): Tx_Phpunit_BackEnd_Module->main()
#23 [...]typo3/mod.php(51): require('/home/.sdb/var/...')
#24 {main}

It’s obvious that t3lib_db does not have any link to the database after a Selenium run (look at the last “0” parameter at #1, line 2). This link is set up by t3lib_db::sql_pconnect(…) and used by calling $GLOBALS[‘TYPO3_DB’]. That’s where this database connection is stored.

The root cause for this failure is inside the PHPUnit_Framework_TestCase class that runs all tests. In some circumstances (I’ll come to the details later on) it backups the $GLOBALS before running a test and restores them back right after the test was executed. It looks like this:

public function runBare()
{
    $this->numAssertions = 0;

    // Backup the $GLOBALS array and static attributes.
    if ($this->runTestInSeparateProcess !== TRUE &&
        $this->inIsolation !== TRUE) {
        if ($this->backupGlobals === NULL ||
            $this->backupGlobals === TRUE) {
            PHPUnit_Util_GlobalState::backupGlobals(
                $this->backupGlobalsBlacklist
            );
        }

        if ($this->backupStaticAttributes === TRUE) {
            PHPUnit_Util_GlobalState::backupStaticAttributes(
                $this->backupStaticAttributesBlacklist
            );
        }
    }

// more test preparation (code skipped...)

    $this->testResult = $this->runTest();

// shutting down the test (code skipped...)

    // Restore the $GLOBALS array and static attributes.
    if ($this->runTestInSeparateProcess !== TRUE &&
        $this->inIsolation !== TRUE) {
        if ($this->backupGlobals === NULL ||
            $this->backupGlobals === TRUE) {
            PHPUnit_Util_GlobalState::restoreGlobals(
                $this->backupGlobalsBlacklist
            );
        }

        if ($this->backupStaticAttributes === TRUE) {
            PHPUnit_Util_GlobalState::restoreStaticAttributes();
        }
    }

// even some more cleanup activities (code skipped...)
}

You see some if-statements steering, if the backup/restore is done or not. In case of my Selenium testcases, the backup/restore was performed, i.e. the following parameters are set:

$this->runTestInSeparateProcess = NULL; // default; = FALSE would be the same
$this->inIsolation = NULL; // default; = FALSE would be the same
$this->backupGlobals = NULL; // default; = TRUE would be the same
$this->backupGlobalsBlacklist = array(); // default

With this setting, the TestCase tries to backup and restore all $GLOBALS values including the $GLOBALS[‘TYPO3_DB’] connection. PHPUnit_Util_GlobalState does this by serialising and deserialising this resource, which in case of the DB-connection does not work. Right after the restore $GLOBALS[‘TYPO3_DB’] is set to 0.

To avoid this, a) someone has to fix the restore-procedure or b) you have to avoid the backup/restore of the DB-connection. Options for b) directly come from the parameters above:

  1. Run the test in a separate process (to be described later on) or…
  2. Set the “backupGlobals” switch to “FALSE” in order to not backup/restore any global variables (which might lead to other problems, if your tests change global values) or…
  3. Exclude the TYPO3_DB variable from being backuped/restored by adding it to the blacklist.

I found a Typo3 Forum post about this issue: http://forum.typo3.org/index.php/t/140961/ It recommends to use solution #3. It works!!!

class Tx_Csevents_Tests_Selenium_BasicSeleniumTestCase extends Tx_Phpunit_Selenium_TestCase {

    // add "TYPO3_DB" to the global backup black list in order to not lose the DB connection
    protected $backupGlobalsBlacklist = array('TYPO3_DB');

    //.... (followed by the rest of this class)
}

That’s it: I already introduced a “BasicSeleniumTestCase” class to handle all Selenium Test cases. Now I use this class to overwrite the $backupGlobalsBlacklist, which is empty by default, using a list that contains “TYPO3_DB”. Now $GLOBALS[‘TYPO3_DB’] is not backuped nor restored any more, thus it’s save from being overwritten by 0 (because resource values are not serializable). Finally fixed! 😀

ValueObjects with lots of (optional) objects

If you want to define ValueObjects with a lot of optional objects using the implementation described in Making a ValueObject NULL-prove, this will lead to a very long constructor and a lot of copy / paste code. Not nice.

Solution #1: divide et impera

The very first idea is to define private setters for each object. These setters are called by the constructor only and every setter is responsible for one object only. This will reduce the length of the constructor code, but it won’t solve the copy-paste issue.

Solution #2: Define a generic (optional) object setter

class MyValueObject {
    protected $param1;
    protected $param2;
    protected $param3;
    protected $param4;
    protected $param5;

    public function __construct($param1, $param2, $param3, $param4, $param5) {
        $this->setParam('param1', $param1, DateTime);
        $this->setParam('param2', $param2, SomeOtherClass);
        //....
    }

    private setParam($name, $value, $class) {
        if($value === NULL) {
            $this->$name = NULL;
        } elseif($value instanceof $class) {
            $this->$name = clone $value;
        } else {
            throw new InvalidArgumentException('expecting parameter ' . $name . ' to be of type ' . $class, 1413833055);
        }
    }
}

Now you can use setParam(…) to set all internal objects including cloning, NULL values (you might switch this functionality on/off for certain parameters to make them mandatory) and type/class checks. This reduces your code, because your constructor uses just one line of code per parameter and your setter is defined just once, thus you don’t do copy/paste and can rely on the centralised functionality. Nice? Nice.

Making a ValueObject NULL-prove

in Testing and Coding a ValueObject we used “clone” to make the ValueObject and it’s internally stored objects immutable. The code for the constructor and the getter was quite simple:

/**
 * @var DateTime
 */
protected $dateTime;

/**
 * @param DateTime $dateTime
 */
public function __construct($dateTime) {
    $this->dateTime = clone $dateTime;
}

Unfortunately this leads to an exception, if you try to set NULL. Furthermore it doesn’t take care of wrong parameters like e.g. an integer, array or objects different from DateTime. To make this ValueObject “NULL-prove” and check the argument’s type, we generate an according test first.

/**
 * @test
 */
public function setAndGetNullWorks() {
    $myValueObject = $this->objectManager->get(MyValueObject, NULL);
    $this->assertNull($myValueObject->getDateTime());
}

/**
 * @test
 * @expectedException InvalidArgumentException
 * @expectedExceptionCode 1412287860
 */
public function constructWithNonDateTimeThrowsException() {
    $myValueObject = $this->objectManager->get(MyValueObject, 42);
}

The first test just calls the constructor with $dateTime=NULL and expects the same “NULL”, if it reads this value back using the getDateTime() method. Using the simple clone implementation above this will fail, because “clone NULL” does not work and will throw an exception.

The second test tries to send an int to the constructor. Please note the annotations: We expect the InvalidArgumentException to be thrown with a specific exception code (1412287860). This will fail as well, if we try to clone the int value.

You can use the constructor with an explicitly named Class like this:

public function __construct(DateTime $dateTime) {
    // ....
}

But this won’t let you use “NULL”, because “NULL” is not accepted as a DateTime object.

Conclusion

Do not use dedicated Class references in parameter lists and use class / type checks inside your value objects, if you

  • want to make the parameter optional (thus it might be “NULL”) or
  • want to use parameters that might be of different classes and/or types (“mixed”).

Do use dedicated Class references in parameter lists, if you

  • want to make the parameter mandatory and
  • want the parameter to be an object of a dedicated class.

You have to ask yourself…

Of course you have to ask yourself how a method shall react in cases like the once above. In my case I accept NULL parameters and I want to get an exception, if the parameter is not a DateTime object. It’s a very good idea to first think about your expectations and to write a unit test reflecting exactly those expectations and not thinking about the actual implementation in the first place. Do the implementation afterwards to meet the functional test, but don’t write a test to cover your planned technical implementation.

The implementation:

One of a variety of solutions to cover my tests above is the following implementation for the constructor:

/**
 * @param DateTime $dateTime
 */
public function __construct($dateTime) {
    if(is_null($dateTime)) {
        $this->dateTime = NULL;
    } elseif($dateTime instanceof DateTime) {
        $this->dateTime = clone $dateTime;
    } else {
        throw new InvalidArgumentException('MyValueObject::__construct must retrieve a DateTime object', 1412287860);
    }
}

One remark regarding the exception code: It’s pretty equal which exception code you use. I totally recommend to use exception codes and to use unique exception codes for every position in the source code where an exception is thrown. Reason: It’s very easy to find those exception locations by searching for the exception code using a simple full text search. This makes support and debugging much more easier.

If you want to generate a unique exception code, just use the current system time stamp when writing your code (e.g. “date +%s” for Unix/Linux systems). I always define the exception code in the test that expects the exception to be thrown, because this is the very first place where the exception code is used in the “@expectedExceptionCode” annotation. The actual implementation just copies this value to meet the test case’s expectation.