Monthly Archives: February 2014

Project Configuration Issue solved

I wrote about an issue on my Eclipse Project Setup: I wasn’t able to setup the autocomplete functionality correctly by including extbase, fluid and phpunit as Eclipse projects. Instead I used a workaround by adding the paths to these extensions to my own project’s build path. This isn’t very comfortable, because the folders keep popping up in my project folder while I am concentrating on my own development. Now I finally solved this issue.

What was wrong?

I added 3 empty projects “extbase”, “fluid” and “phpunit” to my Eclipse workspace. This was just fine. To include the actual source code of the extension, I also added the extension directory to the project’s Include Path. Have a look at the screenshot:The three additional Eclipse ProjectsYou see that e.g. the Eclipse project “fluid” hasn’t got any source path, but has got “/var/www/typo3[…]/fluid” as an additional include path. This is fine for the “fluid” project, because it just needs to include these sources. I won’t change them at all, thus I don’t need to have them inside the project listed as actual development folders. FAIL! The problem is that Eclipse does not recognize these folders if it includes this project as a source reference into another (my) project. Thus including this “fluid” project into my “csevents” project didn’t make Eclipse read the fluid’s include path to find sources for autocompletion.

The solution

Grmpf, this is Eclipse basics! I had to change the project’s build path and not the include path. By default the build path is just the Eclipse workspace path. I had to replace it by the actual extension’s path. Have a look at the dialogue:

Changing the Project's Build PathChose the project’s “PHP Build Path” option (I used phpunit in this example) and klick “Link Source…”. In the upcoming dialogue add the path to the actual location of the referred extension and don’t forget to select “Replace existing project source folder entry to solve nesting”. Klick “Ok”. Your Build Path will now look like this:

The new and correct Build PathNow you can remove this path from the Include Path. The Build Path is just enough. If you do it like this, your Eclipse Project will look like this afterwards:

The correct Eclipse Project ReferenceNow it’s an Eclipse Project “phpunit” with a “phpunit” build path inside (which directly links to the phpunit extension folder on my testserver) and an include path which holds the Eclipse Project itself only. This is the correct project setup to have a working project reference with autocompletion.

Feeling good…

Let’s celebrate our achievements. Just run all tests and have a look at the result:

All green, tests doneThat is a very pleasant and comforting feeling. Have a look at the stats: I made a test of the complete Domain Model and of one Repository, checking all getters, setters, the data mapper configuration and the database treatment. That are 25 tests done. And they will be done every time I just start the unit test – and give me the very good feeling that my fundamental data structure and it’s treatment is still working. That lets you concentrate on your complex business middleware job without hesitating about the over all result in the end. That is indeed a very good status to go to bed and sleep well without having thousands of open points rushing though your mind and keeping you awake.

Please note the two grey bars there, telling me that I haven’t finished implementing all tests I already planned to do. Yes, that’s an open point, but I’m aware of it, the system supports me by granting this information to me and I will go on to fill these gaps asap. Nice to have a scout like this…

Demolition Man: Deleting Data

Ok, we read data, we changed data. That’s enough for the constructive way. It’s time to destroy something. The strategy is quite simple: Create data, retrieve it from the database and destroy it, but now, of course, it is very important to check the database, if the data actually was deleted. The code is straight forward:


public function canDeleteAppointmentFromRepository() {
    $testId = 42;
    $uid = $this->setupTestData($testId);
    $appointment = $this->appointmentRepository->findByUid($uid);
    $this->appointmentRepository->remove($appointment);
    unset($appointment);
    $this->persistenceManager->persistAll();
    $appointment = $this->appointmentRepository->findByUid($uid);
    $this->assertNull($appointment, 'Appointment is not NULL after removing it from the repository');
}

Those who are very familiar to software engineering and British SF might find the answer to all questions in line #2. So you don’t need to read on, because this says it all. 🙂

One special thing to remark here is on line #7. In the lines above I just created and removed the Entity Object. Nevertheless the findByUid in line #8 would find the entity object in the database anyway, because it was marked as “deleted” in the repository’s cache, but was not physically removed from the DB. Thus we have to tell the Persistence Manager to update the database. This is done in line #7. What we need for this is an instance of the persistence manager. We get it by a dependency injection:


/**
* @var Tx_Extbase_Persistence_ManagerInterface
*/
protected $persistenceManager;

public function injectPersistenceManager(Tx_Extbase_Persistence_ManagerInterface $persistenceManager) {
    $this->persistenceManager = $persistenceManager;
}

protected function setUp() {
    parent::setUp();
    $this->testingFramework = new Tx_Phpunit_Framework('tx_csevents');
    $this->setupConfiguration();
    $this->appointmentRepository = $this->objectManager->get(Tx_Csevents_Domain_Repository_AppointmentRepository);
    $this->injectPersistenceManager($this->objectManager->get(Tx_Extbase_Persistence_Manager));
}

The first lines are just classic dependency injection: Declare a property to store the reference to the persistence manager and define a function starting with “inject” to inject the actual instance of it.

Now you can argue, if you want to do your injection within the constructor or within the setUp() method. I chose the setUp() method, because there are 3 constructors available, I have to overwrite, but only one setUp() method I can use as a centralized point for setting up all my environment and Testing Fixture. In my opinion it’s the better solution.

Change the Entity and Write it back to the Repository

When the read test was done successfully, doing the change/write test is simple:


public function canWriteChangedAppointmentToRepository() {
    $testId = 1511;
    $uid = $this->setupTestData($testId);
    $appointment = $this->appointmentRepository->findByUid($uid);
    $appointment->setInternalId('Changed internal id #' . $testId);
    $this->appointmentRepository->update($appointment);
    unset($appointment);
    $appointment = $this->appointmentRepository->findByUid($uid);
    $this->assertNotNull($appointment, 'Wasn\'t able to get back the changed data from the DB at all');
    $this->assertEquals('Changed internal id #' . $testId, $appointment->getInternalId(), 'Retrieved not the expected (changed) internal id from the DB.');

    $this->markTestIncomplete('Change more data?!');
}

Please note that in this case I don’t check, if the data is physically changed in the database. I just rely on the repository. If you want to test the database directly, just read on with the next post, which describes how to do this (using data deletion as an example).

Please also note line 12 where I mark the test as being “incomplete”. I was just struggling, if I should change more data of the entity object. This is (clearly on tests only) another way to mark some “to do-s” like I described in the “Refacoring – Avoiding the Frustration” post.

I read somewhere else in a testing blog that they use assertSame over assertEquals, because this checks the type to be the same as well. I’ll follow this recommendation and change my tests later on in this way.

DB-Test: Read

It’s time to read the first data from the database. Let’s develop a repository and retrieve the created test data from it after configuration is in place. The repository is just standard (i.e. an empty child of Tx_Extbase_Persistence_Repository.

The (empty) Repository ClassNow, if the data mapper configuration is fine and the entity object does it’s job, everything should be alright. Of course I just defined a “read from the database” testcase first:


public function canReadAppointmentFromRepository() {
    $testId = 1876;
    $uid = $this->setupTestData($testId);

    $appointmentsQueryResult = $this->appointmentRepository->findAll();
    $this->assertNotNull($appointmentsQueryResult, 'appointmentsQueryResult is not set');
    $this->assertInstanceOf(Tx_Extbase_Persistence_QueryResultInterface, $appointmentsQueryResult, 'appointmentsQueryResult is of wrong type');

    $appointment = $this->appointmentRepository->findByUid($uid);
    $this->assertNotNull($appointment);
    $this->assertInstanceOf(Tx_Csevents_Domain_Model_Appointment, $appointment);

    $this->assertEquals('Test Title #' . $testId, $appointment->getInternalId(), 'internalId does not match');
    $this->assertEquals('List as #' . $testId, $appointment->getListAs(), 'listAs does not match');
    $this->assertEquals($appointment->getEventStartTime() - 15*60, $appointment->getEventLetInTime(), 'letInTime does not match to startTime');
    $this->assertEquals($appointment->getEventStartTime() + 90*60, $appointment->getEventEndTime(), 'startTime does not match to endTime');
    $this->assertEquals(70, $appointment->getNumOfSeatsAvailable(), 'numOfSeatsAvailable does not match');
    $this->assertFalse($appointment->getIsSoldOut(), 'isSoldOut does not match');

    $this->markTestIncomplete('Repository is not completly tested yet. Link to Description is missing.');
}

So in the first place I create a DB test record and afterwards I just re-read it from the database and compare the entity object’s content with the data that was written to the database in the beginning.

You can of course use another approach by creating the database content through an newly created entity object that’s stored to the database by the repository itself. This means testing the writing (i.e. actually creating) and reading back in one step. I didn’t choose this way, because a) it would mix up two tests and b) I wouldn’t be sure if the data really comes from the database and not from an internal cache of the repository. To prevent b) I wanted to use physically existing data from the database and to ensure this without having a prepared set of test data stored in my database, I just created it on the fly using the phpunit testing framework.

Create Test Data in the Database

In order to test the physical and logical database connect, I don’t use Mockups here (we’ll do that later on), but I create database test data via an instance of the phpunit Test Framework in the setUp() method (see the prior post). Later on (while testing) I can use this data to read it from the database, change it and store it back into the database. While doing this I will check the correctness and consistency of all transferred data.

To create unique test data, I simply use a unique id that defines the data:


protected function setupTestData($number) {
    $now = time();
    $uid = $this->testingFramework->createRecord(
        'tx_csevents_appointments',
        array(
            'internal_id' => 'Test Title #' . $number,
            'listas' => 'List as #' . $number,
            'stime' => $now,
            'letin' => $now - 15*60,
            'etime' => $now + 90*60,
            'places' => 70,
            'soldout' => 0
        )
    );
    return $uid;
}

This is simple, but very efficient. You can use more complex test data generators (e.g. with more varying parameters) to achieve a better test coverage. Think of equivalent class testing for instance.

Note for those, who are not that familiar with testing strategies: If you want to understand Equivalent Class Testing just think of a field where it’s required to enter a price between 0 and 1.000,-€ (including). To test this field, you should test the following values:

  • -1,-€ shall fail
  • 0,-€ shall succeed
  • 1,-€ shall succeed
  • some random value between 1,- and 1.000,-€ shall succeed
  • 999,-€ shall succeed
  • 1.000,-€ shall succeed
  • 1.001,-€ shall fail

…and of course you can just drill this down to € Cents as well – and do tests for $ additionally. There is a lot of room for improvement, but this will tell you the basic idea of Equivalent Class Testing: Test the borders of allowed / valid values from both sides (the allowed and the discarded side) and don’t forget to test the border itself as well (e.g. “0€” including or excluding?!). This is, where most of the GUI defects occur.

setUp and tearDown the Repository Test

The Repository Test needs some specific settings (like the configuration and some test fixtures). They are setup by the – wow – setUp() method (nomen est omen) and all settings are taken back by the tearDown() method. If you overwrite these methods, don’t forget to call the parent’s methods as well in case that the parent class wants to setup and restore some test stuff as well.

I start with the phpunit test framework to setup test data in the database. Afterwards I set the extension’s configuration like described before. In the end I retrieve an instance of the repository class that is to be tested. In the tearDown() method I just do all of this in the opposite way:


protected function setUp() {
    parent::setUp();
    $this->testingFramework = new Tx_Phpunit_Framework('tx_csevents');
    $this->setupConfiguration();
    $this->appointmentRepository = $this->objectManager->get(Tx_Csevents_Domain_Repository_AppointmentRepository);
}

protected function tearDown() {
    unset($this->appointmentRepository);
    $this->restoreConfiguration();
    $this->testingFramework->cleanUp();
    unset($this->testingFramework);
    parent::tearDown();
}

Now everything is ready to create test data in the database, run tests on the repository and check, if all configuration mappings work properly.

The Repository Unit Test

‘Til now we were in a very straigt-forward situation: Setup your environment, read a book to learn the first steps, define your strategy, start programming. Now it’s time to meet the first challenges slowing you down and proving you, if you take your decisions really seriously.

The challenge

I want to go on with the Test First Approach for the Entity Repository. I have to consider that the environment is not a from-scratch-standard-framework-setup: The database tables and columns are not named like the framework would expect it to be. Thus I need a configuration to map the database names and columns to the domain model entity classes and properties.

The configuration itself is not a problem. I just define it in [extkey]/ext_typoscript_setup.txt:

config.tx_csevents {
  persistence {
    classes {
      Tx_Csevents_Domain_Model_Appointment {
        mapping {
          tableName = tx_csevents_appointments
          recordType = Tx_Csevents_Domain_Model_Appointment
          columns {
            listas.mapOnProperty = listAs
            stime.mapOnProperty = eventStartTime
            letin.mapOnProperty = eventLetInTime
            etime.mapOnProperty = eventEndTime
            places.mapOnProperty = numOfSeatsAvailable
            soldout.mapOnProperty = isSoldOut
          }
        }
      }
    }
  }
}

“tabelName” and “recordType” map the database table and the entity object. The values defined in “columns” define the mapping between the database columns and the entity object’s properties.

This mapping will work in the running extension, but if we just run a unit tests, this configuration is ignored at all. Why? It’s because the unit test is not run by the extension itself but by the extbase framework. Thus it uses the extbase configuration and not the [extkey] (=”csevents”) configuration in this case. But to run the unit tests successfully, I need to use my own extension’s configuration.

The Solution

A lot of ideas failed, but finally a semi-official way was successful. Let’s have a look at the solution:

  1. Get the configuration object
  2. Safe the current (extbase) configuration in a local property to restore it at the end
  3. Read the whole TypoScript configuration and extract the own (csevents) configuration
  4. Overwrite the current (extbase) configuration by the extracted (csevents) configuration
  5. Do the tests
  6. Restore the (extbase) configuration to ensure that all following processes use the original configuration

Steps 1-4 are done in the test’s setUp() method, step 5 is just the set of tests in the test classe and step 6 is done by the test’s tearDown() method. To make things a little easier, I just put steps 1-4 and 6 in separate methods instead of just including them directly into the setUp() and tearDown() methods. Let’s have a look:


protected function setupConfiguration() {
    $configurationManager = $this->objectManager->get(Tx_Extbase_Configuration_ConfigurationManager);
    $this->extbaseConfiguration = $configurationManager->getConfiguration(Tx_Extbase_Configuration_ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);
    $fullTypoScript = $configurationManager->getConfiguration(Tx_Extbase_Configuration_ConfigurationManagerInterface::CONFIGURATION_TYPE_FULL_TYPOSCRIPT);
    $myConfig = $fullTypoScript['config.']['tx_csevents.'];
    $configurationManager->setConfiguration($myConfig);
}

protected function restoreConfiguration() {
    $configurationManager = $this->objectManager->get(Tx_Extbase_Configuration_ConfigurationManager);
    $configurationManager->setConfiguration($this->extbaseConfiguration);
}

Line 2: get the configuration manager
Line 3: get the current framework configuration (i.e. the extbase configuration) and safe it internally
Line 4: get the full TS configuration
Line 5: extract my own extension’s configuration
Line 6: overwrite the current framework (=extbase) configuration by the extracted one

This is semi-official, because the method setConfiguration() is marked as an extbase-internal method and therefor shall not be used from the outside, but there was no other effective and efficient solution.

Line 10: get the configuration manager (again, but this is a Singleton, thus you will get the same instance again)
Line 11: overwrite the current (own) configuration by the safed (extbase) configuration

If this would be the only test, restoring the configuration wouldn’t be necessary, but there is a simple rule about testing (it’s the same like on the toilet): leave it in the same status like you found it entering. Thus we restore everything we changed before. Now it’s ensured that all following tests and activities will use the original configuration again. This is pretty and clear.

Dead ends

I tried a lot to get this running. Some notes about things that didn’t work or that may work, but I decided that it’s much too inefficient / complex to do:

  1. Tell the Configuration Manager to use your own configuration
    As noted above, the Configuration Manager reads the extbase’s configuration, because the unit test is running under extbase’s environment. Thus the extension key “extbase” is used to find the configuration. You might try to change this by the Configuration Manager’s method “getConfiguration(CONFIGURATION_TYPE_FRAMEWORK, ‘yourExtensionKey’, ‘pi1’)”, but this will only work for the values you retrieve from this call directly. It won’t work for the repository backend, which still gets it’s information from getConfiguration(…) using “extbase” as extension key. Dead end #1 reached.
  2. Try to setup a (simulated) frontend
    You might think “If the extension works like real life (i.e. in the frontend), it will also use it’s own configuration. So let’s simulate the frontend and the TS configuration will be alright.” 1. It simply doesn’t work. The test is still run in extbase’s environment. 2. It’s bad style to try to simulate a frontend while testing in the backend just to get the right configuration. It’s about testing the domain model and the repository. There is no frontend involved in the test at all. To use a frontend just for configuration purposes is the wrong way. Dead end #2 reached.
  3. Configure extbase in the same way like you configure your own extension
    Idea #3: Copy the TypoScript configuration via “<” or “<=” notation into the extbase configuration. If you do so, extbase uses simply it’s own configuration and this configuration is the same like that one used by your extension in real life. Hmmm. You must do copy&paste and/or you must use a lot of “<” references in TypoScript to include (merge) your extension’s configuration into the existing extbase configuration. Sooner or later you will forget parts of your configuration and a) struggle to find the bug or (worse) b) successfully test with a configuration that does not represent the actual truth. Awful end #3 reached. (Of course you can do it like this, thus it’s not a “dead” end actually, but it’s a very bad style, because of the potential inconsistency and changing another extension’s configuration.)
  4. Maybe that there are much more dead ends, but these were the once, I tried or (at least) though of (actually I never tried to do the 3rd one, because of it’s weaknesses).

Check in

Before we proceed to the repository, let’s just recap about version control and combine this information with the Test First Approach. There is a general principle, a de-facto or best practice rule, how to deal with the version control system. It simply says: “Check in, if all tests are successful.”

If you think this over there are some consequences coming out of this general statement:

  1. Don’t check in, if / as long tests fail.
  2. Try to check in as soon as possible. To achieve this, don’t reduce the tests, but improve the application.
  3. If you write new tests, you might even check the empty test class in into the version control system. Especially if you work in a team, you might use this approach to share just the information that this test (and it’s folder structure) is present and who is working on it.
  4. (or better 3b) I recommend (esp. in teams) to use the check-in-as-soon-as-possible-approach while adding single test cases to a test file (and of course adding the corresponding functionality to the application).

Complete the Domain Model

Now it’s simple to go on and add test cases for the remaining domain model entities. And afterwards just make it “all green” again by implementing the entity objects. But… I remember a recommendation in the book I read. It says: Use a lightweight controller and put most of the business functionality into the model. So if you just think of simple getters and setters, step back, wait a little moment and rethink. If there is any additional functionality inside the domain model, anything that defines / organizes / changes the internal structure of the entities, add a test and implement it afterwards.

This was just “finger exercising” to get familiar with the basic folder structure and the Test First Development. Now it’s getting a little bit more complicated. I start to implement the test for the entity’s repository and I have to consider that the actual domain model does not fit to the database table naming: The table names and the column names are different because of the former table definition, I don’t want to change right now. That is the point where the decision took place to write this blog, because now we extend the basic information given by the book. We have to combine a specific extension setting for the database mappings with the unit tests. Here is where the challenges actually start.