Category Archives: IDE and Tools

Running Typo3 4.5.xx Selenium testcases nowadays

If you want to (or have to) use Typo3 4.5.xx today, you will run in certain trouble trying to get the Selenium tests working, because the related phpunit extension requires an older version of Selenium, which itself requires some older browser versions to run. Here are the details and the solution:

  • Typo3 4.5.xx (actually tested with .32 and .37) comes with phpunit extension 3.7.22
  • phpunit extension 3.7.22 requires Selenium Server 2.32.0 and is not compatible to later versions like 2.43.1
  • Selenium Server 2.32.0 requires Firefox 20.0.1 and is not compatible to later versions like 33.0

Solution:

  • For the Selenium server it’s quite simple: Download the jar of version 2.32.0 from the official repository and run it using java. If you want to run two different Selenium servers on the same machine in parallel, you just have to set up a different port for the second one to listen to (use the -port nnnn option to do so).
  • For the Browser it’s a little more tricky, because (I assume) you want to use the latest browser version for your personal web surfing (and for your Typo3 development and testing environment) while you have to use the former version 20.0.1 for the Selenium WebDriver. I downloaded 20.0.1 from the official repository and installed it to a directory, that’s not part of my PATH declaration (in my case /opt/firefox-20.0.1/).
  • After installation you have to change your test script to make it tell the Selenium WebDriver to use the local Firefox installation. Have a look at the following code sample:
protected function setUp() {
    parent::setUp();
    $this->setDesiredCapabilities(array('firefox_binary' => '/opt/firefox-20.0.1/firefox'));
    $this->setBrowser('firefox');
    $this->setBrowserUrl('http://your.url/');
    // other stuff to set up
}

In line 3 you find the definition for the WebDriver that is telling, which Firefox binary to use. Please note that this refers to the binary directly and not to the directory only.

I wrote my own Selenium Base Test class to do stuff like this in order to not copy and paste this part for each and every test class. I strongly recommend to do so, if you want to do the basic setUp and tearDown jobs centralised.

Project Configuration Issue solved

I wrote about an issue on my Eclipse Project Setup: I wasn’t able to setup the autocomplete functionality correctly by including extbase, fluid and phpunit as Eclipse projects. Instead I used a workaround by adding the paths to these extensions to my own project’s build path. This isn’t very comfortable, because the folders keep popping up in my project folder while I am concentrating on my own development. Now I finally solved this issue.

What was wrong?

I added 3 empty projects “extbase”, “fluid” and “phpunit” to my Eclipse workspace. This was just fine. To include the actual source code of the extension, I also added the extension directory to the project’s Include Path. Have a look at the screenshot:The three additional Eclipse ProjectsYou see that e.g. the Eclipse project “fluid” hasn’t got any source path, but has got “/var/www/typo3[…]/fluid” as an additional include path. This is fine for the “fluid” project, because it just needs to include these sources. I won’t change them at all, thus I don’t need to have them inside the project listed as actual development folders. FAIL! The problem is that Eclipse does not recognize these folders if it includes this project as a source reference into another (my) project. Thus including this “fluid” project into my “csevents” project didn’t make Eclipse read the fluid’s include path to find sources for autocompletion.

The solution

Grmpf, this is Eclipse basics! I had to change the project’s build path and not the include path. By default the build path is just the Eclipse workspace path. I had to replace it by the actual extension’s path. Have a look at the dialogue:

Changing the Project's Build PathChose the project’s “PHP Build Path” option (I used phpunit in this example) and klick “Link Source…”. In the upcoming dialogue add the path to the actual location of the referred extension and don’t forget to select “Replace existing project source folder entry to solve nesting”. Klick “Ok”. Your Build Path will now look like this:

The new and correct Build PathNow you can remove this path from the Include Path. The Build Path is just enough. If you do it like this, your Eclipse Project will look like this afterwards:

The correct Eclipse Project ReferenceNow it’s an Eclipse Project “phpunit” with a “phpunit” build path inside (which directly links to the phpunit extension folder on my testserver) and an include path which holds the Eclipse Project itself only. This is the correct project setup to have a working project reference with autocompletion.

auto-ant-ing

Refer to the previous post to find out about the content of the ant build.xml file. Now we need to tell Eclipse to use this file for “building” (i.e. copying). The only thing Eclipse needs to know, are the settings for ant, the build file and the base directory. Have a look at the screenshot to find out about the details:

  • Use Project > Properties > Builders > New… to setup a new (ant) builder.
  • Just fill in / check the fields in the “Main” tab:
    “Buildfile” must point to the build.xml file. You can use “workspace_loc:” to refer to your local workspace directory. Thus “Buildfiled” is set to “${workspace_loc:/csevents/build.xml}” in my case.
  • Do the same for the “Base Directory” as well (in my case “${workspace_loc:/csevents}”). This variable can be used inside the ant script as the base directory “${basedir}”. It’s the best way to just refer to this variable/property inside the ant script and to define it outside of ant (in the Project Properties) in order to let other developers with other base dirs use the same script just with another setting in their Project Properties.

Bildschirmfoto vom 2014-02-08 00:23:32

After setting up your ant build file together with the ant properties like this, you can simply use Project > Build all (Ctrl+B) to copy your current development version directly from the workspace to the Typo3 extension directory. Fast, simple, efficient. (In fact I am lazy! ;-))

Please note (#1) that Ant just copies the csevents folder of my Eclipse workspace. This is not the project itself completely, but only the csevents subfolder inside this project. Thus all additional Elicpse Project stuff (other referred source folders, the build.xml itself, Subversion directories) are not copied at all. This reflects the official deployment in the best way.

Please note (#2) that Ant in the first step completely deletes the destination directory within the Typo3 installation. Why is that? Just think of the situation that you remove a file (on purpose or accidentally) from your Eclipse Project (e.g. “ext_emconf.php” – accidentally of course). If we now just copy the development stuff to the Typo3 installation, the test won’t fail, because the file is still there, being present from former deployments. Thus the current test installation does not reflect the current development version with (i.e. without) the missing file. To avoid this, we delete the complete installation inside the Typo3 folder and replace it by the current development version. Then we are sure to have a 1:1 representation of our dev-version.

To automatically “build” my extension, I use Ant to do this job for me, because it’s a boring every-time-the-same-steps-to-do job, that a technical automation can do better, faster and more reliable than me. In this case it’s a clean-up/delete-and-copy job. Here is the build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project name="csevents" default="deployFromDev" basedir=".">
    <description>
        Buildfile to deploy the development version of the Typo3
        extension csevents to the local testing system.
    </description>
    <property name="ExtensionName" value="csevents"/>
    <property name="SourceDirectory" location="${basedir}/${ExtensionName}"/>
    <property name="TargetDirectory" location="${basedir}/../.sandbox/${ExtensionName}"/>

    <target name="deployFromDev" depends="clean">
        <copy todir="${TargetDirectory}">
            <fileset dir="${SourceDirectory}"/>
        </copy>
    </target>

    <target name="clean" description="Clean up the local testing directory">
        <delete dir="${TargetDirectory}"/>
    </target>
</project>

Lines 7-9 just define all properties, i.e. folders and files to handle.

In line 11 you find the copy-job which calls “clean” first. “clean” is located in line 17 simply deleting the complete target directory.

The only thing you have to consider: The developer has to have write-access to the ${TargetDirectory}’s parent directory in order to delete and re-create (copy) the target directory. If you use a ‘.sandbox’ directory (or something similar) in your on ‘home’ or ‘workspace’ directory (like I described in “I am lazy (The Server Architecture)“), this is obviously done out of the box.

If you want to learn more about how to add this to the Eclipse IDE and make it run as comfortable as possible, please refer to the next post auto-ant-ing.

I am lazy (The Server Architecture)

This is about my server setup and the way how the IDE, test environment and version control (Subversion) work together. Image first, explanation afterwards…

ServerArchitectureLet’s have a look at the connections:

  1. The webserver Apache links the URL of my localhost to the ‘cms’ directory. (Actually it is “http://localhost/cms/” in my case, because I use some other stuff on the root level.
  2. The ‘cms’ subdirectory of my local web presence is a soft link to a dummy installation of Typo3 in the same directory. The link depends on the Typo3 version I just use. Thus I can switch from e.g. 4.5.32 (dotted line) to 4.5.37 (solid line) just by redefining the soft link.
  3. The Typo3 dummy installation contains a ‘typo3_src’ soft link to the Typo3 core system, which is located at ‘/opt/typo3_src_4.5.xx’ (where ‘xx’ is ’32’ or ’37’ in this case). This is recommended by the Typo3 installation guide in order to split the Typo3 installation into the local system for one web presence and the core. It’s not 100% necessary in this configuration, but it is more flexible, if I want to use the core for some other sites on this server as well.
  4. All 4.5.xx versions of Typo3 use the same DB-connection to a local MySQL database. As long as the DB structure does not change between patches, this is valid. Thus the content of the web site stays the same, no matter if I use the 4.5.32 or 4.5.37 link at 2.
  5. No matter if I use the 4.5.32 or 4.5.37 installation, I just want to make sure that my developed extension is exactly the same in both installations. And in order to update the extension with the latest developed version as easy as possible, I want to have full access rights (esp. write) to the extension directory. Thus I use a central ‘.sandbox’ directory inside my Eclipse workspace, which holds the last deployment. Both Typo3 installations link the typo3conf/ext/csevents directory to this sandbox directory. Now all access rights inside the Typo3 installation stay on wwwrun level and I have full access on the sandbox directory, which is located inside my home directory (underneath ‘workspace’). Eclipse allows to generate all subdirectories despite of ‘.metadata’, thus this is a valid directory.
  6. Inside the Eclipse workspace there is a project directory ‘csevents’ holding the complete Eclipse project with all meta data, additional project data and resources. The ‘.sandbox’ directory stores the deployed version of the extension only. Thus there is no meta data, no additional files etc. The ‘deployment’ is a simple copy job performed by an Ant script. We will have a look at this ant script in the next post. This structure and the deployment process separate the development version from the current testing version, thus I can test on the last deployed version while I’m already changing the code in the development environment.
  7. Finally there is a Subversion connection managed by Eclipse to store all project changes directly to the version control system. My policy is: Check in exactly when (and only when) all tests are successful.

Consequences:

  1. Testing and development can be done separately. The Typo3 extension directory (i.e. the ‘.sandbox’) gets the source code only. There won’t be any SVN files, Eclipse project files or any other administrative development stuff. Thus this is a copy of the “real world”. Good.
  2. It’s well prepared to use it for “Deployment from SVN” later on, if I need to switch back to an older version of the extension in the Typo3 environment, but don’t want to change my current development environment on Eclipse. This would be a direct copy from Subversion to the ‘.sandbox’ directory bypassing Eclipse’s workspace development directory. Very Good.
  3. I can use multiple Typo3 versions for testing (in this case 4.5.32 and 4.5.37). I can access them directly (‘http://localhost/dummy-4.5.32/’) or the current one simply by using the ‘http://localhost/cms/’ shortcut.
  4. I don’t have to build up the content for each installation separatly, because all installations use the same DB connection (as long as table structures are not changed).

Step 6 – the “deployment” from the Eclipse workspace to the sandbox – is the most important one, because it’s used most frequently while developing / testing. Thus it is very important to make this step reliable, fast and easy to use. Because it’s just a lightweight “build” process, let’s use the one-and-only-build-tool-for-Eclipse. That is ant. Yep, I added a very simple ant build script to the project. The only thing it does, is to copy all relevant files from my Eclipse workspace to the ‘.sandbox’ folder. Because this is now the default build script, I can just use Eclipse’ build hotkey (Ctrl+B) to copy all files to the test environment. Simple, fast, easy to remember, efficient. Good! (And I use this instead of a ‘bash cp’, because I am lazy!)

You could ask: Is it the correct way, just to copy the files into the Typo3 extension folder? Isn’t it better to uninstall and re-install the complete extension inside Typo3 ExtManager? The answer is: Yes and no.
If you do severe changes to your extension (adding DB tables, columns etc.), it’s always recommended to use the official extension installation process via the Typo3 backend. I’m sure, somebody can build an ant script for this as well, but I haven’t done this ’til now, because there was no reason to do so.
As long as you do just some simple PHP programming inside your source code, a simple copy process is just enough. You don’t even need to reset the Typo3 caches. While you are working on a new feature or on reengineering (like I do here), you spent much more than 90% of your time on writing just PHP code. Therefor this is a very simple and efficient solution.

So please have a look at the next post to find out about the ant script I use to do the PHP “build” process (just by copying files).

Setting up the Eclipse Project

Bildschirmfoto vom 2014-02-07 23:11:23

Setting up a standard PHP project in Eclipse is a simple process described more than once in the web (e.g. on youtube), but I did some additional work to get a proper autocomplete function for the whole framework. Rau, Kurfürst and Helmich recommended in their book to add the extbase and fluid sources as separate Eclipse projects in order to retrieve their source code and inline documentation. I did so for extbase, fluid and phpunit (i.e. the phpunit extension of typo3 and not phpunit itself).

I faced two challenges:

  1. The autocomplete didn’t work at all in the beginning. Root cause: The upgrade from Eclipse to the newest version corrupted the internal file format for the autocomplete functionality. There are lots of comments regarding this issue. Have a look at http://stackoverflow.com/questions/908489/eclipse-java-code-completion-not-working for instance to find out about the details. In my case it was just the solution to delete the files in workspace/.metadata/.plugins/org.eclipse.jdt.core like it is recommended in one of the comments. Done, good. You only need to know…
  2. Why ever that happens, but my Eclipse ignores the code of related projects completely for the autocompletion. Thus I decided to include the source directories directly into the project via Project > Properties > PHP Build Path > Add Folder…
    The result: All related paths appear in my project directly, thus they spam my project view a little, but it’s feasable. I will live with this situation until I find another solution by chance (or if you give me the solution down in the comments). 😉

Please have a look at the screenshot and the PHP Exlporer’s folder structure. I named the Eclipse Project “csevents” (root level of the folder structure), that’s obviously the name of the Typo3 extension itself, but it is NOT the content of the extension, it’s the Eclipse Project instead. You find another folder underneath the root “csevents”, that’s named “csevents” as well. This IS the actual Typo3 extension directory. You will find the old fashioned subfolders “pi1” and “res” here for instance. On the top level the Eclipse Project specific folders and files can be found that are not part of the actual Typo3 extension. Here you find the referred Typo3 source folders of other extensions (extbase, fluid, phpunit), the build.xml for ant and all other stuff that won’t make it into the official release/deployment.

Finally I finished with an Eclipse Project with full autocomplete functionality and reference to the inline documentation of all relevant extensions. That will make life much easier. Let’s see…

First Steps

Setting up the proper IDE, of course, was the first thing to do. It’s not that relevant for the extension itself, but I spent the first hours on migrating the old CVS to SVN, upgrading Bugzilla, Eclipse and all Plugins (like PDT and mylyn). Unfortunatly I faced a problem on the link between mylyn and Bugzilla that was also described in the Eclipse bug 422353. Currently this bug seems to be fixed, but I haven’t tried the solution yet. This will be one of the next steps, because I’m much more efficient, if I can use mylyn to organize my work. 😉