Tellurium: A Better Functional Test

Lets face it, Selenium has been around for years, it has been a choice tool for functional testing for many because it is free, has good support, and it works. Sure, it hasnt seen many updates as some would have liked, for example it didnt have firefox 3 support for a very long time. No big deal right? The latest beta supports firefox, and every one knows how to use it, the feature set has remained pretty much unchanged for years.

Why would you want to break the mold of functional testing and try something new? Well, until very recently, there hasn't been a compelling enough reason to do this. Fortunately, now there is: Tellurium.

Tellurium is another open source functional testing framework that picks up where Selenium has left off, and packs many new great features. The ideas themselves aren't exactly new: caching, UI modules, DSLs to describe the UI, Data Driven Testing, Using a JS framework instead of xpath, just to name a few. These ideas have been around for ages, but never before have they all been implemented in a mature functional testing framework.

This is the reason you should try Tellurium: new features, and the speed of their implementation.
I didnt know what i was missing with selenium until I implemented a few test cases using Tellurium's UI mapping Groovy DSL. This was a year ago, and I took a gamble in a fairly young testing framework based on Selenium server. Let me tell you, it has paid off. Not only are my tests written in a more Object Oriented way, but since upgrading to Tellurium 0.6.0 my tests run faster.

Lets look at some benchmarks.

Right now Tellurium is built on top of Selenium server, it translates its calls into one or more Selenium calls to do it job. Selenium uses xpath as its primary selector, while Tellurium also supports using the sizzle selector engine, in its jQuery selector.
The following graph shows the sizzle engine speed versus the default xpath engine using by Selenium. The blue is the xpath execution speed, and the red and yellow is jquery without caching and with caching, respectively.

As you can see, the speed up from using siggle is about 30%. I should add that this benchmark was taken on IE6, the slowest javascript platform that is still supported by most sites.

This speed up is going to allow you to have more time to do other things, or write better test cases. It can greatly speed up continuus intergration tests.

For more information about caching see this page. For more information about jquery selector see this page.


Another big performance booster in tellurium 0.6.0 is the bulk data retrival that is possible when using jquery. Every one who has used jquery knows about the convenience of $("table td").text(). Tellurium now performs such calls if you ask it for the text of of all cells in the table, whereas selenium will get you that data cell by cell with overhead for every call.

The above graph shows the awesome speedup that is possible with bulk data retrival. Notice the rightmost group titled "Bulk Data". The yellow bar representing Tellurium performance is so small that it seems like it may be missing. In this case, the speedup from gathering all the data in the javascript, and transporting it in one call gives you an incredible speed up of a few 1000%.


Upcoming performance Improvements in 0.7.0

One nice thing about Tellurium is the speed of development, and new features that make it into the build.

I have been keeping an eye on a performance improvement called "Control Group". The idea behind control group is to group together Selenium server calls into groups, execute them, and return multiple data sets. Since the biggest overhead comes from interacting with the server, this will be a big improvement over the current situation.

So, try tellurium out, you wont be disappointed.