Wow, nearly a month without posting. I haven't really gotten any good excuses, just lack of stuff to write about, I guess. But for the last week I've been struggling and doing something I find really interesting and useful: Web-testing.
Regular readers will know about my love for Selenium. I still believe that Selenium is the way to go about web-testing.
So, the web application in a project was approaching web-test-worthiness quickly (well, actually it was about a month or two past the point where we should've started doing web-testing). Mind that we were already doing continous integration on the project, so it was only for the reason of not having core-developers breaking any of the stuff being out in the web.
Let me clarify a bit on that last point: Modern web frameworks expose core domain objects way out into the web (like the User.lastName property here in Matt Raible's example).
When core developers refactor Java classes - XML files, property files, JSP files and JavaScript files are not brought along in the equation (especially not in Eclipse).
Breakage caused by these changes will most likely not be discovered in your unit tests. You need web-tests.
There are a number of other good reasons for doing web-testing, like the ever increasing amount of JavaScript in your webapp, and testing the web across different browsers. So web-testing is a Good Thing.
I'm not going to go on about the different strategies for getting continous web-testing implemented. The elements I were after were:
1) Getting Maven to test with Selenium scripts (no, not JUnit tests exported from Selenium IDE)
Now normally, this isn't too hard. Alot of projects out there do web-testing by storing Selenium tests as Java-code - JUnit tests. But the trick here is to get Maven run tests based on tests in the Selenese format - pure html tables with web-browser instructions (they look a bit like Fit-tables).
There is an ant-task for doing this, which in later turns has been converted to a Maven plugin goal: Selenium Maven Plugin's selenese goal. It was originally a seperate plugin called Mavenium, but now the project seems to have been absorbed into the other one.
2) Getting Continuum to run these tests.
Once Maven is running the tests properly, it shouldn't be a problem to get Continuum running these. BUT there's the issue of how the web app should be deployed and started for the Selenium plugin to have something to run on. Cargo was the obvious tool for doing this job, but there were some bumps on the road to getting it working.
3) Getting the tests to run headless
As shown here, with the headless option in the selenium plugin. Anyhow, I ended up not using that plugin-goal, instead just starting Xvfb manually on the Linux box and setting the DISPLAY variable to something like 127.0.0.1:1. Then there was a whole bunch of trouble getting Firefox 2.0 in on Fedora Core 6 and so on.. Yech.
I believe these three requirements have not been met together on many other projects, and this combination has not been properly documented anywhere else. This is perhaps because only recently have the plugins involved matured to such a degree that this process is possible at all. I did try achieving the same goals before the summer, but ran into all sorts of problems with Firefox 2.0 instances not being started correctly and problems using the maven-cargo-plugin.
The closest I ever found was this how-to. But still the example uses Selenium's JUnit test (we want HTML-tests, remember?).
So in the next post I'll put together an example that actually does the whole thing my way. I did run into a couple of problems with cargo that I have not been able to solve (ended up hacking the plugins somewhat), but still the example should work on a general basis. Too tired to make it right now though.
Regular readers will know about my love for Selenium. I still believe that Selenium is the way to go about web-testing.
So, the web application in a project was approaching web-test-worthiness quickly (well, actually it was about a month or two past the point where we should've started doing web-testing). Mind that we were already doing continous integration on the project, so it was only for the reason of not having core-developers breaking any of the stuff being out in the web.
Let me clarify a bit on that last point: Modern web frameworks expose core domain objects way out into the web (like the User.lastName property here in Matt Raible's example).
When core developers refactor Java classes - XML files, property files, JSP files and JavaScript files are not brought along in the equation (especially not in Eclipse).
Breakage caused by these changes will most likely not be discovered in your unit tests. You need web-tests.
There are a number of other good reasons for doing web-testing, like the ever increasing amount of JavaScript in your webapp, and testing the web across different browsers. So web-testing is a Good Thing.
I'm not going to go on about the different strategies for getting continous web-testing implemented. The elements I were after were:
1) Getting Maven to test with Selenium scripts (no, not JUnit tests exported from Selenium IDE)
Now normally, this isn't too hard. Alot of projects out there do web-testing by storing Selenium tests as Java-code - JUnit tests. But the trick here is to get Maven run tests based on tests in the Selenese format - pure html tables with web-browser instructions (they look a bit like Fit-tables).
There is an ant-task for doing this, which in later turns has been converted to a Maven plugin goal: Selenium Maven Plugin's selenese goal. It was originally a seperate plugin called Mavenium, but now the project seems to have been absorbed into the other one.
2) Getting Continuum to run these tests.
Once Maven is running the tests properly, it shouldn't be a problem to get Continuum running these. BUT there's the issue of how the web app should be deployed and started for the Selenium plugin to have something to run on. Cargo was the obvious tool for doing this job, but there were some bumps on the road to getting it working.
3) Getting the tests to run headless
As shown here, with the headless option in the selenium plugin. Anyhow, I ended up not using that plugin-goal, instead just starting Xvfb manually on the Linux box and setting the DISPLAY variable to something like 127.0.0.1:1. Then there was a whole bunch of trouble getting Firefox 2.0 in on Fedora Core 6 and so on.. Yech.
I believe these three requirements have not been met together on many other projects, and this combination has not been properly documented anywhere else. This is perhaps because only recently have the plugins involved matured to such a degree that this process is possible at all. I did try achieving the same goals before the summer, but ran into all sorts of problems with Firefox 2.0 instances not being started correctly and problems using the maven-cargo-plugin.
The closest I ever found was this how-to. But still the example uses Selenium's JUnit test (we want HTML-tests, remember?).
So in the next post I'll put together an example that actually does the whole thing my way. I did run into a couple of problems with cargo that I have not been able to solve (ended up hacking the plugins somewhat), but still the example should work on a general basis. Too tired to make it right now though.
Comments
Post a Comment