Hudson is sweet, now the build can tweet

I recently set up a new Hudson build so the Equinox team could test their changes in a branch instead of releasing everything to HEAD. Hudson, like Eclipse, has a rich variety of plugins that can expand the functionality of your build. Hudson has a twitter plugin and thus Eclipse (test) builds tweet here

http://twitter.com/eclipsebuilds

Right now, there are only a few messages about the test builds I started, and stopped when I saw that build was proceeding as normal. In any case, it's pretty cool.

Hudson also has plugins to use additional slave machines on the network to run the build on a faster machine if the current build machine is too busy. It also has plugins to provision Amazon EC2 images and run the build on a slave in the cloud. We currently run about 54,000 JUnit test per platform multipled by five test machines during each build. We run tests on Windows, Mac and Linux machines. But the tests take a long time to complete simply because of the number of tests we run and the lack of additional hardware to run more tests in parallel. It would be fantastic to be able to run the tests on parallel on many machines and finish in fraction of the time they take today. That would allow us to reduce the number of breaking build issues. But of course, this will take significant testing to implement. Sounds like fun!

References
Run JUnit tests in parallel
Hudson build for the p2 branch

Read more...

Test bundles switch from runnable to repo

Earlier this week, I changed the build so that the test bundles are provided in a zipped p2 repository format. A p2 repository looks something like this

...
artifacts.jar
content.jar
features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV.jar
plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735.jar
plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735.jar
binary/
...

The build will install the appropriate test bundles in the eclipse install being tested using the p2 director. Previously, our test bundles were assembled in the runnable format like this

...
eclipse/plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735/...
eclipse/plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735/..
eclipse/features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV
...

and unzipped into the dropins folder. However, the dropins folder is really for legacy purposes. Disadvantages of using the dropins folder include

• It's to help those applications that expect that dropping a bunch of bundles into an install will work (!).
  
• All the bundles in the dropins folder are treated as optional bundles.
• You can't update the bundles in the dropins folder by using the UI, you are responsible for provisioning them.
  

The content of the org.eclipse.test framework bundle hasn't changed. However, if you're rerunning the Eclipse and Equinox projects' tests JUnit tests against your product, you'll need to install them into the eclipse your are testing using the p2 director. Unzipping the our JUnit bundles into the dropins folder won't work. Alternatively, you could use the reporunnable task to transform the repository into the old runnable format.

Why make this change? These days, I would expect that most users are installing bundles via a repository such as Galileo or one offered by a product team. Unzipping files is so Eclipse 1.0. So this will allow our tests to replicate the environment that reflects the reality of our users. As well, as I mentioned in my earlier post, we really need to run our tests in parallel on more machines to speed up the build process. Having the test bundles available in a repository, eventually in a shared location, is a step toward that goal.


References

Test bundles should be packaged as a repo

Supported dropins formats

repo2runnable task

Note: The Eclipse and Equinox projects provide their JUnit test bundles in the eclipse-Automated-Tests-${buildId}.zip in the eclipse-testing/eclipse-junit-tests-${buildId}.zip file that is available with every build.

Read more...