Bundles and Oranges: Dare to Compare

Equinox p2 provides several tools to manage repositories.

Image by Oliver Delgado


The mirroring task is used extensively in the Eclipse and RT Equinox build.  Mirroring a p2 repository allows you to copy the metadata and artifacts to a new location. You can mirror your entire repository or a subset of IUs to a new location. We call mirroring a subset of IUs slicing.

Image by Safari11

The granularity of your slicing operation is dependent on the IUs you specify.

Image by shin0


We run the mirroring task with a comparator.  This allows us to compare the bundles that have just been built with the bundles that already exist from other builds in the composite repository. We want to guarantee that the bundles with the same unique identifier and version have the same binary content.   Do you know which of or your bundles is not like the other?

Image by Stephanie Berghaeuser

A different compiler with the same source could produce different byte code. Using a new builder can change the content of your bundles, for instance if you enable source references.

We use a comparator with a baseline to compare the bundles with the same name and id available in our repositories with the ones that were just created in the current build.  Newer bundles with the same id and version are discarded.   This process guarantees that if a user installs a build from a repository or a zip, they will have the same bundles in their install. Otherwise, you risk inconsistent bundles for your users.  Not good.

We call the p2.mirror task like this:

We are mirroring from the source (unzipped repository of our build time feature containing all features and plugins in the build) to the child repository location.  The IgnoreErrors flag is set to true so the mirroring operation doesn't fail if there are differences. The org.eclipse.equinox.p2.repository.tools.jar.comparator is used to compare the bundles between the two locations and output the differences to a log.  The repository location or baseline is the existing composite repository with content from older builds.   The  comparatorLog is parsed by a JUnit test which generates a failure if the log indicates differences. You can also exclude certain bundles from being compared, as you can see in the exclude stanza.

Other information on p2 repository tools and the comparator
Bug 302283 - add ability to exclude bundles when running comparator with mirror task
Bug 312962 - Exclude doc bundles from comparator
Implementing composite repositories in your build
Slicing and dicing the p2 way
Andrew's comparator slides from EclipseCon 2009


Hey Kim, what's with all the oranges?  I'm very fortunate that I have the opportunity to run my first marathon with my friends on a beautiful course in Ottawa and Gatineau this Sunday.  It will be a challenge. After the race, there will be sweet orange slices in the recovery area for the 39,000 people running in Ottawa this weekend.

Read more...

Webmaster kudos

Sometimes, while working in open source, we don't take time to say thanks. We are so intent on fixing bugs and moving on to the next one. 

I'd like to take the opportunity to say thanks to the webmasters for some work they did this weekend that substantially improved CVS performance. From Denis

"We moved the download.eclipse.org and archive.eclipse.org mounts to the other NFS server (the one that serves pserver and other less important stuff), and that seems to have made an enormous difference".

Our build now takes 40-50 minutes less because of this change. This makes our team much more productive.  And a faster build makes me a happy release engineer. 

Image © psd, http://www.flickr.com/photos/psd/2086641/ licensed under Creative Commons by-nc-sa 2.0

Thank you Matt and Denis for all your hard work improving the eclipse.org infrastructure.

Read more...

Eclipse Top Ten: #10 You can be the one to make a difference

Sometimes, I'm disillusioned with democracy. It's messy and difficult.  Don't get me wrong, I wouldn't want to live in a dictatorship.  But does my vote really matter?  I don't know.  When I email my Member of Parliament to express my opposition to a bill, do they really value my opinion? Judging by the generic form letter that I receive in return, I am deeply skeptical. 

The Eclipse community is small which means that as an individual, you can be the one to initiate change.
At Eclipse, my vote counts.
You bug fixes can help people ship new products, make their work day more productive or  even monitor robots on Mars.
That's really amazing.
People appreciate that and it's great to hear positive feedback on your work on an ongoing basis.
You can be the one to make a difference, positive or negative.
You can be the one to find a critical bug before release day
Or after release day.
Nothing is better that working with people who care about what they do. Hands down.
Just look at all the people who have won the Eclipse Community awards this year. They made a difference.
And that's why I like working in the Eclipse open source community. 
Thanks to all of you for making this a great experience over the years.

Previous Top Ten posts


1, 2, 3, 4, 5, 6,7, 8,9

The entire presentation is now available on slideshare.

Eclipse Top Ten: Important lessons I've learned working on Eclipse

Read more...

Eclipse Top Ten: #9 Teamwork

Image © Alan Chia,  http://www.flickr.com/photos/seven13avenue/2768996935/ licensed under Creative Commons by-nc-sa 2.0

Everyone has to care.
If the webmasters didn’t take care of the servers, we wouldn’t be able to release or build software
If the Eclipse.org IP team didn’t do their job we couldn’t use third party libraries or contributed code
If companies didn’t release products based on Eclipse, there wouldn’t be anyone to pay the bills :-)
If the community wasn’t there to open bugs, work through problems or write articles our software wouldn’t be what it is today.
Last, but not least, if the committers weren’t there, getting up each day and caring about the future of Eclipse, we wouldn’t have much progress.
We ship together as a team.

Previous Top Ten posts

1, 2, 3, 4, 5, 6,7, 8

Read more...