Ada Lovelace Day: One day late

As Gorkem mentioned, yesterday was Ada Lovelace day. Thank you Gorkem for mentioning me among women in the Eclipse community. There are a few more I'd like to thank for their excellent work.

• Kim Horne is uber talented Eclipse UI developer who recently moved on to work at RIM.
• Sonia Dimitrov is an amazing release engineer who now works on Jazz.net by day, while playing violin by night.
  
• Carolyn MacLeod is a very accomplished SWT committer who is an authority on accessibility.
• Susan McCourt is responsible for all the work with the P2 user ui, and in the past provided her many talents to the Platform UI team. The p2 user ui is substantially changed in 3.5 as described in today's talk.

Unfortunately, I can't think of many more women in the eclipse community, although I'm sure there are quite a few more :-). (Please add more in the comments or in your blog). Only about 2% the people in most open source communities are women. This makes me sad. Why does gender matter? There have been many discussions at eclipse with respect to diversity in terms of company representation among committers. Diversity in terms of gender is important too. Diversity lets us see things from an different viewpoints and that's healthy for our community.

In Canada, over 60% of new medical school graduates are women, so obviously it's not an issue with women not having the prerequisites to get into engineering and CS. They simply don't choose to enroll in these degrees. Some of the research regarding this issue can be found here.

So what are steps that we can take as a community to ensure that the committer class of 2020 has more women?

• Talk to your the kids in our your life about working in open source and how much you enjoy it. Computer science is often perceived as a solitary pursuit, when we all know the truth is that the work is very collaborative. Emphasize the community aspects, and how we all work together to solve problems.
• Be a mentor. There are many kids in the community who don't have a exposure to a role model who works in the high tech sector. Therefore, they don't know anything about the industry other than the stereotypes. There are many opportunities to volunteer at local schools for career day, helping kids learn to read, or develop math skills. Some companies such as IBM sponsor a camp each year to expose middle school girls to science and technology. Volunteering at this camp allowed me to get paid to build Lego Mindstorms robots for a day. I highly recommend it!
• Throw away gender based stereotypes. Don't assume your 12 year old niece won't want to learn how to compile a kernel, solder a board, or read a book about web design. You have to start them young :-) Don't call young girl who is interested in engineering or science "geeky", "a tomboy" or "weird". It's normal.
  
  

The purpose of Ada Lovelace day was to for female bloggers to write about a women in technology who inspired them. Here's a list of some other people that have inspired me even though I haven't met them in person.

• Valerie Aurora was one of the first female Linux kernel committers. She is an expert on Linux file systems and writes on women in open source and building communities.
  
• Kathy Sierra is the co-creator of the O'Reilly's excellent Head First line of books. She is also a well known speaker, instructor and blogger who has written on how create users who are passionate about your products.
  
• Scienceblogs is an aggregator for scientists who blog about new research in geology, climatology, medicine, math, physics and many other new frontiers of science. Many of the female bloggers write about how they are breaking down gender stereotypes in the ivory towers of academia. Very interesting!

Read more...

Slicing and dicing the p2 way

We've traditionally used the packager to craft the features that are available on our download page from the signed bundles in our master feature. However, the packager isn't very efficient because it unzips all of the input feature to the packaging process, regardless of the bundles that are actually needed. Also the packager can be confusing for new users to configure. The packager also doesn't include metadata in the resultant zips. As my neighbour Pascal says, metadata matters. We'd like for people to reconsume the metadata that we create during the build instead of recreating it, possibly with errors.



In the M6, the p2.mirror task in the org.eclipse.equinox.p2.repository.tools bundle was introduced. This task provides a much more flexible and efficient mechanism for slicing and dicing bundles from a repo into a smaller repo. For instance, to assemble the components of the Equinox SDK from a repo, this command will do the trick:

<p2.mirror source="file://${buildRepo}">
<destination kind="metadata" location="file://${featureTemp}" name="My Equinox repo" format="file://${buildRepo}" />
<destination kind="artifact" location="file://${featureTemp}" name="My Equinox repo" format="file://${buildRepo}" />
<iu id="org.eclipse.equinox.feature.group" version="" />
<iu id="org.eclipse.equinox.source.feature.group" version="" />
<slicingOptions platformFilter="true" includeOptional="false" includeNonGreedy="false" followStrict="true" />
</p2.mirror>

This resulting content will include all the bundles associated with the org.eclipse.equinox.feature.group and org.eclipse.equinox.source.feature.group IU's as referenced in the content.jar. The version of these IU's is not specified which means that the highest version will be selected from the repo. Conversely, you could specify a version. The platformFilter="true" means that all platform specific fragments associated with this iu will be provisioned.



A simple and fast slicer. For the low low price of $19.95. Wait, it's free. Try it out today.

For more information, see

bug 262083

bug 265208

Read more...

Implementing composite repos in your build

As DJ mentioned, composite repository support is included in 3.5. Our I-builds composite repo looks something like this...

compositeArtifacts.jar
compositeContent.jar
I20090210-0950/
I20090211-0900/

with a directory for each child repo such as
ls I20090211-0900/
artifacts.jar
binary/
content.jar
features/
plugins/
sdkinstaller.properties

Composite repositories allow you to identify which bundles are associated with each build in the repo. This allows you to cleanup your repositories more easily and avoid the wrath of the webmasters. How is this implemented in the build? As a neigbourhood committer would say, let me tell you a story......



One of the most important steps in our build is to create a master feature of all bundles and features that can sign it at the foundation in a single step. Once this feature is available, we extract it to a source location and publish metadata to a repo.

<p2.generator source="${reposource}" compress="true" append="true" flavor="tooling" metadataRepository="file:${repo}" artifactRepository="file:${repo}" metadataRepositoryName="${p2.repo.name}" artifactRepositoryName="${p2.repo.name}" publishArtifacts="true" p2OS="linux" mode="incremental" />

The next step is to run some packaging steps, then use the mirror application task to mirror the metadata and artifacts from the repo to the child repository. Why not just copy the bundles from the repo to the child repository instead of using the artifact.mirror task? One of the things that we want in our build is consistency. If a bundle in build A has the same version and qualifier as the bundle in build B, it should have the same content. Inconsistent bundles are scary.


How can a bundle have the same name but be different?

• New compiler changes the byte code
• New certificate changes the manifest
• Conditioning process changes the timestamp in the manifest
  
• A bundle has a compile error because of one the bundles it depends on has a bug. The dependancy is fixed, but the tag of the original bundle remains unchanged.

So we mirror the metadata and the artifacts instead of a simple copy.

<property name="buildRepo" value="${updateSite}/${buildId}">
<mkdir dir="${buildRepo}">
<p2.metadata.mirror source="file:${repo}" writemode="append" destination="file:${buildRepo}">

The artifact.mirror task has the baseline argument which refers to the existing composite repo. This means that the jars in the baseline repo will be used if the same jar exists both the baseline and the source repo. The new jars won't be used. This replicates the experience that a user will have when updating to the latest build. If the same jar already exists in their install, they won't download a new bundle.

<p2.artifact.mirror source="file:${repo}" baseline="file:${updateSite}" destination="file:${buildRepo}">
<copy todir="${buildRepo}">
<fileset file="${eclipse.build.configs}/../../extras/sdkinstaller.properties">
</fileset>

The next step is to create the composite repository
<p2.composite.artifact.repository.create location="file://${updateSite}" name="${p2.repo.name}" compressed="true" failonexists="false">
<p2.composite.metadata.repository.create location="file://${updateSite}" name="${p2.repo.name}" compressed="true" failonexists="false">

and add the child repository (current build) to the composite repository. The compositeArtifacts.jar and compositeContent.jar will have entries for the child repositories.

<children size='2'>
<child location='http://download.eclipse.org/eclipse/updates/3.5-I-builds/I20090210-0950'/>
<child location='http://download.eclipse.org/eclipse/updates/3.5-I-builds/I20090211-0900'/>
</children>


When we run the p2 director to provision the build zips, we provision from the composite repository.

Consistent metadata and artifacts ensures happiness all around.




This solution isn't perfect yet. I'll also be using the p2 team's comparator to provide warnings for certain scenarios . See bug 263272

Read more...

Eclipse 3.5 M5 now available

The 3.5M5 Eclipse and Equinox builds, as well as the new and noteworthy have been replicated to eclipse.org. The milestones p2 repo has the M5 bundles available.

M5 wasn't without drama. Late Friday afternoon, the UI team asked for a rebuild toward 3.5M5 to include a new splash screen. How could this go wrong? A rebuild with one file change. What could be simpler?

I woke up on Saturday morning confident that the build would be ready to promote. Instead of quality time with my laptop I'd be able to spend time snowshoeing with my favourite mathematician. Unfortunately, this was not to be. The build was missing the eclipse executables (eclipse.exe, eclipsec. exe etc). Hmm, I don't think this is a cosmetic bug. One file change and the executables went away? This seems like overkill.

When running the p2 director to provision the bundles for the build from the repo, there were signing errors in the .logs. I ran a test build with signing. The build had the same error. I ran a test build without signing. The executables were back. I ran jarsigner -verify on the bundles on a signed build. The junit 3.8.2 bundle had a signing error in the manifest. Interestingly enough, Denis had changed the signing certificate on eclipse.org on Friday.

At this point I suspected the problem was that the Orbit bundles were signed by the old certificate and the new bundles that we had just compiled in our build were signed in the old certificate. We resign the Orbit bundles in our build. (I've opened this bug to avoid this in the future).

Martin Oberhuber and David Williams ran a Orbit build on Sunday night which was signed by the new certificate. I ran a test build with this new build. The executables were back! Monday morning I promoted an Orbit build to Stable, and started a new build toward M5. Andrew Niefer determined that the root cause of the problem was that the junit bundle had a nested jar with a bogus signature. The end was near of this milestone saga! I could finally start releasing my patches that I had prepared for M6.

I was pinged on IRC by a certain PDE committer from Austin who will remain nameless. "Kim? Is there another build toward m5?" This PDE bug was approved by the PMC for inclusion in M5 yesterday afternoon. Another rebuild.



Here are the bugs with the details.

Promote 3.5M5
Certificate will expire within six months.
JUnit jars should exclude nested jars from signing and packing
[jarprocessor] Should we be signing nested jars?

I'll celebrate the release of M5 with a cookie. Enjoy.

Read more...

Kite Club: Committer Edition

Here in Ottawa, everyone is looking forward to summer vacations after an extremely busy release cycle. My good friend Sonia has introduced us to sport kiting.


Flying a kite is a very relaxing way to spend lunch hour - thoughts of builds and bugs are swept away. Sonia can really make the kite dance.



I just try to keep it up in the air without crashing spectacularly. In some ways, it's similar to being a buildmeister...



Some people see kiting as an extreme sport. Others go kite skiing.

The other day, we discovered that Prism offers a p2 sport kite. It is described as follows

STYLUS P.2
The P.2 develops impressive pull in a strong breeze but responds with crisp precision to your inputs. It’s astonishingly easy to recover from a collapse or a crash due to its oversized air inlets, and when the wind picks up it’ll really scream along. Want more power? Stack one, two, or three more P.2’s in a train and get ready to go for a ride!

I look forward to seeing p2, Eclipse 3.4 and Ganymede take flight.

Read more...

Platform-releng endorses p2, not paddles

Zx has suggested that the Eclipse project's releng team has become soft. What? In fact, we're in pretty good shape due to the amount of shovelling we've done to move the 4m (13 feet) of snow that has fallen in Ottawa this winter. We don't have to resort to Photoshop for extra muscle tone :-)

I did give out out a clown nose this week. It wasn't well received...although in the process I learned some new French vocabulary from a certain Equinox committer. Integrating p2 into the build was really a group effort. We are using the p2 code compiled in the build to build the SDK..so there are bootstrapping issues. It would have been ideal to have more time to test the integration but with EclipseCon immediately followed by M6 we had to get in this week's build. I'll write later on how we are transforming our build process with p2. It promises to alleviate many pain points. However, as always, change reveals many implementation details. For other teams, the transformation will be much much simpler. We now have a build that works with p2 goodness. In the end, that's all that matters.

I saw a preview of Pascal's p2 talk this afternoon and I'd highly recommend taking the opportunity to attend it at EclipseCon and learn more.

As for using paddles as a deterrent for build breakage...well eclipse.org is rated PG. I'll leave the paddling activities to other websites...

Read more...

Eclipse 3.4M5 now available

Eclipse 3.4M5 is now available for download . Please peruse the New and Noteworthy here.


M6 promises to be even more dramatic milestone, with plans for new Equinox and PDE bundles to hatch from incubation and into the SDK.

Read more...

Happy 6th Birthday Eclipse!

Eclipse is six years old today.

From
1.0 to wow!


Happy birthday!

Read more...

Eclipse 3.3.1.1 now available

Eclipse 3.3.1.1 is now available.

Q. What is Eclipse 3.3.1.1?

A. It is Eclipse 3.3.1 with six additonal bug fixes as listed in the readme



Eclipse 3.3.1.1 is NOT

• Eclipse 3.3.1: Extended edition


• Eclipse 3.3.2


• Platform 9 3/4

Q. How are fixes backported to a maintenance stream?

A. During each build we tag both the map files and the builder projects with the build id such as I20071023-0800. This insures that each build is repeatable. If we need to back port fixes to release, I just branch the releng project from the tag and the developers branch the relevant projects, release their code, and update their map in the new branch.

Q. Why do some of the features in 3.3.1.1 have feature versions listed as 3.3.2?

A. The marketing name of "Eclipse 3.3.1.1" doesn't need to correspond to plugin and feature versions. We could start naming our releases in step with the hurricane names, for instance Eclipse Melissa. However, if you looked at the components of Eclipse Melissa, they would still adhere to the versioning numbering guidelines.


For instance, Eclipse 3.3.1.1 has the following feature versions
cvs -> 1.0.1
equinox -> 3.3.1
jdt -> 3.3.1
pde -> 3.3.2
platform -> 3.3.2
rcp -> 3.3.2
sdk -> 3.3.2

When we release the first bug fix in a new stream of a release, we increment the service version of the bundle and the feature that contains it.

Now on to Eclipse 3.4M3 :-)

Read more...

Stay out of the cold with the latest eclipse I-candy

After all the excitement of EclipseCon, I arrived home to realize that although Ottawa currently lacks Santa Clara's warm breezes and sunshine, spring is definitely on the way because ...
1) The melting snow decided to visit my basement
2) Eclipse 3.3 M6 is next week

M6 represents the API freeze for the Europa release of the top-level Eclipse project. In fact, at this juncture, teams must ask the PMC for approval to make API changes.

If you are adopting 3.3 components as part of the Europa release or for your commercial project, it's a good idea to take a look our integration builds, not just milestone builds. Consuming your dependancies each milestone means that you are in only in a position to understand changes after the milestone has shipped. Taking a look at the builds in the weeks leading up to a milestone allows you to provide feedback on new functionality through bugzilla or mailing lists, specify how it impacts your use case and offer to help out :-) while the code is still under development.

The latest integration build is available here.

Of course, as we gear down to the Europa release at the end of June, the number and scope of changes will start to wane. Why? Submitting major new functionality close to the shipping date puts the release at risk. We like to ship and then everyone takes vacation. It's a tradition around here.

To mitigate against the temptation to release risky code, the rules of engagment in an end game plan specify increasing levels of sign off and verification for changes as we march toward Europa release date. This ensures that there is a conversation among members of the team regarding whether a change should be implemented, the associated risks and the possible impact to downstream teams, instead of just one person with the fervent wish to close another bug.

The M6 build menu looks something like this...

Monday starts off with a series of builds toward a test candidate.

Tuesday is test day, where people test the Monday's test candidate.

Wednesday is another build day where teams submit fixes to the issues they discovered during the test pass.

Thursday we have a final build and then ask teams for sign off. Inevitably, there is a rebuild to address a late breaking issue.

Read more...

Eclipse 3.3M5 now available

As Dejan mentioned earlier, many people were anxiously waiting the release of Eclipse 3.3M5. Milestone week...end is always dramatic and the 2007 edition is no exception. Sexier forms, SWT on the splash, a new launcher, a shiny new pdebuild that fetches prebuilt bundles via http a la Orbit - M5 is more exciting than an astronaut love triangle.

These and many other new goodies are documented in the New and Noteworthy.

Download M5 from you friendly neigbourhood mirror.

Read more...

Who will make a splash at Eclipsecon 2007?

And by splash I don't mean who will jump in the hotel pool. Who will be the lucky ones to have their smiling faces appear in an Eclipse SDK vanity build? Last year it was Steve and the Netbeans girls In 2005, the winner of the Eclipsecon trivia contest had her own Daniela build. Whatever happens, it will be very exciting.

Read more...