Adventures in Publishing

For Eclipse 3.5M7, we changed the build to take advantage of the new p2 publisher to to simplify the creation of the platform, RCP and SDK zips. Why switch to the publisher?

Metadata is generated on the source when pde build is invoked, instead of after the jars are built.

The publisher provides better support for the p2.inf

The metadata generator tasks are being deprecated

The publisher makes adding a new platform to the build much easier because metadata is generated for the root files. (We added five new platforms in the 3.5 cycle).

Also, the Equinox team also wanted to test the publisher in an active build environment and iron out any issues this might reveal. Similar to how Equinox is the reference implementation for OSGi, platform releng is the "reference build" for the Equinox team ;-).

Previous to using the publisher, we built the sdk feature and the associated root file feature to construct the zips available on our build page. Root files are just files that reside at the root of the eclipse install such as the eclipse executables, config.ini, eclipse.ini and license files.

To switch the our build to using the publisher I implemented the following steps

1) Set p2.gathering=true in the build.properties of the builder for our master feature. The master feature contains all the bundles and features we compile in the build. I also set skipMirroring=true, since we'd like to continue to control the mirroring aspects later on in the build.

2) Remove the call the p2.generate task after your bundles are built. While using the publisher, the metadata is generated when pde build is invoked in your headless build. So you don't need the separate generate call anymore.

3) Switch the SDK a feature based to product based build. This isn't necessary for most projects that are switching to the publisher. However, the Eclipse SDK is a collection of artifacts for many platforms and product builds make life easier. We also had to to write a p2.inf file to include some platform specific bits into the product build since the PDE UI's product editor has support for four platforms. We build 16! There's a bug open to support more platforms in product files, hopefully this will be addressed in 3.6.

4) Run the appropriate p2 publisher task. In our case, since we have product builds, I run the p2.publish.product task to publish metadata for the products to the repository.

<p2.publish.product flavor="tooling" repository="file:${reposource}" productFile="${eclipse.build.configs}/sdk/builder/sdk.product" compress="true">
<config os="win32" ws="win32" arch="x86" />
<config os="win32" ws="win32" arch="x86_64" />
<config os="win32" ws="wpf" arch="x86" />
<config os="linux" ws="gtk" arch="x86" />
<config os="linux" ws="gtk" arch="x86_64" />
<config os="linux" ws="gtk" arch="ppc" />
<config os="linux" ws="gtk" arch="s390" />
<config os="linux" ws="gtk" arch="s390x" />
<config os="linux" ws="motif" arch="x86" />
<config os="solaris" ws="gtk" arch="sparc" />
<config os="solaris" ws="gtk" arch="x86" />
<config os="macosx" ws="cocoa" arch="x86" />
<config os="macosx" ws="cocoa" arch="x86_64" />
<config os="macosx" ws="carbon" arch="x86" />
<config os="hpux" ws="motif" arch="ia64_32" />
<config os="aix" ws="motif" arch="ppc" />
<advice kind="featureVersions" file="${buildDirectory}/finalFeaturesVersions.properties" />
<advice kind="pluginVersions" file="${buildDirectory}/finalPluginsVersions.properties" />
</p2.publish.product>


5) Update any custom build scripts with the publisher targets. The pde build included in 3.5 will generate this automatically for you. However, if you have bundles where build.properties indicates custom=true, you'll have to update your static build.xml files to include the new publishing parts. We use custom build scripts for our doc and swt bundles. See this patch on bug 266488 for an example. Basically, you're adding the following to the init target of your build.xml.

<condition property="p2.publish.parts" value="true">
<istrue value="${p2.gathering}">
</istrue>
</condition>

and then this additional target.

<target name="publish.bin.parts" depends="init" if="p2.publish.parts">
<antcall target="gather.bin.parts">
</antcall></target></pre><param name="destination.temp.folder" value="${build.result.folder}">

<eclipse.gatherbundle metadatarepository="${p2.build.repo}" artifactrepository="${p2.build.repo}" buildresultfolder="${build.result.folder}" targetfolder="${build.result.folder}/${full.name}">
</eclipse.gatherbundle>
</target>

After these steps are completed, the build proceeds as normal to provision the zips from the repository using the p2 director.

Further References

If you're contemplating switching to the publisher, these documents are useful references. Our build is quite complicated with it's 16 platforms and associated launchers, the transition for most teams will be much simpler than our scenario.

The meta bug for the changes to reorganize the Eclipse and Equinox build to use the publisher.

Wiki article on the p2 publisher

Wiki article on customizing metadata

Ian's blog entries about the publisher 1 and 2.