AFOL at EclipseCon

>> Monday, March 08, 2010

I read an interesting book over the weekend.  Groundswell  is a book from the Harvard Business Press written by Charlene Li and Josh Bernoff.  It discusses how to use social media to open up communication channels with your customers. For instance, how to harness the wisdom of the crowds to design better products and improve technical support.  The book describes the approaches of several companies and shows the ROI calculations for the investment in social media.


One of the companies that was profiled was the Lego Group.   I learned that Lego is the sixth largest toy manufacturer in the world. Over a billion dollar a year business.  And somewhere between 5-10% of their sales go to AFOLs.  Adult fans of Lego.  In fact, they have an executive in charge of marketing to AFOLs because it is such a significant market.  AFOLs hang out in an online community called LUGNET.  The Lego Group's approach to social media is to have Lego Ambassadors in the AFOL community listen to the needs of the people who view Lego as a building material, not just a toy. And these amabasadors are paid for their services in Lego bricks, not money.  Pretty cool.

So what does this have to do with Eclipse?  EclipseCon 2010 is having a e4-Rover Mars challenge where you have the chance to use Eclipse technology to maneuver Lego robots.  Go to EclipseCon and add four letters to the end of your name without paying for tuition.  AFOL at EclipseCon!

Read more...

Community starts with conversation

>> Friday, March 05, 2010

This week, I've been writing slides for our p2 tutorial at EclipseCon. As part of this effort, I wanted to show the companies that were shipping products based on p2 or that used p2 in their internal products. I asked for suggestions on the p2-dev list. Here's a summary of the responses.


Pretty interesting. It's a testament to the Equinox team and the larger community that so many companies are building products based on p2. If your company, belongs on the slide, let me know.

Pascal has arranged a p2 BOF at EclipseCON. It should make for interesting conversation.

Read more...

Tag after Release

>> Tuesday, March 02, 2010

Last week Eclipse and Equinox 3.5.2 was released as part of the Galileo SR2 release.  So far, so good!    A big thanks goes out to David Williams for keeping all the projects on the release train in line and ready for SR2. Also, kudos to the webmasters for ensuring that the mirrors were ready for the release.

The work of the release engineer isn't done after the release bits are available.  I tag all the projects in the Eclipse and Equinox project with as R3_5_2, as well as our our map files and the builder projects.  This ensures that if required, exactly the same build can be reproduced.  In addition, it's useful for developers to be able to compare against a tag for a previous release, or to branch from a tag.

Our source resides in the /cvsroot/eclipse and /cvsroot/rt repositories. The steps I take to do this are as follows:

1. Start eclipse with a clean workspace.
2. Check out the vM2010211-1343 versions of org.eclipse.releng and the builder projects.  Every time we run a build, the org.eclipse.releng and builder projects are retagged with with the build id. M2010211-1343 is the build id of the final 3.5.2 build so I retag these projects as R3_5_2.



3. I remove the orbit map from the releng project in my workspace. There are prebuilt bundles fetched from the Orbit repository so we don't need to tag them. 
4. Replace  :pserver:anonymous with :extssh:kmoir in the remaining map files in my workspace.  I have commit rights on all the eclipse and equinox projects for this very purpose.
5. Change the connection timeout on the Team CVS client to one larger than the default.  Otherwise, your CVS connection will timeout while tagging all the projects.
6. Select the map files in my workspace.  Right click and select Team and the Tag Map File Projects option.


7.Tag as R3_5_2.


8.  Once all the tagging has completed after several hours, I will check out all the projects from the map files and compare with R3_5_2 to ensure that there aren't any files missing the tag.
9.  Install the releng tools to use the "Tag map file projects" functionality. This allows me to tag the versions of projects defined in your map files as another version without checking them out. Very useful.


Questions?
  • Q.Why don't you do this tagging as part of the build process?
  • A.Eclipse and Equinox is a large project with a lot of source  - over 300 bundles and more than 30 features.  Tagging this as part of the the build each time takes a few hours and is often has CVS timeouts.  We don't need that, the build is slow enough as it is today :-) That being said, if you have a smaller project of just a few bundles, tagging the bundles with the build id could be useful.

Read more...

Private Emails Hide the Details

>> Tuesday, February 16, 2010

Recently, I've been receiving a lot of emails directly to my IBM address regarding bugs, features, releng questions, or building Eclipse for platform XYZ from people in the community.  These people are new to Eclipse and therefore may be unaware of the proper channels to ask questions.  I like to gently remind people ask a question on the forums or open a bug. The internet is forever, and Eclipse is no exception.  If you open a bug, others with the same issue will be able to search and find a solution.  Private emails hide knowledge that's useful to the larger community. As committers, we're very detailed oriented people.



Today, I responded to the latest request as follows:

At eclipse,
We're open source,
Private emails,
hide the details,

Asking in the open
Allows others to see,
the issue described,
the remedy prescribed,
for posterity

Found a problem?
Want to request a feature?
Please, I will ask,
open a bug for the task
https://bugs.eclipse.org/bugs



I look forward to seeing their bug.

Read more...

Trim your builds with p2 repository tools

>> Thursday, February 11, 2010

The Eclipse Project has the unusual distinction of consuming the most download space of any project at eclipse.org.   Therefore, I like bugs with a request to "get rid of stuff" versus "add more stuff".  Recently, I fixed a bug to remove the packed jars from our zipped repositories that are available for download.

For example, we provide RCP binary and RCP source zipped repositories on our download page. Last week, these zips included both packed (*.jar.pack.gz) and unpacked (*.jar) bundles.


We don't need to include both types of jars in these downloads because most people will install the bundles locally from the zip.   The packed jars are included by default because when I run the mirror tool against the full build repository to create the slice, it already includes the packed jars.  You can use the remove.iu tool to query the repository for a list of all bundles and remove the associated packed jars.

<p2.remove.iu>
        <repository location="file://${yourRepo}">
        <iu artifacts="(format=packed)" query="">
</iu>

The remove.iu tool will remove the packed bundles from the repository and update the artifacts.jar accordingly.

As I have written before, repository tools are very useful for creating subcomponents of existing respositories.

Remove IU

Slicing and Dicing the p2 way

Want to learn more about using p2 to assemble products out of pre-existing components? Plan on attending the From build to assembly to deployment: Using p2 to facilitate agile software development tutorial that Ian Bull and I will be presenting at EclipseCon.


Read more...

Committer Collector Cards

>> Thursday, January 28, 2010

I have a new idea for Eclipse swag. Committer collector cards. Why? It would be a fun way to get to know more about members of the Eclipse community. Like hockey cards.  But we're better at debugging, and have more teeth.

Here are some ideas for the sorts of information that this could display.


Special power? Yes, all committers have special powers. As a release engineer, my special power is pain tolerance. Much like Wolverine. Yes, it's true.



I can see this being a popular activity at EclipseCon. Trade a Moir for a Merks, a Bokowski for a Bull. What do you win if you collect a whole project? Bugzilla bingo.  You get a bug assigned to you so you can earn your own commit rights.  Diversify the community. It's all very win-win.

Since I'm in the give away free software business, not the marketing swag business, I thought I'd suggest this idea for any enterprising people out there. It could be a branding activity a company who'd like to sponsor such a unique piece of swag. Go for it!

Read more...

EclipseCon Exercise: Commit to Running

>> Friday, January 22, 2010

EclipseCon 2010 looks like it will have a fantastic program this year with many interesting talks.  For instance, there will be talk about Lego Mindstorms and Eclipse. How much do I love Lego?  Let me show you my keychain.



There will be also be a talk on the Eclipse at NASA which looks really interesting.  I mean, how often is it that the software you build is used to monitor robots travelling across another planet? Mr. Platform Releng is very talented at ordering stuff on the internet.  One day I arrived home and found a Lego Mars Rover being assembled in our living room.  Life is unpredictable.  From lego.com.



Outside the formal program, there will be EclipseCon Exercise early in the morning.  You're invited to sign up for a  run every morning before the talks begin on the wiki. Don't hesitate to put your name down if you've just started running, there will be several pace groups.   Thanks to the work of Darin Swanson, who organized EclipseCon Exercise in previous years, we already have a map.


The generous Eclipse Foundation has also agreed  to provide prizes for the EclipseCon Exercise participants.  There will be Eclipse shirts and running related gifts.  What will you win a prize for?  Well, that's up to you.  Again, you can make suggestions on the wiki for prize categories and I'll set up a vote for the community in early March.   Also, if you work at a company that's interested in sponsoring EclipseCon exercise, please contact Donald Smith.

Running before the talks in the morning is a great way to clear your mind for the day. It's also a way to get rid the jitters before you give a talk or tutorial. Most importantly, it gives you a chance to meet and talk to people in a different context, and get to know them better as a person, not just an Eclipse professional. Learn about what they do outside of work,  their communities, and discover common interests.

So consider committing to EclipseCon Exercise.  Sign up today and remember to pack your running shoes!

Sign up sheet for EclipseCon Exercise 2010

Read more...

p2 tutorial RFC

>> Tuesday, January 12, 2010

The EclipseCon program committee is sending out acceptance notifications after completing the difficult task of selecting the talks that will be presented in March. Thanks for all your hard work - I'm sure it will be an amazing conference!

Ian and I have been selected to give a tutorial entitled "Exploring p2". The best approach to learning is spend time actually getting your hands dirty. The wisdom of xkcd


Originally, the focus of this tutorial was to be on building with p2. However, given time constraints in the build category, the program committee asked us to provide a broader focus on p2, not just building with it.

This is where you come in. If you are considering attending this tutorial, let us know what topics you'd like us to cover on the wiki or leave a comment. We really value your input and so we can tailor the tutorial to you. Also, it will be interesting to learn the p2 use cases in the community. A tutorial is a two way street, and we have just as much opportunity to learn about how you are using the software we build, as we have the chance to share our experiences with you.

Keep in mind that there are other talks in this topic that have been approved. For instance, Pascal will be conducting a talk on p2 APIs. Today, our fantastic Equinox committers are working on merging the newly minted p2 APIs into HEAD for 3.6M5.

Looking forward to seeing you at EclipseCon 2010!

Read more...

Less email, more info

>> Thursday, January 07, 2010

I subscribe to many eclipse mailing lists. This means I get a lot of email that's sorted by various filters so my inbox doesn't explode with all the traffic.

However, there are some Eclipse projects that I'm interested in, but not going to participate actively on the mailing list. This reminds me of the pig and chicken analogy. I'm somewhat involved in the other projects, but I'm committed to others. From implementing Scrum



Luckily, each eclipse.org mailing list also has a RSS feed.


For instance, this time of the year, it's interesting to watch the EclipseCon Program Committee feed. Lots of discussions regarding which talks will get the final +1 for the 2010 program.

Or you may have an interest in what's happening in e4, but don't want all the email that this very active project generates.

Read more...

And the winner is.....

>> Monday, December 14, 2009

Eric Rizzo has won the December splash screen contest with this submission.


Congratulations! Thanks to all the contestants for submitting such creative artwork. The winning design will start appearing in tonight's build (N20091214-2000) and has been tagged for the integration build.

Read more...

Vote now for your December Splash

>> Friday, December 11, 2009

Next week, we'll have the our final integration build before the end of 2009. What better way to end the year than with a custom splash screen? As usual, the very creative Eclipse community has stood up to the challenge with eight different designs for your consideration. Please vote here:



Direct link to poll.

The poll closes on Monday at 5 pm EST. I'll announce the winner on Monday and commit the winning splash. Good luck to all the contest entrants!

This contest is closing a bit earlier than expected because we shutdown our builds over the holiday season. Thus, we need to get the December splash into the integration build on December 15, 2009.

Read more...

Eclipse and Equinox 3.6M4 now available




Ho ho ho.
Ho ho ho.
We are Eclipse's elves.

We are Eclipse's elves,
Filling mirrors with M4,
Another milestone out the door.

Oh, we are Eclipse's elves,
We work hard all day,
Debug and refactor is our play,
Bugs we stamp out,
Hurray, the community shouts!

We are Eclipse's elves,
We ship on time each year,
We don't like to brag,
Just sync, commit and tag,
Install new bundles without lag.

We all know who's been good,
Resolved the bugs you should,
Eclipse is you,
Grab a bug and become a committer too!

We are Eclipse's elves.
Ho ho ho. Ho ho ho.
We fix the code ourselves.
Ho Ho!

The Eclipse and Equinox team are happy to announce the release of the Eclipse 3.6M4 milestone.

New and noteworthy

Update to 3.6M4 by adding this site to your list of available sites
http://download.eclipse.org/eclipse/updates/3.6milestones

Equinox Downloads

Eclipse Downloads

Happy Holidays and New Year from the Eclipse and Equinox projects!

Read more...

December splash screen contest: Prizes++

>> Wednesday, December 09, 2009

Last week, I announced the December splash screen contest. Since then, we've had some entries from Olivier Thomann. Very creative. Look at this


Or this.


We need more people to submit splash screens to to the contest. Here's the bug where you can submit your entry. It doesn't have to be a holiday theme, it can be whatever you'd like.

As an added bonus, the generous folks at the Eclipse foundation (hi Ian and Lynn) have offered to donate some eclipse bling to the prize pot. So the total prize package for the December splash screen contest will be

1. Friends of Eclipse membership
2. Eclipse t-shirt
3. $50 donation to Eclipse in your name

An Eclipse t-shirt? Does it give you special powers like the famous "three wolves t-shirt"?




I guess you'll have to enter the contest to find out. Fire up your favourite graphics editor and submit a splash screen to bug 296918

Read more...

A tale of CVS, NFS and Swordplay

>> Friday, December 04, 2009

Earlier this year, the time that it took our builds to check out code from CVS had slowed to a crawl. Last year, our integration builds that started by 8am were available by noon. However, during the 3.6 cycle, most builds weren't ready until 2pm or even 4pm. Very painful, especially for our European committer friends. Not only were the builds slow, but it took a long time for us to commit code, and timeouts were frequent.

Denis and his merry band of webmasters determined that the source of the problem was twofold.

1) NFS configuration issues causing high CPU load on the servers. (Bug 288293)
2) Anonymous pserver users were holding the cvs lock files for a long time (Bug 293355)

Earlier this week, Denis fixed the NFS configuration issues. This had reduced the time it takes to check out our code significantly. Also, I haven't seen any CVS timeouts lately. Today, our 8am integration build was ready at noon. A big thank you to the webmasters! We are much more productive thanks to your work. Less swordplay while waiting for builds, and more time to fix bugs and implement new features. Again from xkcd.



Really, who doesn't love xkcd? It's so good.

Denis will change the anonymous pserver access to a mirrored copy of the CVS repository on December 11. He sent notes to the committers list with the details. The means that committers won't have to contend with anonymous users for cvs lock files if they check out code via ssh. Psever will be available on a separate file system, but sychronized with the live copy.

In preparation for this change, I modified our build scripts to check out our code using ssh, instead of anonymous pserver. However, our map files remain untouched so anyone can check out our code via pserver. The build scripts modify the map files after they have been checked out. We won't have to deal with file contention for CVS lock files. And ssh connections to eclipse.org have a higher QoS than pserver. Good stuff.

Most teams won't have to make any changes because they run their builds on build.eclipse.org. These users will still pull from the live copy of the data via pserver. And the mirrored copy will be up to date. I'm just trying to do everything I can to avoid infrastructure issues that cause build failures.

See bug 294900 for the platform releng changes.

Read more...

Make a splash in December

The Movember splash was a lot of fun. I'd like to open up submissions for a December splash screen. It will be released after M4. This contest is open from December 4-15. At that time, I'll start a poll and ask the community to vote on the splash screen. I'll commit the winning submission. What will you win? Eyeballs on your art. Respect of the community. Warm and fuzzy feelings. And I'll donate $50 to friends of Eclipse in your name.



The existing splash screen is in org.eclipse.platform in /cvsroot/eclipse. The file is splash.bmp. The photoshop files are in 3_6SplashHeliosPsd.zip in the same project. Attach your entries to bug 296918

Nothing offensive please people. Go for it.

Read more...

Good Eclipse Swag

>> Monday, November 30, 2009

I found this mug in the kitchen at work today. It has Eclipse key bindings on front


and back.



Now, all I need is a sandwich to go with this and I'd be set. From xkcd




Swag is rarely both useful and aesthetically appealing. This is both! Thanks itemis!

Read more...

The end of a three year relationship: Goodbye bug 153429

>> Friday, November 27, 2009

This past week, the Eclipse and Equinox team released changes to support the use the JUnit 4 bundle in the Eclipse Test framework in bug 153429. This bug had over 75 people on the cc list and 40 votes to fix it. It was in my bucket for three years. It's finally fixed so now we and (you) can run your automated tests on JUnit 4.

The changes that we made to fix this bug are described in the wiki. John also sent a message to the cross project list.

I have worked on many bugs during my tenure at Eclipse. This was a tough one. It was very complex, and required a tremendous amount of testing. It was also very much a team effort.

I'd like to especially thank DJ Houghton for all his patches for the test harness and and test bundles. Also, DJ added the org.junit 4.7.0 bundle to Orbit which was very helpful. John Arthorne for fixing the p2 tests and all the good advice during this process. Dani Megert and Markus Keller for their changes to JDT. Darin Wright for his advice regarding PDE. Curtis Windatt for his changes to the pde ui tests. Andrew Niefer for the changes to the pde build tests. Other committers made minor changes to their test bundles to accommodate this change. Thank you!

Goodbye bug 153429. I won't miss you.

Read more...

Committer Reps, we need your help

>> Wednesday, November 25, 2009

Hi Boris, Chris, Doug and Ed

Hey, how's it going. I hope that you are all well.

Like a mall on Black Friday, around here it's peak Eclipse committing season. Lots of bugs to fix. Lots of builds to run. As you know, builds are quite a pain point at Eclipse. I'm excited about the possibilities in the b3 project to make things better. Building software is complex, and Eclipse is no exception. In additional to tooling to make builds easier, we need hardware to make builds faster. Our build today takes about five hours to complete, and an additional 6.5 hours for the tests to complete. Really, it's not pretty.

Many eclipse projects run their builds on Hudson on build.eclipse.org. Hudson is fantastic because there's a rich set of plugins that you can use to enhance the functionality of your build. Also, since this server has local access to the eclipse.org filesystem for code checkouts, you're less prone to network errors which can break the build. It also has ldap integration with your commiter login so you can restrict your build configuration to the commiters on your project. In theory, if you need more build machines to run your build - you can use the Amazon EC2 plugin to provision more machines in the cloud, or other plugins to start builds on local slave machines. Good stuff.

However, one of the things that the foundation doesn't provide today is test machines. This means that we can't run our build at the Eclipse foundation. The Eclipse Project builds zips for 14 different platforms. We run JUnit tests on three native platforms: Windows, Linux and Mac. They are the most commonly downloaded platforms. We need test machines to ensure that we don't have any bugs specific to a platform. Why do our tests take so long? We have 54,000 JUnit tests. You don't produce quality software by skimping on tests.

This isn't just about the Eclipse and Equinox projects. This could be very useful for other projects, for instance, the XSL tools project has expressed interest in using test servers. In addition, these machines could be used as slaves machines for running the build in the event that the main Hudson server is too busy. If we had enough machines, we could run more tests in parallel and reduce the time it takes our build to complete. This would be a big win for the community and our committers.

One thing I investigated in running tests in the cloud. However, most cloud services don't have provide a way to run tests on Macs and we need to make sure that our Mac users are happy. If there is a way, I'd appreciate a link. In addition, one of the advantages of running tests on machines local to the eclipse.org filesystem is that we don't spend time copying stuff back and forth across the network. It's just there.

So, what I'm asking from you is at the next board meeting, please bring up the issue of funding test infrastructure at the Eclipse foundation. It might be even be an advertising opportunity for one of the member companies if they donated hardware. Other companies could donate money to pay for the additional rack space. I don't know right now what the final technical solution will be or what it will cost. All I'm asking right now is to start the conversation.

For many years, the Eclipse project has been criticized for not being open enough. Having our build process fully on eclipse.org servers would make us more open. It would also allow any of the Eclipse and Equinox project committers, regardless of company affiliation, to initiate a build. It we had enough hardware, our build could be faster and we could spend less time waiting for builds, and more time fixing bugs the builds reveal.

Please bring this issue up and the next board meeting.

sincerely,
Kim

P.S. Right now we have the following test machines and our tests take about 6-8 hours to complete. Obviously, if we had more machines running tests in parallel, the build would take less time.
1) JUnit: 2 linux, 2 windows, 1 mac, 1 test cvs server,
2) Performance: 2 windows, 2 linux, 1 database server

Read more...

Happy Birthday Eclipse...now some pictures from your youth

>> Tuesday, November 03, 2009

Saturday November 7th, 2009 is Eclipse's 8th birthday. Eight years ago the first Eclipse downloads were publicly available along with the source code.

Here's a picture of the first www.eclipse.org. The website was slashdotted shortly after the "Eclipse is open source" announcement was made. The hard drive had to be replaced, thus the missing cover. This machine was replaced by a real server a few weeks later. A couple of years later, this was replaced by faster and more fault tolerant hardware managed by our most excellent webmasters.



I went through my desk the other day, and found some emails of the original requirements for the eclipse infrastructure, project structure and commit rights. A fast server has 512MB RAM and 20GB of disk space...really? Okay, I feel old.



Bugzilla was there from the start.



Some of the first mailing lists. A few disappeared but even more new ones were created. Today, we have eclipse on twitter, forums, conferences, marketplace, blogs and the list goes on. Pretty impressive!



In 2001, the linux kernel still fit on a floppy disk. It was useful to have a boot disk in the event that the boot partition became corrupted and the machine wouldn't boot. Much easier than booting from a rescue cd and mounting all the partitions by hand. Especially at 4am. Here's a picture for those who've never worked with a diskette (*ahem* co-ops :).




Initial project structure. Today we have so much much more.


The first commit rights - hello PDE family! Today we have an amazing diversity of committers from around the world and many companies. Eclipse also has friends, but would always like more....who doesn't?




In reality, the past doesn't matter that much, and we need to focus on the future. Sometimes I'm cynical about Eclipse, because as a long time committer, I notice that many people like to play (consume), but fewer want to pay (contribute). And to some degree we make it difficult to contribute without a significant investment of time and a steep learning curve. But most days, I'm absolutely amazed by the work that we as a community can do together, when smart, passionate people strive toward a common goal. Happy Birthday Eclipse!


Read more...

Now I see IU, now IU don't

Eclipse 3.6M3 went out the door over the weekend, along with a lot of Halloween candy.



Sometimes, you can have too much Halloween candy. And sometimes, you can have too many IUs in your p2 repo. Don't believe me - just look at this repo with bogus bundles - scary!





Both scenarios can cause your friendly neighbourhood release engineer pain. This is unusual because we're a very pain tolerant people. To alleviate the suffering, the p2 team added an Ant task in 3.6M3 that allows you to remove bundles from your repo. As much as I love spending quality time at the command line modifying metadata, Ant tasks that automate tedious jobs are even better.

The p2.remove.iu task will remove both the metadata and the bundle from the repository for a specified IU. For example, if you had bogus packed com.ibm.icu.* bundles in your repo, this task would remove them.

<p2.remove.iu>
<repository location="file://${reposource}" />
<iu id="com.ibm.icu" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base" artifacts="(format=packed)" />
<iu id="com.ibm.icu.source" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base.source" artifacts="(format=packed)" />
</p2.remove.iu>

This task is useful if you'd like to remove some built time bundles from your repo. Or just correct a mistake after a release. It happens. In any case, it's all good. Almost as tasty as chocolate.




Related bugs

Support excluding bundles when running p2.process.artifacts task
p2.remove.iu task should have an option to specify to remove packed file only
ICU jars at Eclipse 3.5 update site have size of 0

Read more...

Hudson is sweet, now the build can tweet

>> Wednesday, October 21, 2009

I recently set up a new Hudson build so the Equinox team could test their changes in a branch instead of releasing everything to HEAD. Hudson, like Eclipse, has a rich variety of plugins that can expand the functionality of your build. Hudson has a twitter plugin and thus Eclipse (test) builds tweet here

http://twitter.com/eclipsebuilds

Right now, there are only a few messages about the test builds I started, and stopped when I saw that build was proceeding as normal. In any case, it's pretty cool.

Hudson also has plugins to use additional slave machines on the network to run the build on a faster machine if the current build machine is too busy. It also has plugins to provision Amazon EC2 images and run the build on a slave in the cloud. We currently run about 54,000 JUnit test per platform multipled by five test machines during each build. We run tests on Windows, Mac and Linux machines. But the tests take a long time to complete simply because of the number of tests we run and the lack of additional hardware to run more tests in parallel. It would be fantastic to be able to run the tests on parallel on many machines and finish in fraction of the time they take today. That would allow us to reduce the number of breaking build issues. But of course, this will take significant testing to implement. Sounds like fun!

References
Run JUnit tests in parallel
Hudson build for the p2 branch

Read more...

Test bundles switch from runnable to repo

>> Tuesday, October 20, 2009

Earlier this week, I changed the build so that the test bundles are provided in a zipped p2 repository format. A p2 repository looks something like this

...
artifacts.jar
content.jar
features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV.jar
plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735.jar
plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735.jar
binary/
...

The build will install the appropriate test bundles in the eclipse install being tested using the p2 director. Previously, our test bundles were assembled in the runnable format like this

...
eclipse/plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735/...
eclipse/plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735/..
eclipse/features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV
...

and unzipped into the dropins folder. However, the dropins folder is really for legacy purposes. Disadvantages of using the dropins folder include

  • It's to help those applications that expect that dropping a bunch of bundles into an install will work (!).
  • All the bundles in the dropins folder are treated as optional bundles.
  • You can't update the bundles in the dropins folder by using the UI, you are responsible for provisioning them.

The content of the org.eclipse.test framework bundle hasn't changed. However, if you're rerunning the Eclipse and Equinox projects' tests JUnit tests against your product, you'll need to install them into the eclipse your are testing using the p2 director. Unzipping the our JUnit bundles into the dropins folder won't work. Alternatively, you could use the reporunnable task to transform the repository into the old runnable format.

Why make this change? These days, I would expect that most users are installing bundles via a repository such as Galileo or one offered by a product team. Unzipping files is so Eclipse 1.0. So this will allow our tests to replicate the environment that reflects the reality of our users. As well, as I mentioned in my earlier post, we really need to run our tests in parallel on more machines to speed up the build process. Having the test bundles available in a repository, eventually in a shared location, is a step toward that goal.


References

Test bundles should be packaged as a repo


Supported dropins formats

repo2runnable task

Note: The Eclipse and Equinox projects provide their JUnit test bundles in the eclipse-Automated-Tests-${buildId}.zip in the eclipse-testing/eclipse-junit-tests-${buildId}.zip file that is available with every build.

Read more...

Galileo SR1 has left the station

>> Friday, September 25, 2009




The Eclipse and Equinox teams are pleased to announce that our 3.5.1 release now available as part of the Galielo SR1 release. Congratulations to all contributors and committers whose hard work made this release possible!

Galileo

3.5.1 Readme

Eclipse

Equinox

You can also update to 3.5.1 using Help->Check for Updates

Enjoy!

Read more...

Eclipse Helios: More repos, fewer zips

>> Thursday, July 23, 2009

In the 3.5 cycle, we introduced zipped p2 repositories in the build. These zips could be used to provision your target platform or update your install. We left the per-platform zips on the download page for Galileo. With 15 platforms multiplied by several permutations of features, and runtime versus source downloads, these zips consumed significant space on eclipse.org and the mirrors. Not to mention the fact that platforms other than Windows, Linux and Mac have very low download numbers.

For Helios, I've replaced all the platform specific RCP, JDT, PDE, CVS, examples and releng tools zips with p2 repositories specific to that feature. For instance, instead of 15 platform specific runtime RCP zips, there's a p2 source and runtime feature for each one. The SDK and Platform feature zips for each os.ws.arch combination continue to be built.

More information
How to Consume zipped repositories.
bug 280622

Read more...

Galileo Release: Eclipse and Equinox 3.5 now available

>> Monday, June 22, 2009

After a year of development, Eclipse and Equinox 3.5 are now available for download. Please put your feet up and enjoy our new release.


Galileo, Eclipse, and Equinox pages.


Every release is different and this year is no exception.

There's planning and committing to a goal.



Learning new things. See the New and Noteworthy and the readme.

Discussions with committers and community. How many rebuilds will there be?




Drama and angst.



Making new friends. We'd especially like to thank these people who really contributed to the success of this release.



Other Eclipse friends decide to see what's behind a new door.




Some days are a blur.



The final steps at the end of the release seem to take forever.



But the end is sweet.




Congratulations to the Galileo teams on the release. I hope everyone takes time to relax. Because it won't be long before have to lace up your shoes and get ready to run towards Helios!

Read more...

Adventures in Publishing

>> Friday, June 19, 2009

For Eclipse 3.5M7, we changed the build to take advantage of the new p2 publisher to to simplify the creation of the platform, RCP and SDK zips. Why switch to the publisher?

  • Metadata is generated on the source when pde build is invoked, instead of after the jars are built.

  • The publisher provides better support for the p2.inf

  • The metadata generator tasks are being deprecated

  • The publisher makes adding a new platform to the build much easier because metadata is generated for the root files. (We added five new platforms in the 3.5 cycle).


  • Also, the Equinox team also wanted to test the publisher in an active build environment and iron out any issues this might reveal. Similar to how Equinox is the reference implementation for OSGi, platform releng is the "reference build" for the Equinox team ;-).

    Previous to using the publisher, we built the sdk feature and the associated root file feature to construct the zips available on our build page. Root files are just files that reside at the root of the eclipse install such as the eclipse executables, config.ini, eclipse.ini and license files.

    To switch the our build to using the publisher I implemented the following steps

    1) Set p2.gathering=true in the build.properties of the builder for our master feature. The master feature contains all the bundles and features we compile in the build. I also set skipMirroring=true, since we'd like to continue to control the mirroring aspects later on in the build.

    2) Remove the call the p2.generate task after your bundles are built. While using the publisher, the metadata is generated when pde build is invoked in your headless build. So you don't need the separate generate call anymore.

    3) Switch the SDK a feature based to product based build. This isn't necessary for most projects that are switching to the publisher. However, the Eclipse SDK is a collection of artifacts for many platforms and product builds make life easier. We also had to to write a p2.inf file to include some platform specific bits into the product build since the PDE UI's product editor has support for four platforms. We build 16! There's a bug open to support more platforms in product files, hopefully this will be addressed in 3.6.

    4) Run the appropriate p2 publisher task. In our case, since we have product builds, I run the p2.publish.product task to publish metadata for the products to the repository.

    <p2.publish.product flavor="tooling" repository="file:${reposource}" productFile="${eclipse.build.configs}/sdk/builder/sdk.product" compress="true">
    <config os="win32" ws="win32" arch="x86" />
    <config os="win32" ws="win32" arch="x86_64" />
    <config os="win32" ws="wpf" arch="x86" />
    <config os="linux" ws="gtk" arch="x86" />
    <config os="linux" ws="gtk" arch="x86_64" />
    <config os="linux" ws="gtk" arch="ppc" />
    <config os="linux" ws="gtk" arch="s390" />
    <config os="linux" ws="gtk" arch="s390x" />
    <config os="linux" ws="motif" arch="x86" />
    <config os="solaris" ws="gtk" arch="sparc" />
    <config os="solaris" ws="gtk" arch="x86" />
    <config os="macosx" ws="cocoa" arch="x86" />
    <config os="macosx" ws="cocoa" arch="x86_64" />
    <config os="macosx" ws="carbon" arch="x86" />
    <config os="hpux" ws="motif" arch="ia64_32" />
    <config os="aix" ws="motif" arch="ppc" />
    <advice kind="featureVersions" file="${buildDirectory}/finalFeaturesVersions.properties" />
    <advice kind="pluginVersions" file="${buildDirectory}/finalPluginsVersions.properties" />
    </p2.publish.product>


    5) Update any custom build scripts with the publisher targets. The pde build included in 3.5 will generate this automatically for you. However, if you have bundles where build.properties indicates custom=true, you'll have to update your static build.xml files to include the new publishing parts. We use custom build scripts for our doc and swt bundles. See this patch on bug 266488 for an example. Basically, you're adding the following to the init target of your build.xml.

    <condition property="p2.publish.parts" value="true">
    <istrue value="${p2.gathering}">
    </istrue>
    </condition>

    and then this additional target.

    <target name="publish.bin.parts" depends="init" if="p2.publish.parts">
    <antcall target="gather.bin.parts">
    </antcall></target></pre><param name="destination.temp.folder" value="${build.result.folder}">

    <eclipse.gatherbundle metadatarepository="${p2.build.repo}" artifactrepository="${p2.build.repo}" buildresultfolder="${build.result.folder}" targetfolder="${build.result.folder}/${full.name}">
    </eclipse.gatherbundle>
    </target>

    After these steps are completed, the build proceeds as normal to provision the zips from the repository using the p2 director.

    Further References

    If you're contemplating switching to the publisher, these documents are useful references. Our build is quite complicated with it's 16 platforms and associated launchers, the transition for most teams will be much simpler than our scenario.

    The meta bug for the changes to reorganize the Eclipse and Equinox build to use the publisher.

    Wiki article on the p2 publisher

    Wiki article on customizing metadata

    Ian's blog entries about the publisher 1 and 2.

    Read more...

      © Blogger template Simple n' Sweet by Ourblogtemplates.com 2009

    Back to TOP