EclipseCon Exercise Tuesday: 310K and counting

I expected the crowd to be diminished today since the shirts were distributed yesterday and a lot of people were celebrating late into the night. No. Eclipse community, you delivered. Look at this picture.

About the same number of people as on Monday. Today's prize category was a "Best Running Shoes". Patrick Paulin won with his Vibram Five Fingers.

I'm so happy that people continue to get up and running for EclipseCon exercise.

Tomorrow's theme is "Best Running Shirt". Wear an old shirt, or a new shirt recently acquired at the conference for the chance to win an Eclipse jacket courtesy of the foundation.

If you're wondering where the 310K came from, it's the approximate number of people who attended multiplied by 5K a person.

See you tomorrow morning at 7am in the Hyatt lobby!

Read more...

Eclipse Run Time was a Fun Time

We had 30+ people turn out for EclipseCon exercise this morning.  This is a new EclipseCon Exercise attendance record!  Way to go!

EclipseSource generously provided technical t-shirts. Since so many people showed up, they all were distributed the first day.

Today's contest category for an Eclipse jacket was "Runner from farthest away".  The winning runner was from India.   Congratulations! Thanks to the Eclipse Foundation for supporting this event.

We went for between a 5-6K run on the bike trail behind the conference centre.  There were lots of different pace groups.  Some first time runners.  Great job everyone!  If you had a hard time with the run, remember that pace makes a huge difference.  So slow it down a bit if you didn't feel good when you got back to the conference centre. It's just a fun run not a race.    No need to feel nauseous before breakfast :-)

When we returned to the Hyatt, the staff had water bottles and towels for everyone.

Thanks to those who had run this route last year and led the way. I  look forward to seeing you tomorrow morning.

Read more...

EclipseCon Build Panel: It's not rocket science, it's release engineering

There will be a build panel next Tuesday afternoon at EclipseCon. I'll be one of the panellists and hope that it will be a constructive discussion.  Have a question about release engineering but were afraid to ask? This is your chance!

Photo credit clix at www.sxc.hu

We have gifts from our new friends at the Hudson project to give away at the build panel.   Hudson is an open source project for a continuous integration server that's in widespread use, including on build.eclipse.org.  Like eclipse, it has a plugin architecture that allows you to add functionality incrementally and the community contributes plugins to make Hudson better.   The first thirty people who ask the build panel a question will receive a Hudson sticker to affix to their laptop. 

.

The Hudson committers are also having a hackathon this weekend, if you happen to be in the Santa Clara area this Friday and Saturday.

Read more...

Ten things I wish I'd known as young committer

On Monday afternoon, I'm giving a short talk at EclipseCon entitled Eclipse Top Ten. My presentation will discuss the important lessons I've learned while interacting with the Eclipse community. Release engineers build SDKs but along the way I've learned about building community as well.

I prepared the slides with the help of the excellent book Presentation Zen that that my Eclipse colleague Lars Vogel recommended.  One of the suggestions in the book is to present interesting pictures with as little text as possible.  This is to encourage the audience to listen and interact with the speaker, instead of spending time reading text heavy slides.

Photo credit Gábor Shajda http://www.sxc.hu/photo/21223

If you look closely, you will notice the billiard balls are scratched, and a bit worse for wear.  I think that's fitting, because as a long time Eclipse committers, some of us are in a similar condition. The steady stream of bugzillas that has rained down upon us for years does leave some scar tissue:-)

For the draft of this talk, I wrote down a list of 50 different things I've learned working in our open source community.  I had to do a lot work to condense it down to ten.  Given the material I had, I could have had a completely different talk. The end result is a candid and humourous look at the Eclipse community, and how you can get involved to make it better.

What have you learned from open source?  The talk is from my perspective, but I'm sure others have much to contribute to this discussion. I look forward to speaking with you at EclipseCon.

Read more...

EclipseCon Exercise: Vote for prize categories

There will be Eclipse swag available at EclipseCon exercise. Vote now on the criteria to win. You can vote for four difference categories, one for each day.





Next week is EclipseCon. Interesting talks.  New technology.  Awards.  Surprises.  Lego.  Robots. Chats with old friends and making new ones.  What's the perfect start to such a great day?  Running.  Here are the steps to ensure your bun(dle)s are running at EclipseCon.

1) Sign up on the EclipseCon exercise wiki so we know you're coming.  So far there are 30+ people signed up - fantastic - looking forward to seeing you there!

2) Pack your sneakers, running shoes or trainers. Whatever you call them, make sure they end up in your suitcase. Shorts and a shirt too.  As Chris mentioned, there will be EclipseCon exercise shirts for the first 30 runners.

  
3) Get out of bed for and meet in the Hyatt Lobby at 7am.

The last one is really key.  Many of my running friends say "Getting out of bed is the hardest part of running". There's some truth to that. The good news is that it's California.  Sunny and warm. Palm trees.   Look at this forecast.

Looking forward to seeing you at EclipseCon Exercise!

Read more...

Confessions for the EclipseCon Speaker

I recently read a very funny and informative book about the pitfalls to avoid while speaking in public.  Confessions of a Public Speaker by Scott Berkun.  If you need a great book to read on the plane to EclipseCon, I highly recommend it.

Some of the most important points in the book

• Practice, practice practice your talk. This allows you to optimize the organization of your talk and smooth out your delivery.
• If you are speaking to a small number of people in a large room, try to convince them to move to the front of the room. People who are sitting together will change the dynamic of the talk.  Speaking to a group of people who are scattered around a large room is not optimal.
• Go over the agenda in your introduction. For instance, say "I'm going to talk about five points at three minutes a pop. The final five minutes will be for you to ask questions". This will give people an idea of what to expect.
• Interact with your audience.  Ask trivia questions, for a show of hands or ask the audience to solve a problem.
• I notice that a lot of people are working during talks instead of listening to the talk. My employer paid money to send me to a conference and I'm investing my time.  So I make a point listen instead of doing my regular job even though it will be painful to catch up with work later on.  In the book, he suggests asking people to close their laptops and if they are bored after five minutes, go back to surfing the web.  Again, the audience it gets the audience engaged in your talk and your show that your care about your audience.  Of course, if they are blogging or tweeting about how interesting your talk is, they can keep the laptop open :-)

The final chapter of the book describes some of the worst things that have happened while people giving a presentation. There's fire, water, and SWAT teams.  Hilarious. I don't give that many presentations in my day to day job of building bundles, so I found this book was a great resource . With that note, I must get back to writing slides, practicing my talks, and fixing bugs.

Read more...

Better builds with Hudson, hardware and help

Some might say that the build is one of the engines drive a project.  The Eclipse and Equinox build needs a tune up.

(Image © Paul Gorbould, http://www.flickr.com/photos/gorbould/3531940727/in/set-72157607916475025/, licensed under Creative Commons by-nc-sa 2.0)

A simplified summary of our build process today is as follows.  (Many of these processes occur in parallel.)

1. Checkout code from eclipse.org to an IBM build server.
2. Generate build scripts, compile code and create a master feature of all the bundles used in the build.
3. Copy master feature to eclipse.org for signing, copy back to IBM server when complete.
4. Run the p2 director to provision products and use repository tooling to slice out zipped repositories.
5. Run JUnit and performance tests.

Here are some of the fundamental ways this process can be improved:


Hudson


Problem: The build process takes too long to complete code checkout, compilation, signing and packaging. It's also also too monolithic.

Solution: Take advantage of the local access to the eclipse.org filesystem by running the build on the Hudson install at at the foundation.  Now that we have hardware donations there will be new Hudson slaves for more build cycles.  Also, breaking the build up into smaller builds and chaining them together will let us identify problems earlier. See bug 302436 for details. We run test builds on Hudson today and they work very well.


Hardware

Problem:  There aren't enough test machines to run our tests in a reasonable timeframe.  Committers aren't able to rerun the tests on the same hardware that was used in the build.

Solution: New test  hardware at the Eclipse foundation is a start. We'll probably need more but it's a good beginning. Thank you to all the companies who have donated hardware to the foundation recently.


Help

Problem

Today we run our JUnit tests  on Windows machines by invoking them via rsh.  This allows us to manipulate the display while running ui tests.

Solution

This is where you come in.

I'm not sure how to do this on Hudson.  Sonatype has a series of articles on running tests in a multiple OS environment and they state that this is still a problem for them.  If anyone has any pointers to articles on how to do this it would be appreciated.   Please update bug 305213 with your suggestions.

Also, we need some rack mounted Macs to run JUnit tests on this important platform.  So once we get the new hardware integrated, some more hardware donations would be welcome :-)

Read more...

AFOL at EclipseCon

I read an interesting book over the weekend.  Groundswell  is a book from the Harvard Business Press written by Charlene Li and Josh Bernoff.  It discusses how to use social media to open up communication channels with your customers. For instance, how to harness the wisdom of the crowds to design better products and improve technical support.  The book describes the approaches of several companies and shows the ROI calculations for the investment in social media.

One of the companies that was profiled was the Lego Group.   I learned that Lego is the sixth largest toy manufacturer in the world. Over a billion dollar a year business.  And somewhere between 5-10% of their sales go to AFOLs.  Adult fans of Lego.  In fact, they have an executive in charge of marketing to AFOLs because it is such a significant market.  AFOLs hang out in an online community called LUGNET.  The Lego Group's approach to social media is to have Lego Ambassadors in the AFOL community listen to the needs of the people who view Lego as a building material, not just a toy. And these amabasadors are paid for their services in Lego bricks, not money.  Pretty cool.

So what does this have to do with Eclipse?  EclipseCon 2010 is having a e4-Rover Mars challenge where you have the chance to use Eclipse technology to maneuver Lego robots.  Go to EclipseCon and add four letters to the end of your name without paying for tuition.  AFOL at EclipseCon!

Read more...

Community starts with conversation

This week, I've been writing slides for our p2 tutorial at EclipseCon. As part of this effort, I wanted to show the companies that were shipping products based on p2 or that used p2 in their internal products. I asked for suggestions on the p2-dev list. Here's a summary of the responses.

Pretty interesting. It's a testament to the Equinox team and the larger community that so many companies are building products based on p2. If your company, belongs on the slide, let me know.

Pascal has arranged a p2 BOF at EclipseCON. It should make for interesting conversation.

Read more...

Tag after Release

Last week Eclipse and Equinox 3.5.2 was released as part of the Galileo SR2 release.  So far, so good!    A big thanks goes out to David Williams for keeping all the projects on the release train in line and ready for SR2. Also, kudos to the webmasters for ensuring that the mirrors were ready for the release.

The work of the release engineer isn't done after the release bits are available.  I tag all the projects in the Eclipse and Equinox project with as R3_5_2, as well as our our map files and the builder projects.  This ensures that if required, exactly the same build can be reproduced.  In addition, it's useful for developers to be able to compare against a tag for a previous release, or to branch from a tag.

Our source resides in the /cvsroot/eclipse and /cvsroot/rt repositories. The steps I take to do this are as follows:

1. Start eclipse with a clean workspace.
2. Check out the vM2010211-1343 versions of org.eclipse.releng and the builder projects.  Every time we run a build, the org.eclipse.releng and builder projects are retagged with with the build id. M2010211-1343 is the build id of the final 3.5.2 build so I retag these projects as R3_5_2.



3. I remove the orbit map from the releng project in my workspace. There are prebuilt bundles fetched from the Orbit repository so we don't need to tag them. 
4. Replace  :pserver:anonymous with :extssh:kmoir in the remaining map files in my workspace.  I have commit rights on all the eclipse and equinox projects for this very purpose.
5. Change the connection timeout on the Team CVS client to one larger than the default.  Otherwise, your CVS connection will timeout while tagging all the projects.
6. Select the map files in my workspace.  Right click and select Team and the Tag Map File Projects option.


7.Tag as R3_5_2.


8.  Once all the tagging has completed after several hours, I will check out all the projects from the map files and compare with R3_5_2 to ensure that there aren't any files missing the tag.
9.  Install the releng tools to use the "Tag map file projects" functionality. This allows me to tag the versions of projects defined in your map files as another version without checking them out. Very useful.


Questions?

• Q.Why don't you do this tagging as part of the build process?
• A.Eclipse and Equinox is a large project with a lot of source  - over 300 bundles and more than 30 features.  Tagging this as part of the the build each time takes a few hours and is often has CVS timeouts.  We don't need that, the build is slow enough as it is today :-) That being said, if you have a smaller project of just a few bundles, tagging the bundles with the build id could be useful.

Read more...

Private Emails Hide the Details

Recently, I've been receiving a lot of emails directly to my IBM address regarding bugs, features, releng questions, or building Eclipse for platform XYZ from people in the community.  These people are new to Eclipse and therefore may be unaware of the proper channels to ask questions.  I like to gently remind people ask a question on the forums or open a bug. The internet is forever, and Eclipse is no exception.  If you open a bug, others with the same issue will be able to search and find a solution.  Private emails hide knowledge that's useful to the larger community. As committers, we're very detailed oriented people.

Today, I responded to the latest request as follows:

At eclipse,
We're open source,
Private emails,
hide the details,

Asking in the open
Allows others to see,
the issue described,
the remedy prescribed,
for posterity

Found a problem?
Want to request a feature?
Please, I will ask,
open a bug for the task
https://bugs.eclipse.org/bugs



I look forward to seeing their bug.

Read more...

Trim your builds with p2 repository tools

The Eclipse Project has the unusual distinction of consuming the most download space of any project at eclipse.org.   Therefore, I like bugs with a request to "get rid of stuff" versus "add more stuff".  Recently, I fixed a bug to remove the packed jars from our zipped repositories that are available for download.

For example, we provide RCP binary and RCP source zipped repositories on our download page. Last week, these zips included both packed (*.jar.pack.gz) and unpacked (*.jar) bundles.

We don't need to include both types of jars in these downloads because most people will install the bundles locally from the zip.   The packed jars are included by default because when I run the mirror tool against the full build repository to create the slice, it already includes the packed jars.  You can use the remove.iu tool to query the repository for a list of all bundles and remove the associated packed jars.

<p2.remove.iu>
        <repository location="file://${yourRepo}">
        <iu artifacts="(format=packed)" query="">
</iu>

The remove.iu tool will remove the packed bundles from the repository and update the artifacts.jar accordingly.

As I have written before, repository tools are very useful for creating subcomponents of existing respositories.

Remove IU

Slicing and Dicing the p2 way

Want to learn more about using p2 to assemble products out of pre-existing components? Plan on attending the From build to assembly to deployment: Using p2 to facilitate agile software development tutorial that Ian Bull and I will be presenting at EclipseCon.

Read more...

Committer Collector Cards

I have a new idea for Eclipse swag. Committer collector cards. Why? It would be a fun way to get to know more about members of the Eclipse community. Like hockey cards.  But we're better at debugging, and have more teeth.

Here are some ideas for the sorts of information that this could display.

• Name
• Picture
• Commit rights
• ohloh commit history
• LinkedIn url
• Special power

Special power? Yes, all committers have special powers. As a release engineer, my special power is pain tolerance. Much like Wolverine. Yes, it's true.

I can see this being a popular activity at EclipseCon. Trade a Moir for a Merks, a Bokowski for a Bull. What do you win if you collect a whole project? Bugzilla bingo.  You get a bug assigned to you so you can earn your own commit rights.  Diversify the community. It's all very win-win.

Since I'm in the give away free software business, not the marketing swag business, I thought I'd suggest this idea for any enterprising people out there. It could be a branding activity a company who'd like to sponsor such a unique piece of swag. Go for it!

Read more...

EclipseCon Exercise: Commit to Running

EclipseCon 2010 looks like it will have a fantastic program this year with many interesting talks.  For instance, there will be talk about Lego Mindstorms and Eclipse. How much do I love Lego?  Let me show you my keychain.

There will be also be a talk on the Eclipse at NASA which looks really interesting.  I mean, how often is it that the software you build is used to monitor robots travelling across another planet? Mr. Platform Releng is very talented at ordering stuff on the internet.  One day I arrived home and found a Lego Mars Rover being assembled in our living room.  Life is unpredictable.  From lego.com.

Outside the formal program, there will be EclipseCon Exercise early in the morning.  You're invited to sign up for a  run every morning before the talks begin on the wiki. Don't hesitate to put your name down if you've just started running, there will be several pace groups.   Thanks to the work of Darin Swanson, who organized EclipseCon Exercise in previous years, we already have a map.

The generous Eclipse Foundation has also agreed  to provide prizes for the EclipseCon Exercise participants.  There will be Eclipse shirts and running related gifts.  What will you win a prize for?  Well, that's up to you.  Again, you can make suggestions on the wiki for prize categories and I'll set up a vote for the community in early March.   Also, if you work at a company that's interested in sponsoring EclipseCon exercise, please contact Donald Smith.

Running before the talks in the morning is a great way to clear your mind for the day. It's also a way to get rid the jitters before you give a talk or tutorial. Most importantly, it gives you a chance to meet and talk to people in a different context, and get to know them better as a person, not just an Eclipse professional. Learn about what they do outside of work,  their communities, and discover common interests.

So consider committing to EclipseCon Exercise.  Sign up today and remember to pack your running shoes!

Sign up sheet for EclipseCon Exercise 2010

Read more...

p2 tutorial RFC

The EclipseCon program committee is sending out acceptance notifications after completing the difficult task of selecting the talks that will be presented in March. Thanks for all your hard work - I'm sure it will be an amazing conference!

Ian and I have been selected to give a tutorial entitled "Exploring p2". The best approach to learning is spend time actually getting your hands dirty. The wisdom of xkcd


Originally, the focus of this tutorial was to be on building with p2. However, given time constraints in the build category, the program committee asked us to provide a broader focus on p2, not just building with it.

This is where you come in. If you are considering attending this tutorial, let us know what topics you'd like us to cover on the wiki or leave a comment. We really value your input and so we can tailor the tutorial to you. Also, it will be interesting to learn the p2 use cases in the community. A tutorial is a two way street, and we have just as much opportunity to learn about how you are using the software we build, as we have the chance to share our experiences with you.

Keep in mind that there are other talks in this topic that have been approved. For instance, Pascal will be conducting a talk on p2 APIs. Today, our fantastic Equinox committers are working on merging the newly minted p2 APIs into HEAD for 3.6M5.

Looking forward to seeing you at EclipseCon 2010!

Read more...

Less email, more info

I subscribe to many eclipse mailing lists. This means I get a lot of email that's sorted by various filters so my inbox doesn't explode with all the traffic.

However, there are some Eclipse projects that I'm interested in, but not going to participate actively on the mailing list. This reminds me of the pig and chicken analogy. I'm somewhat involved in the other projects, but I'm committed to others. From implementing Scrum



Luckily, each eclipse.org mailing list also has a RSS feed.


For instance, this time of the year, it's interesting to watch the EclipseCon Program Committee feed. Lots of discussions regarding which talks will get the final +1 for the 2010 program.

Or you may have an interest in what's happening in e4, but don't want all the email that this very active project generates.

Read more...

And the winner is.....

Eric Rizzo has won the December splash screen contest with this submission.


Congratulations! Thanks to all the contestants for submitting such creative artwork. The winning design will start appearing in tonight's build (N20091214-2000) and has been tagged for the integration build.

Read more...

Vote now for your December Splash

Next week, we'll have the our final integration build before the end of 2009. What better way to end the year than with a custom splash screen? As usual, the very creative Eclipse community has stood up to the challenge with eight different designs for your consideration. Please vote here:



Direct link to poll.

The poll closes on Monday at 5 pm EST. I'll announce the winner on Monday and commit the winning splash. Good luck to all the contest entrants!

This contest is closing a bit earlier than expected because we shutdown our builds over the holiday season. Thus, we need to get the December splash into the integration build on December 15, 2009.

Read more...

Eclipse and Equinox 3.6M4 now available

Ho ho ho.
Ho ho ho.
We are Eclipse's elves.

We are Eclipse's elves,
Filling mirrors with M4,
Another milestone out the door.

Oh, we are Eclipse's elves,
We work hard all day,
Debug and refactor is our play,
Bugs we stamp out,
Hurray, the community shouts!

We are Eclipse's elves,
We ship on time each year,
We don't like to brag,
Just sync, commit and tag,
Install new bundles without lag.

We all know who's been good,
Resolved the bugs you should,
Eclipse is you,
Grab a bug and become a committer too!

We are Eclipse's elves.
Ho ho ho. Ho ho ho.
We fix the code ourselves.
Ho Ho!

The Eclipse and Equinox team are happy to announce the release of the Eclipse 3.6M4 milestone.

New and noteworthy

Update to 3.6M4 by adding this site to your list of available sites
http://download.eclipse.org/eclipse/updates/3.6milestones

Equinox Downloads

Eclipse Downloads

Happy Holidays and New Year from the Eclipse and Equinox projects!

Read more...

December splash screen contest: Prizes++

Last week, I announced the December splash screen contest. Since then, we've had some entries from Olivier Thomann. Very creative. Look at this


Or this.


We need more people to submit splash screens to to the contest. Here's the bug where you can submit your entry. It doesn't have to be a holiday theme, it can be whatever you'd like.

As an added bonus, the generous folks at the Eclipse foundation (hi Ian and Lynn) have offered to donate some eclipse bling to the prize pot. So the total prize package for the December splash screen contest will be

1. Friends of Eclipse membership
2. Eclipse t-shirt
3. $50 donation to Eclipse in your name

An Eclipse t-shirt? Does it give you special powers like the famous "three wolves t-shirt"?




I guess you'll have to enter the contest to find out. Fire up your favourite graphics editor and submit a splash screen to bug 296918

Read more...

A tale of CVS, NFS and Swordplay

Earlier this year, the time that it took our builds to check out code from CVS had slowed to a crawl. Last year, our integration builds that started by 8am were available by noon. However, during the 3.6 cycle, most builds weren't ready until 2pm or even 4pm. Very painful, especially for our European committer friends. Not only were the builds slow, but it took a long time for us to commit code, and timeouts were frequent.

Denis and his merry band of webmasters determined that the source of the problem was twofold.

1) NFS configuration issues causing high CPU load on the servers. (Bug 288293)
2) Anonymous pserver users were holding the cvs lock files for a long time (Bug 293355)

Earlier this week, Denis fixed the NFS configuration issues. This had reduced the time it takes to check out our code significantly. Also, I haven't seen any CVS timeouts lately. Today, our 8am integration build was ready at noon. A big thank you to the webmasters! We are much more productive thanks to your work. Less swordplay while waiting for builds, and more time to fix bugs and implement new features. Again from xkcd.



Really, who doesn't love xkcd? It's so good.

Denis will change the anonymous pserver access to a mirrored copy of the CVS repository on December 11. He sent notes to the committers list with the details. The means that committers won't have to contend with anonymous users for cvs lock files if they check out code via ssh. Psever will be available on a separate file system, but sychronized with the live copy.

In preparation for this change, I modified our build scripts to check out our code using ssh, instead of anonymous pserver. However, our map files remain untouched so anyone can check out our code via pserver. The build scripts modify the map files after they have been checked out. We won't have to deal with file contention for CVS lock files. And ssh connections to eclipse.org have a higher QoS than pserver. Good stuff.

Most teams won't have to make any changes because they run their builds on build.eclipse.org. These users will still pull from the live copy of the data via pserver. And the mirrored copy will be up to date. I'm just trying to do everything I can to avoid infrastructure issues that cause build failures.

See bug 294900 for the platform releng changes.

Read more...

Make a splash in December

The Movember splash was a lot of fun. I'd like to open up submissions for a December splash screen. It will be released after M4. This contest is open from December 4-15. At that time, I'll start a poll and ask the community to vote on the splash screen. I'll commit the winning submission. What will you win? Eyeballs on your art. Respect of the community. Warm and fuzzy feelings. And I'll donate $50 to friends of Eclipse in your name.



The existing splash screen is in org.eclipse.platform in /cvsroot/eclipse. The file is splash.bmp. The photoshop files are in 3_6SplashHeliosPsd.zip in the same project. Attach your entries to bug 296918

Nothing offensive please people. Go for it.

Read more...

Good Eclipse Swag

I found this mug in the kitchen at work today. It has Eclipse key bindings on front


and back.



Now, all I need is a sandwich to go with this and I'd be set. From xkcd

Swag is rarely both useful and aesthetically appealing. This is both! Thanks itemis!

Read more...

The end of a three year relationship: Goodbye bug 153429

This past week, the Eclipse and Equinox team released changes to support the use the JUnit 4 bundle in the Eclipse Test framework in bug 153429. This bug had over 75 people on the cc list and 40 votes to fix it. It was in my bucket for three years. It's finally fixed so now we and (you) can run your automated tests on JUnit 4.

The changes that we made to fix this bug are described in the wiki. John also sent a message to the cross project list.

I have worked on many bugs during my tenure at Eclipse. This was a tough one. It was very complex, and required a tremendous amount of testing. It was also very much a team effort.

I'd like to especially thank DJ Houghton for all his patches for the test harness and and test bundles. Also, DJ added the org.junit 4.7.0 bundle to Orbit which was very helpful. John Arthorne for fixing the p2 tests and all the good advice during this process. Dani Megert and Markus Keller for their changes to JDT. Darin Wright for his advice regarding PDE. Curtis Windatt for his changes to the pde ui tests. Andrew Niefer for the changes to the pde build tests. Other committers made minor changes to their test bundles to accommodate this change. Thank you!

Goodbye bug 153429. I won't miss you.

Read more...

Committer Reps, we need your help

Hi Boris, Chris, Doug and Ed

Hey, how's it going. I hope that you are all well.

Like a mall on Black Friday, around here it's peak Eclipse committing season. Lots of bugs to fix. Lots of builds to run. As you know, builds are quite a pain point at Eclipse. I'm excited about the possibilities in the b3 project to make things better. Building software is complex, and Eclipse is no exception. In additional to tooling to make builds easier, we need hardware to make builds faster. Our build today takes about five hours to complete, and an additional 6.5 hours for the tests to complete. Really, it's not pretty.

Many eclipse projects run their builds on Hudson on build.eclipse.org. Hudson is fantastic because there's a rich set of plugins that you can use to enhance the functionality of your build. Also, since this server has local access to the eclipse.org filesystem for code checkouts, you're less prone to network errors which can break the build. It also has ldap integration with your commiter login so you can restrict your build configuration to the commiters on your project. In theory, if you need more build machines to run your build - you can use the Amazon EC2 plugin to provision more machines in the cloud, or other plugins to start builds on local slave machines. Good stuff.

However, one of the things that the foundation doesn't provide today is test machines. This means that we can't run our build at the Eclipse foundation. The Eclipse Project builds zips for 14 different platforms. We run JUnit tests on three native platforms: Windows, Linux and Mac. They are the most commonly downloaded platforms. We need test machines to ensure that we don't have any bugs specific to a platform. Why do our tests take so long? We have 54,000 JUnit tests. You don't produce quality software by skimping on tests.

This isn't just about the Eclipse and Equinox projects. This could be very useful for other projects, for instance, the XSL tools project has expressed interest in using test servers. In addition, these machines could be used as slaves machines for running the build in the event that the main Hudson server is too busy. If we had enough machines, we could run more tests in parallel and reduce the time it takes our build to complete. This would be a big win for the community and our committers.

One thing I investigated in running tests in the cloud. However, most cloud services don't have provide a way to run tests on Macs and we need to make sure that our Mac users are happy. If there is a way, I'd appreciate a link. In addition, one of the advantages of running tests on machines local to the eclipse.org filesystem is that we don't spend time copying stuff back and forth across the network. It's just there.

So, what I'm asking from you is at the next board meeting, please bring up the issue of funding test infrastructure at the Eclipse foundation. It might be even be an advertising opportunity for one of the member companies if they donated hardware. Other companies could donate money to pay for the additional rack space. I don't know right now what the final technical solution will be or what it will cost. All I'm asking right now is to start the conversation.

For many years, the Eclipse project has been criticized for not being open enough. Having our build process fully on eclipse.org servers would make us more open. It would also allow any of the Eclipse and Equinox project committers, regardless of company affiliation, to initiate a build. It we had enough hardware, our build could be faster and we could spend less time waiting for builds, and more time fixing bugs the builds reveal.

Please bring this issue up and the next board meeting.

sincerely,
Kim

P.S. Right now we have the following test machines and our tests take about 6-8 hours to complete. Obviously, if we had more machines running tests in parallel, the build would take less time.
1) JUnit: 2 linux, 2 windows, 1 mac, 1 test cvs server,
2) Performance: 2 windows, 2 linux, 1 database server

Read more...

Happy Birthday Eclipse...now some pictures from your youth

Saturday November 7th, 2009 is Eclipse's 8th birthday. Eight years ago the first Eclipse downloads were publicly available along with the source code.

Here's a picture of the first www.eclipse.org. The website was slashdotted shortly after the "Eclipse is open source" announcement was made. The hard drive had to be replaced, thus the missing cover. This machine was replaced by a real server a few weeks later. A couple of years later, this was replaced by faster and more fault tolerant hardware managed by our most excellent webmasters.



I went through my desk the other day, and found some emails of the original requirements for the eclipse infrastructure, project structure and commit rights. A fast server has 512MB RAM and 20GB of disk space...really? Okay, I feel old.



Bugzilla was there from the start.



Some of the first mailing lists. A few disappeared but even more new ones were created. Today, we have eclipse on twitter, forums, conferences, marketplace, blogs and the list goes on. Pretty impressive!



In 2001, the linux kernel still fit on a floppy disk. It was useful to have a boot disk in the event that the boot partition became corrupted and the machine wouldn't boot. Much easier than booting from a rescue cd and mounting all the partitions by hand. Especially at 4am. Here's a picture for those who've never worked with a diskette (*ahem* co-ops :).




Initial project structure. Today we have so much much more.


The first commit rights - hello PDE family! Today we have an amazing diversity of committers from around the world and many companies. Eclipse also has friends, but would always like more....who doesn't?




In reality, the past doesn't matter that much, and we need to focus on the future. Sometimes I'm cynical about Eclipse, because as a long time committer, I notice that many people like to play (consume), but fewer want to pay (contribute). And to some degree we make it difficult to contribute without a significant investment of time and a steep learning curve. But most days, I'm absolutely amazed by the work that we as a community can do together, when smart, passionate people strive toward a common goal. Happy Birthday Eclipse!

Read more...

Now I see IU, now IU don't

Eclipse 3.6M3 went out the door over the weekend, along with a lot of Halloween candy.



Sometimes, you can have too much Halloween candy. And sometimes, you can have too many IUs in your p2 repo. Don't believe me - just look at this repo with bogus bundles - scary!





Both scenarios can cause your friendly neighbourhood release engineer pain. This is unusual because we're a very pain tolerant people. To alleviate the suffering, the p2 team added an Ant task in 3.6M3 that allows you to remove bundles from your repo. As much as I love spending quality time at the command line modifying metadata, Ant tasks that automate tedious jobs are even better.

The p2.remove.iu task will remove both the metadata and the bundle from the repository for a specified IU. For example, if you had bogus packed com.ibm.icu.* bundles in your repo, this task would remove them.

<p2.remove.iu>
<repository location="file://${reposource}" />
<iu id="com.ibm.icu" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base" artifacts="(format=packed)" />
<iu id="com.ibm.icu.source" artifacts="(format=packed)" />
<iu id="com.ibm.icu.base.source" artifacts="(format=packed)" />
</p2.remove.iu>

This task is useful if you'd like to remove some built time bundles from your repo. Or just correct a mistake after a release. It happens. In any case, it's all good. Almost as tasty as chocolate.




Related bugs

Support excluding bundles when running p2.process.artifacts task
p2.remove.iu task should have an option to specify to remove packed file only
ICU jars at Eclipse 3.5 update site have size of 0

Read more...

Hudson is sweet, now the build can tweet

I recently set up a new Hudson build so the Equinox team could test their changes in a branch instead of releasing everything to HEAD. Hudson, like Eclipse, has a rich variety of plugins that can expand the functionality of your build. Hudson has a twitter plugin and thus Eclipse (test) builds tweet here

http://twitter.com/eclipsebuilds

Right now, there are only a few messages about the test builds I started, and stopped when I saw that build was proceeding as normal. In any case, it's pretty cool.

Hudson also has plugins to use additional slave machines on the network to run the build on a faster machine if the current build machine is too busy. It also has plugins to provision Amazon EC2 images and run the build on a slave in the cloud. We currently run about 54,000 JUnit test per platform multipled by five test machines during each build. We run tests on Windows, Mac and Linux machines. But the tests take a long time to complete simply because of the number of tests we run and the lack of additional hardware to run more tests in parallel. It would be fantastic to be able to run the tests on parallel on many machines and finish in fraction of the time they take today. That would allow us to reduce the number of breaking build issues. But of course, this will take significant testing to implement. Sounds like fun!

References
Run JUnit tests in parallel
Hudson build for the p2 branch

Read more...

Test bundles switch from runnable to repo

Earlier this week, I changed the build so that the test bundles are provided in a zipped p2 repository format. A p2 repository looks something like this

...
artifacts.jar
content.jar
features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV.jar
plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735.jar
plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735.jar
binary/
...

The build will install the appropriate test bundles in the eclipse install being tested using the p2 director. Previously, our test bundles were assembled in the runnable format like this

...
eclipse/plugins/org.eclipse.jdt.debug.tests_3.1.100.N20091019-1735/...
eclipse/plugins/org.eclipse.ua.tests_3.3.300.N20091019-1735/..
eclipse/features/org.eclipse.sdk.tests_3.6.0.N20091019-1735-9J9fG6sFIKSg8a7j2ZwWY0UAV4BV
...

and unzipped into the dropins folder. However, the dropins folder is really for legacy purposes. Disadvantages of using the dropins folder include

• It's to help those applications that expect that dropping a bunch of bundles into an install will work (!).
  
• All the bundles in the dropins folder are treated as optional bundles.
• You can't update the bundles in the dropins folder by using the UI, you are responsible for provisioning them.
  

The content of the org.eclipse.test framework bundle hasn't changed. However, if you're rerunning the Eclipse and Equinox projects' tests JUnit tests against your product, you'll need to install them into the eclipse your are testing using the p2 director. Unzipping the our JUnit bundles into the dropins folder won't work. Alternatively, you could use the reporunnable task to transform the repository into the old runnable format.

Why make this change? These days, I would expect that most users are installing bundles via a repository such as Galileo or one offered by a product team. Unzipping files is so Eclipse 1.0. So this will allow our tests to replicate the environment that reflects the reality of our users. As well, as I mentioned in my earlier post, we really need to run our tests in parallel on more machines to speed up the build process. Having the test bundles available in a repository, eventually in a shared location, is a step toward that goal.


References

Test bundles should be packaged as a repo

Supported dropins formats

repo2runnable task

Note: The Eclipse and Equinox projects provide their JUnit test bundles in the eclipse-Automated-Tests-${buildId}.zip in the eclipse-testing/eclipse-junit-tests-${buildId}.zip file that is available with every build.

Read more...