tag:blogger.com,1999:blog-25023280.post614762324568778893..comments2023-09-10T06:25:56.441-07:00Comments on Releng of the Nerds: Challenges in Release EngineeringKim Moirhttp://www.blogger.com/profile/14700841495895160750noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-25023280.post-32994523558086749702012-06-24T17:45:34.824-07:002012-06-24T17:45:34.824-07:00reprogrammer: Speaking from my experience as an Ec...reprogrammer: Speaking from my experience as an Eclipse release engineer, yes the dependencies are expressed in a machine readable form to compile the source code into OSGi bundles. However, there was not any machine readable output that could be parsed to optimize the signing, testing or performance tests automatically. This was just an Ant script that called these modules manually. So no magic there.<br /><br />Ian: Perhaps I should have used the word releases there to be more clear. However, builds are not free. Intermittent problems can occur - such as hung slaves, network issues, hardware and power failures require human intervention. So in theory, a build should always succeed and has little incremental cost assuming that your infrastructure can handle the additional load. However, in practice, this is not always the case :-)Kim Moirhttps://www.blogger.com/profile/14700841495895160750noreply@blogger.comtag:blogger.com,1999:blog-25023280.post-20158631958311700222012-06-24T15:39:38.172-07:002012-06-24T15:39:38.172-07:00"5. Frequent releases generate faster user fe...<i>"5. Frequent releases generate faster user feedback on new features. However, additional releases are expensive for the release engineering, quality assurance and release management teams. Each additional release eats time, both people and machine. <br> <br>Making the community aware that builds are not free is an ongoing communication exercise."</i><br /><br />I've never committed code to Mozilla, but I'm not sure I understand your point here.<br /><br />I understand that RELEASES need testing etc and therefore use people time, but surely additional BUILDS only use up extra machine time? Surely splitting a commit across two builds could even reduce people time, by making it more obvious which change caused the regression.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-25023280.post-79631218758387718452012-06-24T13:57:12.210-07:002012-06-24T13:57:12.210-07:00Kim Moir: Just trying to understand, do you mean t...Kim Moir: Just trying to understand, do you mean that it's difficult to auto-parallelize the build because not all dependencies are explicitly described in a form that is easy to analyze by a machine? How do you currently improve the parallelism of your build system?reprogrammerhttps://www.blogger.com/profile/13738356713636985774noreply@blogger.comtag:blogger.com,1999:blog-25023280.post-86918628482687494952012-06-24T13:49:32.115-07:002012-06-24T13:49:32.115-07:00Thanks Greg, that's a good point I missed. Th...Thanks Greg, that's a good point I missed. There aren't any formal training programs for this type of work although several Mozilla release engineers who have come from the excellent program at Seneca College.<br /><br />reprogrammer: With respect to parallelization, yes definitely analyzing the dependency graph is useful. I just find it's always something in the back of your mind, how can you parallelize more tasks to reduce total build time? Not just compilation, sigining, packaging etc.Kim Moirhttps://www.blogger.com/profile/14700841495895160750noreply@blogger.comtag:blogger.com,1999:blog-25023280.post-27862379323526878302012-06-24T13:25:54.252-07:002012-06-24T13:25:54.252-07:00What makes auto-parallelizing the build challengin...What makes auto-parallelizing the build challenging? Isn't it possible to automatically parallelize the build by analyzing the dependency graph of the build system?reprogrammerhttps://www.blogger.com/profile/13738356713636985774noreply@blogger.comtag:blogger.com,1999:blog-25023280.post-10576987063553038432012-06-24T13:07:57.831-07:002012-06-24T13:07:57.831-07:00Whoops, copy-and-paste error: the second link in t...Whoops, copy-and-paste error: the second link in the previous comment should be http://www.amazon.com/Software-Build-Systems-Principles-Experience/dp/0321717287Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-25023280.post-41662568571999511552012-06-24T13:07:02.344-07:002012-06-24T13:07:02.344-07:00I think the biggest challenge is getting the peopl...I think the biggest challenge is getting the people responsible for training programmers --- i.e., university profs --- to accept that this is a challenging problem that deserves time in the curriculum (and more serious research attention). The next time I update http://third-bit.com/articles/not-on-the-shelves-2009.pdf, a book like third-bit.com/articles/not-on-the-shelves-2009.pdf on packaging, releasing, and installing will definitely make an appearance.Anonymousnoreply@blogger.com