This is a blog post to sum up some of my thoughts I shared on the Nebula-dev mailing list with Wim Jongmam and later on Twitter with Zoltán Ujhelyi and Dave Carver about naming build types at Eclipse.org and scheduling them.
It is a topic open to debate. My goal with it is to find out practices and names that are relevant and useful for community (contributors and, mainly, consumers) and also to give food for thoughts about how to deal with builds.
Historically, Eclipse has 4 to 5 classical qualifier for binary artifacts:
This wording is specific to Eclipse, only Eclipse people do understand the meaning of it. Even worse, some of these are not used accurately. Although this is more or less official wording, I am not sure it is used by most projects.
Now Eclipse.org provides continuous integration, to automate and manage builds executions, and most of them happen whenever a change happen on your VCS. Continuous integration has highly changed the way binary are produced and made available to consumers, it is now much easier to get builds, lots of project have a ping of less that 10 minutes between a commit and a build ready to be released.
That was the starting point of my thoughts, with the Nebula build job: Why calling nightly a job that does run on a build any time a commit happen?
Requalifying binaries to make consumption clearer
As a producer of builds, here are my opinion on these qualifiers:
- Release: The heartbeat of the product for consumers
- Maintenance: is a release, but with 3rd qualifier digit different than 0
- Stable: Is nothing but an integration build that was put on a temporary update-site for the release train. And to be honest, I don’t use it, I directly point the latest good continuous integration builds to the release train builder
- Integration: The heartbeat of the product for developers, built on each commit
- Nightly: What is in it? What does nighly mean at the same time for a developer in Ottawa and a developper in Beijing? Who cares that it is build nightly and not a 2pm CEST? For most projects, the nightly built artifacts are not at all different from the ones that would have been built with the last commit. What is the added value of artifacts built during a nighly scheduled build over the latest artifact built from last commit? Why building something every night if there was no commit in it for 3 monthes?
So I am in favor of removing Maintenance, Stable and Nightly. That gives
Wow, 2 kinds of builds, reminds me a tool… Oh yes! Maven! Maven has 2 types of binaries: Release and SNAPSHOTS. That’s great, we finally arrive to the same conclusion as Maven, except that what Maven calls “snapshot” is called “integration” in the Eclipse teminology.
But now, let’s consider the pure wording: why having 2 names for the same thing? How to call a binary that is work in progress? An “Integration” or a “snapshot”?
Let’s be pragmatic: This report tells us there are about 9000000 Java developers in the world. This onetells us 53% of this population use Maven. Let’s say that about 60% of Maven users do understand the meaning of SNAPSHOT. That means 9000000 * 53% * 60% = 2,862,000 people do know what SNAPSHOT means. Wayne confirmed me that the size of the Eclipse community is 132,284 people, who might know the meaning of “integration” in an Eclipse update-site name. That’s the number of people who have an account on eclipse.org sites – wiki, forum, bugzilla, marketplace. Even if we assume that 100% of them do understand the differences between the different qualifiers, and I am pretty sure that less than the 600 committers actually do, that makes that Snapshot is 21 times a more popular and understood word than integration.
So, Eclipse projects and update sites would be easier to understand and consume for the whole Java community by accepting the Maven wording, and using it on the websites and update-sites name.
Following the Maven naming and the release/snapshots dogmas would make consumption easier, but also avoid duplication of built artifacts, and also make things go more “continuously”. Your project is on rails, it’s going ahead with Snapshots, and sometimes make stops at a Release. That’s also a step ahead towards the present (see how GitHub works) of software delivery: continuous improvement, continuous delivery.
Requalifying build job to make production clearer
So now let’s speak about build management, no more about delivery.
You need several jobs to keep both quality and fast enough feedback, then set up several jobs!
Dave Carver reminded me about some basics of Continuous Integration: keep short feedback loops, one builds for each thing. That’s true. If you have a “short” build for acceptance and a “long” build for QA, you need to have separated jobs. Developers need acceptance feedback, they also need QA feedback, but they for sure cannot wait for a long build to have short-term feedback. Otherwise, they’ll drink too much coffee while waiting, it is bad for their health.
But do not create several builds until you have real needs (metrics can also help you to see whether you have need). If you have a 40 minutes full-build, that fails rarely, with slow commit activity, and that nobody is synchronously waiting for this build to go ahead, then multiplying builds and separating reports could be expensive for low Return On Investment. That’s the case for GMF-Tooling: we have a 37 minutes build with compile + javadoc + tests + coverage + signing, and we are currently quite happy with it, no need to spend more effort on it now. Let’s see how we feel when there will be a Sonar instance at Eclipse.org and that we’d have enabled static analysis… Maybe it will be time to split job.
Avoid scheduling build jobs, it makes you less agile
Before scheduling a job that could happen on each commit, just think about how long will be the feedback loop between a commit and the feedback you’ll get: it is the time that will happen between the commit and the clock event starting your job. It can be sooooo long! Why not starting it on commit and get results as soon as possible? Also why schedule and run builds when nothing may change between to schedule triggers?
I can only see one use case where scheduling job executions is relevant: when you have lots of build stuff in the pipe of your build server, and you have limited resources. Then, in that specific but common case, you want to set up priorities: you don’t want a long-running QA job to slow down your dev team who is waiting for feedbacks from a faster acceptance build. For this reason, I would use scheduling, but I would do so because I have no better idea on how to ensure the team will get the acceptance feedback when necessary, it is a matter of priority. Maybe inversting in more hardware would be useful. Then you could stop scheduling, and get all your builds giving feedbacks as soon as they can. Nobody would wait for a schedule event to be able to go ahead the project.
As I said to Zoltán on Twitter: “The best is to have all build reports as soon as possible, as often as possible” (and of course only when necessary, don’t build something that has already been built!). Scheduling goes often against those objectives, but it is sometimes helps to avoid some bottlenecks in the build queue, and then save time projects.
Name it to know what it provides!
Ok, at Eclipse, there are “Nightly” jobs. I don’t like this word. Once again, “Nightly” is meaningless in a worldwide community. And the most important issue are “What does this build provide?”, “Why is it nightly, can’t I get one on each commit?”.
If this build is run on each commit, then don’t call it “Nightly”, because it feeds the consumer with false information. You can think about having both “acceptance” and “QA” jobs, then put that in their names rather than a schedule info, that’s a far more relevant information.
Continuous integration has changed the way we produce and deliver software, we must benefit of it and adopt the good practices that come with it. Continuous integration is the first step towards what seems to be the present of near future of software delivery: continuous improvement and continuous delivery. We must not miss that step if we want to stay efficient.
TLWR (Too Long Wont Read):
Very often, one has to define links or mapping between objects. Here is the way I learnt to represent the concept of matching items when I was 4.
Now I’m 26 and I use the concept of mapping everyday at work. It can represent reference, transformation, association… But that’s now more complicated, because I now use structured data!
With the BPEL “Assign” element in mind, I wanted to find a widget that allowed me to map structured data, represented as trees, using the simple “Draw a line” method that I have been using for years. Except that “Draw a line” would be represented by “Drag’n’drop” on a computer. After some research, I could not find an open-source widget for that. Then I started to develop a new one. And here is what it looks like:
Although it is primarly intended to be used in Eclipse BPEL Designer, I made it part of Nebula because I hope it is helpful for lots of other projects in the Eclipse community. I can remember some projects I work(ed) on that have nice use-cases for this widget in the past (for Eclipse JWT and Scarbo, Bonita Studio, Petals Studio…); and I now think about how to simplify the GMFMap editor with it.
If you are interested in it and if you plan to attend EclipseCon 2012, then vote for this submission to learn more: http://www.eclipsecon.org/2012/sessions/%C2%A1new-nebula-treemapper-widget
As I am working on trying to build SWTBot with Tycho, I find out a mistake that is quite common with SWTBot and that makes test failing with Tycho whereas they work with some more “opaque” builders.
When you write a UI test, your test does depend on UI components you use. This dependency is specific to your test bundle, and then must be explicitly defined in your MANIFEST.MF. For example, if your test will click on the “New > Java Project” menu, so it highly depends on org.eclipse,jdt,ui, which provides this contribution. Then do not forget to add this in your dependencies!
It can work in some cases when you are sure your test platform already contains the contribtutor of the UI elements you manipulate (here org.eclipse.jdt.ui) . Then the menu is already there – as a 3rd party contribution-, although you did not add the dependency to it in your test. But that’s more or less a lucky case, or a case that requires rigorous management of your test platform.
With Tycho, your test platform is, by default, made of your test bundle and all its dependencies (computed from MANIFEST.MF). Then if you don’t explicit your dependencies to UI contributions, your test will probably run in a target platform which does not include the UI elements you interact with, and will fail. When you have this depedency to the UI contributor (such as org.eclipse.jdt.ui) in your test MANIFEST.MF, you are sure you’ll have the menu available whenever you execute your test. Moreover, you are sure that all installation of your test bundle with p2 will contain the necessary stuff to get it working.
To sum it up: If your test depends on UI elements, then it depends on plugins that contribute these UI elements. So tell it in its MANIFEST.MF. That’s all!
For those who are living in French Alps, in Lyon area, or even in Geneve area, here is a reminder for an important event that takes place for the first time in Grenoble: an Eclipse Demo Camp! We’ll celebrate the latest release of Eclipse: Indigo.
With this Demo Camp, you’ll have the opportunity to see some demos of what’s new in Indigo, some presentations and demonstrations about some famous tools of Eclipse you might never had the opportunity to try, and last but not least, to chat together between people who are interested or involved in Eclipse development in the area.
The event takes place on Tuesday June 28th, during the afternoon. You can register either on the wiki page of the event, or using the EventBrite ticket system. For the hottest news, you can foolow the Twitter account of the DemoCamp. Amd rememver: all that for FREE!
I hope I’ll meet some of you there!