Archive for category Uncategorized

Goodbye PetalsLink, hello JBoss|Red Hat !

Today is my last day working for PetalsLink.

Working for PetalsLink was a quite interesting experience:

On the technical side, I enjoyed moving all the XML-based tooling of PetalsStudio to a more powerful EMF-based approach for Petals JBI editor – for those who don’t know JBI, it is a standard that allows to define SOA artifacts in your ESB. Moving to EMF allowed us to provide better tooling faster, because most of the complexity in manipulating JBI can be removed with very few efforts leveraging EMF ExtendedMetaData. That was the first time I faced this part of EMF, and I got pretty impressed of how well it works (working with EMF always gives this impression of “being well”). I also improved the ability to plug new JBI components into the Studio, which is a critical point when you have to deal with connectors for almost everything – Mail, SFTP, Talend, XSLT…. So that was an interesting challenge in term of conception and development.
Petals Studio was also the pretext to start using Git, GitHub and Sonar. I am pretty happy to have learnt these 3 tools that clearly improved the way I work.

Also, I had the great opportunity to work closely to several Eclipse projects:

  • I could contribute the Tycho build of GMF Tooling, put it on Hudson, get source moved to Git/mirrored to GitHub, improve wiki… GMF Tooling is a project I’ve used for 3 years now and I often saw in it some critical organization points to improve to make it more dynamic in term of development. Working at PetalsLink gave me the opportunity to do what I think was necessary to keep the project healthy. With the help of Michael Golubev, I now think this was an real success.
  • I could contribute to Nebula the TreeMapper widget, which will probably have some very interesting use-cases soon. As I became a committer, I also helped in improving Tycho build and CI, nad it seems like the project liked it if we look at the new p2 update-sites.
  • I contributed some small improvements to Eclipse BPEL designer, tried (unsuccessfully) to make SWTBot use Tycho, and developed a useful extension for Draw2d.

The only thing I wish I would be able to do here is to push ahead the usage of Sonar at Eclipse, at least for GMF Tooling and Nebula.

But I probably learnt even more things from PetalsLink by discovering another company organisation that is very different from what I could experiment before (OpenWide and BonitaSoft): PetalsLink is focused on the Research about SOA and agility of Systems of Information. It is a wide topic! Petals products are quite good compared to other alternatives in the SOA landscape, but they don’t meet the success they deserve, it was a bit frustrating for a developer.

I enjoyed working for PetalsLink, all the expectations are fulfilled, so it is time for me to go ahead, to find a new experience, a new team, new challenges, new issues… I love discovering new things!

That’s why I’ll start working for JBoss|Red Hat tomorrow, as part of the team developing JBoss Tools and JBoss Developer Studio.

I’ll have the opportunity to work with a great team! My main occupation for the next monthes will be to assist Nick Boldt in making JBoss Tools CI and build infrastructure better and better. I’d also like to open the road towards efficient QA for JBoss Tools, including -among other- usage of Jacoco and Sonar. Then I’ll also work on developing nice stuff for some JBoss Tools modules, most probably on the SOA/BPM part.

That’s gonna be a lot of fun! I’m eager to be tomorrow and actually get started for this new team/employer/project/product/users.

Let’s keep in touch via this blog and twitter ;)

8 Comments

Call a spade a spade, and a Nightly a Snapshot

This is a blog post to sum up some of my thoughts I shared on the Nebula-dev mailing list with Wim Jongmam and later on Twitter with Zoltán Ujhelyi and Dave Carver about naming build types at Eclipse.org and scheduling them.
It is a topic open to debate. My goal with it is to find out practices and names that are relevant and useful for community (contributors and, mainly, consumers) and also to give food for thoughts about how to deal with builds.

Background

Historically, Eclipse has 4 to 5 classical qualifier for binary artifacts:

  • Release
  • Maintenance
  • Stable
  • Integration
  • Nightly

This wording is specific to Eclipse, only Eclipse people do understand the meaning of it. Even worse, some of these are not used accurately. Although this is more or less official wording, I am not sure it is used by most projects.

This guy has driven a revolution in the way we build and deliver software

Now  Eclipse.org provides continuous integration, to automate and manage builds executions, and most of them happen whenever a change happen on your VCS. Continuous integration has highly changed the way binary are produced and made available to consumers, it is now much easier to get builds, lots of project have a ping of less that 10 minutes between a commit and a build ready to be released.

That was the starting point of my thoughts, with the Nebula build job: Why calling nightly a job that does run on a build any time a commit happen?

Requalifying binaries to make consumption clearer

As a producer of builds, here are my opinion on these qualifiers:

  • Release: The heartbeat of the product for consumers
  • Maintenance: is a release, but with 3rd qualifier digit different than 0
  • Stable: Is nothing but an integration build that was put on a temporary update-site for the release train. And to be honest, I don’t use it, I directly point the latest good continuous integration builds to the release train builder
  • Integration: The heartbeat of the product for developers, built on each commit
  • Nightly: What is in it? What does nighly mean at the same time for a developer in Ottawa and a developper in Beijing? Who cares that it is build nightly and not a 2pm CEST? For most projects, the nightly built artifacts are not at all different from the ones that would have been built with the last commit. What is the added value of artifacts built during a nighly scheduled build over the latest artifact built from last commit? Why building something every night if there was no commit in it for 3 monthes?

So I am in favor of removing Maintenance, Stable and Nightly. That gives

  • Release
  • Integration

Wow, 2 kinds of builds, reminds me a tool… Oh yes! Maven! Maven has 2 types of binaries: Release and SNAPSHOTS. That’s great, we finally arrive to the same conclusion as Maven, except that what Maven calls “snapshot” is called “integration” in the Eclipse teminology.

But now, let’s consider the pure wording: why having 2 names for the same thing? How to call a binary that is work in progress? An “Integration” or a “snapshot”?

Let’s be pragmatic: This report tells us there are about 9000000 Java developers in the world. This onetells us 53% of this population use Maven. Let’s say that about 60% of Maven users do understand the meaning of SNAPSHOT. That means 9000000 * 53% * 60% = 2,862,000 people do know what SNAPSHOT means.  Wayne confirmed me that the size of the Eclipse community is 132,284 people, who might know the meaning of “integration” in an Eclipse update-site name. That’s the number of people who have an account on eclipse.org sites – wiki, forum, bugzilla, marketplace. Even if we assume that 100% of them do understand the differences between the different qualifiers, and I am pretty sure that less than the 600 committers actually do, that makes that Snapshot is 21 times a more popular and understood word than integration.

So, Eclipse projects and update sites would be easier to understand and consume for the whole Java community by accepting the Maven wording, and using it on the websites and update-sites name.
Following the Maven naming and the release/snapshots dogmas would make consumption easier, but also avoid duplication of built artifacts, and also make things go more “continuously”. Your project is on rails, it’s going ahead with Snapshots, and sometimes make stops at a Release. That’s also a step ahead towards the present (see how GitHub works) of software delivery: continuous improvement, continuous delivery.

Requalifying build job to make production clearer

So now let’s speak about build management, no more about delivery.

You need several jobs to keep both quality and fast enough feedback, then set up several jobs!

Dave Carver reminded me about some basics of Continuous Integration: keep short feedback loops, one builds for each thing. That’s true. If you have a “short” build for acceptance and a “long” build for QA, you need to have separated jobs. Developers need acceptance feedback, they also need QA feedback, but they for sure cannot wait for a long build to have short-term feedback. Otherwise, they’ll drink too much coffee while waiting, it is bad for their health.


But do not create several builds until you have real needs (metrics can also help you to see whether you have need). If you have a 40 minutes full-build, that fails rarely, with slow commit activity, and that nobody is synchronously waiting for this build to go ahead, then multiplying builds and separating reports could be expensive for low Return On Investment. That’s the case for GMF-Tooling: we have a 37 minutes build with compile + javadoc + tests + coverage + signing, and we are currently quite happy with it, no need to spend more effort on it now. Let’s see how we feel when there will be a Sonar instance at Eclipse.org and that we’d have enabled static analysis… Maybe it will be time to split job.

Avoid scheduling build jobs, it makes you less agile

Before scheduling a job that could happen on each commit, just think about how long will be the feedback loop between a commit and the feedback you’ll get: it is the time that will happen between the commit and the clock event starting your job. It can be sooooo long! Why not starting it on commit and get results as soon as possible? Also why schedule and run builds when nothing may change between to schedule triggers?
I can only see one use case where scheduling job executions is relevant: when you have lots of build stuff in the pipe of your build server, and you have limited resources. Then, in that specific but common case, you want to set up priorities: you don’t want a long-running QA job to slow down your dev team who is waiting for feedbacks from a faster acceptance build. For this reason, I would use scheduling, but I would do so because I have no better idea on how to ensure the team will get the acceptance feedback when necessary, it is a matter of priority. Maybe inversting in more hardware would be useful. Then you could stop scheduling, and get all your builds giving feedbacks as soon as they can. Nobody would wait for a schedule event to be able to go ahead the project.
As I said to Zoltán on Twitter: “The best is to have all build reports as soon as possible, as often as possible” (and of course only when necessary, don’t build something that has already been built!). Scheduling goes often against those objectives, but it is sometimes helps to avoid some bottlenecks in the build queue, and then save time projects.

Name it to know what it provides!

Remember Into the Wild or Doctor Zhivago... "By its right name"...

Ok, at Eclipse, there are “Nightly” jobs. I don’t like this word. Once again, “Nightly” is meaningless in a worldwide community. And the most important issue are “What does this build provide?”, “Why is it nightly, can’t I get one on each commit?”.
If this build is run on each commit, then don’t call it “Nightly”, because it feeds the consumer with false information. You can think about having both “acceptance” and “QA” jobs, then put that in their names rather than a schedule info, that’s a far more relevant information.

Conclusion

Continuous integration has changed the way we produce and deliver software, we must benefit of it and adopt the good practices that come with it. Continuous integration is the first step towards what seems to be the present of near future of software delivery: continuous improvement and continuous delivery. We must not miss that step if we want to stay efficient.

11 Comments

Refreshing GMF documentation

Da Graphical Modeling Framework (GMF) has ever been an amazing, very productive and powerful project to generate diagram for models based on EMF. However, I recently read several things and forums or had some chatting with people that make me think GMF is difficult to use for newcomers. GMF is not that complicated to understand and use, but the feedbacks I got show that it is not easy to get it, it can be a blocking issue for some potential users. The question is “what can we do to make it easier for people to consume GMF?”.

The first answer I have is in refactoring documentation to show more “Getting started” tutorials. Then people need to spend less time to find a way to get started.

What you see on the main GMF wiki page

Also, I thought the tutorial was too monolithic. There are a lot of things you can do with GMF, but not all will be useful for your use-case. Users need to clearly understand what use case of piece of documentation actually resolves, in order either to skip it, or to spend more time on it. This can be achieved by reorganizing the titling of the documentation. Then it becomes easier to find out what are the different steps when creating a diagram, which ones you can skip, which ones interest you.

Structured organization and titling

Finally, by digging into current GMF documentation, I discovered that GMF has lots and lots of resources to help people to leverage it. There are several tutorials, including one that includes Flash videos to show how to generate a diagram. This is a very-high value resource, but it was unfortunately not very easy to find. This resource deserved to be highlighted! A sad thing is that (AFAIK) there is currently no way to integrate these movies into the wiki. I hope Bug 352735 gets fixed soon so that people will really benefit from these tutorials without effort.

If you know anything you could improve to the refreshed documentation, please do it. Remember of your first experience with GMF or of the first time you met a classical use-case: How did you search answer? How could the answer be easier to find? How could the answer be easier to understand? What in the documentation makes it less efficicent? If you have ideas, feel free to edit the documentation accordingly.

I learnt a few lessons from this work:

  • Very new users are really the people to target for tutorials.
  • For tutorials, a video is more attractive and efficient than 5 pages of documentation.
  • Do not try to be exhaustive in tutorials, prefer being modular or pluggable, so that the tutorial remains easy to follow, and that people can extend it by adding a sub-sub-title. It will make the tutorial easier to maintain, without making it easier to read (people are able to skip your “tutorial modules”)
  • Make your tutorial incremental. Resolve use-case one after the other. Be very explicit about the use-case. People need to understand where and why you go before reading the “How”
  • Sometimes, there is alreadly a lot of documentation available, there is no need for more. Instead, it can be useful to spend time on organizing documentation and making it more visible. A documentation that nobody finds or reads is very sad, it is waste.

These are my very first steps with documentation, and my first thoughts after working a little bit on GMF documentation. There are probably some things you’d like to tell me on this topic according to your experience. I’d be glad to learn from you, please tell me ;)

11 Comments

Blog in progress

This blog is not yet ready-to-go. I’ll soon update it to post stuff related to Eclipse development, among other things. But for the moment, there is nothing to read here. See post on http://www.bonitasoft.org/blog/author/mickael-istria-eclipse/ .

Stay tuned!

Leave a comment

Follow

Get every new post delivered to your Inbox.