Release Engineering at Openbravo

Posts Tagged ‘Releases

Openbravo releases: life cycle and support plan

Let’s play a game: find a word that describes in the best possible manner how your ideal ERP should be. Think about it for a minute. Many of you have probably answered Stable. I mean, once you implement it in a customer you want it to behave as you designed it to be. No more, no less. No inconveniences, no surprises.  That’s stability, right? Admittedly this sounds pretty good.

Some of you, on the other hand, might have answered Featureful, Useful, Pleasant or even Gorgeous. But hey, hold on a minute… if you want something to be Stable it can’t possibly be Beautiful, Pleasant and Featureful at the same  time. You know, business software is boring by design. That’s how it’s been up to now and don’t expect it to change in the future. Learn to live with that.

Really? Is this true? How can we find a balance between offering a user a stable but featureful, pleasant and beautiful product? Imagine an ERP that you enjoy using. Is that even possible? We believe it is, and this is why we have redesigned our releases life cycle and support plan. We started applying it in June 2010, now let me explain you how it works.


  • We care about the ERP’s stability: a major release should be there for a long time.
  • We care about eliminating headaches to the system integrator: upgrades should be easy. And don’t force me to upgrade to the next major version, let me skip one if I want to.
  • I want a featureful ERP: one that helps me get the job done in a pleasant manner. Allow me to get the latest features in an easy manner.

Based on these principles, we have defined three phases for every major release:

  1. Preventive Support:
    • Customers receive support according to SLA.
    • Customers are entitled to report defects and expect resolution.
    • Openbravo ships regular maintenance packs.

    This phase starts on the release day and ends 6 months after the second to next release, with a minimum duration of 2.5 years.

    Example: 2.40 was released on November 2008. And 3.0 will be released on March 2011. Therefore the Preventive Support phase for 2.40 lasts till September 2011 (March 2010 + 6 months).

  2. Reactive Support – phase 1:
    • Customers receive support according to SLA.
    • Customers are entitled to report defects and expect resolution (major severity only).

    This phase lasts for 1 year.

    Example: 2.40 finishes the Preventive Support phase in September 2011. And the Reactive support – phase 1 lasts till October 2012.

  3. Reactive Support – phase 2:
    • Customers receive support according to reduced SLA (no critical tickets accepted).

    This phase lasts for 2 years.

    Example: 2.40 finishes the Reactive Support – phase 1 in September 2012. And the Reactive support – phase 2 lasts till September 2014.

Let’s illustrate this so that we can get the idea visually:

Note that Openbravo 3.1 and 3.2 are fictional releases, they are not planned nor developed yet. They are listed here to illustrate the support plans for 2.50 and 3.0 if the future 3.1 and 3.2 were to be released in 03/2012 and 03/2013 respectively.

As you can see in the graph the minimum support length of a release is of 5.5 years. Feel free to check the complete document in our wiki for more details.

We’ve seen there are different user profiles: those more conservative who look for a “Stable” product above all. And those somewhat more aggressive who look for new features and additional stimulus. Which profile fits better to you? The good news is we have a plan for both.


Written by jpabloae

29/11/2010 at 16:01

Posted in openbravo

Tagged with , ,

We’re branching the Openbravo 3.0 code line

Up to now the 2.50 and 3.0 versions of Openbravo have shared the same Core. “Core” is what we call the most essential module of Openbravo, the one that includes standard functionality like Financial or Sales Management.

If 2.50 and 3.0 have shared the same Core, what’s the difference between them? While a default 2.50 installation includes just Core, the 3.0 version includes a distribution of several modules: right now “Core” plus 16 more.

In terms of source code management we decided to keep one code line as much time as possible, simply because it reduces our developer’s backporting work. Backporting basically means that a bug fixed in 3.0 should also be fixed in 2.50 (if it affects 2.50, of course).

The news is that we have now reached the point where the 2.50 and 3.0 Core cannot share the same code any more. So we are planning to branch the 2.50 and 3.0 code-lines around the 15th of November.

And that’s not all about it. According to our plans we’ll be releasing version 3.0RC3 precisely the 15th of November. The extra news is that we’ll be having two 2.50 MPs on November, 2.50MP24 and 2.50MP25. Why? Simple, as the “Core” is still shared between 2.50 and 3.0, version 3.0RC3 will be needing some code not ready yet in 2.50MP23. And this is why we need 2.50MP24.

In terms of timing, we’ll freeze 2.50MP24 the 1st of November and release it the 15th of that month. So are we releasing both 2.50MP24 and 3.0RC3 the 15th of November? That’s right, and starting from that point the 2.50 and 3.0 Core code-lines will follow their own paths.

Lastly a couple of notes about how this affects to the 2.50 to 3.0 upgrade process:

  1. While we are introducing a separate branch to simplify our code management, 3.0 is just another version of core and updating from 2.50 MPx to 3.0 is not any different than updating from 2.50 MPx to 2.50 MPy. In both cases, it is a full code replacement of core and, as long as the system has been configured using the principles of modularity and does not contain any core customization, system configurations and extensions are going to be preserved during the upgrade.
  2. We will be cleaning the 3.0 core more aggressively than we are doing with the 2.50 core. Because of that, we can expect a higher volume of API changes in the 3.0 code line for the next few months and up to general availability date. However, we will continue to be conservative in 3.0 as well and accept only low risk API changes in an effort to minimize the chances to break module dependencies.  3.0 API changes will be listed in the Release Notes and users intending to upgrade from 2.50 are encouraged to monitor those notifications.

Written by jpabloae

18/10/2010 at 17:33

Posted in openbravo

Tagged with , , , ,

Introducing maturity levels to Openbravo ERP

Stability is a keyword inherently attached to ERP systems. System integrators and end users want a system that simply offers them the capabilities they need in a pleasant manner and at any moment. In this case stability means that it always works in the way they expect it to work. Openbravo takes this challenge very seriously, as you can see in our current 2.50 MPs (Maintenance Packs) release process:

  • We run a set of automated tests on every commit, which in practice means a 24/7 job: build tests, sanity tests, upgrade tests, functional tests, etc.
  • As a general rule every commit is related to a resolved issue. The QA team, together with the development team, leads the effort of individually verifying the correct resolution of these issues.
  • The QA team performs complete set of manual tests before releasing a maintenance pack to guarantee the quality of the release.

This is the global picture of our current 2.50 MP release process, delivered at the beginning of every month. Now we would like to go beyond this by offering an additional service level in our MPs. And precisely the Life Cycle Management feature introduced in version 2.50MP20 makes this possible: whenever a released MP stays live without known issues for 40 days we’ll tag that release in a special manner. So that system integrators can choose to either use the current stable versions or the ones that have matured for 40 days.

This Life Cycle Management feature introduces the concept of Maturity level for modules, which allows Openbravo to make use of its statuses as follows:

  • Test: primarily used by the Openbravo QA team. This is the pre-release status.
  • QA Approved (old name: Controlled release): this is the current maturity level of the MPs once they are released. It means they have passed the automated tests, the issues have been individually verified and the QA team has run a comprehensive set of manual tests.
  • Confirmed Stable (old name: General availability): the module has passed 40 days in the QA Approved maturity level without any known issues.

So in practice, what does this mean for a system integrator? Simple: those looking for exactly the same maturity level of the current MPs should use the QA Approved status. And those who would like to go beyond this level should consider using the Confirmed Stable status.

Configuring this is as simple as selecting the desired setting in the Module Management Console:

Note that Confirmed Stable is the default option for all the modules. We’ll apply this policy for Core starting from 2.50MP23.


EDIT, 2011/07/04: the maturity levels have been renamed as follows:

  • Controlled Release → QA Approved.
  • General Availability → Confirmed Stable.

Written by jpabloae

29/09/2010 at 19:03

Posted in openbravo

Tagged with , ,

Openbravo ERP Ubuntu Lucid package available

Ladies and gentlemen, fasten your seat belts. It’s here. Be ready for the new Cloud experience with Openbravo ERP and Ubuntu. For the long awaited Ubuntu release, codename Lucid Lynx 10.04, has been just released today! Two key words you should never forget:

  • LTS: this is a Long Term Support release. A new LTS version is released every 2 years and it gets longer support and more focus on security, stability and maintenance.
  • Cloud: first class cloud computing support in Amazon EC2 and Ubuntu Enterprise Cloud (UEC). It’s never been easier to work with your Openbravo ERP Ubuntu instance on the Cloud.

Fine, this sounds good. But how do I install it? First you need to get Ubuntu Lucid Lynx up and running. You can do this in mainly two ways:

  1. Install Ubuntu on a local machine: your own computer, a local server or a virtual machine.
  2. Run an instance on the Amazon EC2 cloud. Use the official AMIs. If it’s for a production server make sure you choose the EBS powered images (aka persistent storage).

And once this is done, install Openbravo ERP:

  1. Enable the Partner’s Repository:
  2. sudo add-apt-repository "deb lucid partner"
    sudo add-apt-repository "deb-src lucid partner"
  3. Install the openbravo-erp package:
  4. sudo apt-get update
    sudo apt-get install openbravo-erp

You can also install it using Synaptic or the Ubuntu Software Center:

Software Center: Openbravo ERP installation

For more details about the installation process you can check the complete intructions in the user’s manual.

This package is powered by PostgreSQL 8.4, OpenJDK 6, Tomcat 6 and the Apache HTTP Server 2.2.

It’s been a long way and with your help we have improved the stability and overall maturity of the Ubuntu + Openbravo ERP experience. We are excited with new delivery. Enjoy!

Written by jpabloae

29/04/2010 at 23:59

Openbravo ERP Ubuntu Lucid package available for beta testing

Ubuntu Lucid Lynx 10.04 LTS is around the corner. It’s the latest and greatest of the Ubuntu releases and it will be ready the 29th of April. It’s a special release, different to the previous Karmic 9.10, Jaunty 9.04 and Intrepid 8.10 releases. In fact it’s the most important Ubuntu release in the last 2 years. Why? The key lies on the LTS term, which is an abbreviation for Long Term Support. In a nutshell, a new LTS version is released every 2 years and it gets longer support for security and maintenance updates and also more testing. This means that it will be the natural choice for those who want to use Ubuntu on a production server for the next 2 years. Openbravo is aware of this fact and we’re getting ready for this.

Now we need your help for getting it fine tuned. Would you like to help out? Great!

We have just released a beta version of the package, so we encourage you to test it and provide us any feedback you find valuable:

  • Tell us what you like about the package.
  • Tell us what you don’t like.
  • Find what you think that could be improved.
  • Anything else you think is important.

Being a beta package, keep in mind these important notes:

  • Do not use it in production environments.
  • The official release of this package is scheduled for the 29th of April.

What’s new on this package? There is a significant change compared to our Karmic release: we have switched to OpenJDK! For those unfamiliar with OpenJDK, it’s a 100% open/free version of the Sun JDK, as well as 100% compatible. Some months ago we foresaw this need and started working to support it.

So how can I test the Openbravo ERP Lucid Lynx package? First you need to get the beta version of Ubuntu Lucid Lynx up and running. You can do this in mainly two ways:

  1. Install Ubuntu on a local machine: your own computer, a local server or a virtual machine.
  2. Run an instance on the Amazon EC2 cloud. Use the official AMIs.

Once this is ready, add our testing repository and install openbravo-erp:

  1. Add our testing repository:
    sudo apt-key adv --keyserver --recv-keys C2F11D81
    sudo sh -c 'echo "deb lucid main" >> /etc/apt/sources.list'
    sudo sh -c 'echo "deb-src lucid main" >> /etc/apt/sources.list'
  2. Install the openbravo-erp package:
    sudo apt-get update
    sudo apt-get install openbravo-erp

You can also use Synaptic or your favourite package management tool.

This is how you can send us your feedback:

This beta testing phase will last four weeks, primarily because our intention is to release our package together with the final version of Lucid. This is your opportunity to test the long awaited Ubuntu release together with the new Openbravo ERP package. And why not, also a good chance get involved in the project!

UPDATE (2010/04/09): the beta2 version of Lucid is out, download links updated.

Written by jpabloae

07/04/2010 at 16:37

Posted in openbravo

Tagged with , , , ,

Towards a better Openbravo ERP release process

Openbravo ERP 2.50 has a time based release plan. This helps us get bug fixes and new features into our user’s hands more quickly and improve our planning process.

Up to now we’ve been releasing Maintenance Packs on a monthly basis. However we have detected some improvement areas, mainly two:

  1. We want to release like a clock:  same time, same day, every month. Everyone gets benefited by this:
    • End users appreciate knowing the release dates without even reading a release calendar. They simply know that they’ll have an update at the end of every month.
    • Our engineering team can organize more effectively: developers, QA, Support, Release Management.
  2. Developers should know what commits they push go into what release. This way they can organize their work more efficiently.

So let’s start! We already have the list of desired improvements and we know that we need to set the key dates. So first we need to identify the different kind of events involved in the development process:

  • Regular commits (e.g. bugfixes or small features).
  • Large project merges (e.g. a new large feature).
  • QA automatic tests.
  • Feature freeze for the release.
  • Release packaging.
  • QA manual tests.
  • Publish the release.

Now we need to set some dates and deadlines. The QA manual work – bug found – fix it loop lasts around 10 days. So the freeze and packaging should be before the 20th. And we want to provide a guarantee to developers, so that they know which commits go into what release. Here’s a proposal:

  • All the commits pushed before the 15th of a month will go into that month’s release. We’ll make sure all the tests are passed, working together with the developers in case there’s a failure.
  • We’ll freeze the code the 18th and start the packaging right after this moment. This means we have 3 days to make the code pass all the automatic test suite. It should be more than enough. It also means that commits pushed between the 15th and the 18th might go into that month’s release. If it passes the tests of course.
  • From that moment on QA can start the manual work. If they find issues the affected developer will work on the fix and they’ll iterate on this process till QA happy is happy with the result.
  • Afterwards the Release Management team can publish the release.

And last but not least, project merges tend to create instabilities in the code main line, compared to having no reintegrations at all. So taking this into account the project merges must be done a specific time frame: from the 22nd of one month until the 5th of the next one. So that we give it enough room (10 days) to get the code stable for the freeze time.

We can see these policies summarized in the following graph:

Openbravo ERP 2.50 release timeline

Improving a release process is usually an evolution, not a revolution. And I believe this is the first step towards better timely releases.

Written by jpabloae

31/03/2010 at 16:21

Posted in openbravo

Tagged with ,

Openbravo ERP 2.40 continuous releases

Openbravo ERP has currently two major versions: 2.50 is the latest one and it’s actively developed. 2.40 is a maintenance release for Professional Edition users, and it mainly consists of bug fixes, with an average of 20 fixes per month.

These 2.40 updates are distributed as Maintenance Packs (MPs – a collection of bug fixes), with a frequency of one month during the first year and of two months during the second one (since November 2009). Starting from 2.40MP11 we are introducing a new concept of MPs. Instead of releasing bi-monthly updates we will release them on a bi-weekly basis, tested and verified by our Continuous Integration Framework. This is the workflow that this automated process follows:

  • Mike reports an issue and it gets assigned to Ken, a Openbravo ERP developer.
  • Ken fixes the issue and pushes the commit into the 2.40 Mercurial repository.
  • Our build farm runs a set of tests on that changeset:
    • Incremental update test on PostgreSQL and Oracle. This is important to test updates to this changeset.
    • Full clean builds on PostgreSQL and Oracle. This is used to test new installations based on this new changeset.
    • Verify the database consistency. This verifies that the database model matches exactly its definition in the XML files.
    • Functional tests on PostgreSQL and Oracle. This runs a set of graphical on a web browser, powered by Selenium.
    • Other minor sanity tests, such as checking that no database data is entered using the reserved ID ranges.
  • If all the tests pass successfully, we tag and sign the release in the Mercurial repository, and later on replicate it into our Subversion mirror. If one test fails the system waits for new changesets and starts the process again.
  • Mike gets notified about the availability of a new MP that includes a fix for the issue he reported.

These kind of MPs are easily identifiable by it’s version numbering, following the 2.40MPX.Y schema. As an example, we currently have released four, called 2.40MP11.1, 2.40MP11.2, 2.40MP11.3 and 2.40MP11.4.

What does this mean for a 2.40 user?

  • We are reducing the delivery time. You get maintenance updates every two weeks.
  • You get updates of the fixes you reported faster than ever.
  • You update your 2.40 installation using the usual method (Subversion), this does not change.

Note that we are also planning to release two more “traditional” MPs during the 2.40 life cycle, that will consist of additional manual verifications. They will be called 2.40MP12 and 2.40MP13, scheduled for February and August 2010 respectively. Here’s a table that summarizes the 2.40 MP frequencies:

Version Release date
2.40MP11 November 2009 (already released)
2.40MP11.X Bi-weekly
2.40MP12 February 2010
2.40MP12.X Bi-weekly
2.40MP13 August 2010

Written by jpabloae

23/12/2009 at 16:39