Release Engineering at Openbravo

Openbravo releases: life cycle and support plan

Let’s play a game: find a word that describes in the best possible manner how your ideal ERP should be. Think about it for a minute. Many of you have probably answered Stable. I mean, once you implement it in a customer you want it to behave as you designed it to be. No more, no less. No inconveniences, no surprises.  That’s stability, right? Admittedly this sounds pretty good.

Some of you, on the other hand, might have answered Featureful, Useful, Pleasant or even Gorgeous. But hey, hold on a minute… if you want something to be Stable it can’t possibly be Beautiful, Pleasant and Featureful at the same  time. You know, business software is boring by design. That’s how it’s been up to now and don’t expect it to change in the future. Learn to live with that.

Really? Is this true? How can we find a balance between offering a user a stable but featureful, pleasant and beautiful product? Imagine an ERP that you enjoy using. Is that even possible? We believe it is, and this is why we have redesigned our releases life cycle and support plan. We started applying it in June 2010, now let me explain you how it works.


  • We care about the ERP’s stability: a major release should be there for a long time.
  • We care about eliminating headaches to the system integrator: upgrades should be easy. And don’t force me to upgrade to the next major version, let me skip one if I want to.
  • I want a featureful ERP: one that helps me get the job done in a pleasant manner. Allow me to get the latest features in an easy manner.

Based on these principles, we have defined three phases for every major release:

  1. Preventive Support:
    • Customers receive support according to SLA.
    • Customers are entitled to report defects and expect resolution.
    • Openbravo ships regular maintenance packs.

    This phase starts on the release day and ends 6 months after the second to next release, with a minimum duration of 2.5 years.

    Example: 2.40 was released on November 2008. And 3.0 will be released on March 2011. Therefore the Preventive Support phase for 2.40 lasts till September 2011 (March 2010 + 6 months).

  2. Reactive Support – phase 1:
    • Customers receive support according to SLA.
    • Customers are entitled to report defects and expect resolution (major severity only).

    This phase lasts for 1 year.

    Example: 2.40 finishes the Preventive Support phase in September 2011. And the Reactive support – phase 1 lasts till October 2012.

  3. Reactive Support – phase 2:
    • Customers receive support according to reduced SLA (no critical tickets accepted).

    This phase lasts for 2 years.

    Example: 2.40 finishes the Reactive Support – phase 1 in September 2012. And the Reactive support – phase 2 lasts till September 2014.

Let’s illustrate this so that we can get the idea visually:

Note that Openbravo 3.1 and 3.2 are fictional releases, they are not planned nor developed yet. They are listed here to illustrate the support plans for 2.50 and 3.0 if the future 3.1 and 3.2 were to be released in 03/2012 and 03/2013 respectively.

As you can see in the graph the minimum support length of a release is of 5.5 years. Feel free to check the complete document in our wiki for more details.

We’ve seen there are different user profiles: those more conservative who look for a “Stable” product above all. And those somewhat more aggressive who look for new features and additional stimulus. Which profile fits better to you? The good news is we have a plan for both.

Written by jpabloae

29/11/2010 at 16:01

Posted in openbravo

Tagged with , ,

We’re branching the Openbravo 3.0 code line

Up to now the 2.50 and 3.0 versions of Openbravo have shared the same Core. “Core” is what we call the most essential module of Openbravo, the one that includes standard functionality like Financial or Sales Management.

If 2.50 and 3.0 have shared the same Core, what’s the difference between them? While a default 2.50 installation includes just Core, the 3.0 version includes a distribution of several modules: right now “Core” plus 16 more.

In terms of source code management we decided to keep one code line as much time as possible, simply because it reduces our developer’s backporting work. Backporting basically means that a bug fixed in 3.0 should also be fixed in 2.50 (if it affects 2.50, of course).

The news is that we have now reached the point where the 2.50 and 3.0 Core cannot share the same code any more. So we are planning to branch the 2.50 and 3.0 code-lines around the 15th of November.

And that’s not all about it. According to our plans we’ll be releasing version 3.0RC3 precisely the 15th of November. The extra news is that we’ll be having two 2.50 MPs on November, 2.50MP24 and 2.50MP25. Why? Simple, as the “Core” is still shared between 2.50 and 3.0, version 3.0RC3 will be needing some code not ready yet in 2.50MP23. And this is why we need 2.50MP24.

In terms of timing, we’ll freeze 2.50MP24 the 1st of November and release it the 15th of that month. So are we releasing both 2.50MP24 and 3.0RC3 the 15th of November? That’s right, and starting from that point the 2.50 and 3.0 Core code-lines will follow their own paths.

Lastly a couple of notes about how this affects to the 2.50 to 3.0 upgrade process:

  1. While we are introducing a separate branch to simplify our code management, 3.0 is just another version of core and updating from 2.50 MPx to 3.0 is not any different than updating from 2.50 MPx to 2.50 MPy. In both cases, it is a full code replacement of core and, as long as the system has been configured using the principles of modularity and does not contain any core customization, system configurations and extensions are going to be preserved during the upgrade.
  2. We will be cleaning the 3.0 core more aggressively than we are doing with the 2.50 core. Because of that, we can expect a higher volume of API changes in the 3.0 code line for the next few months and up to general availability date. However, we will continue to be conservative in 3.0 as well and accept only low risk API changes in an effort to minimize the chances to break module dependencies.  3.0 API changes will be listed in the Release Notes and users intending to upgrade from 2.50 are encouraged to monitor those notifications.

Written by jpabloae

18/10/2010 at 17:33

Posted in openbravo

Tagged with , , , ,

Sizing Openbravo: the art of choosing the right hardware

Let’s say you’ve been working hard customizing Openbravo for a customer. And now you’d like to deploy it, so that the customer can start using it immediately. Next step? You go ahead and start installing the ERP on a server… wait! Not yet! You first need to think about the hardware, don’t you? Not so relevant nowadays? A commodity? Try to answer these questions and you’ll see why it matters:

  • How many employees does the company have?
  • From these employees, how many of them will use the ERP?
  • From these users, could you classify them depending on the amount of time and intensity they’ll devote to the ERP?
  • How do the most typical user flows look like? Standard ERP flows? Customized ones? Heavy report generation or long processes?

As you can imagine the hardware requirements differ quite significantly depending if you need to support 5, 20 or 100 concurrent users.

And let me ask you two additional questions:

  • Do you care about saving costs?
  • Do you like the feelings (aka headaches) arisen when you notice that your server just cannot cope with all the requests in the ERP? Imagine you’ve just bought some new hardware and it’s not enough. Even worse, the customer can no longer work under those conditions, they’re unhappy because of having to buy new hardware again and they do not trust your abilities to choose the right hardware this time.

“Well, of course I care about costs, my time and the service level I provide to my customers! Are you kidding?”

OK, I might have dramatized it a bit. Have I? If you’ve ever participated in an ERP implementation process, you’ll know that these things happen. Obviously no one chooses to consciously increase the project’s costs, look for headaches and decrease the overall customer satisfaction. But these might be some of the consequences of incorrectly sizing your Openbravo installation.

Sizing? Sizing, you say? A sizing is an approximation of the hardware resources required to support a specific software implementation, in this case Openbravo. And I have good news for you, for all these inconveniences can be avoided by using the Sizing Tool we’ve just developed for you!

In a nutshell, you simply tell the tool how many concurrent users you want to support and it tells you what hardware you can use. And in case you want to calculate it using your own flows, it’s enabled to allow that as well.

To give some of the behind the scenes details, we have written test cases for the most common Openbravo flows using a performance benchmarking tool called JMeter. This, summed to our experience with real customers has outcome in the Sizing Tool and Guidelines.

Additionally there’s some interesting data we have proved with this tool. For instance we’ve been specially pleased to see that Openbravo supports 100-250 concurrent users without a hitch (given the proper hardware, of course).

This tool is available for the Openbravo partners. What are you waiting for? Don’t miss it, check it out, it’s in the Partner Portal, under the Resources & Assets -> Delivery Materials sections.

Written by jpabloae

05/10/2010 at 19:15

Posted in openbravo

Tagged with ,

Introducing maturity levels to Openbravo ERP

Stability is a keyword inherently attached to ERP systems. System integrators and end users want a system that simply offers them the capabilities they need in a pleasant manner and at any moment. In this case stability means that it always works in the way they expect it to work. Openbravo takes this challenge very seriously, as you can see in our current 2.50 MPs (Maintenance Packs) release process:

  • We run a set of automated tests on every commit, which in practice means a 24/7 job: build tests, sanity tests, upgrade tests, functional tests, etc.
  • As a general rule every commit is related to a resolved issue. The QA team, together with the development team, leads the effort of individually verifying the correct resolution of these issues.
  • The QA team performs complete set of manual tests before releasing a maintenance pack to guarantee the quality of the release.

This is the global picture of our current 2.50 MP release process, delivered at the beginning of every month. Now we would like to go beyond this by offering an additional service level in our MPs. And precisely the Life Cycle Management feature introduced in version 2.50MP20 makes this possible: whenever a released MP stays live without known issues for 40 days we’ll tag that release in a special manner. So that system integrators can choose to either use the current stable versions or the ones that have matured for 40 days.

This Life Cycle Management feature introduces the concept of Maturity level for modules, which allows Openbravo to make use of its statuses as follows:

  • Test: primarily used by the Openbravo QA team. This is the pre-release status.
  • QA Approved (old name: Controlled release): this is the current maturity level of the MPs once they are released. It means they have passed the automated tests, the issues have been individually verified and the QA team has run a comprehensive set of manual tests.
  • Confirmed Stable (old name: General availability): the module has passed 40 days in the QA Approved maturity level without any known issues.

So in practice, what does this mean for a system integrator? Simple: those looking for exactly the same maturity level of the current MPs should use the QA Approved status. And those who would like to go beyond this level should consider using the Confirmed Stable status.

Configuring this is as simple as selecting the desired setting in the Module Management Console:

Note that Confirmed Stable is the default option for all the modules. We’ll apply this policy for Core starting from 2.50MP23.


EDIT, 2011/07/04: the maturity levels have been renamed as follows:

  • Controlled Release → QA Approved.
  • General Availability → Confirmed Stable.

Written by jpabloae

29/09/2010 at 19:03

Posted in openbravo

Tagged with , ,

On demand branch testing for Openbravo developers

These have been some hard working but passionate weeks for Ken, an Openbravo ERP Core developer. He’s been working on implementing a couple of features in the accounting engine. In fact he’s pretty excited:

– “Wow, this is really going to be valuable for our users!”.

But hold on Ken, you think you’ve finished the development process but you haven’t. Let’s go back to the beginning, so that you can understand.

Once upon a time Ken and his team detected the need to developing a new feature. So being this a relevant change in the Core of the product, Ken decided to start developing this new feature in a new Mercurial branch. It’s the common sense choice for these cases, because he doesn’t want to integrate these changes into Core until it’s ready for general usage. Then he dived into the interesting part: he spent 2 weeks coding and writing some new tests for this new feature. So there he is, he wants the end users to enjoy this new development as soon as possible. This is legitimate and desirable, but Core has a golden rule which cannot be broken:

Anything pushed into the Core repository (erp/devel/pi) must meet a certain quality defined by our automatic tests. This is what we usually call the integration process, or simply int.

To see this simpler: all the balls in int must always be green. You might be wondering which tests they cover. Click on any of the jobs listed in the integration tab to see a description of what they do.

So Ken is now somehow a bit frightened, because although he has written quality code, and quoting his own words:

You never know for sure if you pass the tests until you pass them

And running the tests in your own machine is not an option, because from one side it’s difficult to set them up and from another side we’re continuously adding new tests to this process. As this is a legitimate concern, we’ve decided to solve this gap: we’ll provide Ken a way of running all the tests in a simulated environment, a real clone of the integration process.

So how can Ken run all the tests on his branch? Simple, just running this command:

hg push -f

And then monitor the jobs in the try tab. As of now only one developer can use try at a time. So you need to click on the Changes link of the active job to see if it’s testing your branch or not.

To make things easier to remember, you can add the following snippet to your main Mercurial configuration file (e.g. $HOME/.hgrc):

try =

So that if you ever need to run the tests in a branch, the process would be as simple as running this command:

hg push -f try

But Ken is confused:

– “How can everyone push to the same repository? We are using different branches. Isn’t this crazy?”.

The key lies on the -f argument we’re passing to the hg push command. If you’re interested in the basic internals of how this works, every time you push your commits you’ll create a new head in the repository, and the last commit of this head will effectively become the tip of the entire repository. And the try tests are run against this tip. Confusing? If you want to have a better understanding I suggest you to play with a local repository and see how it evolves as you push new heads. If you don’t care, just use it and forget about the internals.

So what’s next? This is just the beginning of this try feature, you can consult the reference documentation. Some future plans in our TODO list:

  • Add e-mail notifications when your build starts, ends and with the final result.
  • Increase the number of developers that can use this simultaneously.

This feature is available for all the Core contributors (Openbravo S.L.U staff and external contributors). Would you like to become a Core contributor? Great! Read the following guide.

I’m sure you have suggestions about this, so we’ll be happy to hear about them. You can do it either by adding a comment here or by joining us in the #openbravo IRC channel.

I would like to thank the Mozilla team for the original idea behind this concept.

Written by jpabloae

01/07/2010 at 14:00

Openbravo ERP Ubuntu Lucid package available

Ladies and gentlemen, fasten your seat belts. It’s here. Be ready for the new Cloud experience with Openbravo ERP and Ubuntu. For the long awaited Ubuntu release, codename Lucid Lynx 10.04, has been just released today! Two key words you should never forget:

  • LTS: this is a Long Term Support release. A new LTS version is released every 2 years and it gets longer support and more focus on security, stability and maintenance.
  • Cloud: first class cloud computing support in Amazon EC2 and Ubuntu Enterprise Cloud (UEC). It’s never been easier to work with your Openbravo ERP Ubuntu instance on the Cloud.

Fine, this sounds good. But how do I install it? First you need to get Ubuntu Lucid Lynx up and running. You can do this in mainly two ways:

  1. Install Ubuntu on a local machine: your own computer, a local server or a virtual machine.
  2. Run an instance on the Amazon EC2 cloud. Use the official AMIs. If it’s for a production server make sure you choose the EBS powered images (aka persistent storage).

And once this is done, install Openbravo ERP:

  1. Enable the Partner’s Repository:
  2. sudo add-apt-repository "deb lucid partner"
    sudo add-apt-repository "deb-src lucid partner"
  3. Install the openbravo-erp package:
  4. sudo apt-get update
    sudo apt-get install openbravo-erp

You can also install it using Synaptic or the Ubuntu Software Center:

Software Center: Openbravo ERP installation

For more details about the installation process you can check the complete intructions in the user’s manual.

This package is powered by PostgreSQL 8.4, OpenJDK 6, Tomcat 6 and the Apache HTTP Server 2.2.

It’s been a long way and with your help we have improved the stability and overall maturity of the Ubuntu + Openbravo ERP experience. We are excited with new delivery. Enjoy!

Written by jpabloae

29/04/2010 at 23:59

Openbravo ERP Ubuntu Lucid package available for beta testing

Ubuntu Lucid Lynx 10.04 LTS is around the corner. It’s the latest and greatest of the Ubuntu releases and it will be ready the 29th of April. It’s a special release, different to the previous Karmic 9.10, Jaunty 9.04 and Intrepid 8.10 releases. In fact it’s the most important Ubuntu release in the last 2 years. Why? The key lies on the LTS term, which is an abbreviation for Long Term Support. In a nutshell, a new LTS version is released every 2 years and it gets longer support for security and maintenance updates and also more testing. This means that it will be the natural choice for those who want to use Ubuntu on a production server for the next 2 years. Openbravo is aware of this fact and we’re getting ready for this.

Now we need your help for getting it fine tuned. Would you like to help out? Great!

We have just released a beta version of the package, so we encourage you to test it and provide us any feedback you find valuable:

  • Tell us what you like about the package.
  • Tell us what you don’t like.
  • Find what you think that could be improved.
  • Anything else you think is important.

Being a beta package, keep in mind these important notes:

  • Do not use it in production environments.
  • The official release of this package is scheduled for the 29th of April.

What’s new on this package? There is a significant change compared to our Karmic release: we have switched to OpenJDK! For those unfamiliar with OpenJDK, it’s a 100% open/free version of the Sun JDK, as well as 100% compatible. Some months ago we foresaw this need and started working to support it.

So how can I test the Openbravo ERP Lucid Lynx package? First you need to get the beta version of Ubuntu Lucid Lynx up and running. You can do this in mainly two ways:

  1. Install Ubuntu on a local machine: your own computer, a local server or a virtual machine.
  2. Run an instance on the Amazon EC2 cloud. Use the official AMIs.

Once this is ready, add our testing repository and install openbravo-erp:

  1. Add our testing repository:
    sudo apt-key adv --keyserver --recv-keys C2F11D81
    sudo sh -c 'echo "deb lucid main" >> /etc/apt/sources.list'
    sudo sh -c 'echo "deb-src lucid main" >> /etc/apt/sources.list'
  2. Install the openbravo-erp package:
    sudo apt-get update
    sudo apt-get install openbravo-erp

You can also use Synaptic or your favourite package management tool.

This is how you can send us your feedback:

This beta testing phase will last four weeks, primarily because our intention is to release our package together with the final version of Lucid. This is your opportunity to test the long awaited Ubuntu release together with the new Openbravo ERP package. And why not, also a good chance get involved in the project!

UPDATE (2010/04/09): the beta2 version of Lucid is out, download links updated.

Written by jpabloae

07/04/2010 at 16:37

Posted in openbravo

Tagged with , , , ,