New Role

Bollocks! People realized that the only thing I did as a Community guy was to go from country to country to collect hangovers with other people. Plus, as the GM of webdetails, the only thing I do is ring the bell from time to time and prevent people from killing each other! Apart from that, this amazing group takes things on their own. I've rendered myself useless! (which also leads me to the very disturbing question of... then why the hell do I work over 10 hours a day?! doing what?? Oh well, better stop this line of thought before I get depressed!)

Which is actually absolutely fine by me, don't get me wrong! But for some reason, the folks at the Pentaho ivory towers didn't like this idea as much as me! So, adding that to the fact that I'm always bitch^H^H^H^H^H hum... passionately spreading my ideas, they decided to just through some more work my way.

So starting from... well, 2 or 3 weeks ago, I'm also a Product Designer, which means... well, actually I'm not sure, but has something to do with actually helping designing how the product shapes up. I'll figure something up!

Some important context - Like everything that I write on this blog, this is my personal opinion, and never should be seen as anything official. But I made it absolutely clear to everyone that regardless of what I effectively do I'd always maintain a very open and honest line of communication.

Also, never interpret any of this as me complaining about our current product, which is absolutely not the case. As I wrote very recently I absolutely believe Pentaho is the best Data Integration / Business Analytics tool in the solar system! But what drives me forward is knowing that it's always possible to improve, always possible to do better as opposed to just sit and down and let others pick up the pace.

Positioning

My main goal is to act as a facilitator between the PM team, heavily market and strategy focused, and the product centric Engineering team. It's almost like a translator task, converting PM's use cases and market needs into product-based feature requests that can be more easily processed by our Engineering team.

From the full list of asks, there are some that can be grouped together - due to complexity, similarity, relevance to our vision, or something else -, I'll try to come up with a way to write the specific feature asks so that it not only covers PM's needs, also facilitates the work that needs to be done by the Engineering team. This is not an individual job - it will go through an iterative process that will include PM, Engineering and a representative of each of the other areas, aiming to improve global communication and feedback.

The others will be mapped to standard bugs / feature requests, with no need for bigger epics. The overall balance between these two distinct groups will dictate the final prioritzation on our queue.

We're still working on the proper process, but it'll probably end up in me passing the direct asks as features / bugs and writing epics and stories for the asks that don't fit in the previous category. From that point on, the Engineering team picks up the regular process. Anyway, internal and boring stuff, what really matters are the results.
Vision

All the great work the PM team have done so far has allowed us to come to a product vision that we're pursuing:



This is what we need, in the long term, to accomplish; We need to stop making our users decide if they want to open Analyzer or PIR, if they want to implement a project using Dashboard Designer or Ctools, if they should build a chart in PRD with jFreeChart or with CCC, force them to build the same customer prompt in several different ways depending on the tool they're using.

We shouldn't make them decide; For them it's all datasources, global definitions, and different ways to consume the information. And from a usability perspective, that's exactly what we should be shaped like.

I obviously know that this is easier said than done; It's going to be a slow and gradual process, but we can't lose sight of it.
Identified Goals

Having that goal in mind, there are a set of initial steps that I will aggressively fight for:
Focus and Simplification

To me, the materialization of Pentaho as a Data Orchestration Platform has to start by focusing our offering and message in major cornerstones of our offering: Pentaho Data Integration, the best ETL desktop client tool the world has ever seen, and what I've personally code-named Pentaho Data Platform.

This Pentaho Data Platform (someone will then turn this into an appropriate terminology) is simply the blending of BA and DI. That distinction has to end, as it only creates problems to ourselves. We have to have a single platform offering, and depending on the configuration each customer will have the ability to decide if it's going to use it in a DI Configuration or more dedicated to a content serving platform, more suited to what we usually call our BA Platform.

And the absolute beauty of what we can do is the tight integration between the two. That's where the secret is, and where I'll spend a huge amount of time in. Getting the data flowing seamlessly from one to the other, from data to insights, easy and naturally.... this is what brings us apart from everyone else!

We have a lot of other tools other than these two; They obviously still play a fundamental role, but from a product standpoint, they're exactly that - Tools that will allow us to technically accomplish this vision, but hide as much as possible that complexity from the end user.

During all of this transformations, there are 2 things that we have to keep in mind as non-negociable:

  1. We can't force heavy updates on customers. Whatever we do has to provide customers with a simple a migration path. They must be confident that it's safe to bet on the current technology, and still be able to take advantage of what will come next.
  2. Multi-tenancy, internationalization, embedability, extensability, pluggability, api documentation have to be deeply buried in our DNA. They have to be present in everything we do.

Datasources

This is a topic that requires a big overhaul. We need to stop making people decide if they want PIR or Analyzer. We need to stop making our pre-sales team demo Schema Workbench as the only way to tune a model; We must be able to secure datasources. We must be able to support a new type of data source without having to write several data access layers, drivers, dialects, whatever.

In order to achieve a coherency and simplification throughout a product implementation, we'll need to bring the concept of Entities as first class citizens, most likely backed up by the metastore (engineering will tell). We need to make sure that we don't force our users to, again and again, describe how they use the Date dimension in their company, always ask them what their territories look like, etc.

While this requires a much, much deeper analysis, it's an indication of how I'd like to set the tone on this specific epic.
Visualizations

We produce data - we need to consume it. Tables, charts, components, it's all visualization elements, and we must work into achieving a coherent visualization layer that can be not only used on our stack but also consumed by our OEMs / integrators, fitting our Embedded strategy. We'll need to start thinking about this very soon, even though that from a priority standpoint, we'll need to fix the datasources issue first.
Community

As I've been defending for ages, an open-source / community based strategy is, IMO, the best way to achieve all this goals. Does this means that everything should or will be open source? Absolutely not! Does this also mean that everything new will be closed source and available only to customers? Hell no!

What we need to do is make sure that throughout all this product design we take this into account and achieve a good balance between these two universes. We need to make sure that we give our community users everything they need to be successful and promote engagement and innovation, and offer our enterprise customers the specific added value that they need and expect as a differentiation factor.

It's always a grey area, impossible to completely please everyone, but I honestly think we've been able to do a good job there - and we need to keep it that way!

Release Cadence

Using a silly analogy, it's common knowledge that when we're riding a bike we must be looking at the road ahead of us and not to our front wheel.

I think the same applies here - We must be focused on where we want to achieve. We must make sure that every step we give, every feature we implement, every piece of code that get's checked in gets us closer to our objective. But we also have to be realistic that some of the things I described before are not things that can be done in a simple minor release. It will be a slow process to get there, and most likely things we'll have to deliver only on major releases.

But we can't just go away and develop this kind of big stories for a long time and stop shipping our software to our users; There has to be a balance.

So I'll fight for us to do things a bit differently - Always thinking about the long term, major release asks, we force ourselves to break them in tasks that can be delivered in minor releases; and prioritizing them, when possible, over the features that may not contribute as much to that end goal. Once again easier said than done, but fundamental to make sure we're heading in the right direction.

This has a direct impact with the release cadence; I'll be recommending that we start adopting a time boxed release cadence of 3 releases per year: 1 major release per year and one minor release every 4 months. Feature that are implemented, tested and documented will make it's way on the next release train; the ones that are not, will wait for the next train. Same with major versions. How do we know what will make it in the next version? Simple - PM prioritizes the tasks, engineering puts an effort on them, we know the velocity of the implementation, so we can infer a virtual release line in our queue.

There's one obvious implication in this - We'll stop answering customers "you'll have feature X in version Y", and we'll move to a mode where we'll say "There's a likewood of X% of you getting that feature in the next N version". But again, we're not saying anything very different - either we commit to a list of features and we can't commit to a date, or we commit to a date and we can't commit to a list of features. Heisenberg's principle all along.

Update: I'm not obviously the only one thinking this way, and apparently this conversation has been going out for some time. In Pentaho World it was announced that we'd effectively be adopting something very very similar to this. Yay!



This should give a better idea of the stuff I'll pursue. This is simply the beginning, as the collaboration with PM and Engineering will dictate the next steps and where our destination is.

Am I sure all this will result on a positive improvement? No idea - but I'll keep trying until I'm either fired or killed. Most likely the second.


Cheers!

-pedro

More...