
As companies continue digitizing their operations, data is more valuable than ever. But as we wrote in our article [Where enterprise software is heading?](/blog/where-is-enterprise-software-heading), the number of applications used in an organization keeps increasing. **It is then not surprising to see businesses invest in data platforms to centralized their data.**

Once all the data is in one place, it's possible to have a 360 view of the customer journey, control costs, and have real-time accounting. Data is also the fuel needed to run predictions. In the next years, artificial intelligence models will help companies run accurate finance forecasts and it could detect operational anomalies automatically.

**However, building a data platform is complex.** First, you need to extract the data from all the applications and store it as a single data model. The customer data coming from the CRM and the one coming from the accounting must have the same identifiers and attributes to be consolidated. This is known as Extract Transform Load (ETL). **[It is said that data engineers spend 80% of their time cleaning and organizing data.](https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists)**

Once the data is stored. Whether it's in a data warehouse or a database, the second step is to build analytic views and models on the data to get insights and create value over the raw data. This is heavy work. The tools at play are complex to configure and you need a data team to benefit from your data platform.

Every business deals with the same data models: Customers, Organizations, Invoices, etc. **Yet, they need to redefine manually the data schemas to store their data and create analytics views almost from scratch.** So, could this process be simplified?

**At Datablist, we believe it's time to have common data models that define all those concepts (Customers, Organizations, Finance, etc.) independently of the applications that store them.**

**In the past months, the big players (Microsoft, Salesforce, AWS, and Google Cloud) have announced working on unified data models to federate an ecosystem of tools compatible with the same data schemas.** Are all of them working on a single data model? No, they have decided to go on a separate road!



# What is a data model and why it matters?

First, we need to define what is a data model. It is made of:

  - **Entities** - Classes that represent business concepts. For example Person, Organization, Product. Entities can be drill-down in nested levels. The entity "Person" can have sub-entities like Contact, Lead, Customer for example. Usually, a data model is composed of few root classes but each one has a nested tree of sub-entities.
  - **Properties** - Entities are empty shells. They need Properties (or attributes) to store information. *name* is an example of property and the same property could be used in several entities: Person, Product, Organization. Properties are also defined in a hierarchical tree. We can imagine *firstName* and *lastName* as sub properties of *name*.
  - **Relationships** - With the properties, it is possible to define relations. For example, a *Product* entity would have a *brand* property that is represented by an *Organization* value.
  - **Data validations** - On a generic data model, data validation is loose to avoid imposing constraints on the input data. Nevertheless, a data model could define rules to validate mandatory properties or property value types.

**Together, *Entities*, *Properties* and *Relationships* compose a data model**. It is also called a **vocabulary**.

So, why is it important? **Because it's the first step toward standardization!.** Standardization of data organization and expected structure. **Data cleaning, the most time-consuming task of any data engineer could be automated (at least partially).** When you export data from different applications in a single data model, the consolidation will be automatic.

Then, once the data is stored and consolidated, come the analytics and data visualization parts. **A unified data model would bring ready-to-use tools.** Instead of building analytics views from scratch, generic views would be used to make dashboards that represent common concepts (revenues, customer churn rate, etc.).

**Finally, a unified model would empower non-technical people to work with the data instead of relying on data scientists.**


# Microsoft and the Open Data Initiative

**The first move was made by Microsoft with the "Open Data Initiative" in 2018.** Microsoft was joined by SAP and Adobe to agree on a list of data schemas so their respective applications could speak the same language.

The Open Data Initiative has created the Common Data Model (CDM). When exporting customer data from SAP, Adobe, or Microsoft Dynamics 365, among the formats, the Common Data Model is now available.

[![Common Data Model Entities](/blog_articles_images/2021/01/cdm-entities-v1.png)](https://aka.ms/cdmposter)
*Common Data Model Entities*

The full Data Poster is available [here](https://aka.ms/cdmposter) and an interactive version [here](https://microsoft.github.io/CDM/).

**Microsoft Dynamics 365 internal data structure is the main source for this data model.** It has the advantage of providing a complete and broad model. Entities for banking, education, medical fields are available. **On the other hand, the model is very complex and the inner workings of Dynamics 365 are too visible.**

This is a smart move from Microsoft and the Azure platform. **All the data tools on the Azure platform will be compatible eventually**. Obviously, the Azure Data Lake to store data but also the Artificial Intelligence tools that will benefit from standard data structures.

To read more, visit the Microsoft page on the Open Data Initiative: [https://www.microsoft.com/en-us/open-data-initiative](https://www.microsoft.com/en-us/open-data-initiative) and the Github repository: [https://github.com/microsoft/CDM](https://github.com/microsoft/CDM).


# Salesforce and the Cloud Information Model

Recently, at the end of 2019, Salesforce and Amazon Web Services decided to start their own initiative called the **[Cloud Information Model](https://cloudinformationmodel.org/)** (CIM). Google Cloud, Twilio and other companies have joined the consortium since.

Unlike the Common Data Model, the Cloud Information Model is not a replication of an existing software data structure. Thus, it is lighter and the number of entities and properties is nothing to compare with the Open Data Initiative. It sounds like a weak point but it is not, in the long term. The project is at the beginning and it will grow slowly. In the end, the data model will be easier to understand and to implement.

[![Cloud Information Model - Party](/blog_articles_images/2021/01/cloudinformationmodel-party.png)](https://cloudinformationmodel.org/party.html)
*Cloud Information Model - Party Entities*

Another difference with the Microsoft data model is the governance of the project. It is open-source and claims to welcome any company that wants to contribute.

On the negative side, the project is less active than the Common Data Model. Salesforce has announced it would be available on Salesforce Customer 360 and Salesforce Mulesoft solution is compatible, but that's all. The ecosystem doesn't exist yet.

**With AWS and Google Cloud on board, we will see if they implement it on their big data tools.**

To read more, visit the [Cloud Information Model website](https://cloudinformationmodel.org/) and the Github repository [https://github.com/cloudinformationmodel](https://github.com/cloudinformationmodel).


# Do we need several data models?

**One can ask why Salesforce, AWS, and the other members of the Cloud Information Model haven't joined the Microsoft initiative and decided instead to go alone.** Clearly, this has to do with having control over the course of the project and be at the center of the future ecosystem.

**But it doesn't matter**. Both models share common entities and properties. A property from one will have a matching property on the other. We might see data format converters to translate data from a vocabulary to another one, in the same way, we have PNG, JPEG, etc. to store images and converters to go from one format to the other.
**Competition has always been an accelerator for innovation.**


# What about Datablist?

**At Datablist, we are building our own data model using the W3C standard [JSON-LD](https://www.w3.org/TR/json-ld11/).** The Cloud Information Model (CIM) also uses JSON-LD to structure its data model. Technologically speaking, our data model is quite similar to CIM. Unfortunately, at the moment, the CIM model has a narrow list of business domains (basically Customer, Payment, Product, Order) and is not stable. To move fast, we will continue building our own data model.


**Data interoperability is coming. The 2020 decade will be all about data!**


As companies continue digitizing their operations, data is more valuable than ever. But as we wrote in our article <a href="/blog/where-is-enterprise-software-heading">Where enterprise software is heading?</a>, the number of applications used in an organization keeps increasing. It is then not surprising to see businesses invest in data platforms to centralized their data.
Once all the data is in one place, it's possible to have a 360 view of the customer journey, control costs, and have real-time accounting. Data is also the fuel needed to run predictions. In the next years, artificial intelligence models will help companies run accurate finance forecasts and it could detect operational anomalies automatically.
However, building a data platform is complex. First, you need to extract the data from all the applications and store it as a single data model. The customer data coming from the CRM and the one coming from the accounting must have the same identifiers and attributes to be consolidated. This is known as Extract Transform Load (ETL). <a href="https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists" target="_blank" rel="nofollow">It is said that data engineers spend 80% of their time cleaning and organizing data.</a>
Once the data is stored. Whether it's in a data warehouse or a database, the second step is to build analytic views and models on the data to get insights and create value over the raw data. This is heavy work. The tools at play are complex to configure and you need a data team to benefit from your data platform.
Every business deals with the same data models: Customers, Organizations, Invoices, etc. Yet, they need to redefine manually the data schemas to store their data and create analytics views almost from scratch. So, could this process be simplified?
At Datablist, we believe it's time to have common data models that define all those concepts (Customers, Organizations, Finance, etc.) independently of the applications that store them.
In the past months, the big players (Microsoft, Salesforce, AWS, and Google Cloud) have announced working on unified data models to federate an ecosystem of tools compatible with the same data schemas. Are all of them working on a single data model? No, they have decided to go on a separate road!
<h1 id="what-is-a-data-model-and-why-it-matters"><a href="#what-is-a-data-model-and-why-it-matters" aria-hidden="true" tabindex="-1"></a>What is a data model and why it matters?</h1>
First, we need to define what is a data model. It is made of:
<ul>
<li>Entities - Classes that represent business concepts. For example Person, Organization, Product. Entities can be drill-down in nested levels. The entity "Person" can have sub-entities like Contact, Lead, Customer for example. Usually, a data model is composed of few root classes but each one has a nested tree of sub-entities.</li>
<li>Properties - Entities are empty shells. They need Properties (or attributes) to store information. name is an example of property and the same property could be used in several entities: Person, Product, Organization. Properties are also defined in a hierarchical tree. We can imagine firstName and lastName as sub properties of name.</li>
<li>Relationships - With the properties, it is possible to define relations. For example, a Product entity would have a brand property that is represented by an Organization value.</li>
<li>Data validations - On a generic data model, data validation is loose to avoid imposing constraints on the input data. Nevertheless, a data model could define rules to validate mandatory properties or property value types.</li>
</ul>
Together, Entities, Properties and Relationships compose a data model. It is also called a vocabulary.
So, why is it important? Because it's the first step toward standardization!. Standardization of data organization and expected structure. Data cleaning, the most time-consuming task of any data engineer could be automated (at least partially). When you export data from different applications in a single data model, the consolidation will be automatic.
Then, once the data is stored and consolidated, come the analytics and data visualization parts. A unified data model would bring ready-to-use tools. Instead of building analytics views from scratch, generic views would be used to make dashboards that represent common concepts (revenues, customer churn rate, etc.).
Finally, a unified model would empower non-technical people to work with the data instead of relying on data scientists.
<h1 id="microsoft-and-the-open-data-initiative"><a href="#microsoft-and-the-open-data-initiative" aria-hidden="true" tabindex="-1"></a>Microsoft and the Open Data Initiative</h1>
The first move was made by Microsoft with the "Open Data Initiative" in 2018. Microsoft was joined by SAP and Adobe to agree on a list of data schemas so their respective applications could speak the same language.
The Open Data Initiative has created the Common Data Model (CDM). When exporting customer data from SAP, Adobe, or Microsoft Dynamics 365, among the formats, the Common Data Model is now available.
<a href="https://aka.ms/cdmposter" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/01/cdm-entities-v1.png" alt="Common Data Model Entities"></a>
Common Data Model Entities
The full Data Poster is available <a href="https://aka.ms/cdmposter" target="_blank" rel="nofollow">here</a> and an interactive version <a href="https://microsoft.github.io/CDM/" target="_blank" rel="nofollow">here</a>.
Microsoft Dynamics 365 internal data structure is the main source for this data model. It has the advantage of providing a complete and broad model. Entities for banking, education, medical fields are available. On the other hand, the model is very complex and the inner workings of Dynamics 365 are too visible.
This is a smart move from Microsoft and the Azure platform. All the data tools on the Azure platform will be compatible eventually. Obviously, the Azure Data Lake to store data but also the Artificial Intelligence tools that will benefit from standard data structures.
To read more, visit the Microsoft page on the Open Data Initiative: <a href="https://www.microsoft.com/en-us/open-data-initiative" target="_blank" rel="nofollow">https://www.microsoft.com/en-us/open-data-initiative</a> and the Github repository: <a href="https://github.com/microsoft/CDM" target="_blank" rel="nofollow">https://github.com/microsoft/CDM</a>.
<h1 id="salesforce-and-the-cloud-information-model"><a href="#salesforce-and-the-cloud-information-model" aria-hidden="true" tabindex="-1"></a>Salesforce and the Cloud Information Model</h1>
Recently, at the end of 2019, Salesforce and Amazon Web Services decided to start their own initiative called the <a href="https://cloudinformationmodel.org/" target="_blank" rel="nofollow">Cloud Information Model</a> (CIM). Google Cloud, Twilio and other companies have joined the consortium since.
Unlike the Common Data Model, the Cloud Information Model is not a replication of an existing software data structure. Thus, it is lighter and the number of entities and properties is nothing to compare with the Open Data Initiative. It sounds like a weak point but it is not, in the long term. The project is at the beginning and it will grow slowly. In the end, the data model will be easier to understand and to implement.
<a href="https://cloudinformationmodel.org/party.html" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/01/cloudinformationmodel-party.png" alt="Cloud Information Model - Party"></a>
Cloud Information Model - Party Entities
Another difference with the Microsoft data model is the governance of the project. It is open-source and claims to welcome any company that wants to contribute.
On the negative side, the project is less active than the Common Data Model. Salesforce has announced it would be available on Salesforce Customer 360 and Salesforce Mulesoft solution is compatible, but that's all. The ecosystem doesn't exist yet.
With AWS and Google Cloud on board, we will see if they implement it on their big data tools.
To read more, visit the <a href="https://cloudinformationmodel.org/" target="_blank" rel="nofollow">Cloud Information Model website</a> and the Github repository <a href="https://github.com/cloudinformationmodel" target="_blank" rel="nofollow">https://github.com/cloudinformationmodel</a>.
<h1 id="do-we-need-several-data-models"><a href="#do-we-need-several-data-models" aria-hidden="true" tabindex="-1"></a>Do we need several data models?</h1>
One can ask why Salesforce, AWS, and the other members of the Cloud Information Model haven't joined the Microsoft initiative and decided instead to go alone. Clearly, this has to do with having control over the course of the project and be at the center of the future ecosystem.
But it doesn't matter. Both models share common entities and properties. A property from one will have a matching property on the other. We might see data format converters to translate data from a vocabulary to another one, in the same way, we have PNG, JPEG, etc. to store images and converters to go from one format to the other.
Competition has always been an accelerator for innovation.
<h1 id="what-about-datablist"><a href="#what-about-datablist" aria-hidden="true" tabindex="-1"></a>What about Datablist?</h1>
At Datablist, we are building our own data model using the W3C standard <a href="https://www.w3.org/TR/json-ld11/" target="_blank" rel="nofollow">JSON-LD</a>. The Cloud Information Model (CIM) also uses JSON-LD to structure its data model. Technologically speaking, our data model is quite similar to CIM. Unfortunately, at the moment, the CIM model has a narrow list of business domains (basically Customer, Payment, Product, Order) and is not stable. To move fast, we will continue building our own data model.
Data interoperability is coming. The 2020 decade will be all about data!

Brace yourself, the battle for a unified data model is coming!

# Introduction
Last month, [Twilio bought Segment.io](https://www.forbes.com/sites/alexkonrad/2020/10/09/twilio-to-acquire-cloud-startup-segment-for-3-billion/), a vertical SaaS software that focuses on customer data integration. This move by Twilio comes after buying Sendgrid last year and follows a bundling strategy. Twilio wants to be a 360° platform when it comes to customer value proposition.

[In a famous quote](https://hbr.org/2014/06/how-to-succeed-in-business-by-bundling-and-unbundling), Jim Barksdale said the two ways to make money is by bundling products together or unbundling them and creating targeted offers.

We see both in the software industry today. Microsoft, Salesforce, Oracle and other behemoths keep adding to their software suite, making them ubiquitous. **In the meanwhile, we can see an increase in niche SaaS products**. From image segmentation APIs to content sentiment analysis tools. **The number of SaaS products we use at work keeps increasing.**

Integrated suites have the advantage of an unified experience while specialized products take advantage of their expertise and can be laser focus on the problem they solve.

In my previous article ([where is enterprise software heading](/blog/where-is-enterprise-software-heading)), I discussed the evolutions in enterprise software as we have moved from ERP to a galaxy of SaaS products (also know as best of breed). This is an integrated suite vs best of breed debate we have now.

In a perfect world, we would be able to compose our own suite by making the different softwares work together. Unfortunately, we are not in a perfect world. Few months ago, a Microsoft sales contacted me to make us move from Slack to Microsoft Teams. I told him my experience with Skype Enterprise and Teams, from an external point of view, was a disaster. It was a hell to work with companies using those tools when you were on another suite (aka Google + Slack). He acknowledged it has not been strategic for Microsoft, but he assured the company started to work on more interoperability.

It might come from my desire to stay away from big corporations but I keep dreaming of a world where software collaboration could exist outside of big enterprise suites.

The number of specialized APIs is impressive. In my last count, I found about twenty companies providing an API for invoice PDF parsing, a good hundred of APIs to check the validity of an email address, and hundreds of companies/people enrichment tools.

Each of these APIs focuses on a single problem and addresses it well. But chances are you don't want to implement tens of different APIs to build an internal tool that would automate your accounting or your customer management. The question is: **how can we bundle them to create our own suite?**


# Why workflow automation (Zapier, Integromat, etc.) is not the best solution?

The first wave of tools to address this problematic are called "workflow automation tools" and Zapier, Mulesoft, Integromat, etc. are in this category.

![Integromat Sreenshot](/blog_articles_images/2020/11/integromat-screenshot.png)
*Integromat Workflow Automation*

The idea is simple: APIs are "Apps". They have inputs and outputs. Something goes in, the app does its job, and something else goes out. The workflow automation tool ensures the data moves from one app to the next one. Apps can be chained to make complex workflows.

While this solves a lot of problems, it is not suited for everything. **With workflow automation, the starting point is an event from an app (or a scheduled event)** and the workflow will move the data from one app to another one. **It acts as a plumbing pipe.**

It connects APIs together more than creating a bundle of products. Here are some limitations:

- **It is not possible to trigger manually an action for a specific data item.** Imagine you have an existing item in your CRM and you want to run it though an automation workflow you have. It is not possible without manually adding a new source event in your workflow.
- Data moves as a whole between apps. It is not possible to transform the value for one property and then merge it back to the data object to be send to the next workflow app. You would need a data storage layer in between.


# Connect Apps through a data hub

So how can we bundle APIs/Products together to create value? **The answer is: through data**. Instead of starting from app events like in workflow automation, the data must be at the center.

<div class="content-video">
 <video autoplay muted loop playsinline width="100%" height="auto" class="d-block">
 <source src="https://storage.googleapis.com/public_datablist/website/videos/run-action-datablist.mp4" type="video/mp4" />
 </video>
 <div class="text-center">Datablist screencast</div>
</div>

**Businesses need a centralized data hub for data related to customers, suppliers, employees, etc. from which partial data can be send to external APIs to be processed.** The results of this process then return to the data hub. It is about consolidating several APIs processes into a single data storage.

From this data hub, automations can be configured on data events like with workflow automation. When a new item (customer, invoice, etc.) is added to the data hub, actions run to process the new data and the result is saved for later use. But now, actions can be trigged manually directly from a data item. For example: send a mail to a customer.

**Building with a data first approach opens the possibility of data standardisation.** With workflow automation, every time you integrate an app (Application A) with another one (Application B), you need to define what properties from the data exposed by *Application A* will be used as input in *Application B*. Integration between two apps needs configuration. By starting from the data, it is not how one app integrates with another one, but how each app connects to the data. The data is the source for each integration.

If a data property is of type **email**, all APIs with an **email** input type can be automatically listed and shown.

# Own your platform and use the power of external APIs

The combo Data hub + APIs is a powerful mix. It gives companies the flexibility to build a platform tailored for their needs without re-developing every feature.

Import leads in your data hub, and get access to a catalog of thousands of APIs to enrich them, validate their information, or send them to dedicated tools like a CRM so the Sales team starts a selling process.

At Datablist, we work to make this vision a reality 🚀. [Join the beta](https://www.datablist.com) (and get 30% off on launch!) or subscribe to the newsletter to know more 🤝.

<h1 id="introduction"><a href="#introduction" aria-hidden="true" tabindex="-1"></a>Introduction</h1>
Last month, <a href="https://www.forbes.com/sites/alexkonrad/2020/10/09/twilio-to-acquire-cloud-startup-segment-for-3-billion/" target="_blank" rel="nofollow">Twilio bought Segment.io</a>, a vertical SaaS software that focuses on customer data integration. This move by Twilio comes after buying Sendgrid last year and follows a bundling strategy. Twilio wants to be a 360° platform when it comes to customer value proposition.
<a href="https://hbr.org/2014/06/how-to-succeed-in-business-by-bundling-and-unbundling" target="_blank" rel="nofollow">In a famous quote</a>, Jim Barksdale said the two ways to make money is by bundling products together or unbundling them and creating targeted offers.
We see both in the software industry today. Microsoft, Salesforce, Oracle and other behemoths keep adding to their software suite, making them ubiquitous. In the meanwhile, we can see an increase in niche SaaS products. From image segmentation APIs to content sentiment analysis tools. The number of SaaS products we use at work keeps increasing.
Integrated suites have the advantage of an unified experience while specialized products take advantage of their expertise and can be laser focus on the problem they solve.
In my previous article (<a href="/blog/where-is-enterprise-software-heading">where is enterprise software heading</a>), I discussed the evolutions in enterprise software as we have moved from ERP to a galaxy of SaaS products (also know as best of breed). This is an integrated suite vs best of breed debate we have now.
In a perfect world, we would be able to compose our own suite by making the different softwares work together. Unfortunately, we are not in a perfect world. Few months ago, a Microsoft sales contacted me to make us move from Slack to Microsoft Teams. I told him my experience with Skype Enterprise and Teams, from an external point of view, was a disaster. It was a hell to work with companies using those tools when you were on another suite (aka Google + Slack). He acknowledged it has not been strategic for Microsoft, but he assured the company started to work on more interoperability.
It might come from my desire to stay away from big corporations but I keep dreaming of a world where software collaboration could exist outside of big enterprise suites.
The number of specialized APIs is impressive. In my last count, I found about twenty companies providing an API for invoice PDF parsing, a good hundred of APIs to check the validity of an email address, and hundreds of companies/people enrichment tools.
Each of these APIs focuses on a single problem and addresses it well. But chances are you don't want to implement tens of different APIs to build an internal tool that would automate your accounting or your customer management. The question is: how can we bundle them to create our own suite?
<h1 id="why-workflow-automation-zapier-integromat-etc-is-not-the-best-solution"><a href="#why-workflow-automation-zapier-integromat-etc-is-not-the-best-solution" aria-hidden="true" tabindex="-1"></a>Why workflow automation (Zapier, Integromat, etc.) is not the best solution?</h1>
The first wave of tools to address this problematic are called "workflow automation tools" and Zapier, Mulesoft, Integromat, etc. are in this category.
<img src="/blog_articles_images/2020/11/integromat-screenshot.png" alt="Integromat Sreenshot">
Integromat Workflow Automation
The idea is simple: APIs are "Apps". They have inputs and outputs. Something goes in, the app does its job, and something else goes out. The workflow automation tool ensures the data moves from one app to the next one. Apps can be chained to make complex workflows.
While this solves a lot of problems, it is not suited for everything. With workflow automation, the starting point is an event from an app (or a scheduled event) and the workflow will move the data from one app to another one. It acts as a plumbing pipe.
It connects APIs together more than creating a bundle of products. Here are some limitations:
<ul>
<li>It is not possible to trigger manually an action for a specific data item. Imagine you have an existing item in your CRM and you want to run it though an automation workflow you have. It is not possible without manually adding a new source event in your workflow.</li>
<li>Data moves as a whole between apps. It is not possible to transform the value for one property and then merge it back to the data object to be send to the next workflow app. You would need a data storage layer in between.</li>
</ul>
<h1 id="connect-apps-through-a-data-hub"><a href="#connect-apps-through-a-data-hub" aria-hidden="true" tabindex="-1"></a>Connect Apps through a data hub</h1>
So how can we bundle APIs/Products together to create value? The answer is: through data. Instead of starting from app events like in workflow automation, the data must be at the center.
<div class="content-video">
 <video autoplay muted loop playsinline width="100%" height="auto" class="d-block">
 <source src="https://storage.googleapis.com/public_datablist/website/videos/run-action-datablist.mp4" type="video/mp4" />
 </video>
 <div class="text-center">Datablist screencast</div>
</div>
Businesses need a centralized data hub for data related to customers, suppliers, employees, etc. from which partial data can be send to external APIs to be processed. The results of this process then return to the data hub. It is about consolidating several APIs processes into a single data storage.
From this data hub, automations can be configured on data events like with workflow automation. When a new item (customer, invoice, etc.) is added to the data hub, actions run to process the new data and the result is saved for later use. But now, actions can be trigged manually directly from a data item. For example: send a mail to a customer.
Building with a data first approach opens the possibility of data standardisation. With workflow automation, every time you integrate an app (Application A) with another one (Application B), you need to define what properties from the data exposed by Application A will be used as input in Application B. Integration between two apps needs configuration. By starting from the data, it is not how one app integrates with another one, but how each app connects to the data. The data is the source for each integration.
If a data property is of type email, all APIs with an email input type can be automatically listed and shown.
<h1 id="own-your-platform-and-use-the-power-of-external-apis"><a href="#own-your-platform-and-use-the-power-of-external-apis" aria-hidden="true" tabindex="-1"></a>Own your platform and use the power of external APIs</h1>
The combo Data hub + APIs is a powerful mix. It gives companies the flexibility to build a platform tailored for their needs without re-developing every feature.
Import leads in your data hub, and get access to a catalog of thousands of APIs to enrich them, validate their information, or send them to dedicated tools like a CRM so the Sales team starts a selling process.
At Datablist, we work to make this vision a reality 🚀. <a href="https://www.datablist.com" target="_blank" rel="nofollow">Join the beta</a> (and get 30% off on launch!) or subscribe to the newsletter to know more 🤝.

How to use a Data Hub to connect APIs


Since the beginning of digital transformation in the 1960s, enterprise software has played a key role in increasing organization productivity. They have moved from custom development, to ERP, to the current best-of-breed approach. Automation and AI are around the corner to drive the next revolution. If the different shifts in technologies that happened in the past are any clues to look forward, chances are the future of enterprise software will look very different than today.

# 1. The ERP age

Before the 1980s, most of the enterprise softwares was custom built. The 1980s brang software standardization with applications developed to manage inventory and manufacturing processes in the industries. Applications spread to all the business needs including finance and human resources. In 1990, the advisory firm Gartner coined the term ERP (for "Enterprise Resource Planning") to refer to this category of business management softwares.

The following decade, the 1990s, was the golden age for ERPs with a rapid growth in adoption across medium and large companies in a market dominated by firms like SAP, Oracle, and HP with on-premise solutions.

![Odoo ERP](/blog_articles_images/2020/10/example-erp-screenshot-odoo.png)
*A screenshot of Odoo, an open source ERP*

## 1.1. Where ERPs shine

### Data centralisation

The major innovation introduced with ERP is a central and real time view over the company data. Inventory, Finance, HR and other applications share a unified data storage. This data consolidation enables cross departments analytics and simplifies company planning.

### Extensible

Moreover, ERPs are extensible. All the applications living in the ERP share a common underlying architecture and have the same interface components. All the ERP market leaders expose their platform to external developers. This creates opportunities for third parties applications, and internal processes that are too specific to the company can be built as applications on top of the platform.

## 1.2. ERP limitations

### Complexity

**This globally consistent view of company financials and inventory come at the expenses of complexity.** A triple complexity! Complexity to implement, to use and to extend.

ERP systems come in a one-size-fit-all monolith strategy. **To be implemented in a company, the company must comply with how the ERP works.** ERP applications are not flexible enough to map custom processes the company might be using. When you add that the implementation must cover several departments, therefore several applications, to be valuable, you end up with scary stories. In 2017, the german grocery chain Lidl, [after 6 years and $500 million spent in SAP implementation](https://www.handelsblatt.com/today/companies/programmed-for-disaster-lidl-software-disaster-another-example-of-germanys-digital-failure/23582902.html), decided to roll back the entire inventory system to their previous custom made platform. More recently in Australia, [a SAP implementation in Australia](https://www.youngwitness.com.au/story/6938588/qld-health-still-fixing-inventory-system/) cost 33,5 Millions $ (AUD) more than planned [with mitiged results](https://www.theregister.com/2020/09/28/queensland_health_sap_s4hana_rollout/).

It is not surprising that small companies, looking for fast implementation and flexibility, are only few to use ERPs.

When implemented, ERPs are hard to use. Because all applications are built with the same components, they all share the same User Interface patterns. **The interface is more a representation of the underlying data than a targeted interface for a defined application.**

To finish on the complexity issue, unlike the promise, extending an ERP is not easy. First of all, they are monoliths. Meaning all the applications and data live in a single system. Developing over it can impact the stability of other applications. **Motivation to develop an application on the ERP must be validated by the company.** A single team wouldn't take the responsibility to extend the company ERP without asking permission. In addition, if an existing application doesn't suit the organization, customizing the code prevents the organization from automatically updating the application when a new version is available. Resulting in maintenance time and instability.

![ERP Complexity](/blog_articles_images/2020/10/erp-complexity.jpg)
*Someone, somewhere, trying to implement an ERP*


### Closed world

ERPs live in a closed world. They are only designed to interact with the company employees. **Suppliers, partners and customers, although modeled in the ERP database, do not interact directly with it.**

The isolation is also true for the applications. Only applications designed for the ERP will play nicely with the data and the processes integrated in it. **Interconnecting an ERP with external HR or Finance software is complex and this is not the way ERP vendors want you to use their product.**

As a result, you need a SAP software developer to build an application on the SAP platform. Some technologies are proprietary and diverged from other programming languages and practices used elsewhere. As a result, salaries for ERP developers are higher than average.

### Lock-in

Long implementation, centralized data and closed world inevitably introduce vendor lock-in. The average ERP life cycle in a company is 15 years (time between an implementation and a new implementation). As a result, the market is concentrated in the hands of a few big players. And it is not to displease them.


# 2. The best of breed revolution

Starting in the 2000s, enterprises moved from vertically integrated organizations toward agile and domain specialized entities. **This move was accompanied with companies opening to the outside word, especially to interact with customers.** The one-size-fits-all ERP systems started to lose momentum in favor of specialized cloud native softwares. **From a single system, the organizations were now choosing to have different softwares, often from different vendors, to meet their ever growing needs.** Over the past few years, an increasing number of companies have switched to this approach, called “best of breed”.

![Odoo ERP](/blog_articles_images/2020/10/SaaSSpending.png)
*SaaS Spending Evolution (source: Blissfully)*


## 2.1. The benefits of best of breed

### Cloud Native

Rising of best of breed philosophy is concomitant with the rise of SaaS (Software as A Service) that live in the cloud. This new wave of enterprise softwares embraced the possibilities created by the cloud and offered a totally different experience.

**Compared to on premise hosting, cloud native softwares are fast to implement.** The setup no longer involves buying servers and sizing the infrastructure, but is reduced to configuration and interconnection with the company's other systems, shortening the time to market. Software update responsibility returns to the software vendor, saving the company maintenance cost.

It also changes the business model of enterprise software vendors. They switched from set up fees + product licences to on demand pricing. How many employees do you have? How many customers? **The goal being to capture value as the company grows.**

With this fast implementation and linear pricing, a myriad of new software editors were created to target all company size. From small companies to multinationals.

### Flexibility

Single purpose applications bring flexibility to the company. Because it is hosted by the vendor, **decision to use an application is slowly pushed back to the team responsibility.** With companies embracing agile organization, teams are empowered to test and decide on their own the tools they want to use. Once implemented, if the results are insufficient, the team can switch to another vendor while keeping the other applications it might use. This “test fast, change fast” is another principle of agility and it is meant to accelerate innovation.

Most of the best of breed applications expose API to connect them with other applications. The enterprise processes are then scattered across multiple applications.


### Easy to use

Because applications address specific needs, **the user interface can be adapted to the intended user who is using the applications.** Applications meant for sales people can have a different user interface than the ones for human resources people. With all the existing devices (smartphones, tablets, laptops, smart watches, etc.), software vendors can offer personalized user experiences.

### Open to the external world

Finally, best of breed applications unveil a first step toward opening the enterprise softwares to the external world. **Where ERPs focuses on company members, applications now interact directly with external parties: customers, suppliers, partners, etc.**

This opening is still limited but the trend is accelerating.


## 2.2. Best of breed downsides

The best of breed approach comes with its own challenges, especially the consequences of having a system made of multiple softwares, data storages and vendors.

### Siloed Data

Ironically, while ERPs main advantage is a central data storage, the biggest issue with best of breed patterns is data silos. Every software is independent and has a dedicated database. **On a company scale, data silos reflect the different business units and teams.**

This has consequences. First, generic data types like customer information might be stored in a CRM only accessible for a part of the company, preventing people from having access to important information. And even if the CRM is connected to other softwares to expose customer information, it can lead to data duplication and inconsistencies. This is referred to as the "source of truth" problem.

Finally, when the data is kept in several places, the company doesn't have a global view of its activity. Running analytics to find correlation between data is not possible. Aligning vision, automation, analytics and planning get complicated without a global view on the data.

![Data Silos](/blog_articles_images/2020/10/data-silos.jpg)
*Data Silos in an organization*

### Integration Complexity

In the best of breed world, software vendors focus on their application. **Complexity is moved to integrating the applications together.** How to connect data between applications? How to keep it secure? How to synchronize common objects (like customers) across them? These questions are left to the companies to answer.

In a 2019 survey conducted by Mulesoft, a provider of integration solutions, [IT leaders reported that only 29% of their applications are currently integrated together](https://www.mulesoft.com/press-center/technology-trends-2019-connectivity-benchmark).

There are diverse reasons for this lack of integration. Data models are different between applications. Customer attributes in a CRM can be different than the one needed in a financial application. They are different and the data models can change. The frequency of deployments (and changes) is increasing with the use of agile methodologies adopted by more and more software vendors. Furthermore, some software vendors don't want to expose their data. In the data gold rush, it is better to receive data from other applications than expose its own data for others to create value with it.

**Companies are far from reaching a global view of their operations with this scattered data.**

![Number of applications per company](/blog_articles_images/2020/10/number-app-per-company.png)
*Number of applications per company has skyrocketed in the last years. (Source Blissfully 2019)*

### Hard to build on top of an applications mesh

From an ERP market with few big players, the best of breed approach has enabled new vendors to join the enterprise software dance. This leads to vendors offering specialized applications to many niche markets.** Yet, often, an organization digital system can't be only made of vendor applications. Custom development is needed to answer specific needs.**

Best of breed makes it difficult to build a custom application on top of this distributed system. Integration with vendor applications is not always possible. **Some applications offer packaged integration with other market leaders but don't open their API to custom development.** Hosting, testing and maintenance lead to high cost for internal application development.


### Duplication of costs

To conclude on the downsides, the costs of all the applications can add up to become quite expensive. Keeping track of all the applications used in the company is not easy, some vendor licences keep being paid while not used. Hundred or thousands of business applications can drain down the profitability of the organization.

![Duplication Subscription](/blog_articles_images/2020/10/duplication-cost.png)
*Number of applications paid several times by different teams. (Source Blissfully 2019)*



# 3. The future of enterprise software

Marc Andreessen famously said in 2011 that ["software is eating the world"](https://a16z.com/2011/08/20/why-software-is-eating-the-world/). Companies that embrace softwares both in their operations and as a driver of growth will continue to lead the digital transformation. Predicting the evolution of enterprise softwares is not easy, but we can look at the current trends to predict what will happen in the market.

The following paragraphs are what we, [at Datablist](/blog/announcing-datablist), believe will shape the future of enterprise software. We can see four strong tendencies:

- **Agility and the power of autonomous teams.**
- **AI, automation and analytics in data driven organizations.**
- **Leveraging collaboration with the external world**
- **Security concerns in a competitive and hostile world**


## 3.1. Agility and the power of autonomous teams

Agility can be defined as an employee's capacity to analyze information in the environment and respond to that information quickly and autonomously. **It has been proven that decentralization is key to maintaining innovation and adaptation in an ever changing world.**

We believe the enterprise softwares landscape will continue to be trusted by best of breed vendors instead of all in one ERP solutions.

- **SaaS solution living in the cloud and pay as you go**. Flexibility and fast implementation is where SaaS software shine. Cloud native applications bring automatic updates, few maintenance costs and predictable pricing. Data ownership and security with the cloud are growing concerns for companies but the SaaS vendors are adapting to those needs and we see no reason the cloud would lose it current momentum.
- **Customer centric experience**. Agility means the buyers are now the users. Developers can choose the tools they want to use, sales people can decide what CRM they will work with. This need of flexibility can only be achieved with the best of breed approach. They choose their products based on online reviews and network feedback. They are informed buyers and want to choose and not be sold to. User experience and design will be more and more important in enterprise softwares.



## 3.2. AI, automation and analytics in data driven organizations

Productivity increase in an organization comes from both employee productivity gain and automation of organization tasks. Automation relies on data. **The businesses that will benefit from automation are the ones that will build a consolidated data view on top of their mesh of best of breed applications.**

### A centralized data space

Connecting best of breed applications together is not enough. First, you are limited by the connectors the applications provide. Applications have connectors to market leaders. But what about small vendors? Worse, what about your internal custom applications? **Second, it is not in the best of breed vendors DNA to make sense of the other applications data.**

> Your CRM doesn’t care about your employees data

A centralized data space gives the organization a consolidated view over the data, whether it is users generated data or organization generated data.

**We believe enterprise data warehouses will gain popularity as a foundation for automation and internal custom applications.**


### Platform to build on top of the data

Despite the ever growing number of enterprise softwares, there will still be times where custom development would make more sense. Those internal applications will sit on top of the centralized data space and use the data to deliver value to the organization.

Modern development tools are shortening the development life cycle. Serverless and container technologies are perfectly suited to embrace this trend.

**Components and APIs that solve a single purpose will work as building blocks for those internal applications. OCR technologies, text analysis, company verification services are generic needs that will be provided off the shelf.**

![Datablist Centralized Data Space](/blog_articles_images/2020/10/datablist-centralized-data-space.png)
*A centralized data space for unified analytic views and automation*

## 3.3. Leveraging collaboration with the external world

Innovation comes when the best idea wins regardless of whether the idea comes from. This first means organization must aggregate insights and ideas from everywhere. We have seen companies opening up to customers to listen to their feedback. This trend will accelerate. Contributions can come from employees, customers but also vendors or partners. **Organizational boundaries will be porous so all the community can participate in the company mission and be involved in the decision making process.**

We already see tech companies betting on open source, sharing their working methodology, the design patterns and overall being transparent on what they learn. It engages employees and creates purpose.

Enterprise applications will foster collaboration, with people inside the company and outside of it.

## 3.4. Security concerns in a competitive and hostile world

The more organizations become digital and open, the more they are vulnerable. Losing trust from customers in this flexible market is not acceptable.

Organizations will use automatic tools to constantly audit their assets and vulnerabilities. Developers codes, vendor softwares, and products will be mapped in a global view of the organization systems.



# Conclusion

As companies move from a single one-size-fit-all ERP world to a galaxy of SaaS applications, new problems arise:

- **Data is siloed** between Sales, R&D, Operations, etc.
- Processes based on multiple data sources **cannot be automated**
- **Companies can’t build** applications over this mesh of SaaS tools

[At Datablist](/blog/announcing-datablist), we believe those problems can be solved with a secure centralized data space coupled with a platform to build custom applications and automation.

If you share our vision, subscribe to our newsletter bellow and follow us on [Twitter](https://twitter.com/datablist) 🤝.


Where is enterprise software heading?

Datablist idea comes from a personal frustration in my previous company. I was CTO of an e-commerce software company. One of the goals for the product team was to automate everything that was redundant. We had built a development pipeline to streamline and automate all the actions between a developer code push and the deployment in production of our software. On every deployment, a Slack bot pushed the release note it has built on its own using the Jira backlog. The DevOps philosophy we followed encourages automation to speed deployment and to remove human error from it. **My frustration came from the gap between the level of automation we had in our software development processes compared with all the manual tasks that needed to be done to "run" the company.** We used tools for sales, human resources, customers management, etc. **Those tools accentuate the existing organization silos by keeping the control over the data.**

I'm a coder and I tried to aggregate the data by building data bridges and workflows using Google Spreadsheet, Zapier or other workflow automation softwares. I found myself to be quickly limited by those tools.
I needed to have more control over the data and I had to store it in an external database, with computing instances to read files and images. I finally gave up when I realized I was building an infrastructure as complex as the one we had for our e-commerce software.

I thought about this problem for a long time. Now that we have digitized everything from supplier invoices to customer contracts, why don't we move toward more automation for the organization processes?


- Supplier invoices scattered in our emails or their websites must be gathered together
- Employees onboarding with welcoming emails, account configurations must be automated
- Reconciliation between invoices and bank transactions must be automatic
- **Customers and business data must be stored in a data warehouse fully accessible to the whole company**


Same as in a software development team, the organization processes and repetitive tasks must move to a digital operating model. A digital representation of the processes made with softwares and algorithms.

**This vision cannot simply be done with pipes that connect softwares together. Nor with bots that mimic human actions. Complete access to the data and a platform to create tailored algorithms on this data is mandatory.**

**A digital operating model is infinitely scalable**, can be improved at an impressive rate and is more resilient to human error. **The organization data, when aggregated and consolidated, give people information about the situation so they can take initiative and build on this data platform.** This is what Agile organizations is about.

It is about moving human activity from performing the processes to designing and monitoring the automation softwares that perform the processes.

**This is just the beginning, if you share our vision and are interested in influencing the direction of Datablist, subscribe to our newsletter bellow and follow us on [Twitter](https://twitter.com/datablist) 🤝.**


Datablist idea comes from a personal frustration in my previous company. I was CTO of an e-commerce software company. One of the goals for the product team was to automate everything that was redundant. We had built a development pipeline to streamline and automate all the actions between a developer code push and the deployment in production of our software. On every deployment, a Slack bot pushed the release note it has built on its own using the Jira backlog. The DevOps philosophy we followed encourages automation to speed deployment and to remove human error from it. My frustration came from the gap between the level of automation we had in our software development processes compared with all the manual tasks that needed to be done to "run" the company. We used tools for sales, human resources, customers management, etc. Those tools accentuate the existing organization silos by keeping the control over the data.
I'm a coder and I tried to aggregate the data by building data bridges and workflows using Google Spreadsheet, Zapier or other workflow automation softwares. I found myself to be quickly limited by those tools.
I needed to have more control over the data and I had to store it in an external database, with computing instances to read files and images. I finally gave up when I realized I was building an infrastructure as complex as the one we had for our e-commerce software.
I thought about this problem for a long time. Now that we have digitized everything from supplier invoices to customer contracts, why don't we move toward more automation for the organization processes?
<ul>
<li>Supplier invoices scattered in our emails or their websites must be gathered together</li>
<li>Employees onboarding with welcoming emails, account configurations must be automated</li>
<li>Reconciliation between invoices and bank transactions must be automatic</li>
<li>Customers and business data must be stored in a data warehouse fully accessible to the whole company</li>
</ul>
Same as in a software development team, the organization processes and repetitive tasks must move to a digital operating model. A digital representation of the processes made with softwares and algorithms.
This vision cannot simply be done with pipes that connect softwares together. Nor with bots that mimic human actions. Complete access to the data and a platform to create tailored algorithms on this data is mandatory.
A digital operating model is infinitely scalable, can be improved at an impressive rate and is more resilient to human error. The organization data, when aggregated and consolidated, give people information about the situation so they can take initiative and build on this data platform. This is what Agile organizations is about.
It is about moving human activity from performing the processes to designing and monitoring the automation softwares that perform the processes.
This is just the beginning, if you share our vision and are interested in influencing the direction of Datablist, subscribe to our newsletter bellow and follow us on <a href="https://twitter.com/datablist" target="_blank" rel="nofollow">Twitter</a> 🤝.

Blog

Brace yourself, the battle for a unified data model is coming!

How to use a Data Hub to connect APIs

Where is enterprise software heading?

Announcing Datablist!