
# Introduction

In a 1937 article, the Nobel Prize-Winning author Ronald Coase wrote the ["Nature of the Firm"](https://onlinelibrary.wiley.com/doi/full/10.1111/j.1468-0335.1937.tb00002.x). In this article, he laid down the conditions for how people organize to produce goods and services. For Coase, the two main ways of producing were with individuals in a market ruled by prices, and employees grouped in a firm. Markets are an efficient means of creating wealth but Coase argues the best way to serve the requirement of customers was through the firm. **Indeed, according to Coase, the firm structure allows reducing the coordination cost mandatory for any collaborative project. But this theory was only valid until the digital age!**

**A new form of collaborative work has emerged with the digital age where individuals contribute without the border of a firm.** These collaborative productions can be open-source software like Linux, Node.js (a javascript webserver), culture, or information like Wikipedia. Following Coase article and his 2 models of production, [Yochai Benkler](https://en.wikipedia.org/wiki/Yochai_Benkler) wrote in 2002 an essay called ["Coase’s Penguin, Or, Linux and ‘The Nature of the Firm.’"](http://www.benkler.org/CoasesPenguin.PDF) to add a third model, now possible with online collaboration, that he called ["commons-based peer production (CBPP)"](https://en.wikipedia.org/wiki/Commons-based_peer_production). This new model of production relies on decentralized information gathering, self-organization, and most of the time, no financial incentive. This is an incredible age, where thousands of people can participate in projects beyond the limit of their firm. **After the first wave, driven by individuals, companies now embrace this philosophy as it brings a sense of belonging to employees and serves the companies as they benefit from contributions from outside their labor force.**

Open-source produces code libraries to be used as building blocks in larger projects but it also produces standalone applications. In the latter, modularisation and the cloud are transforming how we develop web applications and as a result, will change how people collaborate on open source projects.


# Web applications are now distributed

When I started coding fifteen years ago, a web application was a big program that handled everything. User accounts, email sendings, file storage, etc. were all coded in a single program.

Two limitations emerged with this approach:
- First, when several people work on the same program, the code base grows quickly, and the complexity increases. Developer changes can conflicts when they edit the same code and it brings overhead in code maintenance. Furthermore, at some point, it becomes impossible for a single mind to keep the whole program modeled in his head.
- And it became obvious developers were reinventing the wheel and spending too much time on basic features.

**The solution was to split the application into several services. Each service is responsible for a subset of features and can be maintained without interfering with other service's code. Services can be developed by the application developers or can be maintained (and sometimes host) by other providers.**

Amazon and its [Amazon Web Services (AWS)](https://aws.amazon.com/) understood this shift before anyone. AWS started with infrastructure services but now provides high-level services to deal with authentication, emails, push notifications, analytics, machine learning, etc. Amazon teams develop the services, update them when security fixes are needed and take care of the availability.

[![Amazon Web Services Infographic](/blog_articles_images/2021/02/aws-toolkit.png)](https://aws.amazon.com/products/)
*AWS products sample*

**With this transformation, the infrastructure is now an integral part of the whole application. It is not enough to have the software code idle in Github to be able to run it, the intelligence from the cloud infrastructure is required.** How do the services communicate with each other? How do you share settings between services? How errors are handled when a service is down? How do you deploy a new service in this mesh?

This transformation has been going on for some years in closed source applications. For open-source applications, it is harder to define this infrastructure intelligence in the code repository. **Today, serverless technologies can change how open-source distributed applications are built and move code collaboration to the cloud.**


# Serverless technologies will change how we consume open source software...

The open-source software distribution model currently consists of choosing between starting from the source code and self-deploying it, or using a SaaS model some companies have built on top of open-source software. In the latter, the software is hosted on instances paid by the editor and the editor generates revenues with licenses. The editor's role is to make a profit between what the instances cost and the revenues the licenses generate.

> For example, the WordPress software code is open source and can be self-deployed or used directly using the SaaS plan provided by [Wordpress.com](https://wordpress.com/pricing/).

Now, imagine what the [serverless infrastructure](https://en.wikipedia.org/wiki/Serverless_computing) brings.
Take an application built around serverless technologies. The application is hosted on a cloud platform, ready to run. Obviously, it is made of compute serverless services to run code, but also serverless databases, serverless cache instances, serverless queue services, etc.

> Serverless computing is a cloud computing execution model in which the cloud provider allocates machine resources on demand, taking care of the servers on behalf of their customers.
> [Check the Serverless definition on Wikipedia.](https://en.wikipedia.org/wiki/Serverless_computing)

When a customer runs the applications, the cloud platform allocates resources on-demand to launch and run the needed services. At the end of the execution, it is possible to count each service invocation and bill the customer.

**In this model, the starting point is not a Github repository but a ready to run application hosted in a cloud platform App Store.**

# ... and bring collaborative cloud platforms

Serverless technology is still new, yet we can already witness the change.

## AWS Serverless Application Repository

Amazon Web Services platform's primary role is to distribute AWS services but they [release in 2018](https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/) their [Serverless Application Reposity](https://aws.amazon.com/serverless/serverlessrepo/). An example is [this service to handle image manipulation](https://serverlessrepo.aws.amazon.com/applications/us-east-1/526515951862/image-resizer-service). AWS has a leading role in serverless and it would make sense for the other cloud providers to follow. Google currently has a [marketplace](https://cloud.google.com/marketplace) to run third-party solutions but the marketplace doesn't provide software with serverless pricing.


[![AWS Serverless Application Reposity](/blog_articles_images/2021/02/aws-serverless-store.png)](https://serverlessrepo.aws.amazon.com/applications)
*AWS Serverless Application Reposity*


## Algorithmia
Another example is the [Algorithmia services marketplace](https://algorithmia.com/algorithms). Alorithmia is a serverless cloud platform to host machine learning models. Anyone can publish its trained models on the marketplace for others to use. A published model exposes an API. Hosting and scaling of the underlying service are managed by Algorithmia. **Publishers can also charge a commission fee for the execution of their services.**

[![Algorithmia](/blog_articles_images/2021/02/algorithmia.png)](https://algorithmia.com/algorithms)
*Algorithmia Algorithms store*


## Glitch

I love [Glitch](https://glitch.com/). I love [the story](https://medium.com/glitch/welcome-to-glitch-fe161d0fc39b) and the company culture. Glitch is a collaborative coding environment where the applications are always "runnable". Users can create applications from scratch or duplicate open source applications from other users. Once someone accesses the application, Glitch runs it on a serverless platform and the instance is stopped after 5 minutes of inactivity. Glitch blends both code hosting and application hosting in a collaborative cloud platform.

[![Glitch](/blog_articles_images/2021/02/glitch.png)](https://glitch.com/)
*Glitch*


## Datablist

[Datablist is a database platform](https://www.datablist.com) to store any kind of items and to trigger actions on them (manually or using automation). For example, you can trigger enrichment actions on people items, web scrapping actions on URLs, OCR (optical character recognition) actions on PDF, etc.

Under the hood, every action on Datablist is a serverless service called whenever a user runs the action. All the actions available on Datablist are listed in an "Actions Directory".

Users can use actions from this Actions Directory or code their own actions that will be hosted and run on Datablist.

An important concept on Datablist is to have [common data types shared by all users](/blog/common-data-model-vs-cloud-information-model). It allows having generic actions on data types and properties: People, PDF, URL, etc. Because they are generic, an action created by a user can be published on the "Actions Directory" and be available to everyone.

**By using tens, hundreds, or thousands of actions on your data, and by automating when and how each action is triggered, you end up building applications. An application being a mesh of actions. The goal is to have applications that suit your needs while you benefit from community actions, so you don't have to maintain each service.**


If you share our vision, subscribe to our newsletter below and join our [beta program](https://www.datablist.com).


<h1 id="introduction"><a href="#introduction" aria-hidden="true" tabindex="-1"></a>Introduction</h1>
In a 1937 article, the Nobel Prize-Winning author Ronald Coase wrote the <a href="https://onlinelibrary.wiley.com/doi/full/10.1111/j.1468-0335.1937.tb00002.x" target="_blank" rel="nofollow">"Nature of the Firm"</a>. In this article, he laid down the conditions for how people organize to produce goods and services. For Coase, the two main ways of producing were with individuals in a market ruled by prices, and employees grouped in a firm. Markets are an efficient means of creating wealth but Coase argues the best way to serve the requirement of customers was through the firm. Indeed, according to Coase, the firm structure allows reducing the coordination cost mandatory for any collaborative project. But this theory was only valid until the digital age!
A new form of collaborative work has emerged with the digital age where individuals contribute without the border of a firm. These collaborative productions can be open-source software like Linux, Node.js (a javascript webserver), culture, or information like Wikipedia. Following Coase article and his 2 models of production, <a href="https://en.wikipedia.org/wiki/Yochai_Benkler" target="_blank" rel="nofollow">Yochai Benkler</a> wrote in 2002 an essay called <a href="http://www.benkler.org/CoasesPenguin.PDF" target="_blank" rel="nofollow">"Coase’s Penguin, Or, Linux and ‘The Nature of the Firm.’"</a> to add a third model, now possible with online collaboration, that he called <a href="https://en.wikipedia.org/wiki/Commons-based_peer_production" target="_blank" rel="nofollow">"commons-based peer production (CBPP)"</a>. This new model of production relies on decentralized information gathering, self-organization, and most of the time, no financial incentive. This is an incredible age, where thousands of people can participate in projects beyond the limit of their firm. After the first wave, driven by individuals, companies now embrace this philosophy as it brings a sense of belonging to employees and serves the companies as they benefit from contributions from outside their labor force.
Open-source produces code libraries to be used as building blocks in larger projects but it also produces standalone applications. In the latter, modularisation and the cloud are transforming how we develop web applications and as a result, will change how people collaborate on open source projects.
<h1 id="web-applications-are-now-distributed"><a href="#web-applications-are-now-distributed" aria-hidden="true" tabindex="-1"></a>Web applications are now distributed</h1>
When I started coding fifteen years ago, a web application was a big program that handled everything. User accounts, email sendings, file storage, etc. were all coded in a single program.
Two limitations emerged with this approach:
<ul>
<li>First, when several people work on the same program, the code base grows quickly, and the complexity increases. Developer changes can conflicts when they edit the same code and it brings overhead in code maintenance. Furthermore, at some point, it becomes impossible for a single mind to keep the whole program modeled in his head.</li>
<li>And it became obvious developers were reinventing the wheel and spending too much time on basic features.</li>
</ul>
The solution was to split the application into several services. Each service is responsible for a subset of features and can be maintained without interfering with other service's code. Services can be developed by the application developers or can be maintained (and sometimes host) by other providers.
Amazon and its <a href="https://aws.amazon.com/" target="_blank" rel="nofollow">Amazon Web Services (AWS)</a> understood this shift before anyone. AWS started with infrastructure services but now provides high-level services to deal with authentication, emails, push notifications, analytics, machine learning, etc. Amazon teams develop the services, update them when security fixes are needed and take care of the availability.
<a href="https://aws.amazon.com/products/" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/02/aws-toolkit.png" alt="Amazon Web Services Infographic"></a>
AWS products sample
With this transformation, the infrastructure is now an integral part of the whole application. It is not enough to have the software code idle in Github to be able to run it, the intelligence from the cloud infrastructure is required. How do the services communicate with each other? How do you share settings between services? How errors are handled when a service is down? How do you deploy a new service in this mesh?
This transformation has been going on for some years in closed source applications. For open-source applications, it is harder to define this infrastructure intelligence in the code repository. Today, serverless technologies can change how open-source distributed applications are built and move code collaboration to the cloud.
<h1 id="serverless-technologies-will-change-how-we-consume-open-source-software"><a href="#serverless-technologies-will-change-how-we-consume-open-source-software" aria-hidden="true" tabindex="-1"></a>Serverless technologies will change how we consume open source software...</h1>
The open-source software distribution model currently consists of choosing between starting from the source code and self-deploying it, or using a SaaS model some companies have built on top of open-source software. In the latter, the software is hosted on instances paid by the editor and the editor generates revenues with licenses. The editor's role is to make a profit between what the instances cost and the revenues the licenses generate.
<blockquote>
For example, the WordPress software code is open source and can be self-deployed or used directly using the SaaS plan provided by <a href="https://wordpress.com/pricing/" target="_blank" rel="nofollow">Wordpress.com</a>.
</blockquote>
Now, imagine what the <a href="https://en.wikipedia.org/wiki/Serverless_computing" target="_blank" rel="nofollow">serverless infrastructure</a> brings.
Take an application built around serverless technologies. The application is hosted on a cloud platform, ready to run. Obviously, it is made of compute serverless services to run code, but also serverless databases, serverless cache instances, serverless queue services, etc.
<blockquote>
Serverless computing is a cloud computing execution model in which the cloud provider allocates machine resources on demand, taking care of the servers on behalf of their customers.
<a href="https://en.wikipedia.org/wiki/Serverless_computing" target="_blank" rel="nofollow">Check the Serverless definition on Wikipedia.</a>
</blockquote>
When a customer runs the applications, the cloud platform allocates resources on-demand to launch and run the needed services. At the end of the execution, it is possible to count each service invocation and bill the customer.
In this model, the starting point is not a Github repository but a ready to run application hosted in a cloud platform App Store.
<h1 id="-and-bring-collaborative-cloud-platforms"><a href="#-and-bring-collaborative-cloud-platforms" aria-hidden="true" tabindex="-1"></a>... and bring collaborative cloud platforms</h1>
Serverless technology is still new, yet we can already witness the change.
<h2 id="aws-serverless-application-repository"><a href="#aws-serverless-application-repository" aria-hidden="true" tabindex="-1"></a>AWS Serverless Application Repository</h2>
Amazon Web Services platform's primary role is to distribute AWS services but they <a href="https://aws.amazon.com/blogs/aws/now-available-aws-serverless-application-repository/" target="_blank" rel="nofollow">release in 2018</a> their <a href="https://aws.amazon.com/serverless/serverlessrepo/" target="_blank" rel="nofollow">Serverless Application Reposity</a>. An example is <a href="https://serverlessrepo.aws.amazon.com/applications/us-east-1/526515951862/image-resizer-service" target="_blank" rel="nofollow">this service to handle image manipulation</a>. AWS has a leading role in serverless and it would make sense for the other cloud providers to follow. Google currently has a <a href="https://cloud.google.com/marketplace" target="_blank" rel="nofollow">marketplace</a> to run third-party solutions but the marketplace doesn't provide software with serverless pricing.
<a href="https://serverlessrepo.aws.amazon.com/applications" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/02/aws-serverless-store.png" alt="AWS Serverless Application Reposity"></a>
AWS Serverless Application Reposity
<h2 id="algorithmia"><a href="#algorithmia" aria-hidden="true" tabindex="-1"></a>Algorithmia</h2>
Another example is the <a href="https://algorithmia.com/algorithms" target="_blank" rel="nofollow">Algorithmia services marketplace</a>. Alorithmia is a serverless cloud platform to host machine learning models. Anyone can publish its trained models on the marketplace for others to use. A published model exposes an API. Hosting and scaling of the underlying service are managed by Algorithmia. Publishers can also charge a commission fee for the execution of their services.
<a href="https://algorithmia.com/algorithms" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/02/algorithmia.png" alt="Algorithmia"></a>
Algorithmia Algorithms store
<h2 id="glitch"><a href="#glitch" aria-hidden="true" tabindex="-1"></a>Glitch</h2>
I love <a href="https://glitch.com/" target="_blank" rel="nofollow">Glitch</a>. I love <a href="https://medium.com/glitch/welcome-to-glitch-fe161d0fc39b" target="_blank" rel="nofollow">the story</a> and the company culture. Glitch is a collaborative coding environment where the applications are always "runnable". Users can create applications from scratch or duplicate open source applications from other users. Once someone accesses the application, Glitch runs it on a serverless platform and the instance is stopped after 5 minutes of inactivity. Glitch blends both code hosting and application hosting in a collaborative cloud platform.
<a href="https://glitch.com/" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/02/glitch.png" alt="Glitch"></a>
Glitch
<h2 id="datablist"><a href="#datablist" aria-hidden="true" tabindex="-1"></a>Datablist</h2>
<a href="https://www.datablist.com" target="_blank" rel="nofollow">Datablist is a database platform</a> to store any kind of items and to trigger actions on them (manually or using automation). For example, you can trigger enrichment actions on people items, web scrapping actions on URLs, OCR (optical character recognition) actions on PDF, etc.
Under the hood, every action on Datablist is a serverless service called whenever a user runs the action. All the actions available on Datablist are listed in an "Actions Directory".
Users can use actions from this Actions Directory or code their own actions that will be hosted and run on Datablist.
An important concept on Datablist is to have <a href="/blog/common-data-model-vs-cloud-information-model">common data types shared by all users</a>. It allows having generic actions on data types and properties: People, PDF, URL, etc. Because they are generic, an action created by a user can be published on the "Actions Directory" and be available to everyone.
By using tens, hundreds, or thousands of actions on your data, and by automating when and how each action is triggered, you end up building applications. An application being a mesh of actions. The goal is to have applications that suit your needs while you benefit from community actions, so you don't have to maintain each service.
If you share our vision, subscribe to our newsletter below and join our <a href="https://www.datablist.com" target="_blank" rel="nofollow">beta program</a>.

Serverless will move software collaboration to the cloud!


As companies continue digitizing their operations, data is more valuable than ever. But as we wrote in our article [Where enterprise software is heading?](/blog/where-is-enterprise-software-heading), the number of applications used in an organization keeps increasing. **It is then not surprising to see businesses invest in data platforms to centralized their data.**

Once all the data is in one place, it's possible to have a 360 view of the customer journey, control costs, and have real-time accounting. Data is also the fuel needed to run predictions. In the next years, artificial intelligence models will help companies run accurate finance forecasts and it could detect operational anomalies automatically.

**However, building a data platform is complex.** First, you need to extract the data from all the applications and store it as a single data model. The customer data coming from the CRM and the one coming from the accounting must have the same identifiers and attributes to be consolidated. This is known as Extract Transform Load (ETL). **[It is said that data engineers spend 80% of their time cleaning and organizing data.](https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists)**

Once the data is stored. Whether it's in a data warehouse or a database, the second step is to build analytic views and models on the data to get insights and create value over the raw data. This is heavy work. The tools at play are complex to configure and you need a data team to benefit from your data platform.

Every business deals with the same data models: Customers, Organizations, Invoices, etc. **Yet, they need to redefine manually the data schemas to store their data and create analytics views almost from scratch.** So, could this process be simplified?

**At Datablist, we believe it's time to have common data models that define all those concepts (Customers, Organizations, Finance, etc.) independently of the applications that store them.**

**In the past months, the big players (Microsoft, Salesforce, AWS, and Google Cloud) have announced working on unified data models to federate an ecosystem of tools compatible with the same data schemas.** Are all of them working on a single data model? No, they have decided to go on a separate road!



# What is a data model and why does it matter?

First, we need to define what a data model is. It is made of:

  - **Entities** - Classes that represent business concepts. For example Person, Organization, Product. Entities can be drill-down in nested levels. The entity "Person" can have sub-entities like Contact, Lead, Customer for example. Usually, a data model is composed of few root classes but each one has a nested tree of sub-entities.
  - **Properties** - Entities are empty shells. They need Properties (or attributes) to store information. *name* is an example of property and the same property could be used in several entities: Person, Product, Organization. Properties are also defined in a hierarchical tree. We can imagine *firstName* and *lastName* as sub properties of *name*.
  - **Relationships** - With the properties, it is possible to define relations. For example, a *Product* entity would have a *brand* property that is represented by an *Organization* value.
  - **Data validations** - On a generic data model, data validation is loose to avoid imposing constraints on the input data. Nevertheless, a data model could define rules to validate mandatory properties or property value types.

**Together, *Entities*, *Properties* and *Relationships* compose a data model**. It is also called a **vocabulary**.

So, why is it important? **Because it's the first step toward standardization!.** Standardization of data organization and expected structure. **Data cleaning, the most time-consuming task of any data engineer could be automated (at least partially).** When you export data from different applications in a single data model, the consolidation will be automatic.

Then, once the data is stored and consolidated, come the analytics and data visualization parts. **A unified data model would bring ready-to-use tools.** Instead of building analytics views from scratch, generic views would be used to make dashboards that represent common concepts (revenues, customer churn rate, etc.).

**Finally, a unified model would empower non-technical people to work with the data instead of relying on data scientists.**


# Microsoft and the Open Data Initiative

**The first move was made by Microsoft with the "Open Data Initiative" in 2018.** Microsoft was joined by SAP and Adobe to agree on a list of data schemas so their respective applications could speak the same language.

The Open Data Initiative has created the Common Data Model (CDM). When exporting customer data from SAP, Adobe, or Microsoft Dynamics 365, among the formats, the Common Data Model is now available.

[![Common Data Model Entities](/blog_articles_images/2021/01/cdm-entities-v1.png)](https://aka.ms/cdmposter)
*Common Data Model Entities*

The full Data Poster is available [here](https://aka.ms/cdmposter) and an interactive version [here](https://microsoft.github.io/CDM/).

**Microsoft Dynamics 365 internal data structure is the main source for this data model.** It has the advantage of providing a complete and broad model. Entities for banking, education, medical fields are available. **On the other hand, the model is very complex and the inner workings of Dynamics 365 are too visible.**

This is a smart move from Microsoft and the Azure platform. **All the data tools on the Azure platform will be compatible eventually**. Obviously, the Azure Data Lake to store data but also the Artificial Intelligence tools that will benefit from standard data structures.

To read more, visit the Microsoft page on the Open Data Initiative: [https://www.microsoft.com/en-us/open-data-initiative](https://www.microsoft.com/en-us/open-data-initiative) and the Github repository: [https://github.com/microsoft/CDM](https://github.com/microsoft/CDM).


# Salesforce and the Cloud Information Model

Recently, at the end of 2019, Salesforce and Amazon Web Services decided to start their own initiative called the **[Cloud Information Model](https://cloudinformationmodel.org/)** (CIM). Google Cloud, Twilio and other companies have joined the consortium since.

Unlike the Common Data Model, the Cloud Information Model is not a replication of an existing software data structure. Thus, it is lighter and the number of entities and properties is nothing to compare with the Open Data Initiative. It sounds like a weak point but it is not, in the long term. The project is at the beginning and it will grow slowly. In the end, the data model will be easier to understand and to implement.

[![Cloud Information Model - Party](/blog_articles_images/2021/01/cloudinformationmodel-party.png)](https://cloudinformationmodel.org/party.html)
*Cloud Information Model - Party Entities*

Another difference with the Microsoft data model is the governance of the project. It is open-source and claims to welcome any company that wants to contribute.

On the negative side, the project is less active than the Common Data Model. Salesforce has announced it would be available on Salesforce Customer 360 and Salesforce Mulesoft solution is compatible, but that's all. The ecosystem doesn't exist yet.

**With AWS and Google Cloud on board, we will see if they implement it on their big data tools.**

To read more, visit the [Cloud Information Model website](https://cloudinformationmodel.org/) and the Github repository [https://github.com/cloudinformationmodel](https://github.com/cloudinformationmodel).


# Do we need several data models?

**One can ask why Salesforce, AWS, and the other members of the Cloud Information Model haven't joined the Microsoft initiative and decided instead to go alone.** Clearly, this has to do with having control over the course of the project and be at the center of the future ecosystem.

**But it doesn't matter**. Both models share common entities and properties. A property from one will have a matching property on the other. We might see data format converters to translate data from a vocabulary to another one, in the same way, we have PNG, JPEG, etc. to store images and converters to go from one format to the other.
**Competition has always been an accelerator for innovation.**


# What about Datablist?

**At Datablist, we are building our own data model using the W3C standard [JSON-LD](https://www.w3.org/TR/json-ld11/).** The Cloud Information Model (CIM) also uses JSON-LD to structure its data model. Technologically speaking, our data model is quite similar to CIM. Unfortunately, at the moment, the CIM model has a narrow list of business domains (basically Customer, Payment, Product, Order) and is not stable. To move fast, we will continue building our own data model.


**Data interoperability is coming. The 2020 decade will be all about data!**


As companies continue digitizing their operations, data is more valuable than ever. But as we wrote in our article <a href="/blog/where-is-enterprise-software-heading">Where enterprise software is heading?</a>, the number of applications used in an organization keeps increasing. It is then not surprising to see businesses invest in data platforms to centralized their data.
Once all the data is in one place, it's possible to have a 360 view of the customer journey, control costs, and have real-time accounting. Data is also the fuel needed to run predictions. In the next years, artificial intelligence models will help companies run accurate finance forecasts and it could detect operational anomalies automatically.
However, building a data platform is complex. First, you need to extract the data from all the applications and store it as a single data model. The customer data coming from the CRM and the one coming from the accounting must have the same identifiers and attributes to be consolidated. This is known as Extract Transform Load (ETL). <a href="https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists" target="_blank" rel="nofollow">It is said that data engineers spend 80% of their time cleaning and organizing data.</a>
Once the data is stored. Whether it's in a data warehouse or a database, the second step is to build analytic views and models on the data to get insights and create value over the raw data. This is heavy work. The tools at play are complex to configure and you need a data team to benefit from your data platform.
Every business deals with the same data models: Customers, Organizations, Invoices, etc. Yet, they need to redefine manually the data schemas to store their data and create analytics views almost from scratch. So, could this process be simplified?
At Datablist, we believe it's time to have common data models that define all those concepts (Customers, Organizations, Finance, etc.) independently of the applications that store them.
In the past months, the big players (Microsoft, Salesforce, AWS, and Google Cloud) have announced working on unified data models to federate an ecosystem of tools compatible with the same data schemas. Are all of them working on a single data model? No, they have decided to go on a separate road!
<h1 id="what-is-a-data-model-and-why-does-it-matter"><a href="#what-is-a-data-model-and-why-does-it-matter" aria-hidden="true" tabindex="-1"></a>What is a data model and why does it matter?</h1>
First, we need to define what a data model is. It is made of:
<ul>
<li>Entities - Classes that represent business concepts. For example Person, Organization, Product. Entities can be drill-down in nested levels. The entity "Person" can have sub-entities like Contact, Lead, Customer for example. Usually, a data model is composed of few root classes but each one has a nested tree of sub-entities.</li>
<li>Properties - Entities are empty shells. They need Properties (or attributes) to store information. name is an example of property and the same property could be used in several entities: Person, Product, Organization. Properties are also defined in a hierarchical tree. We can imagine firstName and lastName as sub properties of name.</li>
<li>Relationships - With the properties, it is possible to define relations. For example, a Product entity would have a brand property that is represented by an Organization value.</li>
<li>Data validations - On a generic data model, data validation is loose to avoid imposing constraints on the input data. Nevertheless, a data model could define rules to validate mandatory properties or property value types.</li>
</ul>
Together, Entities, Properties and Relationships compose a data model. It is also called a vocabulary.
So, why is it important? Because it's the first step toward standardization!. Standardization of data organization and expected structure. Data cleaning, the most time-consuming task of any data engineer could be automated (at least partially). When you export data from different applications in a single data model, the consolidation will be automatic.
Then, once the data is stored and consolidated, come the analytics and data visualization parts. A unified data model would bring ready-to-use tools. Instead of building analytics views from scratch, generic views would be used to make dashboards that represent common concepts (revenues, customer churn rate, etc.).
Finally, a unified model would empower non-technical people to work with the data instead of relying on data scientists.
<h1 id="microsoft-and-the-open-data-initiative"><a href="#microsoft-and-the-open-data-initiative" aria-hidden="true" tabindex="-1"></a>Microsoft and the Open Data Initiative</h1>
The first move was made by Microsoft with the "Open Data Initiative" in 2018. Microsoft was joined by SAP and Adobe to agree on a list of data schemas so their respective applications could speak the same language.
The Open Data Initiative has created the Common Data Model (CDM). When exporting customer data from SAP, Adobe, or Microsoft Dynamics 365, among the formats, the Common Data Model is now available.
<a href="https://aka.ms/cdmposter" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/01/cdm-entities-v1.png" alt="Common Data Model Entities"></a>
Common Data Model Entities
The full Data Poster is available <a href="https://aka.ms/cdmposter" target="_blank" rel="nofollow">here</a> and an interactive version <a href="https://microsoft.github.io/CDM/" target="_blank" rel="nofollow">here</a>.
Microsoft Dynamics 365 internal data structure is the main source for this data model. It has the advantage of providing a complete and broad model. Entities for banking, education, medical fields are available. On the other hand, the model is very complex and the inner workings of Dynamics 365 are too visible.
This is a smart move from Microsoft and the Azure platform. All the data tools on the Azure platform will be compatible eventually. Obviously, the Azure Data Lake to store data but also the Artificial Intelligence tools that will benefit from standard data structures.
To read more, visit the Microsoft page on the Open Data Initiative: <a href="https://www.microsoft.com/en-us/open-data-initiative" target="_blank" rel="nofollow">https://www.microsoft.com/en-us/open-data-initiative</a> and the Github repository: <a href="https://github.com/microsoft/CDM" target="_blank" rel="nofollow">https://github.com/microsoft/CDM</a>.
<h1 id="salesforce-and-the-cloud-information-model"><a href="#salesforce-and-the-cloud-information-model" aria-hidden="true" tabindex="-1"></a>Salesforce and the Cloud Information Model</h1>
Recently, at the end of 2019, Salesforce and Amazon Web Services decided to start their own initiative called the <a href="https://cloudinformationmodel.org/" target="_blank" rel="nofollow">Cloud Information Model</a> (CIM). Google Cloud, Twilio and other companies have joined the consortium since.
Unlike the Common Data Model, the Cloud Information Model is not a replication of an existing software data structure. Thus, it is lighter and the number of entities and properties is nothing to compare with the Open Data Initiative. It sounds like a weak point but it is not, in the long term. The project is at the beginning and it will grow slowly. In the end, the data model will be easier to understand and to implement.
<a href="https://cloudinformationmodel.org/party.html" target="_blank" rel="nofollow"><img src="/blog_articles_images/2021/01/cloudinformationmodel-party.png" alt="Cloud Information Model - Party"></a>
Cloud Information Model - Party Entities
Another difference with the Microsoft data model is the governance of the project. It is open-source and claims to welcome any company that wants to contribute.
On the negative side, the project is less active than the Common Data Model. Salesforce has announced it would be available on Salesforce Customer 360 and Salesforce Mulesoft solution is compatible, but that's all. The ecosystem doesn't exist yet.
With AWS and Google Cloud on board, we will see if they implement it on their big data tools.
To read more, visit the <a href="https://cloudinformationmodel.org/" target="_blank" rel="nofollow">Cloud Information Model website</a> and the Github repository <a href="https://github.com/cloudinformationmodel" target="_blank" rel="nofollow">https://github.com/cloudinformationmodel</a>.
<h1 id="do-we-need-several-data-models"><a href="#do-we-need-several-data-models" aria-hidden="true" tabindex="-1"></a>Do we need several data models?</h1>
One can ask why Salesforce, AWS, and the other members of the Cloud Information Model haven't joined the Microsoft initiative and decided instead to go alone. Clearly, this has to do with having control over the course of the project and be at the center of the future ecosystem.
But it doesn't matter. Both models share common entities and properties. A property from one will have a matching property on the other. We might see data format converters to translate data from a vocabulary to another one, in the same way, we have PNG, JPEG, etc. to store images and converters to go from one format to the other.
Competition has always been an accelerator for innovation.
<h1 id="what-about-datablist"><a href="#what-about-datablist" aria-hidden="true" tabindex="-1"></a>What about Datablist?</h1>
At Datablist, we are building our own data model using the W3C standard <a href="https://www.w3.org/TR/json-ld11/" target="_blank" rel="nofollow">JSON-LD</a>. The Cloud Information Model (CIM) also uses JSON-LD to structure its data model. Technologically speaking, our data model is quite similar to CIM. Unfortunately, at the moment, the CIM model has a narrow list of business domains (basically Customer, Payment, Product, Order) and is not stable. To move fast, we will continue building our own data model.
Data interoperability is coming. The 2020 decade will be all about data!

Brace yourself, the battle for a unified data model is coming!

# Introduction
Last month, [Twilio bought Segment.io](https://www.forbes.com/sites/alexkonrad/2020/10/09/twilio-to-acquire-cloud-startup-segment-for-3-billion/), a vertical SaaS software that focuses on customer data integration. This move by Twilio comes after buying Sendgrid last year and follows a bundling strategy. Twilio wants to be a 360° platform when it comes to customer value proposition.

[In a famous quote](https://hbr.org/2014/06/how-to-succeed-in-business-by-bundling-and-unbundling), Jim Barksdale said the two ways to make money is by bundling products together or unbundling them and creating targeted offers.

We see both in the software industry today. Microsoft, Salesforce, Oracle and other behemoths keep adding to their software suite, making them ubiquitous. **In the meanwhile, we can see an increase in niche SaaS products**. From image segmentation APIs to content sentiment analysis tools. **The number of SaaS products we use at work keeps increasing.**

Integrated suites have the advantage of an unified experience while specialized products take advantage of their expertise and can be laser focus on the problem they solve.

In my previous article ([where is enterprise software heading](/blog/where-is-enterprise-software-heading)), I discussed the evolutions in enterprise software as we have moved from ERP to a galaxy of SaaS products (also know as best of breed). This is an integrated suite vs best of breed debate we have now.

In a perfect world, we would be able to compose our own suite by making the different softwares work together. Unfortunately, we are not in a perfect world. Few months ago, a Microsoft sales contacted me to make us move from Slack to Microsoft Teams. I told him my experience with Skype Enterprise and Teams, from an external point of view, was a disaster. It was a hell to work with companies using those tools when you were on another suite (aka Google + Slack). He acknowledged it has not been strategic for Microsoft, but he assured the company started to work on more interoperability.

It might come from my desire to stay away from big corporations but I keep dreaming of a world where software collaboration could exist outside of big enterprise suites.

The number of specialized APIs is impressive. In my last count, I found about twenty companies providing an API for invoice PDF parsing, a good hundred of APIs to check the validity of an email address, and hundreds of companies/people enrichment tools.

Each of these APIs focuses on a single problem and addresses it well. But chances are you don't want to implement tens of different APIs to build an internal tool that would automate your accounting or your customer management. The question is: **how can we bundle them to create our own suite?**


# Why workflow automation (Zapier, Integromat, etc.) is not the best solution?

The first wave of tools to address this problematic are called "workflow automation tools" and Zapier, Mulesoft, Integromat, etc. are in this category.

![Integromat Sreenshot](/blog_articles_images/2020/11/integromat-screenshot.png)
*Integromat Workflow Automation*

The idea is simple: APIs are "Apps". They have inputs and outputs. Something goes in, the app does its job, and something else goes out. The workflow automation tool ensures the data moves from one app to the next one. Apps can be chained to make complex workflows.

While this solves a lot of problems, it is not suited for everything. **With workflow automation, the starting point is an event from an app (or a scheduled event)** and the workflow will move the data from one app to another one. **It acts as a plumbing pipe.**

It connects APIs together more than creating a bundle of products. Here are some limitations:

- **It is not possible to trigger manually an action for a specific data item.** Imagine you have an existing item in your CRM and you want to run it though an automation workflow you have. It is not possible without manually adding a new source event in your workflow.
- Data moves as a whole between apps. It is not possible to transform the value for one property and then merge it back to the data object to be send to the next workflow app. You would need a data storage layer in between.


# Connect Apps through a data hub

So how can we bundle APIs/Products together to create value? **The answer is: through data**. Instead of starting from app events like in workflow automation, the data must be at the center.

<div class="content-video">
 <video autoplay muted loop playsinline width="100%" height="auto" class="d-block">
 <source src="https://storage.googleapis.com/public_datablist/website/videos/run-action-datablist.mp4" type="video/mp4" />
 </video>
 <div class="text-center">Datablist screencast</div>
</div>

**Businesses need a centralized data hub for data related to customers, suppliers, employees, etc. from which partial data can be send to external APIs to be processed.** The results of this process then return to the data hub. It is about consolidating several APIs processes into a single data storage.

From this data hub, automations can be configured on data events like with workflow automation. When a new item (customer, invoice, etc.) is added to the data hub, actions run to process the new data and the result is saved for later use. But now, actions can be trigged manually directly from a data item. For example: send a mail to a customer.

**Building with a data first approach opens the possibility of data standardisation.** With workflow automation, every time you integrate an app (Application A) with another one (Application B), you need to define what properties from the data exposed by *Application A* will be used as input in *Application B*. Integration between two apps needs configuration. By starting from the data, it is not how one app integrates with another one, but how each app connects to the data. The data is the source for each integration.

If a data property is of type **email**, all APIs with an **email** input type can be automatically listed and shown.

# Own your platform and use the power of external APIs

The combo Data hub + APIs is a powerful mix. It gives companies the flexibility to build a platform tailored for their needs without re-developing every feature.

Import leads in your data hub, and get access to a catalog of thousands of APIs to enrich them, validate their information, or send them to dedicated tools like a CRM so the Sales team starts a selling process.

At Datablist, we work to make this vision a reality 🚀. [Join the beta](https://www.datablist.com) (and get 30% off on launch!) or subscribe to the newsletter to know more 🤝.

<h1 id="introduction"><a href="#introduction" aria-hidden="true" tabindex="-1"></a>Introduction</h1>
Last month, <a href="https://www.forbes.com/sites/alexkonrad/2020/10/09/twilio-to-acquire-cloud-startup-segment-for-3-billion/" target="_blank" rel="nofollow">Twilio bought Segment.io</a>, a vertical SaaS software that focuses on customer data integration. This move by Twilio comes after buying Sendgrid last year and follows a bundling strategy. Twilio wants to be a 360° platform when it comes to customer value proposition.
<a href="https://hbr.org/2014/06/how-to-succeed-in-business-by-bundling-and-unbundling" target="_blank" rel="nofollow">In a famous quote</a>, Jim Barksdale said the two ways to make money is by bundling products together or unbundling them and creating targeted offers.
We see both in the software industry today. Microsoft, Salesforce, Oracle and other behemoths keep adding to their software suite, making them ubiquitous. In the meanwhile, we can see an increase in niche SaaS products. From image segmentation APIs to content sentiment analysis tools. The number of SaaS products we use at work keeps increasing.
Integrated suites have the advantage of an unified experience while specialized products take advantage of their expertise and can be laser focus on the problem they solve.
In my previous article (<a href="/blog/where-is-enterprise-software-heading">where is enterprise software heading</a>), I discussed the evolutions in enterprise software as we have moved from ERP to a galaxy of SaaS products (also know as best of breed). This is an integrated suite vs best of breed debate we have now.
In a perfect world, we would be able to compose our own suite by making the different softwares work together. Unfortunately, we are not in a perfect world. Few months ago, a Microsoft sales contacted me to make us move from Slack to Microsoft Teams. I told him my experience with Skype Enterprise and Teams, from an external point of view, was a disaster. It was a hell to work with companies using those tools when you were on another suite (aka Google + Slack). He acknowledged it has not been strategic for Microsoft, but he assured the company started to work on more interoperability.
It might come from my desire to stay away from big corporations but I keep dreaming of a world where software collaboration could exist outside of big enterprise suites.
The number of specialized APIs is impressive. In my last count, I found about twenty companies providing an API for invoice PDF parsing, a good hundred of APIs to check the validity of an email address, and hundreds of companies/people enrichment tools.
Each of these APIs focuses on a single problem and addresses it well. But chances are you don't want to implement tens of different APIs to build an internal tool that would automate your accounting or your customer management. The question is: how can we bundle them to create our own suite?
<h1 id="why-workflow-automation-zapier-integromat-etc-is-not-the-best-solution"><a href="#why-workflow-automation-zapier-integromat-etc-is-not-the-best-solution" aria-hidden="true" tabindex="-1"></a>Why workflow automation (Zapier, Integromat, etc.) is not the best solution?</h1>
The first wave of tools to address this problematic are called "workflow automation tools" and Zapier, Mulesoft, Integromat, etc. are in this category.
<img src="/blog_articles_images/2020/11/integromat-screenshot.png" alt="Integromat Sreenshot">
Integromat Workflow Automation
The idea is simple: APIs are "Apps". They have inputs and outputs. Something goes in, the app does its job, and something else goes out. The workflow automation tool ensures the data moves from one app to the next one. Apps can be chained to make complex workflows.
While this solves a lot of problems, it is not suited for everything. With workflow automation, the starting point is an event from an app (or a scheduled event) and the workflow will move the data from one app to another one. It acts as a plumbing pipe.
It connects APIs together more than creating a bundle of products. Here are some limitations:
<ul>
<li>It is not possible to trigger manually an action for a specific data item. Imagine you have an existing item in your CRM and you want to run it though an automation workflow you have. It is not possible without manually adding a new source event in your workflow.</li>
<li>Data moves as a whole between apps. It is not possible to transform the value for one property and then merge it back to the data object to be send to the next workflow app. You would need a data storage layer in between.</li>
</ul>
<h1 id="connect-apps-through-a-data-hub"><a href="#connect-apps-through-a-data-hub" aria-hidden="true" tabindex="-1"></a>Connect Apps through a data hub</h1>
So how can we bundle APIs/Products together to create value? The answer is: through data. Instead of starting from app events like in workflow automation, the data must be at the center.
<div class="content-video">
 <video autoplay muted loop playsinline width="100%" height="auto" class="d-block">
 <source src="https://storage.googleapis.com/public_datablist/website/videos/run-action-datablist.mp4" type="video/mp4" />
 </video>
 <div class="text-center">Datablist screencast</div>
</div>
Businesses need a centralized data hub for data related to customers, suppliers, employees, etc. from which partial data can be send to external APIs to be processed. The results of this process then return to the data hub. It is about consolidating several APIs processes into a single data storage.
From this data hub, automations can be configured on data events like with workflow automation. When a new item (customer, invoice, etc.) is added to the data hub, actions run to process the new data and the result is saved for later use. But now, actions can be trigged manually directly from a data item. For example: send a mail to a customer.
Building with a data first approach opens the possibility of data standardisation. With workflow automation, every time you integrate an app (Application A) with another one (Application B), you need to define what properties from the data exposed by Application A will be used as input in Application B. Integration between two apps needs configuration. By starting from the data, it is not how one app integrates with another one, but how each app connects to the data. The data is the source for each integration.
If a data property is of type email, all APIs with an email input type can be automatically listed and shown.
<h1 id="own-your-platform-and-use-the-power-of-external-apis"><a href="#own-your-platform-and-use-the-power-of-external-apis" aria-hidden="true" tabindex="-1"></a>Own your platform and use the power of external APIs</h1>
The combo Data hub + APIs is a powerful mix. It gives companies the flexibility to build a platform tailored for their needs without re-developing every feature.
Import leads in your data hub, and get access to a catalog of thousands of APIs to enrich them, validate their information, or send them to dedicated tools like a CRM so the Sales team starts a selling process.
At Datablist, we work to make this vision a reality 🚀. <a href="https://www.datablist.com" target="_blank" rel="nofollow">Join the beta</a> (and get 30% off on launch!) or subscribe to the newsletter to know more 🤝.

How to use a Data Hub to connect APIs


Since the beginning of the digital transformation in the 1960s, enterprise software has played a key role in increasing companies productivity. They have moved from custom development, to ERP, to the current best-of-breed approach. Automation and AI are around the corner to drive the next revolution. If the different shifts in technologies that happened in the past are any clues to look forward to, chances are the future of enterprise software will look very different than today.

# 1. The ERP age

Before the 1980s, most of enterprise software programs were custom built. The 1980s brought software standardization with applications developed to manage inventory and manufacturing processes in the industries. Applications spread to all the business needs including finance and human resources. In 1990, the advisory firm Gartner coined the term ERP (for "Enterprise Resource Planning") to refer to this category of business management software programs.

The following decade, the 1990s, was the golden age for ERPs with rapid growth in adoption across medium and large companies in a market dominated by firms like SAP, Oracle, and HP with on-premise solutions.

![Odoo ERP](/blog_articles_images/2020/10/example-erp-screenshot-odoo.png)
*A screenshot of Odoo, an open-source ERP*

## 1.1. Where ERPs shine

### Data centralisation

The major innovation introduced with ERP is a central and real-time view of the company data. Inventory, Finance, HR and other applications share a unified data storage. This data consolidation enables cross departments analytics and simplifies company planning.

### Extensible

Moreover, ERPs are extensible. All the applications living in the ERP share a common underlying architecture and have the same interface components. All the ERP market leaders expose their platform to external developers. This creates opportunities for third parties applications, and internal processes that are too specific to the company can be built as applications on top of the platform.

## 1.2. ERP limitations

### Complexity

**This globally consistent view of company financials and inventory come at the expense of complexity.** A triple complexity! Complexity to implement, to use, and to extend.

ERP systems come in a one-size-fits-all monolith strategy. **To be implemented in a company, the company must comply with how the ERP works.** ERP applications are not flexible enough to map custom processes the company might be using. When you add that the implementation must cover several departments, therefore several applications, to be valuable, you end up with scary stories. In 2017, the German grocery chain Lidl, [after 6 years and $500 million spent in SAP implementation](https://www.handelsblatt.com/today/companies/programmed-for-disaster-lidl-software-disaster-another-example-of-germanys-digital-failure/23582902.html), decided to roll back the entire inventory system to their previous custom made platform. More recently in Australia, [a SAP implementation in Australia](https://www.youngwitness.com.au/story/6938588/qld-health-still-fixing-inventory-system/) cost 33,5 Millions $ (AUD) more than planned [with mixed results](https://www.theregister.com/2020/09/28/queensland_health_sap_s4hana_rollout/).

It is not surprising that small companies, looking for fast implementation and flexibility, are only a few to use ERPs.

When implemented, ERPs are hard to use. Because all applications are built with the same components, they all share the same User Interface patterns. **The interface is more a representation of the underlying data than a targeted interface for a defined application.**

To finish on the complexity issue, unlike the promise, extending an ERP is not easy. First of all, they are monoliths. Meaning all the applications and data live in a single system. Developing over it can impact the stability of other applications. **Motivation to develop an application on the ERP must be validated by the company.** A single team wouldn't take the responsibility to extend the company ERP without asking permission. In addition, if an existing application doesn't suit the organization, customizing the code prevents the organization from automatically updating the application when a new version is available. Resulting in maintenance time and instability.

![ERP Complexity](/blog_articles_images/2020/10/erp-complexity.jpg)
*Someone, somewhere, trying to implement an ERP*


### Closed world

ERPs live in a closed world. They are only designed to interact with the company employees. **Suppliers, partners, and customers, although modeled in the ERP database, do not interact directly with it.**

The isolation is also true for the applications. Only applications designed for the ERP will play nicely with the data and the processes integrated with it. **Interconnecting an ERP with external HR or Finance software is complex and this is not the way ERP vendors want you to use their product.**

As a result, you need an SAP software developer to build an application on the SAP platform. Some technologies are proprietary and diverged from other programming languages and practices used elsewhere. As a result, salaries for ERP developers are higher than average.

### Lock-in

Long implementation, centralized data, and closed world inevitably introduce vendor lock-in. The average ERP life cycle in a company is 15 years (time between a first implementation and a new implementation). As a result, the market is concentrated in the hands of a few big players. And it is not to displease them.


# 2. The best of breed revolution

Starting in the 2000s, enterprises moved from vertically integrated organizations toward agile and domain specialized entities. **This move was accompanied by companies opening to the outside world, especially to interact with customers.** The one-size-fits-all ERP systems started to lose momentum in favor of specialized cloud-native applications. **From a single system, the organizations were now choosing to have different software programs, often from different vendors, to meet their ever-growing needs.** Over the past few years, an increasing number of companies have switched to this approach, called “best of breed”.

![Odoo ERP](/blog_articles_images/2020/10/SaaSSpending.png)
*SaaS Spending Evolution (source: Blissfully)*


## 2.1. The benefits of best of breed

### Cloud Native

The rising of best of breed philosophy is concomitant with the rise of SaaS (Software as A Service) that live in the cloud. This new wave of enterprise software embraced the possibilities created by the cloud and offered a totally different experience.

**Compared to on-premise hosting, cloud-native applications are fast to implement.** The setup no longer involves buying servers and sizing the infrastructure but is reduced to configuration and interconnection with the company's other systems, shortening the time to market. Software update responsibility returns to the software vendor, saving the company maintenance cost.

It also changes the business model of enterprise software vendors. They switched from set up fees + product licenses to on-demand pricing. How many employees do you have? How many customers? **The goal is to capture value as the company grows.**

With this fast implementation and linear pricing, a myriad of new software editors was created to target all company size. From small companies to multinationals.

### Flexibility

Single-purpose applications bring flexibility to the company. Because it is hosted by the vendor, **decision to use an application is slowly pushed back to the team responsibility.** With companies embracing agile organization, teams are empowered to test and decide on their own the tools they want to use. Once implemented, if the results are insufficient, the team can switch to another vendor while keeping the other applications it might use. This “test fast, change fast” is another principle of agility and it is meant to accelerate innovation.

Most of the best-of-breed applications expose API to connect them with other applications. The enterprise processes are then scattered across multiple applications.


### Easy to use

Because applications address specific needs, **the user interface can be adapted to the intended user who is using the applications.** Applications meant for salespeople can have a different user interface than the ones for human resources people. With all the existing devices (smartphones, tablets, laptops, smartwatches, etc.), software vendors can offer personalized user experiences.

### Open to the external world

Finally, best-of-breed applications unveil a first step toward opening the enterprise software to the external world. **Where ERPs focuses on company members, applications now interact directly with external parties: customers, suppliers, partners, etc.**

This opening is still limited but the trend is accelerating.


## 2.2. Best of breed downsides

The best of breed approach comes with its own challenges, especially the consequences of having a system made of multiple programs, data storages, and vendors.

### Siloed Data

Ironically, while ERPs main advantage is a central data storage, the biggest issue with best-of-breed patterns is data silos. Every software is independent and has a dedicated database. **On a company scale, data silos reflect the different business units and teams.**

This has consequences. First, generic data types like customer information might be stored in a CRM only accessible for a part of the company, preventing people from having access to important information. And even if the CRM is connected to other programs to expose customer information, it can lead to data duplication and inconsistencies. This is referred to as the "source of truth" problem.

Finally, when the data is kept in several places, the company doesn't have a global view of its activity. Running analytics to find a correlation between data is not possible. Aligning vision, automation, analytics and planning get complicated without a global view of the data.

![Data Silos](/blog_articles_images/2020/10/data-silos.jpg)
*Data Silos in an organization*

### Integration Complexity

In the best-of-breed world, software vendors focus on their application. **Complexity is moved to integrating the applications together.** How to connect data between applications? How to keep it secure? How to synchronize common objects (like customers) across them? These questions are left to the companies to answer.

In a 2019 survey conducted by Mulesoft, a provider of integration solutions, [IT leaders reported that only 29% of their applications are currently integrated together](https://www.mulesoft.com/press-center/technology-trends-2019-connectivity-benchmark).

There are diverse reasons for this lack of integration. Data models are different between applications. Customer attributes in a CRM can be different than the ones needed in a financial application. They are different and the data models can change. The frequency of deployments (and changes) is increasing with the use of agile methodologies adopted by more and more software vendors. Furthermore, some software vendors don't want to expose their data. In the data gold rush, it is better to receive data from other applications than expose its own data for others to create value with it.

**Companies are far from reaching a global view of their operations with this scattered data.**

![Number of applications per company](/blog_articles_images/2020/10/number-app-per-company.png)
*Number of applications per company has skyrocketed in the last years. (Source Blissfully 2019)*

### Hard to build on top of an applications mesh

From an ERP market with few big players, the best-of-breed approach has enabled new vendors to join the enterprise software dance. This leads to vendors offering specialized applications to many niche markets.** Yet, often, an organization's digital system can't be only made of vendor applications. Custom development is needed to answer specific needs.**

Best of breed makes it difficult to build a custom application on top of this distributed system. Integration with vendor applications is not always possible. **Some applications offer packaged integration with other market leaders but don't open their API to custom development.** Hosting, testing, and maintenance lead to high cost for internal application development.


### Duplication of costs

To conclude on the downsides, the costs of all the applications can add up to become quite expensive. Keeping track of all the applications used in the company is not easy, some vendor licenses keep being paid while not used. Hundred, or thousands of business applications can drain down the profitability of the organization.

![Duplication Subscription](/blog_articles_images/2020/10/duplication-cost.png)
*Number of applications paid several times by different teams. (Source Blissfully 2019)*



# 3. The future of enterprise software

Marc Andreessen famously said in 2011 that ["software is eating the world"](https://a16z.com/2011/08/20/why-software-is-eating-the-world/). Companies that embrace software both in their operations and as a driver of growth will continue to lead the digital transformation. Predicting the evolution of enterprise software is not easy, but we can look at the current trends to predict what will happen in the market.

The following paragraphs are what we, [at Datablist](/blog/announcing-datablist), believe will shape the future of enterprise software. We can see four strong tendencies:

- **Agility and the power of autonomous teams.**
- **AI, automation and analytics in data-driven organizations.**
- **Leveraging collaboration with the external world**
- **Security concerns in a competitive and hostile world**


## 3.1. Agility and the power of autonomous teams

Agility can be defined as an employee's capacity to analyze information in the environment and respond to that information quickly and autonomously. **It has been proven that decentralization is key to maintaining innovation and adaptation in an ever-changing world.**

We believe the enterprise software landscape will continue to be trusted by best-of-breed vendors instead of all-in-one ERP solutions.

- **SaaS solution living in the cloud and pay as you go**. Flexibility and fast implementation is where SaaS software shine. Cloud-native applications bring automatic updates, few maintenance costs, and predictable pricing. Data ownership and security with the cloud are growing concerns for companies but the SaaS vendors are adapting to those needs and we see no reason the cloud would lose it current momentum.
- **Customer-centric experience**. Agility means the buyers are now the users. Developers can choose the tools they want to use, salespeople can decide what CRM they will work with. This need for flexibility can only be achieved with the best-of-breed approach. They choose their products based on online reviews and network feedback. They are informed buyers and want to choose and not be sold to. User experience and design will be more and more important in enterprise software.



## 3.2. AI, automation, and analytics in data-driven organizations

Productivity increase in an organization comes from both employee productivity gain and automation of organization tasks. Automation relies on data. **The businesses that will benefit from automation are the ones that will build a consolidated data view on top of their mesh of best-of-breed applications.**

### A centralized data space

Connecting best-of-breed applications together is not enough. First, you are limited by the connectors the applications provide. Applications have connectors to market leaders. But what about small vendors? Worse, what about your internal custom applications? **Second, it is not in the best-of-breed vendor's DNA to make sense of the other application's data.**

> Your CRM doesn’t care about your employees data

A centralized data space gives the organization a consolidated view over the data, whether it is users generated data or organization-generated data.

**We believe enterprise data warehouses will gain popularity as a foundation for automation and internal custom applications.**


### Platform to build on top of the data

Despite the ever-growing number of enterprise software programs, there will still be times where custom development would make more sense. Those internal applications will sit on top of the centralized data space and use the data to deliver value to the organization.

Modern development tools are shortening the development life cycle. Serverless and container technologies are perfectly suited to embrace this trend.

**Components and APIs that solve a single purpose will work as building blocks for those internal applications. OCR technologies, text analysis, company verification services are generic needs that will be provided off the shelf.**

![Datablist Centralized Data Space](/blog_articles_images/2020/10/datablist-centralized-data-space.png)
*A centralized data space for unified analytic views and automation*

## 3.3. Leveraging collaboration with the external world

Innovation comes when the best idea wins regardless of whether the idea comes from. This first means organization must aggregate insights and ideas from everywhere. We have seen companies opening up to customers to listen to their feedback. This trend will accelerate. Contributions can come from employees, customers but also vendors or partners. **Organizational boundaries will be porous so all the community can participate in the company mission and be involved in the decision-making process.**

We already see tech companies betting on open source, sharing their working methodology, the design patterns, and overall being transparent on what they learn. It engages employees and creates purpose.

Enterprise applications will foster collaboration, with people inside the company and outside of it.

## 3.4. Security concerns in a competitive and hostile world

The more organizations become digital and open, the more they are vulnerable. Losing trust from customers in this flexible market is not acceptable.

Organizations will use automatic tools to constantly audit their assets and vulnerabilities. Developer's codes, vendor applications, and products will be mapped in a global view of the organization systems.


# Conclusion

As companies move from a single one-size-fits-all ERP world to a galaxy of SaaS applications, new problems arise:

- **Data is siloed** between Sales, R&D, Operations, etc.
- Processes based on multiple data sources **cannot be automated**
- **Companies can’t build** applications over this mesh of SaaS tools

[At Datablist](/blog/announcing-datablist), we believe those problems can be solved with a secure centralized data space coupled with a platform to build custom applications and automation.

If you share our vision, subscribe to our newsletter below and follow us on [Twitter](https://twitter.com/datablist) 🤝.


Where is enterprise software heading?

Datablist idea comes from a personal frustration in my previous company. I was CTO of an e-commerce software company. One of the goals for the product team was to automate everything redundant. We had built a development pipeline to streamline and automate all the actions between a developer code push and the deployment in production of our software. On every deployment, a Slack bot pushed the release note it has built on its own using the Jira backlog. The DevOps philosophy we followed encourages automation to speed deployment and to remove human error from it. **My frustration came from the gap between the level of automation we had in our software development processes compared with all the manual tasks that needed to be done to "run" the company.** We used tools for sales, human resources, customer management, etc. **Those tools accentuate the existing organization silos by keeping control over the data.**

I'm a coder and I tried to aggregate the data by building data bridges and workflows using Google Spreadsheet, Zapier, or other workflow automation software. I found myself to be quickly limited by those tools.
I needed to have more control over the data and I had to store it in an external database, with computing instances to read files and images. I finally gave up when I realized I was building a web infrastructure as complex as the one we had for our e-commerce software.

I thought about this problem for a long time. Now that we have digitized everything from supplier invoices to customer contracts, why don't we move toward more automation for the organization processes?


- Supplier invoices scattered in our emails or their websites must be gathered together
- Employees onboarding with welcoming emails, account configurations must be automated
- Reconciliation between invoices and bank transactions must be automatic
- **Customers and business data must be stored in a data warehouse fully accessible to the whole company**


Same as in a software development team, the organization processes and repetitive tasks must move to a digital operating model. A digital representation of the processes made with software and algorithms.

**This vision cannot simply be done with pipes that connect software programs together. Nor with bots that mimic human actions. Complete access to the data and a platform to create tailored algorithms on this data is mandatory.**

**A digital operating model is infinitely scalable**, can be improved at an impressive rate, and is more resilient to human error. **The organization data, when aggregated and consolidated, give people information about the situation so they can take initiative and build on this data platform.** This is what Agile organizations are about.

It is about moving human activity from performing the processes to designing and monitoring the automation applications that perform the processes.

**This is just the beginning. If you share our vision and are interested in influencing the direction of Datablist, subscribe to our newsletter below and follow us on [Twitter](https://twitter.com/datablist) 🤝.**


Datablist idea comes from a personal frustration in my previous company. I was CTO of an e-commerce software company. One of the goals for the product team was to automate everything redundant. We had built a development pipeline to streamline and automate all the actions between a developer code push and the deployment in production of our software. On every deployment, a Slack bot pushed the release note it has built on its own using the Jira backlog. The DevOps philosophy we followed encourages automation to speed deployment and to remove human error from it. My frustration came from the gap between the level of automation we had in our software development processes compared with all the manual tasks that needed to be done to "run" the company. We used tools for sales, human resources, customer management, etc. Those tools accentuate the existing organization silos by keeping control over the data.
I'm a coder and I tried to aggregate the data by building data bridges and workflows using Google Spreadsheet, Zapier, or other workflow automation software. I found myself to be quickly limited by those tools.
I needed to have more control over the data and I had to store it in an external database, with computing instances to read files and images. I finally gave up when I realized I was building a web infrastructure as complex as the one we had for our e-commerce software.
I thought about this problem for a long time. Now that we have digitized everything from supplier invoices to customer contracts, why don't we move toward more automation for the organization processes?
<ul>
<li>Supplier invoices scattered in our emails or their websites must be gathered together</li>
<li>Employees onboarding with welcoming emails, account configurations must be automated</li>
<li>Reconciliation between invoices and bank transactions must be automatic</li>
<li>Customers and business data must be stored in a data warehouse fully accessible to the whole company</li>
</ul>
Same as in a software development team, the organization processes and repetitive tasks must move to a digital operating model. A digital representation of the processes made with software and algorithms.
This vision cannot simply be done with pipes that connect software programs together. Nor with bots that mimic human actions. Complete access to the data and a platform to create tailored algorithms on this data is mandatory.
A digital operating model is infinitely scalable, can be improved at an impressive rate, and is more resilient to human error. The organization data, when aggregated and consolidated, give people information about the situation so they can take initiative and build on this data platform. This is what Agile organizations are about.
It is about moving human activity from performing the processes to designing and monitoring the automation applications that perform the processes.
This is just the beginning. If you share our vision and are interested in influencing the direction of Datablist, subscribe to our newsletter below and follow us on <a href="https://twitter.com/datablist" target="_blank" rel="nofollow">Twitter</a> 🤝.

Blog

Serverless will move software collaboration to the cloud!

Brace yourself, the battle for a unified data model is coming!

How to use a Data Hub to connect APIs

Where is enterprise software heading?

Announcing Datablist!