Every company is competing in markets that are becoming more data-driven. According to Jeff Pollock, VP of Product Development at Oracle, with each year that passes, it becomes clearer that there is a direct link between economic winners and organisations that excel at using data for innovation and efficiency. As a result, Jeff explained that data mesh is generating so much interest now because it is the most modern way to make data easily accessible across modern multi-cloud IT architectures.
The term ‘Data Mesh’ may sound familiar to some but what does the concept really mean? According to Jeff, in IT, a mesh can mean a decentralised approach – not hub and spoke – for example, as seen in a Wi-Fi mesh or a service mesh. Typically, a mesh is known to provide better speed and coverage. He added that the word “mesh” itself conjures up the image of a fabric – think of the weave on a loom – where the threads of an interlaced structure give many paths for something to travel across it.
“At a basic level, you can take this image across to the concept of a data mesh. Fundamentally, it is about recognising that enterprise data has value and releasing it from its current rigid, often quite disconnected and monolithic data stores into decentralised data architecture, making it better, faster, cheaper to achieve business transformation objectives,” he explained.
As regular consumers, each of us may buy products and services every day. From a morning coffee to a computer or a car, we intrinsically understand their value. But what about data? Today, data is an asset with explicit and implicit value. To take advantage of this value and maximise its potential, a Data Mesh will strive to make data as liquid as possible – making data events easy to detect, replicate, transform, and analyse across multiple clouds and at great distances.
It also means being able to work with data even as it is generated in real-time and, therefore, the mesh must be capable of detecting events in any part of the network where data is born.
Achieving Business Transformation Objectives by Enhancing Data Availability
Since every single company is competing in markets that are increasingly data-driven, so much of the value that they are creating is digital. According to IDC, the digital economy is accelerating as organisations race to recover; over 65% of APAC GDP is expected to be digitalised between now and 2023.
“Business leaders are leveraging technologies to adapt for survival, accelerate for growth and reimagine to lead. APAC is already leading the race with digital core investments to strengthen organisational foundations and digital innovation initiatives. Increasingly, business leaders are thinking about data as an asset – as a kind of capital. This isn’t a metaphor like “data is the new oil”. Data fulfils the literal, economic textbook definition of capital. Capital is a produced good; it is not a natural resource. You have to invest to create it, not just dig it out of the ground.”
What has brought the data mesh concept to the fore is the need for a change in thinking of how companies manage data and to make it more easily available because many can’t. The older, more monolithic ways of doing data integration or using isolated data lakes are no longer sufficient to meet the needs of leading organisations today. There is a broad recognition across the industry that data needs to become less centralised and more real-time.
Since a lot of the innovation today relies on data, everyone is looking to use data to spot the gap, improve customer service, and deliver products and services online more compellingly. This relies on both traditional, highly skilled developers and increasingly on those working in the business doing rapid application development using new low- or no-code tools to bring ideas to life.
Businesses also want to make data access more self-service, particularly around analytics that democratises data and lets people in the business get the insight they need. This removes the need for them to rely on it to be provided through often process-intensive projects, which is a win-win for both the business and IT.
Jeff also mentioned that “Another reason for wanting all data to be quickly and easily accessible is today’s use of Artificial Intelligence (AI) and machine-learning (ML). This needs access to massive amounts of data to run well and new data is essential for ongoing learning. This is essential not only for the insights that can be revealed and potential predictions that can be made but also to support the extreme automation of our digital systems and increasing connectedness to the physical world – think 5G, robotics, biotech, internet of things, etc.”
As the digital economy transforms more and more industries, the opportunities for businesses to use data productively are multiplying each year. Basically, all products and services are going to be radically transformed by the availability of data and data products.
With Many Paths Leading to Data-Centricity, Why Data Mesh?
Data mesh is a decentralised architecture that helps unleash value from distributed data stores. So, not only medium to large enterprises with data across on-premises and/or multi-cloud can benefit from this, but also industry solutions that run across networks. For example, data-centric IoT applications in energy grids, oil and gas field operations and sporting events require real-time telemetry such as races, signal intelligence for military operations, etc.
“Even small businesses can be interested in exploring the value proposition of a decentralised data mesh to future-proof for growth and to support agile application development. Here perhaps you would start with a little bit of mesh thinking by investing in tools that do mesh or hub patterns. That way you can start with a small hub but grow into a mesh,” explained Jeff.
Data mesh encourages businesses to focus on data products and the outcomes that these data products produce. The data mesh approach emphasises cross-functional ‘business domains’; think of this in terms of the finance department, or marketing, sales or HR, and as an item of value to be managed and cared for within a lifecycle.
This is in stark contrast to how IT organisations have typically treated data management, which has usually been aligned to the organisational structure of the company. But business domains and data domains are usually cross-cutting and span multiple organisations. A mature data mesh implementation will be supported by cross-functional teams and data stewardship.
A data mesh reunites data and purpose. For many decades, the industry has separated its stores of data, from the analytic processing carried out to make sense of it. This has meant that data has had to be moved around typically in batches on a daily, weekly, or monthly basis, between these systems bringing a range of problems – delays in getting insight from data, lack of consistency, greater complexity of systems and heavy workloads for those doing the manual tasks of moving it around, and a lack of understanding and commonality of purpose across the IT team who manages the data and the business that uses the data.
To help organisations develop and benefit from the data mesh concept, Jeff said Oracle is able to provide world-class cloud services that support event-driven, streaming data and that can run in decentralised architectures. He singled out the GoldenGate platform as an important part of Oracle’s multi-cloud and on-premises data strategy as it is certified on non-Oracle public clouds and can help customers connect their streaming data events globally across networks.
Going forward, he said Oracle plans to continue investing in GoldenGate as the world’s most powerful platform for data mesh capabilities.