New study confirms: the need for decentralized data base module

Second study done this year by Starburst and Red Hat companies discovered that organizations have higher demands regarding data accessibility and flexibility

New market study ordered by Starburst and Red Hat companies discovered that 55% of organizations claim that Covid had raised substantially the importance of data accessibility. This data reflects a minor rise compared to results of last year study.
As a result, organizations are planning to prioritize flexibility that comes from multiple cloud infrastructure when coming to pick solutions for their data infrastructure.

An annual report, done by independent research company Enterprise Management Associates (EMA), titled “Data status and what’s next”, had found that transition to an infrastructure that enables agility and flexibility when accessing data is essential for business success and its ability to provide an optimal user experience in today’s fast changing environment.

The new study points that global organizations are relying more and more on data and are being pressured to make data accessibility better to comply with costumers demands in a fast, progressive digital age. The gap between the need for quicker access to data and the need for data centralization in a single data base, produces three major challenges:

1. The spread of data is expending in a complex manner: 43% of study participants had stated that their organizations are using four to six data platforms on average. 11% of the participants had stated that they are using 10-12 platforms simultaneously. Furthermore, the organizations are using new kinds of data that are adding up to their work environment in an increasing pace. 65% of the survey participants stated that most of the data they are about to add to their work environment is streaming data. And 60% of the participants stated that they are planning to add to their work environment video data and data from events.
2. Increasing problems in liner work process that traces pipeline: due to the Covid epidemic organizations have to accelerate their data dependent decisions making process to withstand clients fast changing expectations. However, 48% of the participants stated that they take more than 24 hours to create a data pipeline. 51% of the participants stated that it takes another 24 hours to pump the data through said pipeline until production. The amount of time is making it very difficult for real time business response. All of that, combining with the need for a quicker data access, is causing companies to abandon the pipeline workflow and move to a more decentralized module, or as it called – Data Mesh.
3. AI and machine learning are increasing pressure on different programs: research company EMA has recognized that scientific data (machine learning and neural network) had made data the most important thing in the work process, which further increases the load on already complex data platforms. Organizations need to process a huge amount of AI and machine learning data to start work process. 31% of the participants said that the fact that data is constantly moving and changing makes it even harder to find. This, added to the effort to find sources to scientific data, requires a better workflow automation and better data accessibility that will improve the AI and machine learning modules, and will decrease the need of finding new sources.

“Data is every business breath in digital economy”, says Starburst CEO and co-founder, Justin Borgman. “The result of the most recent study shows that organizations have a clear demand for a quicker, more secure data access, but the traditional approach that controls organizations is hindering the efficiency. As there are more and more different kinds of data, and as traditional work process aren’t providing solutions for data expansions challenges, organizations are interested in new approaches; for example, data Mesh architecture which produces a decentralized data module. Starburst allows its clients to access all their data, regardless of where it is, thus helping them to make better and faster business decisions, without relying on an ETL (Extract, transform, Load) process that takes a lot of time and resources.”

The study also discovered that organizations are interested in certain practices to withstand the challenges they are facing:

1. Transition to a more decentralized data module: there are many advantages to a centralized data base approach, including a more comfortable cost, higher data control level, and better management. Nevertheless, data centralization also comes with an increasing risk and limited flexibility. This lack of flexibility might hurt the business ability to adapt to a fast-changing environment.
2. Essential programs automation: the main challenge which organizations are dealing with is a multiple cloud hybrid work environment (40% of the participants stated this a challenge). Organizations are interested in programs automation for different reasons. 38% of the study participants stated the need of automation to keep the organization competition ability, focusing on automation of basic data infrastructure and data transfer. 33% had stated the need of creating an intelligent search engine, and 32% stated the need of integrating AI and machine learning data to improve business decisions.
3. Flexibility enabling multitude cloud environment: in a study conducted last year the participants stated that 56% of the company data was in the cloud, comparing to 44% of the data remaining in the company website. In a study conducted this year the participants stated that 59% of the organization data was already in the cloud compared to 41% in the company website. The rise of data transferred to the cloud explains the fact that 43% of the study participants stated that a flexibility enabling multitude cloud environment is the most substantial factor in purchase decisions. 34% of the participants stated that the most substantial factor in purchase decisions is the ability to mutually incorporate the systems. Last year only 26% of the participants saw that as an important factor.

“Clients who produces AI and machine learning supported applications have to depend on a data system that allows accessibility to accelerate the module development and the creation of smart applications in a multitude cloud hybrid environment”, says Red Hat program engineering senior director, Steven Hules. “By creating a base for data and applications in cloud architecture, developers and data scientists can withstand their business destinations in a faster, more efficient way, and to provide data based, smart applications.”

Methodology

The study was conducted in January 2022 among 400 participants in the USA, Britain, Canada, France, Germany, Australia, and Singapore. The company conducting the study is the same independent research company called Enterprise Management Associates. The participants were decisions makers in the field of data systems and data management professionals from companies from an array of sectors, including, industry, manufacturing, retail, telecommunication, finance, and more.

About Starburst

Starburst is a SQL based engine manufacturing a virtual layer for quick and efficient extraction of decentralized data from each DB. The tool allows instant data analysis without performing transfer or duplication processes. The company provides the fastest and most efficient analysis engine for Data Warehouses, Data Lakes, or Data Mesh, which allows quick data access, regardless of where it is. Starburst is making data queries in each data base and allows instant use of the data for data-based organizations. Starburst saves organizations costs of infrastructure investment and data analysis ways. In addition, it makes it easier for clients to work in a hybrid work environment and uses tools that exists in the organization. Among Starburst clients there are companies such as Comcast, FINRA, And Condé Nast.

About Aqurate

Aqurate, a subsidiary of ULC, provides BI, Big Data, and end to end analytics solutions. The company represents Starburst and four more leading data platforms allowing the organizations to be data driven: Talend, DBT, Trifacta, and HVR.
The company adjusts its solutions to the business needs of its clients and enables them to get insights, to make better decisions, and increase their business value. In addition, the company offers professional services in data areas in an array of technological jobs and projects, while focusing on the different data roles. The company works directly and by secondary distributors such as Taldor, Malam and Matrix BI. The company’s clients include government sector organizations, health organizations, insurance companies, finance organizations, and high-tech companies.