DataOps: The Holy Grail of Data Management

October 9th, 2020

Zaloni’s recent DataOps Virtual Event featured a Q&A session with Principal Consultant and Founder of the Eckerson Group, Wayne Eckerson. Eckerson Group helps organizations turn data into insights and action. They examine five key architectures – business, organization, analytics, data and governance – and recommend ways to improve each and stitch them together into a coherent whole. During the virtual event session, Wayne shared his expert opinions and advice on data automation through DataOps, describing it as holding the promise of the holy grail of data management: better, faster, cheaper. Watch the full video below and read on for the seven essential keys to success Wayne recommends in a DataOps platform.

Video: Wayne Eckerson and Amy King discuss DataOps and the Convergence of Data Management Capabilities 

Achieving Data Automation with DataOps: People, Process, Technology

How are companies achieving success in DataOps? Wayne advised using the PPT (People, Process, Technology) methodology to drive action. (Side note: Leilani Moll, VP of Data & Analytics Services at Bremer Bank gave the same advice on her webinar about creating a successful Customer 360 initiative using a DataOps approach).

People: IT and the business need to get together, educate themselves on DataOps and how it can solve common goals.

Process: Identify the bottlenecks and go after them systematically with continuous improvement.

Technology: The tool used to implement DataOps should be an end-to-end solution, bringing all the diverse data management components into a single AI-powered, metadata-driven, and collaborative platform. Data automation processes like continuous improvement (data quality) should be built into the platform.

The Convergence of IT and Data Science = Better Outcomes

It’s clear that an end-to-end view of your organization’s data supply chain, no matter where the data  resides (cloud, multi-cloud, on-premise, hybrid) is the key to success. A platform designed for both IT and business users where workflows are automated, processes are streamlined and people can collaborate will always lead to better outcomes. Let’s dive deeper into Eckerson’s seven recommendations for selecting DataOps technology. 

Seven Essential Keys to Data Automation Success in a DataOps Platform

1. End-to-End Functionality

Look for a tool with the functionality you need from end-to-end, however, Wayne cautions that “end-to-end” is both a benefit and liability. The convergence of point-solution products into end-to-end platforms has made modern DataOps possible, but if the platform is not extensible you could find yourself in a vendor lock-in situation. Will the platform adequately support your developers? Does it have open APIs for 3rd party plug-ins? 

2. Extensibility

In the article on Tech Native titled The 7 Benefits of DataOps You’ve Never Heard, Susan Cook, CEO of Zaloni writes “Extensibility is the crux of today’s DataOps platforms. To achieve streamlined, accelerated, optimized data ecosystems, transparency across the entire supply chain is essential. The only way to deliver complete data lineage, standardized enterprise-wide governance, and ML-based workflows and recommendations, is to have a platform that connects to every technology and vendor in the data ecosystem. The best DataOps companies are able to take extensibility one step further by enabling enterprises to keep what is working in their data architecture and replace only what is necessary. This “stay and play” approach to both data and vendors reduces costs, accelerates timelines, and often overcomes hurdles that have previously blocked data project success.”

3. Broad Partner Ecosystem

As mentioned above, the only way to deliver true end-to-end DataOps is to have a platform that connects to every technology and vendor in the data ecosystem. Look for a platform from a company with a broad partner ecosystem you can easily rely on now and as you scale.

4. Portability – Cloud, Storage and Compute Agnostic

“In order to successfully unify data sources, discover any entity within the catalog, and provision to the sandbox or destination of their choice, companies need a single pane of glass DataOps platform that connects to every cloud or on-premises source and scales across new technologies over time.” (Also quoted from The 7 Benefits of DataOps You’ve Never Heard)

5. Standardized Governance

Enterprises need to secure data without hindering business goals. Maintaining regulatory compliance shouldn’t slow down the time to data delivery. A good DataOps platform should give the enterprise confidence that the right data is being seen by the right people. It should provide the right people with governed, self-service data access which reduces the burden on IT and speeds up the time to insights.

6. Scalability and Elasticity

A modern cloud architecture provides both the scalability and elasticity needed to automatically adapt to workload changes, thereby maximizing resources and minimizing costs. Look for a DataOps platform with native integrations to cloud services that provide both scalability and elasticity, as they are not synonymous. 

7. Modularity

This concept ties in with extensibility and portability. Look for a DataOps platform that allows you to pick and choose what you want to deploy and integrate with what you already have. A good DataOps platform shouldn’t force you to “rip and replace” systems and tools you’re already happy with.

Have courage

When asked where the DataOps momentum is emerging from within the enterprise, Wayne has seen it start in IT, under the constant pressure to find the holy grail: to be faster, better and cheaper. And now they can finally achieve it! But it takes courage from both sides – IT and business. The most successful practitioners of DataOps spend time together reviewing processes and technologies systematically to figure out how to do thing faster, better and cheaper. It takes courage to slow down so you can go faster.

For more information, watch the full DataOps Virtual event, now available on-demand.