Zaloni Arena 6.0: the dawn of the modern DataOps

Ben Sharma June 24th, 2020

Meet Zaloni Arena, the modern DataOps platform from Zaloni

Meet Zaloni Arena – the modern DataOps platform that optimizes and secures data pipelines to enable better, faster analytics, reduce the burden on IT, and lower data costs. Our new Arena 6.0 release dramatically improves the ways data engineers and data consumers catalog, control and consume data in hybrid and multi-cloud environments. Arena addresses many enterprise data pain points with an end-to-end solution that combines data unification, discovery,  governance, active metadata management, collaboration, mastering, and self-service provisioning, into one extensible platform.

The name “Arena” drives the mindset of a gathering place – in this case for data and analytics. Using a modern approach, Arena is based on addressing key requirements of increased demand for enterprises to move their data platforms to cloud, need for strong governance to meet compliance requirements, manage and contain data sprawl while enabling agility, collaboration and self-service. 

Why organizations fail with data

Traditional enterprise data management is a manual,  tedious process and one of the main bottlenecks to analytics speed and agility. This is compounded by the fact that enterprises also have to demonstrate compliance with a range of security and governance requirements. As the addition of new data sources, the conveniences of the cloud, and multiplication of vendors continues to rise, the complications of data sprawl multiply. Business use cases fail to gain momentum as every data project demands already limited IT resources needed to source data, ensure its quality, and make it available in a consumable format. 

Rinse and repeat this for various lines of business, development, stage, production, etc. A popular research report by IDC, shows that the 80-20 rule still rings true with 80% of analyst time being spent on find, preparing and protecting data and only 20% spent on analytics. 

Similar problems are prevalent on the consumption side. Analysts often wait weeks, if not months, to gain access to requested data, only to find it has become stale, is not what they needed, or is of poor quality.  “Time to insight” – one of the key metrics for analytics organizations is often too long, not able to meet the timeliness of the insights for making critical decisions. The increased need to make swift decisions using IoT data, streaming data, and location data have only heightened the challenge.

Enter the Zaloni Arena

With the launch of Arena, Zaloni brings to the market a unified DataOps platform for modern data architectures that are decentralized, require strong governance, self-service, and require deployment across multiple environments. Arena is the only platform in the market today that brings together management of cloud native data services from multiple cloud providers using a Kubernetes based deployment model. With built-in data governance and self service access, Arena is able to provide a single pane of glass with full visibility and control of all the data assets in a decentralized architecture. 

Arena’s features divide into three primary areas:  Catalog, Control and Consume. With the data sprawl facing enterprises today, the ability to inventory and catalog data from any source is critical. Arena allows enterprises not just to connect to various data sources wherever they exist, but also captures active metadata for these data sets, bringing together business, technical and operational metadata. Data governance is front and center of Arena. Arena’s Control pillar provides integrated data governance functions such as data quality, role-based access control, and the ability to secure sensitive data with built-in masking and tokenization. Of great value to risk and compliance departments, Arena also captures end-to-end lineage across these hybrid, decentralized data platforms, including data flow through other vendors, bringing provable trust and agility to data initiatives. 

In addition to Catalog and Control areas, Arena fulfills the end-to-end DataOps optimization with advanced, self-service data consumption. Consumption begins with an intuitive, Google-like data search experience to discover relevant data sets and introspect their metadata, including quality metrics, lineage, classification, profile, and enrichment..Data science teams quickly become productive with data by not only swift discovery, but also by adding selected datasets to a shopping cart for provisioning sandboxes for various downstream use cases. The provisioning process is tied with governance workflows and security aspects so that data is managed securely, while still providing the agility for experimentation and ad hoc analytics to drive business insights.Arena facilitates the creation of sandboxes within on-premises infrastructures as well as cloud platforms such as AWS, Azure and Snowflake.

Key modern DataOps platform features

Automation

Create no-code or low-code data pipelines to automate steps in the data management pipeline. Data pipelines can be reused to support various patterns for processing batch or streaming data in diverse formats. Observability and management of these data pipelines meets enterprise scale and performance requirements. Automation improves data quality and reduces time to analytics-ready data. 

Unified Governance and DataOps platform

Manage the entire data supply chain from ingestion to consumption with a common metadata and security layer. Integrated functions for inventory of metadata and data, data ingestion, data quality, profiling, masking and tokenization, data catalog, provisioning allows organizations to quickly onboard new datasets and make them available to the data consumers in a trusted and governed manner. 

Self-Service Access

An easy to use interface to search and find data, introspect datasets for various use cases and then provision it to analytics tools or sandbox environments. Once a data set or multiple sets are found, users can add them to a shopping cart with simple checkout function, enabling  various AI/ML initiatives without needing to involve IT,  but with proper controls and governance in place. 

Multi-cloud support

As workloads become increasingly decentralized and built on heterogeneous environments, support for a consistent dataops abstraction across multiple cloud platforms is a requirement for modern data platforms. Arena’s governance includes support for Cloud PaaS offerings, Cloud based data warehouses, NoSQL databases as a service and others for a variety of use cases.

Containerized and Elastic compute

Support for elastic and on-demand compute environments for various DataOps functions is a key requirement supported by Arena. This includes on-demand compute infrastructures that can be provisioned in containerized deployment architectures like Kubernetes. These capabilities help improve performance and agility, while reducing costs. 

Extensibility

As a requirement in any enterprise scale data platform, Arena provides an open API for integration with third party sources, metadata repositories, workflow tools and various tools. Metadata can be exchanged 2-way between Arena and other repositories to provide a consistent view across the organization. Data engineering functions and ML algorithms can be plugged into a robust data pipeline layer reusing existing and proprietary implementations.

Data Intelligence

One of the key value propositions of Arena, is to provide intelligent system features for efficient DataOps. This includes leveraging built-in Machine Learning based algorithms for various functions, examples – data mastering and de-duplication, data classification for security and governance, insights on usage and data quality. 

See Arena in Action

Arena is available now and is ready for deployment. It is currently available for Amazon AWS, Microsoft Azure and On-prem deployments. It can be deployed in bare metal, IaaS or Kubernetes environments. Please reach out to the Zaloni team to schedule a demo at zaloni.com/get-a-demo.

about the author

Ben Sharma

Ben Sharma is the Co-founder and Chief Product Officer of Zaloni, a published author, and holds two patents for his innovative Big Data, Enterprise Infrastructure, and Analytics solutions. His impressive range of knowledge across data and business software disciplines has led him to leadership roles at leading companies like Fujitsu and NetApp before Zaloni.