Cloud Data Migration Simplified and DataOps Optimized with Zaloni’s Arena Data Platform for MongoDB

Nikhil Goel September 2nd, 2020

NoSQL and document databases have fundamentally changed the way organizations approach data modeling while building a modern, scalable, data platform. Users are no longer limited to the best practices used in traditional relational databases (RDBMS). With modernized cloud data migration, new analytical and operational use cases can be realized through data modernization in a denormalized form to document data stores.

Data platforms are being deployed in cloud computing environments at a rapid pace. This migration to the cloud provides organizations with several options, such as cloud-native or cloud-agnostic data stores, as well as a wide range of tools to migrate data from on-premises environments to cloud environments.

Zaloni, together with MongoDB, created an innovative solution for modernizing data from on-premises relational databases to the cloud-based MongoDB Atlas platform. This solution allows enterprises to realize new use cases such as creating golden records for 360 degree customer views or enabling faster time to data monetization.

Architecting Cloud Environments with DBaaS from MongoDB

MongoDB has pioneered the leading non-relational database over the last decade, with 110M+ downloads, 1M+ registrations for courses in MongoDB University, 18,400+ customers, and four years running as the “Most Wanted” database among developers surveyed by Stack Overflow. MongoDB Atlas, the fully managed global cloud database, is the best way to use MongoDB. Atlas is available on 70+ regions across AWS, GCP, and Azure. Best-in-class infrastructure and database automation, and proven practices guarantee continuous availability, elastic scalability, and compliance with the most demanding data security and privacy standards. Advanced features like auto-scaling and intelligent guidance to help optimize your deployments ensure you find the right balance between performance and efficiency.

Optimizing DataOps in the Cloud with Zaloni and MongoDB

Migrating from on-premises to cloud-based data management is not a trivial task, and realizing new analytical and operational use cases requires a solid foundation in data management and a DataOps strategy with a strong focus on governance.

Zaloni’s Arena software provides a unified data management platform that can be easily deployed to manage such hybrid environments. Organizations can use out-of-the-box governance and automation capabilities to migrate, transform, and denormalize data into MongoDB Atlas. Data engineers can further implement analytical and reporting use-cases using no-code/low-code capabilities in Arena. Arena’s machine learning based data mastering and data provisioning capabilities can help realize use cases such as building Customer 360 data stores with low latency access to data using MongoDB Atlas and MongoDB Atlas Search.

The following architecture diagram and demo video show how Arena integrates with MongoDB.

Demo Video

Arena and MongoDB: Storing Relational Tables and Nested Documents

Data Management & Cloud Data Migration in Action

Use Case: Cloud Data Migration

Challenges

• On-premises databases do not provide the needed scalability or flexibility.
• Storage and compute need to be decoupled, for the ability to scale one or the other, depending upon growth. Basically, to best optimize costs, you need two knobs to dial instead of one.
• The on-premise model entails the high costs and resource-intensive effort of maintaining, upgrading, and provisioning new servers. Many companies are realizing the value of shifting costs from capital expenditures (CAPEX) to a more predictable and controllable operating expenditure (OPEX) model.
• Digital transformation initiatives require data and application modernization.

Solution

• Migrate to the cloud for an OPEX-only model, saving precious time, resources and costs. 
• Provide low code/no-code data pipelines with data quality checks, profiling, masking and tokenization.
• Allow one time and recurring data migration.
• Enable data consumption from MongoDB.

Use Case: Customer 360 

Challenges

• Customer data is scattered in multiple silos.
• Governance and data quality checks are essentially non-existent.
• Enterprises are unequipped to support digital transformation use cases without unified data for 360 degree customer views
• Rigid schemas slow down or block enhancements of customer 360 views over time

Solution

• Easily source and unify data from multiple sources, no matter where it resides.
• Automatically check for data quality and identify sensitive data using machine learning
• Create master records for Customer 360 views.
• Provision to MongoDB for consumption.
• Enable faster and better analytics for differentiated customer experiences (right offer, right time), improved CLV, new revenue streams, and other sales and marketing successes. 

Customer Success Story: Securities 360 view

A large asset management firm, TIAA/Nuveen uses Zaloni Arena to rapidly onboard financial data from various data providers to create ESG-based investment products for their Responsible Investing Platform. (ESG stands for Environmental, Social, and Corporate Governance) Traditional financial metrics and alternative data from various non-traditional sources are ingested into Arena to create a Securities 360 view for ESG scoring. A holistic view for each Security along with the ESG scores are provisioned from the data lake to MongoDB. Portfolio Managers and other data consumers are able to access the 360 views for creating new investment products.

Optimize DataOps with Zaloni Arena 

DataOps Insights

Arena provides insights about your data and metadata based on various metrics that may be relevant to Data Engineers, Stewards and Data Consumers.

DataOps Automation 

Low code and no-code pipelines with a drag-and-drop interface allow creation of reusable DataOps pipelines that are integrated with governance.

Data Ingestion

An intuitive data ingestion wizard is provided to move data from source systems to target data platforms using a metadata-driven approach.

Catalog and Governance

Arena provides a rich marketplace of datasets and other assets with a shopping cart experience. The actionable catalog provides a holistic view with business, technical and operational metadata.

Data Lineage

End-to-end lineage is captured as data moves across the pipeline.

Provisioning

Self-service provisioning with approvals can be used for creating analytics sandboxes for experimental, ad hoc data science projects to enable rapid time to insights.

Data Mastering

Using ML algorithms, data from multiple sources can be matched, linked and merged into a 360 view. This can be used for Customer 360 and other related use cases.

Modernize with Cloud Data Migration

Zaloni’s Arena platform helps companies transition from on-premises to MongoDB Atlas as they modernize their data architecture to the cloud. In addition to simplifying cloud data migration, Arena provides end-to-end management and governance of the data supply chain, building a DataOps foundation to support various analytics use cases. To learn more about how Arena and MongoDB can help you overcome your data challenges, schedule a custom demo.

about the author

Nikhil Goel

Nikhil was a part of the Zaloni team as a data solutions architect then a seniro product manager. He now is Director of Product at Varo Bank.