Building a Modern Data Architecture with Ashwin Nayak

Haley Teeples November 4th, 2021

Propel data-driven business objectives

Zaloni’s marketing team sat down with Vice President of Engineering at Zaloni, Ashwin Nayak, to discuss the importance of building a modern data architecture. From his firsthand experience in the data industry, this blog dives deep into Ashwin’s commentary that details how companies can work towards creating such an architecture to help propel their data-driven business objectives to new heights. 

Data architectures have changed drastically over the past 5 to 10 years, especially in the context of data analytics and DataOps. The most profound change in data architecture in the last few years has been the Lakehouse architecture, a combination of data lake and data warehouse. It’s an architecture pattern and technology combining compute, storage, and database, and each layer can scale independently. It allows businesses to process structured, unstructured, batch, and real-time data to support data product and analytics applications. A zone-based architecture, a logical layer on top of a Lakehouse, can be designed to isolate data as it moves through a data pipeline from a raw zone to a trusted or transformed zone with the right level of abstraction and security. 

In terms of what type of data architecture that businesses are seeking and are highly responsive to is a flexible data architecture. A complex data ecosystem in a large organization with many business units needs the flexibility to organize data to meet their business needs. Via a data fabric architecture pattern, data can be sourced from a well-governed data lake and be stitched together with various governance policies. Each business unit can source governed data and apply transformation rules to create a logical data structure or design it with a data mesh architecture pattern. The data mesh pattern, a domain layer, allows each business unit to have its transformation layer, business rules, data security, and access layer for their end-users. 

During data architecture planning, the conversation must include a few important designed outcomes, the first being self-service access. The self-service feature will allow users to search for data and request access, additionally, users should be able to get access to that dataset in their preferred tool like a query tool, analytics workbench or BI tools. Secondly, defining processes and technologies to serve data citizens such as data engineers, data stewards, data analysts, data scientists and data consumers is another desired outcome companies should strive for in the planning process. Last but not least, companies should also apply consistent data security and governance policies across hybrid and multi-cloud data environments to ensure compliance and any other privacy requirements. 

The final stage of the planning process is the implementation of a data platform and various tools that push the company towards a modern data architecture. Ashwin describes that when a company is at this final stage, it’s always crucial to consider “hybrid” as the norm when it comes to an organizations’ decisions of their data environment, data management tools, and architecture patterns. The fundamentals of data architecture haven’t changed in decades i.e. enabling business with the right data at the right moment. It shouldn’t be confused with a variety of data sources, technologies, and end-users tools. In most cases, the tools and platforms can be divided into: 

1. Data sourcing 
2. Orchestration 
3. Governance and provisioning 
4. Consumption tools 
5. Deployment platform 
6. Access and security. 

Ultimately, depending on organizations’ maturity on current data strategy, technologies, and skills, companies may consider the “best of breed” technologies that are interconnected to deliver business value. 

Zaloni is proud to implement its DataOps platform, Arena, with some of the world’s largest companies to provide a data management and governance solution that allows them to reach their business goals. Make sure to visit our website to schedule a demo to discover your customized solution with our data experts.

about the author

Haley Teeples

Haley Teeples is a recent graduate from North Carolina State University and has been working at Zaloni for over a year. She initially joined Team Z as a Marketing Intern and then worked as a Technical Documentation Intern on the Engineering team. With a clear passion for content creation and learning in the data management space, today, Haley serves as an associate on Zaloni's Product Marketing team.