AWS Cloud Data Management: Conquer Data Sprawl

Amy King December 9th, 2019

Data and analytics success relies on providing data analysts and data scientists with quick, easy access to accurate, quality data. There’s no better solution currently on the market to achieve this than Zaloni Arena paired with AWS for better AWS cloud data management.

In a recent project, together with AWS, we helped the TMX Group (a Canadian financial services company that operates equities, fixed income, derivatives, and energy markets exchanges) manage their complex data sprawl into a consolidated and enriched self-service data catalog.

This allowed TMX Group to use their data for such cases as monetizing data for revenue growth and providing 360-degree customer views to improve customer experience and uncover cross-sell and up-sell opportunities.

Building a Data Lake on AWS

When building a data lake on AWS, we recommend a zone-based architectural approach. This helps control how data is moved and processed while also providing governance and security controls through role-based access. This also provides data lineage that shows where data is coming from, where it’s going and what’s happened to it over time.

Understanding the data architecture is one thing, but what about actually deploying a data lake? How can you ensure success?

Data Lake Deployment Best Practices we Learned from TMX Group

1. Connect more data from more sources

Connecting to a variety of distributed and siloed data sources including cloud and on-prem data, and easily adding these sources to the catalog as they become available is essential to future-proofing your AWS data lake.

2. Catalog data for accurate, trusted, and repeatable use

To gain insights from your data, you need to know what data you have. A data catalog that focuses on automation with machine learning and artificial intelligence along with detailed and active metadata for easy consumption can help to get you answers fast so you can act accordingly.

3. Govern data for security and traceability

Data governance through role-based access control is critical for compliance with industry regulations around privacy and security along with masking and tokenization capabilities. With so much attention on protecting customer data, data governance is a must-have for any organization.

4. Provide business users with self-service AWS cloud data access

What good is a data catalog and data governance without allowing your business users access to the data they need? Granting self-service data access will allow them to see the data they want, when they need it, without needing to request it from IT. That’s a win-win!

Wish this blog was more detailed? This was only a short overview of a much more in-depth version on the AWS blog.

Ready to get started leveraging Zaloni Arena on AWS? Learn more, visit the AWS marketplace or request a custom demo today!

about the author

Amy King

Amy King was Zaloni’s Vice President of Marketing and now is CMO at Relias.