Chemical Company Reduces Cost and Boosts Productivity with AWS Data Lake

Challenge

A global chemical and ingredients distributor found it challenging to store and analyze data because of its legacy data storage platform. Other problems with legacy data storage platform include the inability to process real-time data, sequential data hops, data duplication and lack of uniform data standards across multiple divisions within the company. The client needed an efficient data storage platform that offers more agility and flexibility than traditional data management systems. The client approached iLink because of our prior success in delivering similar projects, technology expertise, and understanding of the industry landscape.

Approach

Link’s experts interviewed key stakeholders from business and technology teams to get a holistic understanding of the process and their pain points.

After extensive research, focusing on key metrics – cost reduction, efficiency, productivity and customer acquisition/retention, iLink proposed AWS data lake solution to the client.

Solution

We created an AWS data lake for the client which included the following technical architectural : 

  • S3 acts as the data hub for serving data
  • Data Catalog crawls S3 objects to generate schema definitions and integrates with EMR, Athena and Redshift Spectrum
  • DynamoDB stores S3 object index values as well as processes control metadata
  • Lambda and EMR serves as the data integration layer to serve data to data marts like RDS, Aurora, SQL Server, Redshift, SFDC
  • Athena used for S3 in-place queries
  • S3 data is organized and readily available for easy access to Machine Learning and SageMaker
  • CloudWatch and CloudTrail for audit and logging

Outcome

Reduced costs, improved efficiency and customer acquisition/retention for the client.

Our proposed AWS data lake solution reduced costs, boosted productivity and enabled following new capabilities to the client:

  • Provided a single source of truth for all data needs, tighter data Integrity, improved accuracy, and reduced data redundancy
  • Ability to scale to high data volumes in a cost-effective manner
  • Supported different types of analytics such as machine learning, ad-hoc queries, big data analytics, full text search, and real-time analytics over multiple data sources stored in the data lake
  • Allowed the client to generate effective insights including reporting on historical data, and predictive analytics through machine learning models to provide forecasting, recommendations, etc.
  • Allowed various roles within the organization such as data scientists, data analysts, and business analysts to access data with their choice of analytic tools

    Etiam magna arcu, ullamcorper ut pulvinar et, ornare sit amet ligula. Aliquam vitae bibendum lorem. Cras id dui lectus. Pellentesque nec felis tristique urna lacinia sollicitudin ac ac ex. Maecenas mattis faucibus condimentum. Curabitur imperdiet felis at est posuere bibendum. Sed quis nulla tellus.

    ADDRESS

    63739 street lorem ipsum City, Country

    PHONE

    +12 (0) 345 678 9

    EMAIL

    info@company.com

    Cart