Creating a Lakehouse End-to-End Architecture

Lakehouse Architecture

Project Information

  • Category: Datalake, Pipeline, Microsoft Fabric
  • Client: Viatris
  • Project Date: Ongoing
  • MS Fabric: Microsoft Fabric

Tools MS Fabric Used Power Automate workflow, Power BI , Synapse Data Science , Synapse Data Engineering , Synapse Data Warehouse

Description

In this project, I designed and implemented a robust data architecture using Microsoft's Fabric, comprising both a Lakehouse and a Data Warehouse. My work focused on automating data workflows and integrating advanced logic for improved data management and analysis.

Key Achievements

  • Data Lakehouse and Warehouse Implementation:
    • Designed and set up a comprehensive Lakehouse and Data Warehouse using Microsoft Fabric.
    • Ensured efficient data storage and retrieval by organizing data into structured and semi-structured formats.
  • Automated Data Pipelines:
    • Developed and optimized data pipelines to automate the refresh and ingestion of data into delta tables.
    • Streamlined data workflows, reducing manual intervention and ensuring up-to-date data availability.
  • Power Automate Integration:
    • Leveraged Power Automate to send automated emails, enhancing communication and alert systems.
    • Automated routine tasks, significantly improving operational efficiency.