نبذة عني
Seeking a challenging role as an Azure Data Engineer where I can leverage my overall 4.5 years in IT under relevant 3.4 years of experience in data integration, transformation, and analytics to build efficient data solut…
Seeking a challenging role as an Azure Data Engineer where I can leverage my overall 4.5 years in IT under relevant 3.4 years of experience in data integration, transformation, and analytics to build efficient data solutions using Cloud services (Azure) like Data Factory and Databricks. I aim to contribute to optimizing data-driven decision-making processes while continuing to expand my skills in cloud technologies and data engineering.
الخبرة
Azure data Engineer
Developed end-to-end data pipelines utilizing Azure Databricks, Azure Data Factory, and Azure SQL to efficiently manage and process large volumes of structured and unstructured data from diverse sources.
Integrated Azure Data Lake with external systems to facilitate seamless data transfer, enabling real-time analytics and reporting capabilities.
Enhanced query performance in Azure SQL and Delta Lake by applying partitioning, indexing, and caching strategies.
Developed and managed ETL/ELT processes for data ingestion from diverse retail systems, including inventory management, e-commerce platforms, and customer databases, ensuring high data quality and integrity throughout the pipeline.
Managed role-based access controls (RBAC) and implemented data encryption in Azure, ensuring the protection of sensitive customer and financial data while adhering to compliance standards.
Automated data workflows and established CI/CD pipelines using Azure DevOps, reducing manual intervention and improving deployment efficiency across development and production environments.
Conducted continuous data quality checks and resolved data-related issues to ensure high levels of data integrity and reliability across systems.
Azure data Engineer
Developed end-to-end data pipelines utilizing Azure Databricks, Azure Data Factory, and Azure SQL, Integrated Azure Data Lake with external systems, Enhanced query performance in Azure SQL and Delta Lake, Developed and managed ETL/ELT processes for data ingestion from diverse retail systems, Managed role-based access controls (RBAC) and data encryption in Azure, Automated data workflows and established CI/CD pipelines using Azure DevOps, Conducted continuous data quality checks and resolved data-related issues
Senior Software Engineer
Designed and implemented a cloud-based Electronic Health Record (EHR) system using Azure services, ensuring secure management and storage of patient medical records.
Developed ETL pipelines using Azure Data Factory to automate data movement between Azure Blob Storage and Azure SQL Database, optimizing the management of patient records and enhancing data integration processes.
Implemented automated appointment scheduling and secure communication channels using Azure Logic Apps, improving operational efficiency and enhancing patient engagement.
Managed sensitive data using Azure Key Vault, ensuring compliance with HIPAA regulations and significantly enhancing data security through secure secrets management.
Integrated Application Insights for end-to-end monitoring of application dependencies, request rates, response times, and failure rates, enhancing system observability.
Collaborated with development teams to manage code repositories using Git for version control, ensuring efficient code management, seamless collaboration, and improved team productivity.
Implemented Spark using Python in Databricks, leveraging Data Frames and Spark-SQL API to optimize data processing and achieve faster data transformation and analysis.
Utilized various data formats, including CSV, JSON, and Parquet, to store and manage data within Azure Data Lake Storage (ADLS) location using Databricks for data processing and transformation.
Designed and implemented DDL, DML, and DQL scripts, along with optimized stored procedures for data manipulations and queries, to enhance database performance and maintain data consistency.
Senior Software Engineer
Designed and implemented a cloud-based Electronic Health Record (EHR) system using Azure services, Developed ETL pipelines using Azure Data Factory, Implemented appointment scheduling and secure communication channels using Azure Logic Apps, Managed sensitive data using Azure Key Vault, Integrated Application Insights for end-to-end monitoring, Collaborated with development teams to manage code repositories using Git, Implemented Spark using Python in Databricks, Utilized various data formats to store and manage data within Azure Data Lake Storage, Designed and implemented DDL, DML, and DQL scripts for data manipulations & queries