نبذة عني
Results-driven data engineer, senior data Analyst with (8+) experienced in database design, Relational database and strong SQL developer. specialized in data migration, data integration, Proven expertise in leading end-t…
Results-driven data engineer, senior data Analyst with (8+) experienced in database design, Relational database and strong SQL developer. specialized in data migration, data integration, Proven expertise in leading end-to-end data management initiatives, driving successful data migration projects, data Visualization and dashboard design using Microsoft BI Stacks (SSMS, SSIS, SSAS, SSRS and Power BI), Tableau, designed large scale data solutions on the Azure platform (Data storage, Data factory, CI/CD, Azure Synapse Analytics, Azure Databrick, CI/CD), Databricks, Auto-loader and DLT, Azure streaming
الخبرة
Data Engineer
• Built and maintained scalable data pipeline real-time and batch processing, ensuring data quality and integrity.
• Experienced and knowledge of Medallion architecture such as bronze, silver and gold layers
• Collaborated with the Enterprise Analytics team to create and maintain ELT processes
• Experienced and worked closely with the Enterprise Analytics team to create and maintain ELT processes
• Implemented data migration projects from on-premises databases to Azure, ensuring data accuracy and minimal downtime.
Senior Data Engineer
Built and maintained scalable data pipeline real-time and batch processing, ensuring data quality and integrity.
Experienced and knowledge of Medallion architecture such as bronze, silver and gold layers
Collaborated with the Enterprise Analytics team to create and maintain ELT processes
Experienced and worked closely with the Enterprise Analytics team to create and maintain ELT processes
Implemented data migration projects from on-premises databases to Azure, ensuring data accuracy and minimal downtime.
Developing scripts UNIX, Python etc. to Extract, Load and Transform data, as well as other utility functions.
Designed and implement end-to-end data processing solutions on Azure, Leveraging services such as Azure Data Factory, Azure Databrick, Pyspark
Experienced working with in Auto-loader, Data Live Table and format files Delta and Parquet
Collaborated with Azure DevOps for version control, continuous integration, and deployment of data solutions.
Created data visualizations and dashboards using Power BI for data analysis and reporting.
Experienced in relational data processing technology like MS SQL, Delta Lake, Spark SQL, SQL Server,
used Azure Databricks and Azure Data Factory for batch and real-time data integration.
Implemented data ingestion, preparation, integration, and operationalization techniques in optimally addressing data
Proficient in using Snowflake Cloud for data warehousing and analytics.
Experience with designing and implementing data pipelines in Snowflake.
Strong understanding of Snowflake architecture, including virtual warehouses, databases, and schemas.
Familiarity with SQL for querying and managing data in Snowflake.
Knowledge of Snowflake's security features and best practices for data protection.
Experience with data loading, transformation, and integration using Snowflake.
Thorough analysis, unit testing and integrated testing with other applications of database objects & T-SQL statements before deployment to the production server.
Designed and implemented end-to-end data pipelines integrating various data sources into Snowflake, ensuring seamless data flow and maintaining data integrity.
Wrote complex SQL queries using advanced SQL concepts like aggregate functions,
while ensuring data accuracy and version control
Worked with OLAP and OLTP tools such as ETL, data warehousing and Modeling.
Participated in analyzing business requirements, Use Case Modeling, Data Model Design, Data flow from various Sources to Target.
Senior Business Intelligence Analyst
Developed and maintain data visualization dashboard using tool such as Power BI and Tableau for effective communication
Expertise in creating visually appealing and insightful dashboards, reports, and data visualizations to support informed decision-making.
Created and maintained documentation for Power BI solutions, including data models, report specifications, and user guides
Employed standard data modelling techniques in identifying, extracting and analyzing business data for senior management in a user-friendly dashboard, using data visualization and reporting tools (SSRS, Power BI, Tableau, Power BI App).
Develop comprehendible KPI measures and dashboards based on client’s defined business requirements.
Responsible for end-to-end management of reporting and visualization software servers (installation, upgrades and scaling, troubleshooting, migration and testing)
Analyze target production resources for performance issues, capacity planning and storage and make recommendations to management Created Parameterized, Cascaded, Drill-down, Cross-tab and Drill-through Reports
Collaborated with stakeholder to define business requirement and translate them into analytical solutions
Developed and maintained automated data pipelines using Azure Data Factory and Auto-loader and DLT.
Experienced with Snowflake features such as; Python; SQL, Tasks; Streams; Time travel; Zero Copy Snow SQL; Snow Pipe Cloning; Optimizer; Metadata Manager; data sharing; and stored procedures
Developed ETL pipelines in and out of data warehouse using combination of Data bricks, Python and SQL/Snow SQL.
Design interactive dashboard report using SSRS, SSRS, Power BI, Tableau, and Excel
Provided analytical insight for data-driven decision-making and reporting at the department and corporate levels.
Generated Power BI Dashboard with quick/context/global filters, parameters and calculated fields on Tableau reports.
Created Power BI Dashboards with interactive views, trends and drill downs along with user level security
Senior Data Engineer | Data Analyst
Managed end-to-end operations of ETL data pipelines, Power BI, Tableau
Wrote and coded logical and physical database descriptions, specifying identifiers of database to management systems
presented the solution to the business, providing technical expertise and explaining the trade-offs with various data design alternatives.
Recommends approach for data migration strategies to best align with business needs and to optimize maintainability and performance.
Identifies the key technology interactions and dependencies across data systems/platforms impacting the organization’s ability to achieve integration, compatibility, and performance targets.
Conducted research and analysis and provides recommendations for the design and delivery of the data architectures required for the delivery of IT-based solutions, services, and support.
Designed solutions for high availability, connectivity, and disaster recovery, as required.
Provided technical advice and support to Data Office, Systems, and management relating to data architectures and schemas.
Ensured data architecture designs follow established security policies.
Ensured technical designs comply with architecture guiding principles.
Writes professional/technical proposals, reports, and manuals.
Collaborated with other Data Office and Systems teams on enterprise-wide IT strategies and technologies, as require Data Solution Design & Development
Partners with members of the Data Office team, specifically the Director, Data Platform Solution Architect and the Data Engineers, setting standards and modeling in our Azure data platform for operational and analytical data, structured and unstructured data, data in motion, and data at reset.
Partners with members of the Data Office team and IT stakeholders to define data processing requirements and dependencies across the data platform for data ingestion, data standardization, data integration, and data consumption.
Designed, develops, publishes, and maintains the Azure data platform semantic layer and maps its data lineage back through the various data propagation zones of the data platform and subsequently back to the physical operational source systems where the source data is
Lead the design and implementation of end-to-end data solutions on Azure, including data ingestion, storage, processing, and visualization.
Develop and optimize ETL processes using AWS, Azure Data Factory, Azure Databricks, and other Azure data services.
Collaborate with data scientists and business analysts to understand data requirements and implement data solutions that meet business objectives.
Implemented and managed data lakes and data warehouses on Azure for structured and unstructured data.
Design and implement real-time data streaming solutions using Azure Stream Analytics.
Ensure data security and compliance with regulatory standards, implementing access controls and encryption measures.
Conduct performance tuning and optimization of Azure SQL databases and data warehouse solutions.
Developed Optimized SQL queries and scripts for data extraction and manipulation.
Implemented Event Handlers and Error Handling in SSIS packages and notified process results to various user communities
Tested ETL/ data flow structures and SSIS Packages for data integration and reporting
Designed SSIS packages and T-SQL scripts to extract data, transform and load (ETL) from Source Systems for enterprises data Warehouse (EDW)
Implemented and managed data warehousing solutions using Snowflake, a cloud-based platform, for scalable and high-performance data storage and analytics.
Python libraries such as Pandas and NumPy for efficient data processing, manipulation, and analysis, contributing to streamlined data workflows
Developed and maintained Extract, Transform, Load (ETL) scripts in Python to seamlessly integrate and transform data from various sources into a unified format for storage and analysis
Leveraged Python libraries like SQL Alchemy for interacting with databases, executing SQL queries, and managing database connections, contributing to efficient data retrieval and storage processes.
Business Intelligence Analyst
Created, designed and built an end-to-end enterprise data Warehousing
From business requirement gathering, Stakeholder matrix, Bus matrix, source to target mapping, staging database, Control framework database, Process flow, Test Plan and Test Case for the Data Warehouse Include staging SQL script and Enterprise Data Warehouse SQL script
Analyzed business requirements and designed conceptual and logical diagrams
Designed and analyzed source-to-target mapping, transformation, and data quality.
Developed SSIS packages for ETL tasks, including data extraction, transformation, and loading
Implemented slowly changing dimension (SCD) Type 1 and 2 for data warehouses
Tested, validated and reformulated models to foster accurate prediction of outcomes
Applied feature selection algorithms and machine learning models for predictive analytics.
Analyze, process data, data mining, Quantitative analysis using Scikit-learn, NumPy, pandas, Matplotlib
Conducted exploratory data analysis, feature engineering, and data preparation.
Developed and implemented machine learning models using Python libraries (Scikit-Learn, NumPy, Pandas, Matplotlib)
Created and maintained data visualizations and dashboards using Tableau and Power BI