نبذة عني
Overall 5 years of experience and 2 years in Machine Learning and Data Engineer Projects. Currently working with Makonis Software Solutions as Data Engineer. Good understanding of Data Analysis fundamentals and experienc…
Overall 5 years of experience and 2 years in Machine Learning and Data Engineer Projects. Currently working with Makonis Software Solutions as Data Engineer. Good understanding of Data Analysis fundamentals and experience in designing and implementing solutions is looking for a similar role and position in a progressive organization.
الخبرة
Data Engineer
* Design and develop ETL integration patterns using Python on Spark.
* Data acquisition from internal/external data sources
* Migration of data warehouse from SQL Server to Hadoop and Hive.
* Designed and built Data Quality frameworks for covering Data Quality aspects like
Completeness, Accuracy, coverage using Python and Spark and Kafka.
* Optimization of Hive queries using best practices and right parameters and using
technologies like Hadoop, YARN, Python, PySpark.
* Developed spark applications in python(PySpark) on distributed environment to load
huge number of CSV files with different schema in to Hive ORC tables.
* Worked on reading and writing multiple data formats like JSON, ORC, Parquet on HDFS
using PySpark.
* Involved in converting Hive Queries into various Spark Actions and Transformations by
Creating RDD and Dataframe from the required files in HDFS.
* Responsible for data ingestion into Bigdata using Spark Streaming and Kafka.
Machine Learning
Web Scraping, Data Visualization & Regression Modeling.
Compiled pricing data for competitive analysis by performing web scraping in python.
Created charts in Jupyter Notebook to perform preliminary analysis & visualize data using Matplotlib.
Clustering , Classification & Creating FastAPI Framework
Conceptualized & implemented a sentiment analysis tool to rate the financial competence of companies
Deploy Machine Learning models as a Microservice using FastAPI
Machine Learning
Web Scraping, Data Visualization & Regression Modeling.
Compiled pricing data for competitive analysis by performing web scraping in python.
Created charts in Jupyter Notebook to perform preliminary analysis & visualize data using Matplotlib.
Clustering , Classification & Creating FastAPI Framework
Conceptualized & implemented a sentiment analysis tool to rate the financial competence of companies
Deploy Machine Learning models as a Microservice using FastAPI
SQL Developer and Data Analysis
Plan, design, and implement application database code objects, such as stored procedures and views.
Build and maintain SQL scripts, indexes, and complex queries for data analysis and extraction.
Analyzing product, market, and share trends to build data.
Design, build and deploy BI solutions.
Designing, developing and maintaining business intelligence solutions.
Presenting information through reports and visualization.
Create visualizations and reports for requested projects.
Develop and execute database queries and conduct analysis.
Create data objects (DDL).
Prepare documentations for database applications.
Memory management for database systems.
Develop database schemas, tables and dictionaries.
Ensure the data quality and integrity in databases.
Create complex functions, scripts, stored procedures and triggers to support application development.
Take regular database backup.
SQL Developer and Data Analysis
Plan, design, and implement application database code objects, such as stored procedures and views.
Build and maintain SQL scripts, indexes, and complex queries for data analysis and extraction.
Analyzing product, market, and share trends to build data.
Design, build and deploy BI solutions.
Designing, developing and maintaining business intelligence solutions.
Presenting information through reports and visualization.
Create visualizations and reports for requested projects.
Develop and execute database queries and conduct analysis.
Create data objects (DDL).
Prepare documentations for database applications.
Memory management for database systems.
Develop database schemas, tables and dictionaries.
Ensure the data quality and integrity in databases.
Create complex functions, scripts, stored procedures and triggers to support application development.
Take regular database backup.
Data Engineer
Design and develop ETL integration patterns using Python on Spark.
Data acquisition from internal/external data sources
Migration of data warehouse from SQL Server to Hadoop and Hive.
Designed and built Data Quality frameworks for covering Data Quality aspects like Completeness, Accuracy, coverage using Python and Spark and Kafka.
Optimization of Hive queries using best practices and right parameters and using technologies like Hadoop, YARN, Python, PySpark.
Developed spark applications in python(PySpark) on distributed environment to load huge number of CSV files with different schema in to Hive ORC tables.
Worked on reading and writing multiple data formats like JSON,ORC,Parquet on HDFS using PySpark.
Involved in converting Hive Queries into various Spark Actions and Transformations by Creating RDD and Dataframe from the required files in HDFS.
Responsible for data ingestion into Bigdata using Spark Streaming and Kafka
Data Engineer
Design and develop ETL integration patterns using Python on Spark.
Data acquisition from internal/external data sources
Migration of data warehouse from SQL Server to Hadoop and Hive.
Designed and built Data Quality frameworks for covering Data Quality aspects like Completeness, Accuracy, coverage using Python and Spark and Kafka.
Optimization of Hive queries using best practices and right parameters and using technologies like Hadoop, YARN, Python, PySpark.
Developed spark applications in python(PySpark) on distributed environment to load huge number of CSV files with different schema in to Hive ORC tables.
Worked on reading and writing multiple data formats like JSON,ORC,Parquet on HDFS using PySpark.
Involved in converting Hive Queries into various Spark Actions and Transformations by Creating RDD and Dataframe from the required files in HDFS.
Responsible for data ingestion into Bigdata using Spark Streaming and Kafka