Duman Shrestha

Duman Shrestha

Python Data Engineer
United States of America

نبذة عني

Around 6+ years of experience as a Python Developer, proficient in multiple languages and environments including Python, JavaScript, Node.JS, SQL, Bash, REST Api, AWS, Google Cloud, C, C++ and more. Proficient in Fronten…

الخبرة

Data Engineer

verizon
Mar 2023 - حتى الآن · 3 سنوات 4 أشهر

• Utilized Python programming language to design and develop scalable ETL processes for ingesting large volumes of telecommunications data from various sources, including call records, network logs, and customer interactions.
• Wrote a python program to efficiently process and transform raw data into a structured and usable format.
• Designed and implemented data models to ensure optimal storage and retrieval of telecom data. Optimized database performance for quick access to critical information.
• Implemented real-time data streaming solutions using technologies such as Apache Kafka to process and analyze data as it arrives, enabling timely insights for operational decision-making.
• Developed and implemented data quality checks and validation procedures to ensure the accuracy and integrity of telecom data.

Python Data Engineer

VERIZON, BOSTON, MA
Mar 2023 - حتى الآن · 3 سنوات 4 أشهر

Utilized Python programming language to design and develop scalable ETL processes for ingesting large volumes of telecommunications data from various sources, including call records, network logs, and customer interactions.
Wrote a python program to efficiently process and transform raw data into a structured and usable format.
Designed and implemented data models to ensure optimal storage and retrieval of telecom data.
Optimized database performance for quick access to critical information.
Implemented real-time data streaming solutions using technologies such as Apache Kafka to process and analyze data as it arrives, enabling timely insights for operational decision-making.
Developed and implemented data quality checks and validation procedures to ensure the accuracy and integrity of telecom data.
Collaborated with cross-functional teams to resolve data quality issues and discrepancies.
Implemented automation scripts (bash and python) to schedule and orchestrate data workflows, reducing manual intervention and improving overall efficiency.
Integrated workflow orchestration tools like Apache Airflow to streamline and automate data processing tasks.
Engineered a scalable ETL pipeline that reduced data processing time significantly, enhancing the speed at which actionable insights could be derived.
Made web-based dashboard for data visualization.
Used cloud computing to train machine learning models forecasting the network demand.
Implemented a real-time data processing solution, resulting in a 20% improvement in the company's ability to respond to network anomalies and customer issues promptly.
Received recognition for designing and implementing a data quality framework, leading to a 25% reduction in data-related errors and discrepancies.

Python Data Engineer / QA

FINRA, ROCKVILLE, MD
Mar 2022 - Feb 2023 · 11 أشهر

Integrated data sets to create a comprehensive and accurate representation of companies registered under FINRA.
Wrote script to merge data from multiple sources.
Also wrote a script to test the validation of data.
Tested if the correct data is imported from the correct sources or not.
Introduced an automated data validation process, reducing errors by 20% and significantly improving data accuracy and saving manual time by 50%.
Wrote ETL pipeline to collect data from various sources (including databases, spreadsheets, external APIs), filter the relevant information, conducted thorough data cleaning processes, and save it on relational database server.
Wrote python script and SQL queries for extracting meaningful insights from vast financial datasets to support informed decision-making.
Developed a python program to assure data quality protocols are being followed and to identify and rectify anomalies and detect fraud.
Generated fake testing data to validate this system.
Utilized statistical techniques to analyze financial trends and performance indicators.
Created user friendly interactive dashboards and reports using seaborn to present complex financial information in a comprehensible manner.

Research Assistant

UNIVERSITY OF SOUTH FLORIDA
Jan 2019 - Dec 2021 · 2 سنوات 11 أشهر

Used Python Pandas library to analyze the experimental data obtained by experimental physicists.
Used scikit-learn library (Python) to model the data and verify (or reject) the hypothesis.
Used pingouin library for hypothesis testing and statistical analysis.
Converted the dynamical problem with higher order differential equations to linear algebra problems.
Used Python NumPy to solve the problem.
Used NumPy, SciPy, Matplotlib, Scikit-learn to implement a machine learning model on a biophysical theory.
Wrote test cases on PyTest to test if the data supports the theoretical prediction or it deviates from the theoretical prediction.
Wrote an optimization logic in C program to run from Linux command line, which helps to analyze and filter out the necessary data from raw data of millions of rows.
Studied the effect of discretization on natural continuous systems.
Did planetary simulation to verify discretization will lead to violation of the law of conservation of energy.

Python Web Developer

INSIGHT WORKSHOP, KATHMANDU, NEPAL
Jan 2016 - Nov 2018 · 2 سنوات 10 أشهر

This is an All-in-one solution web app for a drug testing facility based in United States.
Integrated CI/CD pipeline with git version control to develop, test and deploy the web app.
Used Linux CRON to automatically send notification reminders for patients/clients.
Added 2 step verification using text message or email for security reasons.
To make sure a user can only access information and features related to their role, a user is divided based on their role and each role has separate dashboard layout.
Various AWS services were used and integrated for making cost effective and reliable hosting.
This was a full featured web app made for US healthcare company.
I Modified SQL Alchemy for making a true ORM (object relational mapping) to connect MySQL.
Created encrypted RESTful API endpoints for sending classified user data.
implemented token-based authentication API.
Used Containerized development (inside docker) and Containerized Hosting
I Made a dashboard where user can see the summary of their business and analysis and charts.
Export the report to various formats including CSV, PDF, Excel, Word, HTML
Wrote test cases to verify web app is working correctly.
Which helped developers to know adding other features did not break the previously working feature.
And made development faster.
Used faker library to generate fake data for testing.

المهارات

البيانات الضخمة مستودع البيانات هادوب (Hadoop) سكالا سبارك أباتشي هايف أباتشي بيغ خدمات أمازون ويب (AWS) كافكا نوسكيو إل ردشفت (خدمة قاعدة بيانات) إس 3 (خدمة تخزين البيانات على الإنترنت) Python JavaScript Node.JS SQL Bash REST API Google Cloud C++ HTML CSS SCSS TypeScript Bootstrap React Vue JS Django Flask FastAPI Pandas NumPy SciPy Matplotlib Seaborn Statistics PySpark Excel Pyright Pytest Pyunit Selenium Playwright SQLite MySQL MS SQL Server PostgreSQL MongoDB XML XSL YAML CSV AJAX Model-View-Controller Functional Programming Object-Oriented Programming HTTP Axios Jupyter Notebook
الإبلاغ عن هذا الملف الشخصي؟