نبذة عني
Greetings! I'm Gopinath, a dedicated Senior Data Engineer and Technical Lead with a rich professional journey spanning over Ten years. My expertise lies in shaping the data landscape through innovative ETL solutions, dat…
Greetings! I'm Gopinath, a dedicated Senior Data Engineer and Technical Lead with a rich professional journey spanning over Ten years. My expertise lies in shaping the data landscape through innovative ETL solutions, data warehousing, and pipeline automation, tailored for global business needs.
Core Competencies & Experience:
• Data Engineering Mastery: Skilled in creating robust data pipelines using Databricks, Snowflake, and Delta Lake. My proficiency extends to managing complex data transfers and migrations with a keen eye for quality and efficiency.
• Technical Proficiency: Adept in SQL, Python, and Pyspark, with a strong foundation in various data visualization tools like Power BI, SSRS, and Databricks. I bring a blend of technical knowledge and practical application that drives data-driven decision-making.
• Cloud & Big Data Expertise: My experience with cloud technologies, particularly AWS, Azure, and data integration tools, positions me as a frontrunner in modern data solutions. This expertise is instrumental in deploying scalable and secure data environments.
• Leadership & Collaboration: Having led diverse teams, I believe in fostering a culture of collaboration and continuous learning. My ability to adapt and communicate effectively makes me a valuable asset in any global team setting.
Global Readiness:
• Cultural Fit: My career has been a blend of local and global interactions, enabling me to develop a deep understanding of diverse work cultures. This sensitivity to multicultural environments makes me an ideal candidate for international projects.
• Innovative Problem Solver: I approach challenges with a solution-oriented mindset, whether it's enhancing data processes or navigating cross-cultural team dynamics. My global exposure has honed my ability to think creatively and deliver results in varied settings.
✅ Open to exciting opportunities in the UAE/ Europe
✅ Valid Passport, Immediate availability
✅ Collaborative leader with Agile/Scrum
الخبرة
Senior Data Engineer
Architected and optimised scalable PySpark and Python data pipelines targeting Snowflake and PostgreSQL, cutting end-to-end data processing time by 45%.
Built end-to-end analytics platforms using Databricks and Delta Lake with Medallion Architecture, boosting pipeline reliability by 40% and enabling real-time analytics at enterprise scale.
Standardised a Python-based ingestion framework for PostgreSQL sources, reducing new source onboarding effort by 50%.
Optimised complex SQL and PL/SQL workloads, improving query performance by 40% and enhancing reporting SLAs across business units.
Designed optimised SQL views and dimensional data models powering Power BI and Tableau dashboards, significantly reducing report refresh times.
Orchestrated and monitored production pipelines using Apache Airflow, managing DAG dependencies, retries, and alerting to ensure reliable end-to-end data workflows.
Technical Lead
Developed data pipelines for transforming and loading data to Postgres using Pyspark and Python., Design and construct Data Warehouse projects, including structures in Snowflake and Postgres., Analysed large data files, including Parquet, JSON, and CSV formats, using Python Pandas and Pyspark., Execute data analysis and aggregate calculations through data frame construction., Create PL SQL to perform the business logic and other aggregate functions.
Senior Data Engineer
Architected and optimised scalable PySpark and Python data pipelines targeting Snowflake and PostgreSQL, cutting end-to-end data processing time by 45%, Built end-to-end analytics platforms using Databricks and Delta Lake with Medallion Architecture, boosting pipeline reliability by 40% and enabling real-time analytics at enterprise scale, Standardised a Python-based ingestion framework for PostgreSQL sources, reducing new source onboarding effort by 50%, Optimised complex SQL and PL/SQL workloads, improving query performance by 40% and enhancing reporting SLAs across business units, Designed optimised SQL views and dimensional data models powering Power BI and Tableau dashboards, significantly reducing report refresh times, Orchestrated and monitored production pipelines using Apache Airflow, managing DAG dependencies, retries, and alerting to ensure reliable end-to-end data workflows
Lead Engineer / Senior Systems Engineer
Developed cloud-native data pipelines using Azure Databricks and Snowflake, boosting overall data throughput by 40%.
Automated ingestion and transformation workflows with PySpark and Python, eliminating manual effort by 50%.
Orchestrated end-to-end pipelines via Azure Data Factory (ADF), reducing job failures and processing delays by 30%.
Analysed large-scale Parquet, JSON, and CSV datasets using Pandas and PySpark, applying complex aggregations to drive business analytics.
Led migration of on-premises data from SQL Server to Amazon Aurora using AWS SCT and custom Python migration scripts.
Delivered optimised data models and SQL views supporting high-performance Power BI and Tableau reporting dashboards.
Technical Lead
Develop data pipelines using Azure Databricks, Snowflake, and Delta Lake., Orchestrate the job in Azure Data Factory., Involved in Migration of data from SQL Server to Oracle using AWS SCT and python for data migration., Analysed large data files, including Parquet, JSON, and CSV formats, using Python Pandas and Pyspark., Constructed data frames and perform comprehensive aggregate function calculations., Analysed and debugged user-reported defects in existing Python code.
Lead Engineer / Senior Systems Engineer
Developed cloud-native data pipelines using Azure Databricks and Snowflake, boosting overall data throughput by 40%, Automated ingestion and transformation workflows with PySpark and Python, eliminating manual effort by 50%, Orchestrated end-to-end pipelines via Azure Data Factory (ADF), reducing job failures and processing delays by 30%, Analysed large-scale Parquet, JSON, and CSV datasets using Pandas and PySpark, applying complex aggregations to drive business analytics, Led migration of on-premises data from SQL Server to Amazon Aurora using AWS SCT and custom Python migration scripts, Delivered optimised data models and SQL views supporting high-performance Power BI and Tableau reporting dashboards
Senior Software Engineer
Executed full database migration from SQL Server to MariaDB, reducing infrastructure costs by 35% and improving data load performance by 45%.
Engineered scalable Python (Pandas) ETL workflows integrating web APIs, SQL Server data, and multiple file formats (CSV, TXT, Excel) into MariaDB for accurate business reporting.
Built and managed SQL Server database objects and MSBI components; optimised execution plans to drive significant performance improvements.
Tuned SQL stored procedures for BI reporting, reducing report generation time by 30%.
Senior Software Engineer
Involved in Migration of data from SQL Server to Maria DB as part of cost reduction and provide the better solution for management using AWS SCT., Involved in Migration of data from SQL Server to AWS Aurora using AWS SCT and python for data migration., Develop data pipelines using Azure Databricks, Snowflake, and Delta Lake., Writing Store Procedures for Tableau reporting and involved in query tuning process., Analysed large data files like Parquet, JSON, and CSV using Python Pandas and Pyspark., Integrating with web API to pull the data into maria db database., Primarily working on Python pandas library for ETL process with various data source like csv, txt, excel to SQL server & Maria db destination., Collaborated closely with engineering and business teams, employing Scrum/Agile methodologies., Managed and prioritized defect and bug backlogs, ensuring timely delivery.
Senior Software Engineer
Executed full database migration from SQL Server to MariaDB, reducing infrastructure costs by 35% and improving data load performance by 45%, Engineered scalable Python (Pandas) ETL workflows integrating web APIs, SQL Server data, and multiple file formats (CSV, TXT, Excel) into MariaDB for accurate business reporting, Built and managed SQL Server database objects and MSBI components; optimised execution plans to drive significant performance improvements, Tuned SQL stored procedures for BI reporting, reducing report generation time by 30%
Senior Software Engineer
Maintained and supported large-scale SQL Server and Oracle ETL systems, achieving 99.9% production availability.
Improved ETL pipeline and query performance by 40% through indexing, query rewrites, and execution plan optimisation.
Automated repetitive data operations, cutting operational overhead by 50%.
Senior Software Engineer
Handling the SQL Server & Oracle related application which is primarily ETL process., Developed and managed MS SQL Server Database tools, including packages, procedures, functions, and triggers, leveraging MSBI technology., Enhanced query performance through strategic execution plan optimizations., Working on Production support environment for SQL Server and Oracle related applications., Working on the basis of Incident & Requests tickets within given SLA., Identifying the Root Cause of the issue and provide necessary fix., Adhere to the ITSM policy in the support process., Automate the Manual data load process.
Senior Software Engineer
Maintained and supported large-scale SQL Server and Oracle ETL systems, achieving 99.9% production availability, Improved ETL pipeline and query performance by 40% through indexing, query rewrites, and execution plan optimisation, Automated repetitive data operations, cutting operational overhead by 50%
Analyst Programmer
Developed SSIS ETL pipelines for Oracle-to-SQL Server integrations, reducing data load time by 35%.
Built SSRS reporting solutions, improving reporting efficiency by 30%.
Analyst Programmer
Developed tables, views, triggers, stored procedures, and user-defined functions, alongside crafting diverse SQL statements for multiple applications., Engineered ETL packages for data transfer from Oracle to SQL Server utilizing SSIS., Produced a range of visualization reports employing SSRS.
Analyst Programmer
Developed SSIS ETL pipelines for Oracle-to-SQL Server integrations, reducing data load time by 35%, Built SSRS reporting solutions, improving reporting efficiency by 30%
Software Engineer
Designed ETL workflows and SQL database objects, improving data integration performance by 30%.
Developed tables, views, triggers, stored procedures, and user-defined functions for multiple enterprise applications.
Software Engineer
Developed tables, views, triggers, stored procedures, and user-defined functions, alongside crafting diverse SQL statements for multiple applications., Engineered ETL packages for data transfer from Oracle to SQL Server utilizing SSIS., Produced a range of visualization reports employing SSRS.
Software Engineer
Designed ETL workflows and SQL database objects, improving data integration performance by 30%, Developed tables, views, triggers, stored procedures, and user-defined functions for multiple enterprise applications