Thiruppathikumar S

Thiruppathikumar S

Hadoop | Spark | Python | Scala | Java | Hive | Kafka | AWS | S3 | EMR | Glue | Athena | Redshift | Cloud Data Engineer
India

نبذة عني

An astute leader with 12+ years’ experience which embraces 6+ years in building Big Data Pipe line, Data Warehouse & Implementing Data Migration Projects in On-Premises/Cloud platform and 6 years in Application Programme…

الخبرة

Data Engineer

Samsung
Apr 2020 - حتى الآن · 6 سنوات 2 أشهر

Data Engineer | Samsung, Chennai, India | May 2020 – Present
• Gather business requirements and translate them into technical solutions. Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation.
• Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring. Validate and qualify solutions. Identify, troubleshoot, and resolve technical issues.
• Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings. Work with data from multiple data sources to build integrated views that will drive decisions.
• Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs. Monitor daily execution and fix business critical pipelines to ensure SLAs are met.
• Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program. Rewire hive queries and spark application to boost the performance.

Data Engineer

Aspire Systems (Samsung), Chennai, India
May 2020 - حتى الآن · 6 سنوات 2 أشهر

Gather business requirements and translate them into technical solutions.
Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation.
Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring.
Validate and qualify solutions.
Identify, troubleshoot, and resolve technical issues.
Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings.
Work with data from multiple data sources to build integrated views that will drive decisions.
Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs.
Monitor daily execution and fix business critical pipelines to ensure SLAs are met.
Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program.
Rewire hive queries and spark application to boost the performance.

Data Engineer

Aspire Systems (Samsung), Chennai, India
May 2020 - حتى الآن · 6 سنوات 2 أشهر

Gather business requirements and translate them into technical solutions
Design, develop, test, improve, and maintain new and existing solutions for data integration, data preparation
Incorporate deep data management expertise into solutions, including integrity, recovery, security, and monitoring
Validate and qualify solutions
Identify, troubleshoot, and resolve technical issues
Work with business stakeholders, IT groups and third-party vendors to understand source systems and enterprise infrastructure offerings
Work with data from multiple data sources to build integrated views that will drive decisions
Work with business stakeholders to assist with data-related technical issues and support the data infrastructure needs
Monitor daily execution and fix business critical pipelines to ensure SLAs are met
Ensure data reliability, efficiency, quality to enable analytics team to run the sophisticated analytics program
Rewire hive queries and spark application to boost the performance

Sr Consultant

Flextronics Technologies, Chennai, India
Jul 2016 - Apr 2020 · 3 سنوات 9 أشهر

Create, maintain and monitor an efficient data pipeline architecture, including working with scheduled jobs, replication, monitoring and partitioning.
Assemble complex data to meet business requirements.
Setup and maintain infrastructure to solve high volume data engineering problems for stream processing and batch processing.
Ensure data integrity and consistency throughout, and enable teams to easily verify this in end analysis.
Follow the proven innovation in the implementation to layout the optimized solutions.
Closely work with client and associate in preparing the sprint plan.
Identify, design, and implement internal process improvements automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the libraries and frameworks that support large, complex data process business rules.
Participate in value stream mapping, backlog grooming, sprint planning, team retros and demos.

Sr. Associate Consultant

Cognizant, Chennai, India
Apr 2016 - Jun 2016 · 2 أشهر

Identify, Design and build the proof of concept and prototype data pipeline to ease the business process and optimize performance.

Sr. Engineer

Servion Global Solution Ltd, Chennai, India
Sep 2014 - Apr 2016 · 1 سنة 7 أشهر

Produce the optimized data pipe line and ensuring data is collected and managed in the central data warehouse system to enable the analytics platform.
Addressing team difficulties to implement the solutions.
Own database and data flow documentation.
Contribute to code, design reviews and suggest improvements.
Follow quality development rules and recommendations.
Suggests and implements improvements and optimisations to development process.

Lead Engineer

HCL Technologies Ltd, Chennai, India
Mar 2011 - Aug 2014 · 3 سنوات 5 أشهر

Design, develop, test, deploy, maintain, and improve data and new user-facing features.
Write client and server-side code for service-oriented applications, create fast, easy-to-use, high volume production applications, and develop prototypes quickly.
Define the effort estimates, risks and dependencies during or after release planning.
Integrate, build and release software stack; Responsible for delivering component.
Update all the relevant and required technical documents.
Create highly readable & maintainable software.
Provide efficient code.

Sr. Engineer

OJoin Marketing Pvt Ltd, Bangalore, India
May 2010 - Feb 2011 · 9 أشهر

Design, build and support monitoring and observability tools and policies.
Write code that is tested, readable, and maintainable.
Perform code reviews, evaluate implementations, and provide feedback for improvements.
Engage in technical discussion and decision making to shape the future of making things.
Collaborate with stakeholders to understand requirements, understand use cases and build towards a cohesive technical strategy.

Software Engineer

LB InfoTech Pvt Ltd, Chennai, India
Nov 2008 - Apr 2010 · 1 سنة 5 أشهر

Design and implement re-usable software components.
Ensure the product is highly configurable and extensible.
Work collaboratively with a core team of architects and developers spread across different locations.
Research and build POCs using available frameworks to ensure feasibility.
Create technical design documents and present the details to a larger audience.
Participate in architecture and design reviews for projects that require complex technical solutions.

Sr Consultant

Flextronics Technologies, Chennai, India
Jul 2016 - Apr 2020 · 3 سنوات 9 أشهر

Create, maintain and monitor an efficient data pipeline architecture, including working with scheduled jobs, replication, monitoring and partitioning
Assemble complex data to meet business requirements
Setup and maintain infrastructure to solve high volume data engineering problems for stream processing and batch processing
Ensure data integrity and consistency throughout, and enable teams to easily verify this in end analysis
Follow the proven innovation in the implementation to layout the optimized solutions
Closely work with client and associate in preparing the sprint plan
Identify, design, and implement internal process improvements automating processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc
Build the libraries and frameworks that support large, complex data process business rules
Participate in value stream mapping, backlog grooming, sprint planning, team retros and demos

Sr. Associate Consultant

Cognizant, Chennai, India
Apr 2016 - Jun 2016 · 2 أشهر

Identify, Design and build the proof of concept and prototype data pipeline to ease the business process and optimize performance

Sr. Engineer

Servion Global Solution Ltd, Chennai, India
Sep 2014 - Apr 2016 · 1 سنة 7 أشهر

Produce the optimized data pipe line and ensuring data is collected and managed in the central data warehouse system to enable the analytics platform
Addressing team difficulties to implement the solutions
Own database and data flow documentation
Contribute to code, design reviews and suggest improvements
Follow quality development rules and recommendations
Suggests and implements improvements and optimisations to development process

Lead Engineer

HCL Technologies Ltd, Chennai, India
Mar 2011 - Aug 2014 · 3 سنوات 5 أشهر

Design, develop, test, deploy, maintain, and improve data and new user-facing features
Write client and server-side code for service-oriented applications, create fast, easy-to-use, high volume production applications, and develop prototypes quickly
Define the effort estimates, risks and dependencies during or after release planning
Integrate, build and release software stack; Responsible for delivering component
Update all the relevant and required technical documents
Create highly readable & maintainable software
Provide efficient code

Sr. Engineer

OJoin Marketing Pvt Ltd, Bangalore, India
May 2010 - Feb 2011 · 9 أشهر

Design, build and support monitoring and observability tools and policies
Write code that is tested, readable, and maintainable
Perform code reviews, evaluate implementations, and provide feedback for improvements
Engage in technical discussion and decision making to shape the future of making things
Collaborate with stakeholders to understand requirements, understand use cases and build towards a cohesive technical strategy

Software Engineer

LB InfoTech Pvt Ltd, Chennai, India
Nov 2008 - Apr 2010 · 1 سنة 5 أشهر

Design and implement re-usable software components
Ensure the product is highly configurable and extensible
Work collaboratively with a core team of architects and developers spread across different locations
Research and build POCs using available frameworks to ensure feasibility
Create technical design documents and present the details to a larger audience
Participate in architecture and design reviews for projects that require complex technical solutions

المهارات

البيانات الضخمة تحليل البيانات المحترف تنقيب البيانات إدارة البيانات قاعدة بيانات SQL تيراداتا (نظام قواعد البيانات) Hadoop Apache Spark Apache Hive Apache Sqoop Map Reduce Kafka Python Scala Java AWS S3 EMR EC2 Glue Redshift Athena Data pipelines Data Warehouse Data Migration MapReduce Amazon S3 Amazon EMR Amazon EC2 AWS Glue Amazon Redshift Amazon Athena OOP SOA Apache Airflow PySpark HDFS Spark Structured Streaming IBM Aspera AWS Transfer for SFTP Oracle Maven Eclipse Spark Streaming Apache Solr HBase TDCH TPT BTEQ Shell scripting Tivoli Informatica Hortonworks Cloudera Agile
الإبلاغ عن هذا الملف الشخصي؟