ETL Development lead
Job Location:
Monthly Salary:
Posted on:
6 days ago
Posted on:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Overview:
TekWissen is a global workforce management provider throughout India and many other countries in the world. The below job opportunity is to one of our clients who is a part of a trusted global innovator of IT and business services headquartered in Tokyo. We help clients transform through consulting industry solutions business process services IT modernization and managed services. This client enables us to move confidently into the digital future. This client committed to Long Term success and combine global reach with local client attention to serve them in over 50 Countries.
Position: ETL Development lead
Location: Chennai Hyderabad Bangalore
Work Type: Hybrid
Job Type: Full Time
Job Description:
- We are seeking a skilled Offshore ETL/Data Engineer to support enterprise data integration transformation and modernization initiatives.
- The candidate will build scalable data pipelines and contribute to next-gen data solutions incorporating AI/ML capabilities along with legacy ETL modernization.
Key Responsibilities:
- Design develop and maintain ETL pipelines using Informatica Azure Data Factory (ADF) and Databricks
- Perform migration of legacy ETL workflows (Informatica) to Databricks using Python/PySpark
- Analyze existing ETL workflows and re-engineer into optimized Spark-based transformations
- Develop data processing and transformation solutions using Python and PySpark
- Apply AI/ML techniques for data enrichment anomaly detection and predictive insights (where applicable)
- Build and optimize SQL queries data models and transformations
- Schedule and monitor jobs using AutoSys
- Integrate data from multiple sources:
- Relational databases (SQL Server DB2)
- Files (CSV XML JSON)
- Mainframe systems
- Streaming platforms like Kafka
- Perform data validation reconciliation and ensure data quality
- Troubleshoot ETL/pipeline failures and optimize performance
- Collaborate closely with onshore teams for development and production support
- Work Schedule Requirement
- Offshore support coverage required up to 3:00 PM Central Time (CT) i.e. 2:30AM IST
- Ensure strong overlap for batch monitoring issue resolution and critical support
Required Skills:
- Strong experience in Informatica ETL development
- Proven experience in Informatica to Databricks migration
- Strong programming skills in Python and PySpark
- Hands-on experience with Databricks and Azure Data Factory (ADF)
- Proficient in SQL (complex query development and optimization)
- Experience with AutoSys job scheduling
- Experience integrating data from:
- DB / DB2 / Mainframe systems
- Files and streaming platforms (Kafka)
- Solid understanding of ETL re-engineering transformation logic conversion and Spark optimization
- AI / ML Skills (Preferred)
- Basic to intermediate understanding of AI/ML concepts and data pipelines for ML workloads
- Experience using Databricks ML / MLflow / notebooks for model tracking and experimentation
- Ability to support data preparation for machine learning models (feature engineering dataset curation)
- Exposure to Python ML libraries (e.g. Pandas Scikit-learn)
Experience or familiarity with:
- Anomaly detection / predictive analytics use cases
- Data pipelines supporting AI workflows
- Understanding of LLM / GenAI integration with data platforms (preferred but optional)
Preferred Qualifications
- Experience with Azure cloud data ecosystem
- Exposure to Data Lake / Lakehouse architecture
- Familiarity with CI/CD and DevOps practices
- Experience in batch streaming data processing patterns
Soft Skills
- Strong communication and coordination with offshore/onshore teams
- Ability to work independently in a fast-paced environment
- Proactive in identifying modernization and AI-driven opportunities
TekWissen Group is an equal opportunity employer supporting workforce diversity.
