Senior Talend Developer Lead Data Engineer

Role Overview

We are seeking a highly skilled Senior Talend Developer / Lead Data Engineer to drive complex data integration initiatives across SAP ERP Talend Kinaxis RapidResponse and modern cloud data platforms. The ideal candidate will combine deep handson technical expertise with solution architecture capabilities and team leadership.

This role involves designing and optimizing endtoend data pipelines modernizing legacy integrations and enabling scalable governed and cloud-native data platforms leveraging Azure Data Services and Databricks. The engineer will play a key role in supporting global supply chain planning systems and enterprise data initiatives.
Key Responsibilities

1. Data Integration & ETL Development (Talend)

• Design and implement robust ETL workflows using Talend Data Integration (Open Studio).

• Hands-on expertise in designing and implementing template-driven architecture for reusable pipelines with minimal code changes.

• Integrate data from diverse sources including SAP databases APIs flat files cloud repositories MQs and XML-based systems.

• Apply best practices for data profiling cleansing validation transformation mapping and auditing.

• Build scalable batch and near-real-time data pipelines.

• Implement strong monitoring alerting and logging for reliable production operations.

2. Integration Architecture & Delivery

• Architect and develop endtoend integration solutions across SAP ERP (S/4HANA and ECC) Talend Kinaxis RapidResponse MongoDB Solace/RabbitMQ and other enterprise systems.

• Lead migration of legacy file transfers and XML interfaces to modern automated frameworks.

• Define and enforce data mapping standards enrichment logic and transformation rules.

• Ensure alignment with business requirements and integration best practices.

• Strong understanding of version control and branching strategies using Talend Git.

3. Talend Administration Center (TAC) & Runtime Operations

• Manage deployments job scheduling execution logs and monitoring via TAC and TMC.

• Configure tasks triggers resource utilization sequencing and error-recovery workflows.

• Manage environment promotions versioning and dependency resolution.

• Deploy jobs on Talend JobServer and ESB Runtime across Dev Test UAT and Production environments.

• Optimize memory parallelism and runtime configurations for performance.

4. SAP Data Integration

• Develop ETL workflows using SAP integration technologies: IDoc BAPI RFC OData and structured extract files.

• Understand SAP data models business logic and core master/transactional tables such as MARA EKPO BKPF MSEG etc.

• Work closely with SAP functional and BASIS teams to ensure reliable validated and secure data extraction.

5. Cloud Modernization (Azure & Databricks)

• Architect and implement solutions using Azure Data Factory Azure Synapse Azure Data Lake Storage.

• Utilize Databricks (PySpark Delta Lake) for advanced transformations metadata-driven orchestration and ML readiness.

• Integrate Talend with Azure CI/CD pipelines using Azure DevOps or GitHub Actions.

• Promote cloud-native modernization and best practices.

6. Governance Observability & Security

• Implement Data Quality frameworks validation layers and automated data health checks.

• Build endtoend data lineage metadata and traceability frameworks.

• Develop enterprise observability dashboards using Grafana Azure Monitor and related tools.

• Ensure compliance with RBAC encryption key vault usage audit trails and secure configuration standards.

7. Leadership & Cross-functional Collaboration

• Mentor and guide junior and mid-level data engineers.

• Perform code reviews enforce development standards and maintain technical excellence.

• Collaborate with SAP teams Kinaxis consultants global business stakeholders and platform architects.

• Manage change requests prioritization issue resolution and agile delivery workflows.

8. Future-Ready Initiatives

• Evaluate and implement streaming architectures using Kafka or Azure Event Hub.

• Explore AI-driven data quality anomaly detection and pattern recognition using Databricks ML.

• Drive cost optimization scalability improvements and multi-region deployment strategies.
   

Qualifications :

Educational qualification:

Bachelors degree in Computer Science Information Technology Software Engineering or a related field.

Core Technical Expertise (Mandatory)

• Advanced experience in Talend DI/ESB job design parameterization error handling and optimization.

• Minimum 6 years in ETL and enterprise data integration.

• Minimum 6 years working with Talend TAC/TMC administration.

• Minimum 3 years integrating SAP via IDoc BAPI RFC OData.

• Minimum 3 years deploying and managing jobs on Talend Runtime.

• Minimum 6 years strong SQL experience (Oracle MySQL SQL Server etc.).

Kinaxis & SAP Integration

• Strong expertise with Kinaxis RapidResponse data model and file-based integration.

• Deep experience in XML XSD schema evolution and large-file processing.

Cloud & Big Data Technologies

• Hands-on experience with Azure Data Factory Azure Synapse Azure Data Lake Storage.

• Databricks experience using PySpark Delta Lake optimization techniques.

Programming & Scripting

• Proficiency in Python SQL and Groovy for automation and custom ETL logic.

Governance & Security

• Experience with DQ frameworks metadata/lineage tools and auditing.

• Strong grasp of secrets management encryption and compliance frameworks.

Preferred Skills

• Java development for custom Talend components (2 years).

• Git experience (3 years) with branching strategies PR workflows and code reviews.

Soft Skills

• Strong analytical and problem-solving abilities.

• Excellent communication and stakeholder engagement.

• Ability to lead teams drive architecture decisions and deliver large-scale programs.

• High attention to detail documentation discipline and a quality-first mindset.

Additional Information :

Over all experience of 8-12 years

Remote Work :

No

Employment Type :

Full-time