Senior Integration Engineer - Kafka
Senior Integration Engineer - Kafka
Posted on :
19-05-2025
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
We are seeking a highly skilled and experienced Senior Integration Engineer to join our dynamic team. This role will be pivotal in designing developing and maintaining robust integration solutions with a strong focus on leveraging Apache Kafka for realtime data streaming and AWS integration services particularly AWS Glue for ETL and data cataloging. The ideal candidate will have a deep understanding of enterprise integration patterns eventdriven architectures and cloudnative solutions. You will collaborate closely with various engineering product and business teams to ensure seamless and efficient data flow across our diverse system landscape.
Key Responsibilities:
- Design develop implement and maintain scalable and resilient integration solutions using Apache Kafka and its ecosystem (Kafka Connect Kafka Streams Schema Registry).
- Monitor troubleshoot and optimize Kafka cluster performance including throughput latency and resource utilization to ensure stability and efficiency.
- Architect and build ETL/ELT pipelines using AWS Glue for data extraction transformation and loading from various sources into data lakes data warehouses and other target systems.
- Develop and manage AWS Glue Data Catalog and ensure data quality consistency and governance across integrated systems.
- Integrate a variety of cloudbased applications databases and thirdparty services using appropriate AWS services (e.g. Lambda S3 API Gateway SNS SQS EventBridge) and other integration technologies.
- Champion and implement best practices for eventdriven architecture microservices integration and API development.
- Collaborate with architects data engineers software developers and business stakeholders to understand integration requirements and translate them into technical designs and solutions.
- Optimize and troubleshoot existing integration solutions to improve performance reliability and scalability.
- Develop and maintain comprehensive documentation for integration designs processes and data flows.
- Implement robust monitoring alerting and logging for integration solutions to ensure operational excellence.
- Mentor junior engineers and provide technical leadership within the integration domain.
- Stay current with emerging technologies and industry trends in data integration Kafka and AWS services.
- Ensure security best practices are implemented and maintained for all integrationsolutions.
RequiredQualifications:
- Bachelors degree in computer science Engineering Information Technology or a related field (or equivalent practical experience).
- 5 years of handson experience in designing and developing enterpriselevel integration solutions.
- Proven indepth experience with Apache Kafka including designing and implementing Kafka producers consumers topics connectorsand performance tuning.
- Strong experience with AWS cloud services particularly AWS Glue for ETL data cataloging and job orchestration.
- Proficiency with other AWS integration and data services such as S3 Lambda API Gateway Kinesis Redshift RDS DynamoDB.
- Solid understanding of data modeling data warehousing concepts and ETL/ELT processes.
- Experience with various data formats (e.g. JSON Avro Parquet XML) and API protocols (e.g. REST SOAP gRPC).
- Proficiency in one or more programming languages commonly used in integration development (e.g. Python Java Scala).
- Experience with SQL and NoSQL databases.
- Strong understanding of integration patterns eventdriven architecture and microservices principles.
- Experience with version control systems (e.g. Git) and CI/CD pipelines.
- Excellent analyticalproblemsolving and troubleshooting skills.
- Strong communication and interpersonal skills with the ability to collaborate effectively withtechnical and nontechnicalteams.
Preferred Qualifications:
- Confluent Certified Developer for Apache Kafka (CCDAK) or similar Kafka certification.
- AWS Certifications (e.g. AWS Certified Data Analytics SpecialtyAWS Certified Developer Associate/Professional AWS Certified Solutions Architect Associate/Professional).
- Experience with other ETL tools (e.g. Apache Airflow Talend Informatica).
- Familiaritywith containerization technologies (e.g. Docker Kubernetes).
- Experience with Infrastructure as Code (IaC) tools (e.g. TerraformAWS CloudFormation).
- Knowledge of data governance and data security best practices.
Required Experience:
Senior IC
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
