Software Engineer Kafka & NiFi Admin with Ansible & Terraform

Position Description:

Software Engineer - Kafka & NiFi Admin with Ansible & Terraform
Missions
Position Overview
We are seeking an experienced Senior Kafka & NiFi Administrator to design deploy manage and optimize enterprise-scale data streaming and data flow platforms. The ideal candidate will have deep expertise in Apache Kafka Kafka Connect Schema Registry and Apache NiFi along with strong automation scripting CI/CD and cloud-native experience. This role ensures high availability secure configurations performance tuning and operational excellence for real-time data pipelines.
Key Responsibilities
1. Kafka Administration
Install configure and manage Apache Kafka clusters (on-premise or cloud-native: Azure HDInsights).
Manage Kafka ecosystem components: Kafka Connect Schema Registry Kafka Streams ZooKeeper etc.
Perform cluster scaling partition rebalancing topic management and multi-DC replication (MirrorMaker).
Implement monitoring alerting and logging using tools such as Prometheus Grafana ELK/ECE.
Ensure Kafka security (TLS encryption SASL RBAC Kerberos OAuth).
Troubleshoot broker issues performance bottlenecks consumer lag and message serialization errors.
2. NiFi Administration
Install manage and maintain Apache NiFi and NiFi Registry.
Design optimize and troubleshoot complex NiFi data flows.
Manage NiFi cluster configuration back-pressure settings tuning and provenance repository.
Integrate NiFi with Kafka S3 HDFS RDBMS REST APIs and cloud services.
Implement access control SSL/TLS security policies and NiFi user/group management.
3. Automation & DevOps
Develop automation using Ansible Terraform or Bash.
Build CI/CD pipelines for Kafka NiFi and data flow deployments using Jenkins Azure DevOps or GitHub Actions.
Automate cluster provisioning and configuration using Ansible and Terraform.
Create reusable templates and automation for topic creation ACL management connector deployment and flow lifecycle.
4. Operations & Support
Provide L3 support for streaming platforms including incident analysis and root cause identification.
Establish and enforce best practices for data governance data flow reliability and operational standards.
Maintain detailed documentation for configurations architectures and runbooks.
Collaborate with platform engineering data engineering security SRE and cloud teams.
Key Skills Required
Must Have
610 years of experience in Kafka & NiFi administration.
Strong knowledge of Big Data architecture and administrator responsibilities.
Strong knowledge of Hadoop Kafka internals NiFi flow design performance tuning various data formats Schema Registry Kafka Connect etc.
Configuration and performance tuning of Kafka & NiFi clusters.
Application deployment and disaster recovery.
Automation of Big Data infrastructure using Ansible and Terraform.
Good to Have
Java and shell scripting.
Hadoop administration.
Excellent communication skills.
Profile
Must Have
610 years of experience in Kafka & NiFi administration.
Strong knowledge of Big Data architecture and administrator responsibilities.
Strong knowledge of Hadoop Kafka internals NiFi flow design performance tuning various data formats Schema Registry Kafka Connect etc.
Configuration and performance tuning of Kafka & NiFi clusters.
Application deployment and disaster recovery.
Automation of Big Data infrastructure using Ansible and Terraform.
Good to Have
Java and shell scripting.
Hadoop administration.
Excellent communication skills.

CGI provides a reasonable estimate of the salary range for this position. The calculation of this range depends on various factors including skill level geographic market experience education as well as professional licenses and certifications. Compensation decisions are based on the specifics of each case. A reasonable estimate for this salary range is between $60000 and $115000. This position is currently open.

#LI-AM1

Bilingualism (French and English) is required for this position due to the nature of the role requiring interaction with national and global clients.

Use of the term engineering in this job posting refers to the technical sense related to Information Technology (IT) and does not imply that the individual practices engineering or possesses the requisite license as prescribed by the applicable provincial or territorial engineering regulator. We are seeking individuals with expertise in IT engineering-related functions but licensure from an engineering regulator is not a prerequisite for this position. Engineering is a regulated profession in Canada which is restricted in terms of use of titles and designation.

Skills:

• English
• French
• Apache Kafka
• Grafana
• Nifi
• Prometheus

What you can expect from us:

Together as owners lets turn meaningful insights into action.

Life at CGI is rooted in ownership teamwork respect and belonging. Here youll reach your full potential because

You are invited to be an owner from day 1 as we work together to bring our Dream to life. Thats why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our companys strategy and direction.

Your work creates value. Youll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas embrace new opportunities and benefit from expansive industry and technology expertise.

Youll shape your career by joining a company built to grow and last. Youll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons.

At CGI we value the strength that diversity brings and are committed to fostering a workplace where everyone belongs. We collaborate with our clients to build more inclusive communities and empower all CGI partners to thrive. As an equal-opportunity employer being able to perform your best during the recruitment process is important to us. If you require an accommodation please inform your recruiter.

To learn more about accessibility at CGI contact us via email. Please note that this email is strictly for accessibility requests and cannot be used for application status inquiries.

Come join our teamone of the largest IT and business consulting services firms in the world.