Data Engineer

Job Title -Data Engineer
Experience: 2 Years
Employment Type: Full - Time
Location: Thrissur (Onsite)
Budget - upto 6 LPA
Male Immediate Joiners preferred

Job Summary

We are looking for an experienced Senior Data Engineer with strong expertise in designing implementing and maintaining scalable data pipelines and distributed messaging systems such as Apache Kafka RabbitMQ and similar technologies.

The ideal candidate should have hands-on experience in real-time data streaming ETL/ELT processes data integration infrastructure management and production support. The candidate should also be capable of independently handling data engineering activities troubleshooting critical issues and supporting high-volume data processing systems.

Key Responsibilities

Data Pipeline Architecture & Development
Design develop and maintain scalable real-time and batch data pipelines.
Build reliable ETL/ELT workflows for processing structured and unstructured data.
Ensure high availability scalability and fault tolerance of data systems.

Messaging & Streaming Platform Management
Set up configure and maintain:
Apache Kafka
RabbitMQ
Similar messaging/event-driven platforms
Manage:
Topics
Queues
Producers
Consumers
Brokers
Clusters
Monitor message delivery throughput and system health.

Data Integration & Processing
Integrate data from multiple systems including:
APIs
ERP systems
Databases
Third-party platforms
Implement real-time and asynchronous data synchronization mechanisms.

Monitoring Troubleshooting & Optimization
Monitor pipeline performance and system stability.
Identify and resolve bottlenecks failures and performance issues.
Implement logging alerting backup and recovery mechanisms.
Optimize processing speed reliability and infrastructure utilization.

Database & Infrastructure Management
Work with relational and NoSQL databases.
Perform query optimization indexing and performance tuning.
Support deployment and maintenance on:
Linux servers
Cloud platforms
Containerized environments

Collaboration & Leadership
Coordinate with:
Development teams
QA teams
DevOps teams
Business stakeholders
Provide technical guidance to junior engineers.
Participate in architecture discussions and technical planning.

Required Technical Skills

Mandatory Skills
Strong hands-on experience with:
Apache Kafka
RabbitMQ
Similar messaging or streaming platforms
Experience in:
Data pipeline development
ETL/ELT processes
Real-time data streaming
Distributed systems

Programming & Scripting
Proficiency in one or more:
Python
Java

Scala (preferred)

Database Knowledge
Strong knowledge of:
MySQL
PostgreSQL
MSSQL
MongoDB or other NoSQL databases

Infrastructure & DevOps
Experience with:
Linux environment
Docker
Kubernetes
CI/CD pipelines

Exposure to cloud platforms such as:
AWS
Azure
GCP

Preferred Qualifications
Bachelors degree in Computer Science IT or related field.
Experience handling production-scale distributed systems.
Knowledge of data warehousing and analytics platforms is an advantage.

Soft Skills
Strong troubleshooting and analytical skills
Good communication and coordination abilities
Ability to independently manage critical tasks and priorities
Documentation and process-oriented mindset

Additional Expectations
Ability to provide production support and handle critical incidents.
Ensure data reliability security and operational efficiency.
Maintain proper technical documentation and knowledge-sharing practices.

Required Skills:

Apache KafkaRabbitMQETLELTNoSQL