Location: Bangalore (Hybrid)
Experience: 28 years
Type: Full-time
What Were Looking For
Our Client KJBN Labs is looking for a Data Engineer who thrives in building scalable data pipelines and is passionate about transforming raw data into meaningful insights. You should have hands-on experience handling large-scale data automating ETL workflows and working with modern data lake/lakehouse architectures.
Key Responsibilities
- Design develop and optimize ETL pipelines using Spark Python and modern data engineering tools.
- Work with data lakes/lakehouses (Delta Lake Apache Iceberg etc.) to manage and process structured and semi-structured data.
- Ensure high-quality data ingestion transformation and validation processes.
- Optimize pipelines to handle millions of records/day with high reliability and performance.
- Collaborate with analysts data scientists and product teams to understand data needs and build reusable data models.
- Monitor debug and enhance workflows with robust error handling and logging.
- Work with complex datasets across domains like customer behavior real-time logs transactions or sensor data.
- Data Engineering & Architecture: Design and optimize scalable secure data pipelines and storage solutions (data lakes warehouses) integrating structured and unstructured data from diverse sources.
- Performance Governance & Automation: Ensure system performance cost-efficiency and compliance through automation best practices and data governance policies.
- Collaboration & Leadership: Collaborate with cross-functional teams to support analytics and ML needs while mentoring junior engineers and resolving complex data issues
Key Skills & Technologies
- Strong programming skills in Python
- Deep experience with ETL development and Spark (PySpark/Scala)
- Hands-on experience working with Data Lake or Lakehouse architectures (S3 Delta Lake Iceberg Hudi etc.)
- Familiarity with streaming data and batch processing
- Experience working with large volumes (e.g. 100M rows/day) of diverse datasets
- Bonus: Exposure to Rust is a plus but not mandatory
Why Join KJBN
- Work on impactful large-scale data systems powering real business outcomes
- Collaborate with a high-performing data team
- Flexible work environment with opportunities for ownership and innovation
Ready to build the future of data infrastructure with us Apply now!
Required Experience:
Manager