Senior Data Engineer

Who We Are

Verve has created a more efficient and privacy-focused way to buy and monetize advertising. Verve is an ecosystem of demand and supply technologies fusing data media and technology together to deliver results and growth to both advertisers and publishersno matter the screen or location no matter who what or where a customer is. With 30 offices across the globe and with an eye on servicing forward-thinking advertising customers Verves solutions are trusted by more than 90 of the United States top 100 advertisers 4000 publishers globally and the worlds top demand-side platforms. Learn more at.

Who You Are

Were looking for a Senior Data Engineer to join our Data Processing team at Verve. This team is responsible for designing and maintaining scalable real-time and batch data processing pipelines that enable high-quality insights and analytics. The Data Processing team works closely with the Data Analytics and Data Science teams to ensure efficient data flows across the organization.

In this role youll focus on developing and optimizing data processing workflows ETL pipelines and streaming solutions. Precision efficiency and high-quality deliverables are essential and youll report to the Engineering Manager Data Processing.

If you are passionate about working with big data technologies real-time data processing and optimizing large-scale ETL workflows wed love to hear from you!

This is a hybrid position with three days per week in our Bengaluru office.

What You Will Do

Domain Expertise:

• Design develop and maintain scalable efficient and cost-effective data pipelines for real-time and batch data processing.

• Build high-performance ETL workflows leveraging Kafka Apache Spark and Scala.

• Implement best practices for data modeling storage and retrieval ensuring high availability and reliability.

• Work with GCP-based data services including BigQuery Dataflow and Cloud Storage.

• Optimize the performance and cost efficiency of data processing jobs.

Data Processing & Engineering

• Architect and implement scalable streaming and batch processing solutions.

• Develop real-time data processing pipelines using Kafka and Apache Spark Structured Streaming.

• Ensure the stability and reliability of high-volume data ingestion pipelines.

• Write clean testable and efficient code using Scala.

• Ensure compliance with data privacy and security best practices.

Collaborative Work

• Work closely with Data Analytics and Data Science teams to ensure data is processed stored and accessed efficiently.

• Collaborate with the Data Analytics team to provide structured data for data warehouses reporting APIs and UIs.

• Support the Data Science team by ensuring high-quality data availability for machine learning models.

• Work cross-functionally with engineering and product teams to design end-to-end data solutions.

• Coordinate with global teams in different time zones to deliver projects successfully.

• Extend optimized solutions to sister companies within Verve Group.

What Will You Bring

Must-Have Qualifications:

• 5 years of experience as a Data Engineer working on large-scale ETL/data processing pipelines.

• Experience handling daily terabytes of data efficiently.

• Strong knowledge of Apache Kafka for streaming data processing.

• Strong expertise in Apache Spark (Scala) for batch and real-time processing.

• Proficiency in orchestration tools like Apache Airflow.

• Proficiency in GCP including services like BigQuery Dataflow and Dataproc.

• Solid understanding of data modeling techniques including Star/Snowflake schema and state of the art Lakehouses.

• Strong SQL skills including query optimization and performance tuning.

• Familiarity with Apache Druid or other OLAP engines.

• Strong knowledge of infrastructure as code (Terraform).

Nice-to-Have Qualifications:

• Experience with AdTech or high-velocity data processing environments.

• Hands-on experience with Hadoop ecosystem tools.

• Knowledge of containerization (Docker Kubernetes).

• Familiarity with BI tools like Looker

• Basic DevOps knowledge

What We Offer

Just a few of the benefits waiting for you at Verve:

• Keep healthy through our Apollo Health Check-ups and stay covered with our Medical Insurance for you and your family

• Pick what matters most to you in our INR 4100/month Personalized Benefits Platform: Travel entertainment food fitness and healthcare

• Have us cover your office lunches 3 times a week through Zomato and enjoy office sweets snacks and beverages

• Boost your professional knowledge with our annual training budget & internal webinars and level up your language skills through our German/English classes

• Recharge with 19 paid vacation days 3 Wellness days throughout the year in addition to the public holidays

• Strengthen team connections while exploring new cultures through our monthly Work&Travel raffle offering a chance to work from one of our global offices (after 2 years of employment)

• and even more reasons to join us!

Verve provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race color religion age sex national origin disability status genetics protected veteran status sexual orientation gender identity or expression or any other characteristic protected by federal state or local laws.

#LI-Hybrid