Data Engineer Scala
Share
Job Location:
Monthly Salary:
Posted on:
21 hours ago
Vacancies:
1 Vacancy
Job Summary
We are looking for an enthusiastic Data Engineer - Scala to work to work at the forefront of our cloud modernisation within our Credit & Verification Services. This is a hybrid role requiring travelling to Hyderabad office 40% per month.
Background:
This is an incredibly exciting time for the Experian UKI Region as we look to build our presence in the UK and Hyderabad and work on a technology transformation to meet our aspiration to significantly scale our business over the next five years. This an opportunity to join Credit & Verification Services on this journey and be part of a collaborative team that uses Agile DevSecOps principles to deliver business value.
Credit and Verification Services currently comprises nearly 100 engineering teams who deliver over 200 products achieving significant revenue per annum for our UK Business. Our unique culture and agile ways of working offer a great opportunity to those looking to join an experienced diverse set of problem solvers to design build and maintain our products. We pride ourselves in excellence adopting best practices and holding ourselves to the highest standards.
Main responsibilities:
- You will be #LI-hybrid based in Hyderabad and reporting to Director Engineering.
- Develop and maintain batch data processing pipelines using Scala and Apache Spark.
- Optimize Spark jobs for performance and scalability in a cloud-native environment (AWS).
- You will collaborate with data engineers and analysts to understand data requirements and deliver high-quality solutions.
- Ensure data integrity reliability and consistency across systems.
- Participate in code reviews design discussions and agile development practices.
- Monitor and troubleshoot production jobs ensuring resolution of issues.
Qualifications :
- 36 years of software development experience with at least 2 years in Scala.
- Write clean scalable and re-usable code that implements SOLID principles common design patterns where applicable and adheres to published coding standards
- Strong understanding of functional programming and distributed computing.
- Strong experience with DevOps & IaC tooling - Terraform CI/CD pipelines Git Jenkins
- Hands-on experience with Apache Spark for batch processing.
- Familiarity with AWS services such as S3 AWS Glue Lambda Step Functions AWS Batch and CloudWatch.
- Proficiency in working with structured and semi-structured data formats (e.g. Parquet JSON).
- Experience of the full development lifecycle/application Lifecycle Management Tools BitBucket Jira
- Experience with data lake architectures and ETL frameworks.
- Exposure to containerization tools (Docker Kubernetes)
- Knowledge of data governance and security best practices
- Relevant certifications in AWS Spark or Scala.
Additional Information :
Our uniqueness is that we celebrate yours. Experians culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI work/life balance development authenticity collaboration wellness reward & recognition volunteering... the list goes on. Experians people first approach is award-winning; Worlds Best Workplaces 2024 (Fortune Global Top 25) Great Place To Work in 24 countries and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site and Glassdoor to understand why.
Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is a critical part of Experians DNA and practices and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work irrespective of their gender ethnicity religion color sexuality physical ability or age. If you have a disability or special need that requires accommodation please let us know at the earliest opportunity.
Benefits
Experian care for employees work life balance health safety and support of this endeavor we offer best-in-class family well-being benefits enhanced medical benefits and paid time off.
Experian Careers - Creating a better tomorrow together
Find out what its like to work for Experian by clicking here
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, ... View more
