Senior Data Engineer (Databricks) 6 Month Contractor
Share
Job Location:
Monthly Salary:
Posted on:
13 hours ago
Vacancies:
1 Vacancy
Job Summary
We are looking for a skilled Data Engineer with substantial experience to design develop and maintain scalable data pipelines and data solutions using Databricks. Youll collaborate closely with cross-functional teams to ensure data quality availability and efficiency to support analytics and business intelligence.
You Will:
- Design and implement scalable data architectures using Databricks Unity Catalog Delta Lake and Apache Spark
- Develop and maintain complex ETL/ELT pipelines processing terabytes of data daily
- Architect medallion architecture (Bronze Silver Gold) data lakehouses with optimal performance patterns
- Implement real-time and batch data processing solutions using Structured Streaming and Delta Live Tables
- Design data mesh architectures with proper data governance and lineage tracking
- Optimize Spark jobs for cost efficiency and performance including cluster auto-scaling and resource management
- Implement advanced Delta Lake features including time travel vacuum operations and Z-ordering
- Build robust data quality frameworks with automated testing and monitoring
- Design and implement CI/CD pipelines for data workflows using Databricks Asset Bundles or similar tools
- Develop custom solutions using Databricks APIs and integrate with external systems
#LI-Hybrid
Qualifications :
You Have:
- A Bachelors degree in computer science or a related field.
- 6 years of experience in data engineering including designing and deploying production-quality data pipelines and ETL/ELT solutions.
- A Databricks Certified Professional Data Engineer certification
- Proficiency in Python PySpark and SQL.
- Strong hands-on experience with cloud services such as AWS (Glue Lambda Redshift S3).
- Experience with data governance and management platforms (e.g. AWS Sagemaker Unified Studio and Unity Catalog.
- Experience with data migration projects and legacy system modernization
- Familiarity with containerization and orchestration technologies (Docker Kubernetes) is a plus.
Additional Information :
Whats in it for you to join MUFG Investor Services
Take a look at our careers site and youll find everything youd expect working with one of the fastest-growing businesses at one of the worlds largest financial groups. Now take another look. Because its how we defy expectations that really defines us. Youll feel that difference in all kinds of ways. Our vibrant CULTURE. Connected team. Love of innovation laser client focus.
So why settle for the ordinary Apply now for your next Brilliantly Different opportunity.
We thank all candidates for applying; however only those proceeding to the interview stage will be contacted.
MUFG is an equal opportunity employer
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Were more than just fund administration. Our offerings include a scope of solutions not historically provided by asset servicing companies. From fund financing to subscription lines of credit, securities lending, foreign exchange (FX) services, consulting services and beyond, we help ... View more
