Senior Data Engineer with Databricks

About the Client

Our client is one of the Big Four accounting firms and the worlds largest professional services

network. Headquartered in London they operate in 150 countries with 460000 professionals

delivering excellence in audit tax consulting and advisory.

The team you will join designs develops and deploys innovative enterprise technology and

AI-driven tools to support the delivery of tax services. It is a dynamic group combining expertise

in tax software engineering change management and project management working on

initiatives from tool design and deployment to training development and engagement

management.

Project Tech Stack

Azure Cloud Microservices 8 Core services Python MongoDB

Azure SQL Angular 18 Kendo GitHub Enterprise with Copilot LangGraph LangChain RAG

Pipelines Multi-modal LLMs

What Youll Do

Define and enforce best practices and coding standards across the project

Conduct thorough code reviews to ensure adherence to established guidelines and

maintain high code quality

Working both independently and in close collaboration with others in the team

Communicating clear instructions to team members and helping manage the flow of day-to-day operations

Communicating with the client regularly

Design develop and maintain robust and scalable Spark applications

Write clean maintainable and efficient code following best practices and coding standards

Optimize code for performance and scalability ensuring efficient data handling

Work closely with cross-functional teams to deliver high-quality software solutions

Identify and resolve technical issues ensuring the reliability and performance of applications

Create and maintain comprehensive documentation for code processes and workflows

What You Bring

5 years of hands-on experience in software development

Practical experience developing SQL stored procedures and migrating them into Spark SQL or PySpark environments

Extensive expertise using PySpark on Databricks encompassing Delta tables cluster management and workflow automation

Proven track record of optimizing Spark performance within production-level projects

Strong proficiency in Python specifically for complex data manipulation utilizing libraries like Polars or Pandas

Solid understanding of columnar data storage formats particularly Parquet with practical experience using Delta Tables

Proven expertise in data processing analysis and transformation workflows

Strong analytical and problem-solving abilities with a detail-oriented mindset

Practical and pragmatic approach to balancing standardized processes with flexibility to meet project goals effectively

Solid understanding of microservices architecture and its implementation in scalable systems

Nice to have

Experience working with Azure Cloud services (or other major cloud platforms)

including a range of SaaS offerings such as Service Bus Data Lake Blob Storage Redis and more Familiarity with FastAPI

Expertise in containerization and orchestration tools such as Docker and Kubernetes

English level

Advanced

What we offer

• Model: B2B contract with Bonapolia - one contract one monthly invoice

• Rate: negotiated per candidate seniority and client budget

• Payment: monthly invoice paid on the same schedule the client pays us (usually 30 days)

• Contract written under your countrys law

• We handle the client-side paperwork: MSA NDA invoicing

About Bonapolia

Bonapolia is a family-owned IT staff augmentation team based in Lisbon. We build engineering teams for product companies and IT consultancies across Europe and the US.

More at


Ready to Join
We look forward to receiving your application and welcoming you to our team!