Data Engineer Databricks

IT Developer Data Engineer

Clark Philippines

About Citco

Citco is a global leader in fund services corporate governance and related asset services with staff across 80 offices worldwide. With more than $1 trillion in assets under administration we deliver end-to-end solutions and exceptional service to meet our clients needs.

For more information about Citco please visit

About the Team & Business Line:

Proprietary software solutions and innovation are at the core of what differentiates Citco in the alternative investment space. Through our network of global development centers Citco invests heavily in technology development security and infrastructure to ensure our clients continue to receive award-winning products that underpin our commitment to service excellence.

As a core member of our Innovation Development team you will work with dedicated professionals to ensure our clients maintain access to their critical information assets while keeping Citco ahead of industry trends.

Your Role:

• Design develop and maintainETL/ELT pipelinesusing Databricks Delta Lake and Apache Spark
• Supportdata platform migration activities including migrating pipelines workflows and data assets from legacy or alternative platforms (e.g. on-premise other cloud platforms)to Databricks on AWS
• Assess plan and executeend-to-end migration strategiesensuring minimal disruption and data integrity throughout
• Build and manageAWS-based data pipelinesusingAWS GlueS3 and related ETL services
• Implement data workflows using Databricks notebooks jobs andAWS Glue workflows
• Create and optimize data models for bothbatch and streamingdata processing
• Develop and maintaindata quality frameworksand monitoring systems during and post-migration
• Collaborate with cross-functional teams to understand data requirements and implement scalable solutions
• Write efficientSQL queriesand optimize performance of existing queries
• Implementdata security and governancepractices aligned with organizational standards
• Build and maintain documentation for data processes migration runbooks and architectures (e.g.Medallion Architecture)
• Explore and applyGen AI / Machine Learningcapabilities on data pipelines where applicable
• Stay current with new and upcoming features ofDatabricks AWS and emerging data tools

About You:

Experience and Technical Skills

• 35 yearsof hands-on experience in data engineering with at least23 yearsofdedicated Databricks hands-on experienceincluding notebooks clusters jobs Delta Live Tables and Unity Catalog

• Strong hands-on experience withDatabricks core components:

  • Delta Lake & Delta Live Tables (DLT)

  • Databricks Workflows & Job Orchestration

  • Unity Catalog & Data Governance

  • Databricks AutoLoader & Structured Streaming

• In-depth knowledge ofSQL Python PySpark and/or Scala within the Databricks environment

• Strong expertise integrating Databricks withAWS servicesincludingS3 AWS Glue Athena andLambda

• Hands-on experience building and managingETL/ELT jobs in AWS(Glue Jobs Glue Crawlers Glue Data Catalog)

• Demonstrable experience inreal-time and batch data processingusing Databricks

• Experience withPL/SQL programming creating SQL scripts and writing stored procedures

  • Familiarity withBI Toolssuch as Power BI and/or QLIK

Education

• Bachelors degree or equivalent experience
• Demonstrated commitment to continuous improvement and training

Desired skills and expertise

• Databricks Data Science/Engineer certifications
• PowerBI Predictive Analytics AI/ML Medallion architecture

• Exposure to or working knowledge ofGenerative AI LLMs or Machine Learningin a data context (e.g. MLflow feature stores model serving) nice to have

Our Benefits

Your wellbeing is of paramount importance to us and central to our success. We provide a range of benefits training and education support and flexible working arrangements to help you achieve success in your career while balancing personal needs. Ask us about specific benefits in your location.

We recognize diversity as a source of organizational pride and strength. We have made it a priority to reflect our nations evolving diversity in the people we hire and the culture we create in our organization.

Citco welcomes and encourages applications from people with disabilities. Accommodations are available upon request for candidates taking part in all aspects of the selection.