I’ve worked in data for over ten years. In different environment such as a digital transformation at Santander, re-writing an analytical data lake an JP Morgan, calculating the sending or holding back of mail at Royal Ma…
I’ve worked in data for over ten years. In different environment such as a digital transformation at Santander, re-writing an analytical data lake an JP Morgan, calculating the sending or holding back of mail at Royal Mail, geofencing in real time with spark streaming. I’m comfortable being a single contributor, a coach for more junior member or being relied on to lead and do senior stakeholder management.
The stack is Bigquery, cloud functions, workflows, Terraform.
Moving their 300+ table estate to DBT.
Identifying processes that are no longer needed.
Re-creating processes in a more simple way in DBT.
GCP, bigquery, terraform, DBT, etc
I'm an AWS or GCP data engineer with years of experience. Working on ingestion, transformations, data quality etc. Having worked for the Royal Mail, JP Morgan, Hotels.com
Completed a DQ framework in pyspark/glue.
Built the process to be config driven.
Produced output used by another team in a web app.
Looked after a spring boot java app.
Fixed a long term bug discovered in the app.
Found the code and fixed it under time pressure.
Engaged in stakeholder management with teams in the UK, India and America.
Ingested data, transformed it and created a PowerBI dashboard.
Used GCP (bigquery), Terraform and PowerBI.
Helped streamline manual data processes.
Updated infrastructure in GCP using postgres.
Migrated a source for their API from postgres JSON storage to a noSQL database.
Moved a team with no deployment process to github actions and terraform.
Created a simple devOps process to get them started.
Tried to train up the team who were mostly previously DBAs.
Created a financial data migration and reconciliation.
Moved data from Teradata into s3 using Spark.
Created a simple config driven solution used by other less technical teams.
Refactored a Java DQ app to improve performance.
Got involved in pull requests and coding standards for other teams.
Created spark jobs around IOT / Device events ingestion and transformation.
Built Spark Structured streaming from Kafka to Kudu.
Moved data from huge legacy SQL estate into hadoop using sqoop automated in Python.
Enriched data in Spark.
Built a streaming and batch pipeline.
Helped transform the team into a TDD team using functional and OO design patterns.
Kept a real time dashboard of event aggregations up to date using spark streaming and Kudu.
Made the pipeline fault tolerant.
Loaded old data back into memory on system failures so the aggregations are accurate.
Created a library to speed up development for the team.
Created an easier way of interacting with Kudu without a Spark context.
Created a Table abstract class.
Added unit tests and very high coverage.
In charge of data quality in the data lake for Santander UK.
Reported to the head of financial data quality.
Advised on design and usability of lake warehouses.
Performed accounting reconciliations and data completeness tests for vast amounts of transactional data.
Thought about the performance of spark jobs for massive data sets.
Wrote Spark/Scala production code in Santander’s data lake.
Created generic frameworks and tools based around all areas of the ETL process in Scala and Python.
Created tools for other teams to use to make them productive in the data lake.
Performed programming tasks such as processing JSON, Excel and access databases.
Acted as the technical lead on many agile projects.
Joined as a database expert.
Taught 2 accountants how to use SAS and general database training.
Was put in charge of delivering 2 regulatory projects.
Acted as technical lead on both projects.
Delivered a KPI project and Risk Data Aggregation.
Created C# WPF apps for the BAU team to streamline their data warehouse maintenance.
Created bespoke reports.
Helped clients balance books.
Looked into accounting issues.
Learned a decent level of accounting.
Configured IIS.
Created many triggers and stored procedures.
Performed detailed testing.
Booked balancing database fixes in production.
Worked on GCP with strong CI/CD pipelines.
Created ETL processes with NiFi, cloud functions, dataflow, pubsub and Airflow.
Streamed from pubsub or batch using NiFi to storage.
Used cloud functions to kick off DataFlow jobs.
Built ETL using dataflow again, airflow with SQL scripts or spark in dataproc.
Used Teraform for schema/table creation.
Modelled data correctly.
Took partitioning and clustering into account to keep costs low and data moving quickly.
نظام التوظيف التشغيلي المعتمد على الذكاء الاصطناعي أولاً، الذي يوحّد ويؤتمت دورة التوظيف بالكامل للمؤسسات الحديثة حول العالم.
د.جوب منطقة حرة ذ.م.م. 2026 © جميع الحقوق محفوظة
