<p>Migrated OFSDF workloads from on-prem Hadoop to GCP using Dataproc, BigQuery, GCS, Cloud Composer, and Cloud Logging.</p><p> Designed and developed PySpark-based ingestion pipelines to load data into GCS Landing Zone.</p><p> Built scalable transformation pipelines on Dataproc and optimised analytical datasets in BigQuery for reporting and downstream analytics.</p><p> Used Pub/Sub for event-driven data ingestion and triggering downstream processing workflows.</p><p> Implemented secure access using IAM, service accounts, encryption, and GCP governance best practices.</p><p> Developed monitoring and alerting using Cloud Logging & Monitoring.</p><p> Built data products for Commercial Loan Customers & Management, integrating data from multiple systems.</p><p> Designed and implemented multi-layer architecture: Raw → Schema → Consolidation → Curation → Consumption.</p><p> Built Spark-Scala transformation framework supporting daily/monthly/FTS ingestion patterns.</p><p> Performed joins, unions, aggregations, and curated datasets for finance consumption.</p><p> Applied standardisation rules (datatype conversions, curated IDs, naming conventions).</p><p> Versioned and deployed code using Git-UP, Jenkins, UrbanCode, Spinnaker, and TWS.</p><p> Designed and optimised Databricks + Spark pipelines supporting analytics, ML, and regulatory reporting across a large enterprise environment (POC).</p><p> Built scalable ETL workflows using Python, PySpark, AWS Glue, Lambda, and S3, ensuring accuracy and reliability (POC).</p><p> Implemented Delta Lake–based lakehouse layers with schema governance, lineage, and automated data quality checks (POC).</p><p> Developed ingestion pipelines integrating APIs, RDBMS, Kafka/MSK, and cloud storage, delivering high-throughput and resilient data flows (POC).</p><p> Ensured compliance with GDPR, auditability, lineage, and financial regulatory controls.</p><p> Collaborated with architects, product owners, and downstream consumers to define SLAs, monitoring, and operational readiness.</p><p> Authored architecture documentation for ingestion patterns, cluster sizing, and governance.</p>
