Santos Saenz Ferrero

<p>Designing the overall new architecture of the platform in terms of better data engineering, infrastructure design and CI/CD.<br>
Looking after long-term effectiveness and optimization of the platform.<br>
Pushing new features and best practises.<br>
Close communication with clients and stakeholders.<br>
Pressing collaboration with teams with a similar software stack, Platform team, Security team and teams within the division.<br>
Supervising, providing oversight and advice to other teammates.<br>
Bringing data from different clients to be consumed in the platform by developing data engineering and data modeling tasks with Azure Data Factory and Databricks (PySpark, Delta Lake, Mosaic).<br>
Developing IaC with Terraform for the platform.<br>
Collaboration in infrastructure at an enterprise level through communication and synchronization with the cloud team and security team.<br>
Development of modules made available in Terraform Private Registry and policies with Sentinel.<br>
Managing and developing CI/CD with GitHub, different GitHub Settings, GitHub Actions and GitHub Webhooks.</p>

Division Digital Solutions

•Design of the overall new architecture of our platform, in terms of better data engineering, infrastructure & CI/CD. Looking after long-term effectiveness & optimization of our platform. Pushing new features and best practises

•Communication with clients & stakeholders

•Bringing data to be consumed in our platform by developing data engineering & data modelling tasks with Azure Data Factory & Databricks (PySpark, Delta Lake, Mosaic)

•Developing IaC with Terraform for our platform, & collaboration in Infrastructure at an enterprise level through communication & synchronization with the cloud team & security team

•Managing & developing CI/CD with GitHub, different GitHub Settings, GitHub Actions & GitHub Webhooks

•Governance at scale with Pureview, Azure Governance & Unity Catalog

•Assessing organization wide best practises and compliance through Azure Policies, Terraform Private Registry and Landing Zones

<p>Helping in the design of the overall architecture of the platform in terms of data engineering and infrastructure.<br>
Looking after long-term effectiveness and optimization of the systems.<br>
Close communication with clients and stakeholders.<br>
Self responsible in the selection of the most appropriate tools and strategies for developing the client demands.<br>
Supervising, providing oversight and advice to other teammates.<br>
Bringing the customer data to the Databricks Lakehouse Platform through medallion architecture ETL and data modeling using Data Factory and Databricks.<br>
Using Scala Spark API and Delta Live Tables through Python in both batch and streaming processing paradigms.<br>
Consuming data from different data systems like Relational Databases (SQL Database, SQL Server, MySQL), CRM Salesforce, API-REST, SFTP, Non-Relational Databases (InfluxDB).<br>
Managing data governance through Unity Catalog, implementing data object access control, definition of service principals.<br>
Creating and managing work clusters, all purpose clusters and DBSQL warehouses.<br>
Handling Azure Data Lake Storage and Key Vault.<br>
Managing CI/CD through Databricks Repos.<br>
Using Git, Maven, PyPI, Jenkins and Artifactory.<br>
Performed maintenance and development of new features for a Maven project built on Scala that through Azure SDK and Databricks SDK performs API calls to these platforms.<br>
Used for non technical users to develop ETL in the lakehouse through the use of JSON files.<br>
Developed and scheduled pipelines that handled and enriched millions of movements and transactions daily for consumption by clients from BBVA in Spain and Mexico.<br>
Project responsibilities evolved to developing most complex, critical and time constraint modules of the project.<br>
Supervising, providing oversight and advice to the rest of the teammates.<br>
Close communication with client and other data producers and data consumer teams.<br>
Collaboration with production and infrastructure team.<br>
ETL developed using Spark with Scala.<br>
Developed ETL using Java applications, SQL applications, Python scripts and Shell scripts through the Batch Processing Architecture.<br>
Processing structured and semi-structured high volume data (Parquet, JSON, CSV).<br>
Optimized performance of Spark and other ETL jobs by checking execution DAG and logs with tools like Atenea.<br>
Used different data stores like S3 buckets, Oracle DB instances, Elasticsearch instances and HDFS.<br>
Managed and checked job orchestration and execution through Control-M and scheduled job execution with a Scheduler as a Service.<br>
Used Jenkins, Artifactory, Git and Maven.<br>
Used containerization for local development with Docker.<br>
Used CLI for managing different services.<br>
Analysed quality of code with SonarQube.<br>
Developed tests with JUnit and Mockito.<br>
Used corporate tools for consultation and management of data governance, Data Quality and Traceability.</p>

<p>Developed REST API using Python, SQL and AWS Lambda.<br>
Ingested data from customers, service stations and the stock market through API calls.<br>
Transformed data and provided customer recommendations for refueling and predictions related to fuel prices using gradient boosted decision trees.<br>
Developed different machine learning models.<br>
Performed hyperparameter tuning using Bayesian optimization.<br>
Developed ETL/ELT pipelines and calculated KPIs using Pandas and Python.<br>
Used Git and PyPI.<br>
Interacted with AWS through REST API.<br>
Performed exploratory data analysis on different datasets, mostly related with fuel and time series data using Python, Plotly, Matplotlib, Seaborn, NumPy and Pandas.<br>
Performed univariant and multivariant analysis.<br>
Performed analysis of tendency, stationality, stationarity and noise.<br>
Performed preprocessing of data and analysed best machine learning approaches and algorithms for forecasting.<br>
Developed REST API using Python, SQL and AWS Lambda that reads financial structured data from customers, transforms it, summarizes the result and sends it by email automatically.<br>
Worked with NumPy, SQL and Pandas for cleaning data from customers, transforming it and obtaining different insights about Return of Inversion, Sales and Investments.<br>
Developed HTML documents with CSS that visually provide the insights for the customers.</p>