Mohammad Siddiqi

Mohammad Siddiqi

Data Engineer
United Arab Emirates
English

نبذة عني

On the data engineering side, I have experience orchestrating Spark workflows in Databricks. Experienced with Linux/Python/SQL/Docker/Terraform, and various AWS/Azure/GCP cloud services. I also have professional experien…

الخبرة

Part-time research for Digivate Labs (Azure Gen AI)

Digivate Labs
Dec 2025 - Jan 2026 · 1 شهر

Part-time research looking into and advising on generative AI app hosting using the new Microsoft Foundry.
Primarily focusing on meeting UAE data residency requirements for prospective clients.
Exploring alternatives for fine-tuning that respect UAE data residency requirements.
Exploring alternatives for voice-to-voice inference processing because the Microsoft Foundry Voice Live API does not support regional deployment for UAE-north.

Technical Consultant

Digivate Labs
Sep 2024 - Oct 2025 · 1 سنة 1 شهر

Worked primarily as a data engineer with a heavy focus on Databricks.
Later involved with research and development for a low-latency speech-to-speech LLM app development on GCP.
Set up an unstructured text streaming pipeline to process scraped websites and apply a Databricks registered ML model for classification.
Set up an Azure SQL database and pushed the text data into stream using Azure Logic Apps and Azure Event Hub.
Automated reports for management.
Built a batch pipeline in Databricks to ingest invoice PDF files and extract data using the open-AI 4o model configured in Databricks Mosaic Gateway.
Parsed PDF invoice files using OCR technology into unstructured text and extracted structured field data using an LLM.
Read through, analyzed and implemented Databricks’ unofficial X12 EDI Parser from GitHub to ingest EDI data for X12 835 and 837 transaction types.
Refactored complex Python pipelines including geospatial data into Databricks PySpark, reducing execution time by 60% in some cases.
Did a proof-of-concept for an AWS DynamoDB ingestion pipeline on Databricks.
Familiarity with the AWS SDK for Python (Boto3 library).
Refactored a pipeline that wrote data into Aerospike.
Designed workflows to migrate and validate 100TB of data from AWS S3 into Databricks Delta Tables.
Discussed AWS/Azure cloud architectures with clients to understand their current systems and requirements and how their pipelines could be migrated to Databricks.
Implemented a phone voicebot solution integrating Twilio and Gemini Flash 2.5 Live via a websocket developed using the Google gen-AI Python SDK.
Deployed the solution as a container on Google Cloud Run.
Implemented bidirectional audio byte streaming to enable low latency conversations.
Set up a FreePBX/Asterisk instance on a Linux VM as an autodialer component routing the call to a telephony service via SIP.

Technical Consultant

Digivate Labs, Dubai, UAE · Gurugram , Haryana , India
Sep 2024 - Oct 2025 · 1 سنة

Set up an unstructured text streaming pipeline to process scraped websites and apply a Databricks registered ML model for classification. Set up an Azure SQL database and pushed the text data into stream using Azure Logic Apps and Azure Event Hub (which provides Kafka connector protocol for data sinks). Automated reports for management., Batch pipeline in Databricks to ingest invoice pdf files and extract data using open-AI 4o model configured in Databricks Mosaic Gateway. PDF invoice files parsed using OCR technology into unstructured text from which structured field data such as invoice number, invoice line item, billing address, quantity, tax type & tax amount were extracted using an LLM., Read through, analyzed and implemented Databricks’ (unofficial) X12 EDI Parser (available on Github) in order to implement a pipeline to ingest EDI data for X12 835 and 837 transaction types., Refactored complex python pipelines including Geospatial data into Databricks PySpark, in some cases reducing the execution time by 60%., Did a proof-of-concept for an AWS DynamoDB (NoSQL) ingestion pipeline on Databricks. Familiarity with the AWS SDK for Python (Boto3 library). Also refactored a pipeline that wrote data into Aerospike (another NoSQL key-value database)., Designed workflows to migrate and validate 100TB of data from AWS S3 into Databricks Delta Tables., Discussed AWS/Azure cloud architectures with clients to understand their current systems and requirements and how their pipelines could be migrated to Databricks., Implemented a phone voicebot solution which integrated Twilio & Gemini Flash 2.5 Live via a websocket developed using the Google gen-AI python SDK. The solution was deployed as a container on Google Cloud Run and involved bidirectional audio byte streaming in order to enable low latency conversations. Set up a FreePBX/Asterisk instance on a Linux VM as an autodialer component routing the call to a telephony service via SIP.

Data Engineer

EFS Facility Services, Dubai, UAE
Jun 2023 - Sep 2024 · 1 سنة 3 أشهر

Worked as a data engineer.
Performed system integrations.
Designed ETL pipelines.
Coordinated with software vendors to discuss API requirements for a newly launched datahub project.
Participated in discussions with various departments including Operations, Finance, HR, and Procurement to understand their data models.
Designed pipelines to integrate over 15 systems with a proprietary datahub platform.
Was responsible for the initial implementation of the company’s datahub project.
Automated reports for management.
Designed a psychometric report in PowerBI for onboarding new employees.
Involved in integration of a new procurement system (GEP) as a replacement for the outdated customized procurement modules in JDE.
Exposed to various JDE modules and tools such as orchestrators.
Worked with functional basics of procurement such as Bidding/PR/PO, 2-way voucher and 3-way voucher matching, and approval processes.

Data Engineer

EFS Facility Services, Dubai, UAE · Dubai , Dubai , United Arab Emirates (UAE)
Jun 2023 - Sep 2024 · 1 سنة 3 أشهر

Participated in discussions with various departments (Operations/Finance/HR/Procurement) to understand their data models., Designed pipelines to integrate over 15 systems with a proprietary datahub platform and was responsible for the initial implementation of the company’s datahub project., Automated reports for management., Designed a psychometric report in PowerBI for onboarding new employees., Involved in integration of a new procurement system (GEP) as a replacement for the outdated (customized) procurement modules in JDE. Exposure to various JDE modules and tools such as orchestrators, and also functional basics of procurement such as Bidding/PR/PO, 2-way voucher & 3-way voucher matching, approval processes.

Data Science Intern

Mashreq Bank, Dubai, UAE
Oct 2021 - Dec 2021 · 2 أشهر

Developed a neural network model from scratch to detect clients’ risk of default.
The model performed competitively with existing models.
Established an automated system to periodically analyze new data on a daily basis.
Performed data preprocessing including data normalization, one-hot encoding, imputation, and data wrangling.
Performed customer segmentation on Mashreq’s client base using traditional machine learning techniques such as K-means clustering to enable more specific targeting for advertisements and notifications.

Data Science Intern

Mashreq Bank, Dubai, UAE · Dubai , Dubai , United Arab Emirates (UAE)
Oct 2021 - Dec 2021 · 2 أشهر

Developed a neural network model from scratch to detect clients’ risk of default which performed competitively with existing models. Established an automated system to periodically analyze new data on a daily basis., Performed data preprocessing including data normalization, one-hot encoding, imputation, and data wrangling., Performed customer segmentation on Mashreq’s client base using traditional machine learning techniques such as K-means clustering to enable more specific targeting for advertisements and notifications.

Intern

Emirates NBD Group, Dubai, UAE
Sep 2017 - Dec 2017 · 3 أشهر

Researched and analyzed companies and their financial sheets to evaluate their eligibility for corporate loans.
Assisted the Relationship Managers on client visits.
Observed relationship managers discuss new opportunities and follow up with clients.
Assisted with business proposals and handled relations with other banks during time in the financial institutions department.
Worked with wholesale banking relationship managers to identify and develop new markets to increase the asset book.

المهارات

لينكس مايكروسوفت أوفيس بايثون (لغة برمجة) دمج البيانات علم البيانات جبر خطي الشبكات العصبية الطوبولوجيا التصور البياني للبيانات تعلم الآلة Dimensional Data Modelling Data Engineering Data Preprocessing Data Warehousing and Analytics Reinforcement Learning Architecture discovery Analytical Problem Solving Databricks Spark & Delta Lake AWS Glue Vector Databases Compound AI systems SQL Tensorflow PyTorch Git SKLearn Pandas Numpy SparkSQL/PySpark LaTex Mathematica Vega-lite Snowflake PowerBI Dynamical Systems Metric Spaces Complex Analysis Multivariate Regression Random Forests Gaussian Processes Q Learning Multi-agent pathfinding Denoising / Variational autoencoders Dimensional Modelling ETL Pipelines Cloud Computing AWS Azure Agentic AI Docker Terraform GCP Vertica Data Vault Modeling Kafka Spark Databricks Workflows DBT LLM Infrastructure Containerized Applications RAG KV Caching Quantization Azure SQL Azure Logic Apps Azure Event Hub OCR PySpark AWS SDK for Python Boto3 AWS DynamoDB Aerospike AWS S3 Delta Tables Twilio Gemini Flash 2.5 Live WebSockets Google Cloud Run SIP FreePBX Asterisk Data Warehousing Communication
الإبلاغ عن هذا الملف الشخصي؟