Software Engineer (Java Azure Data Engineering) Expert Polyglot and AIDriven Developer
Overview
A highly experienced engineer with 5 years of deep programming expertise who thrives on solving complex problems independently and in teams. Primarily skilled in Java 8 complemented by strong proficiency in Azure data engineering (ADF Databricks ADLS PySpark) and for front-end development. Comfortable navigating multiple programming paradigms and equipped to work on fullstack cloudnative and dataintensive development at pace and quality.
Key Responsibilities
- Backend & APIs (Java): Design and develop scalable microservices and APIs using Java (Spring Boot) with maintainable and secure code.
- Frontend: Create responsive modular front-end applications with (or similar) following sound UI/UX and component design.
- Cloud DevOps: Deploy and manage cloud infrastructure on Azure leveraging DevOps pipelines containers (Docker/Kubernetes) and infrastructure as code.
- Data Engineering Batch & Orchestration:
- Build and operate Azure Data Factory (ADF) pipelines (triggers activities mapping data flows) for ingestion from APIs databases and files.
- Develop scalable data transformations in Azure Databricks using PySpark and Delta Live Tables (DLT) for reliable declarative pipelines.
- API Data Consumption: Design and implement robust ingestion frameworks to consume data from REST APIs handle pagination authentication and error recovery for large-scale data loads.
- Data Lake & Storage:
- Implement Medallion Architecture (Bronze Silver Gold) on ADLS Gen2 with proper folder hierarchy ACLs governance and cost optimization.
- Integrate Azure Blob Storage and Azure SQL Database for curated and serving layers.
- RealTime & Streaming: Design and maintain streaming pipelines using Azure Kafka integration (e.g. Azure Event Hubs with Kafka protocol) and Spark Structured Streaming for low-latency data products.
- Performance Engineering :
- Optimize PySpark jobs for partitioning caching shuffle reduction and broadcast joins.
- Tune ADF pipelines for efficient data movement and concurrency.
- Apply SQL query optimization techniques (indexes joins window functions) for faster retrieval.
- Design and optimize database views for downstream analytics and BI consumption.
- Data Modeling & Performance: Design schemas (star/snowflake) optimize queries (SQL window functions CTEs) and ensure efficient retrieval for downstream analytics.
- BI & Analytics Enablement:
- Prepare analytics-ready datasets and semantic models for visualization tools like Power BI and Apache Superset; ensure proper data contracts for downstream consumers.
- Partner with web teams to implement Google Tag Manager (GTM) tagging and telemetry that power Google Analytics and downstream analytics.
- Collaboration & Leadership: Collaborate globally with product owners architects data scientists and developers; mentor peers on clean code data best practices and continuous learning. Own commitments and deliver high-quality software within deadlines.
Essential Skills and Tools
- Java (8): In-depth knowledge of core Java concurrency JVM internals and functional programming paradigms.
- Frameworks: Expertise in Spring Boot Spring Security Hibernate and reactive frameworks (WebFlux).
- Frontend: Strong skills in modern JavaScript/TypeScript and CSS preprocessors.
Data Engineering
Programming & Scripting: Java or Python (data ingestion transformation) SQL (advanced joins window functions CTEs) PySpark (large-scale transformations).
- Azure Data Services:
- Azure Data Factory (ADF): Pipelines triggers activities Mapping Data Flows CI/CD with Azure DevOps.
- Azure Databricks: Notebooks (PySpark) Delta Lake Delta Live Tables job clusters Unity Catalog (governance).
- Storage: ADLS Gen2 (hierarchy ACLs) Azure Blob Azure SQL Database.
- RealTime Processing: Azure Kafka integration (e.g. Event Hubs Kafka endpoint) Spark Structured Streaming.
- Data Warehousing & Modeling: Medallion Architecture (Bronze/Silver/Gold) dimensional modeling (star/snowflake) surrogate keys CDC patterns.
- Visualization Tools: Power BI Apache Superset and integration with downstream analytics platforms.
- Web Analytics: Google Tag Manager (GTM) for event tagging; collaboration on Google Analytics instrumentation and taxonomy.
Performance Engineering
- PySpark optimization (partitioning caching shuffle reduction).
- ADF pipeline tuning (parallelism concurrency).
- SQL query optimization (indexes joins views).
- Efficient view design for BI and reporting.
Platform & Engineering Excellence
- Cloud Platforms: Proficient with Microsoft Azure services including App Services AKS Azure DevOps.
- Build & Dependency: Maven Gradle with effective repository management.
- Version Control: Strong knowledge of Git (branching strategies pull requests merge workflows).
- CI/CD: Jenkins/GitHub Actions/Azure Pipelines.
- Containers & Orchestration: Docker Kubernetes (AKS) Helm.
- Database Technologies: PostgreSQL MongoDB Azure SQL Database; schema design indexing partitioning and query optimization.
- Collaboration & Agile: Experience in distributed teams using Scrum/Agile; excellent verbal and written communication.
Attributes
- Selfdisciplined and reliable delivering commitments on schedule with excellence.
- Fast learner with a passion for new technologies and data performance optimization.
- Strong problem solver who values code quality data reliability and maintainability.
- Team player willing to take on challenges and mentor others across application and data disciplines.
Maersk is committed to a diverse and inclusive workplace and we embrace different styles of thinking. Maersk is an equal opportunities employer and welcomes applicants without regard to race colour gender sex age religion creed national origin ancestry citizenship marital status sexual orientation physical or mental disability medical condition pregnancy or parental leave veteran status gender identity genetic information or any other characteristic protected by applicable law. We will consider qualified applicants with criminal histories in a manner consistent with all legal requirements.
We are happy to support your need for any adjustments during the application and hiring process. If you need special assistance or an accommodation to use our website apply for a position or to perform a job please contact us by emailing .
Required Experience:
IC
Software Engineer (Java Azure Data Engineering) Expert Polyglot and AIDriven DeveloperOverviewA highly experienced engineer with 5 years of deep programming expertise who thrives on solving complex problems independently and in teams. Primarily skilled in Java 8 complemented by strong proficiency ...
Software Engineer (Java Azure Data Engineering) Expert Polyglot and AIDriven Developer
Overview
A highly experienced engineer with 5 years of deep programming expertise who thrives on solving complex problems independently and in teams. Primarily skilled in Java 8 complemented by strong proficiency in Azure data engineering (ADF Databricks ADLS PySpark) and for front-end development. Comfortable navigating multiple programming paradigms and equipped to work on fullstack cloudnative and dataintensive development at pace and quality.
Key Responsibilities
- Backend & APIs (Java): Design and develop scalable microservices and APIs using Java (Spring Boot) with maintainable and secure code.
- Frontend: Create responsive modular front-end applications with (or similar) following sound UI/UX and component design.
- Cloud DevOps: Deploy and manage cloud infrastructure on Azure leveraging DevOps pipelines containers (Docker/Kubernetes) and infrastructure as code.
- Data Engineering Batch & Orchestration:
- Build and operate Azure Data Factory (ADF) pipelines (triggers activities mapping data flows) for ingestion from APIs databases and files.
- Develop scalable data transformations in Azure Databricks using PySpark and Delta Live Tables (DLT) for reliable declarative pipelines.
- API Data Consumption: Design and implement robust ingestion frameworks to consume data from REST APIs handle pagination authentication and error recovery for large-scale data loads.
- Data Lake & Storage:
- Implement Medallion Architecture (Bronze Silver Gold) on ADLS Gen2 with proper folder hierarchy ACLs governance and cost optimization.
- Integrate Azure Blob Storage and Azure SQL Database for curated and serving layers.
- RealTime & Streaming: Design and maintain streaming pipelines using Azure Kafka integration (e.g. Azure Event Hubs with Kafka protocol) and Spark Structured Streaming for low-latency data products.
- Performance Engineering :
- Optimize PySpark jobs for partitioning caching shuffle reduction and broadcast joins.
- Tune ADF pipelines for efficient data movement and concurrency.
- Apply SQL query optimization techniques (indexes joins window functions) for faster retrieval.
- Design and optimize database views for downstream analytics and BI consumption.
- Data Modeling & Performance: Design schemas (star/snowflake) optimize queries (SQL window functions CTEs) and ensure efficient retrieval for downstream analytics.
- BI & Analytics Enablement:
- Prepare analytics-ready datasets and semantic models for visualization tools like Power BI and Apache Superset; ensure proper data contracts for downstream consumers.
- Partner with web teams to implement Google Tag Manager (GTM) tagging and telemetry that power Google Analytics and downstream analytics.
- Collaboration & Leadership: Collaborate globally with product owners architects data scientists and developers; mentor peers on clean code data best practices and continuous learning. Own commitments and deliver high-quality software within deadlines.
Essential Skills and Tools
- Java (8): In-depth knowledge of core Java concurrency JVM internals and functional programming paradigms.
- Frameworks: Expertise in Spring Boot Spring Security Hibernate and reactive frameworks (WebFlux).
- Frontend: Strong skills in modern JavaScript/TypeScript and CSS preprocessors.
Data Engineering
Programming & Scripting: Java or Python (data ingestion transformation) SQL (advanced joins window functions CTEs) PySpark (large-scale transformations).
- Azure Data Services:
- Azure Data Factory (ADF): Pipelines triggers activities Mapping Data Flows CI/CD with Azure DevOps.
- Azure Databricks: Notebooks (PySpark) Delta Lake Delta Live Tables job clusters Unity Catalog (governance).
- Storage: ADLS Gen2 (hierarchy ACLs) Azure Blob Azure SQL Database.
- RealTime Processing: Azure Kafka integration (e.g. Event Hubs Kafka endpoint) Spark Structured Streaming.
- Data Warehousing & Modeling: Medallion Architecture (Bronze/Silver/Gold) dimensional modeling (star/snowflake) surrogate keys CDC patterns.
- Visualization Tools: Power BI Apache Superset and integration with downstream analytics platforms.
- Web Analytics: Google Tag Manager (GTM) for event tagging; collaboration on Google Analytics instrumentation and taxonomy.
Performance Engineering
- PySpark optimization (partitioning caching shuffle reduction).
- ADF pipeline tuning (parallelism concurrency).
- SQL query optimization (indexes joins views).
- Efficient view design for BI and reporting.
Platform & Engineering Excellence
- Cloud Platforms: Proficient with Microsoft Azure services including App Services AKS Azure DevOps.
- Build & Dependency: Maven Gradle with effective repository management.
- Version Control: Strong knowledge of Git (branching strategies pull requests merge workflows).
- CI/CD: Jenkins/GitHub Actions/Azure Pipelines.
- Containers & Orchestration: Docker Kubernetes (AKS) Helm.
- Database Technologies: PostgreSQL MongoDB Azure SQL Database; schema design indexing partitioning and query optimization.
- Collaboration & Agile: Experience in distributed teams using Scrum/Agile; excellent verbal and written communication.
Attributes
- Selfdisciplined and reliable delivering commitments on schedule with excellence.
- Fast learner with a passion for new technologies and data performance optimization.
- Strong problem solver who values code quality data reliability and maintainability.
- Team player willing to take on challenges and mentor others across application and data disciplines.
Maersk is committed to a diverse and inclusive workplace and we embrace different styles of thinking. Maersk is an equal opportunities employer and welcomes applicants without regard to race colour gender sex age religion creed national origin ancestry citizenship marital status sexual orientation physical or mental disability medical condition pregnancy or parental leave veteran status gender identity genetic information or any other characteristic protected by applicable law. We will consider qualified applicants with criminal histories in a manner consistent with all legal requirements.
We are happy to support your need for any adjustments during the application and hiring process. If you need special assistance or an accommodation to use our website apply for a position or to perform a job please contact us by emailing .
Required Experience:
IC
View more
View less