Senior Architect

Position Overview

We are seeking a highly motivated Data Platform Engineer to join our Platform Engineering team responsible for the roadmap and evolution of enterprise-scale hybrid data platforms.

The Platform Engineering team manages large-scale data and analytics ecosystems that enable data engineering data warehousing AI/ML governance observability and operational intelligence workloads across on-premises and cloud environments.

The role offers the opportunity to work with modern data platform technologies including Cloudera CDP base platform Kubernetes cloud-native infrastructure AI/ML platforms Elastic stack and advanced automation frameworks.

The ideal candidate combines strong Linux and infrastructure engineering expertise with a passion for learning automation scalability reliability and continuous improvement.

Key Responsibilities

Platform Engineering

• Design deploy configure and manage large-scale enterprise data platforms.
• Support Engineering lifecycle management activities including provisioning IaC upgrades patching capacity expansion delivering on engineering roadmap
• Ensure platform reliability availability resiliency and operational excellence across our environments.
• Infrastructure Automation & DevOps
• Develop and maintain Infrastructure-as-Code (IaC) solutions using Ansible Terraform Gitlab and CI/CD pipelines.
• Create reusable automation frameworks and engineering tools to improve operational efficiency and reduce manual effort.
• Build and operate hybrid data platform services spanning on-premises infrastructure and cloud environments.
• Support modern data platform technologies including:
  • Cloudera CDP Private Cloud
  • Kubernetes
  • Apache Spark
  • Hive
  • Impala
  • Iceberg
  • Ozone
  • NiFi
  • Kafka
  • AI/ML enablement
  • Hybrid Cloud setup (Azure/AWS on-premises)
• Assist with platform modernization and migration initiatives.

Security & Compliance

• Implement and maintain enterprise-grade security controls.
• Manage authentication authorization encryption and certificate-based security models.
• Support Kerberos LDAP SSO Ranger TLS RBAC and regulatory compliance requirements.
• Conduct security reviews and platform hardening activities.

Performance Engineering

• Perform performance analysis and tuning of platform services.
• Optimize compute storage network and application layer configurations.
• Conduct capacity planning forecasting and scalability assessments.
• Assist application teams with performance troubleshooting and platform best practices.

Platform Integration & Customer Enablement

• Integrate data platforms with enterprise services such as Active Directory monitoring tools observability platforms cloud storage and Kubernetes ecosystems.
• Collaborate with architecture application security and infrastructure teams.
• Provide technical guidance to internal customers leveraging the platform.

Required Qualifications

• Bachelors degree in Computer Science Information Technology or related field.
• 10 years of experience in platform engineering Systems Engineering role
• Strong Linux administration experience.
• Experience supporting distributed systems and enterprise infrastructure on multi-petabytes scale managing clusters spanning 1000s of servers.
• Experience with scripting and automation using Python Bash or similar languages.
• Experience with Gitlab and CI/CD methodologies.
• Strong troubleshooting and problem-solving skills.
• Excellent communication and collaboration abilities.

Desired Technical Skills

Technologies

• Cloudera CDP
• Elasticsearch
• Microsoft Azure
• Spark
• Hive
• Impala/Trino
• Kafka
• NiFi
• Iceberg
• Ozone
• Python & Shell Scripting

Container & Cloud Technologies

• Kubernetes
• Docker
• Microsoft Azure/AWS

Automation & Infrastructure

• Ansible
• Terraform
• Shell Scripting & Python
• GitLab CI/CD

Security

• Kerberos
• LDAP
• SSO
• Ranger
• TLS/SSL - Certificate Management

Databases

• PostgreSQL
• NoSQL Platforms

Observability

• Elasticsearch
• Kibana
• Grafana
• Prometheus
• Operational Monitoring Platforms

Preferred Qualifications

• Experience operating enterprise-scale production platforms.
• Experience with AI/ML platforms and MLOps ecosystems.
• Experience with hybrid cloud architectures.
• Understanding of Data Lakehouse Data Mesh and modern data platform concepts.
• Experience supporting highly regulated environments (HIPAA GDPR GxP etc.).
• Experience working with multi petabyte-scale data ecosystems.

What Youll Work On

• Large-scale hybrid data platforms supporting critical business workloads.
• Hybrid Cloud platform modernization and migration initiatives.
• AI/ML enablement platforms.
• Infrastructure automation and operational excellence programs.
• Platform observability governance and security enhancements.
• Next-generation technologies including Iceberg Kubernetes/Docker object storage and AI-assisted operations.