Senior Data Engineer TimescaleDB

Guadalajara - Mexico

Monthly Salary: Not Disclosed

Posted on: 18 hours ago

Vacancies: 1 Vacancy

Department:

Engineering

Job Summary

Were looking for a highly skilled Senior Data Engineer to join our team. This critical role involves managing massive volumes of high-velocity high-cardinality data generated by real-time processing systems primarily in the Computer Vision domain. The ideal candidate possesses deep expertise in TimescaleDB for efficient storage and querying proficiency with cloud-based object storage and familiarity with the specialized NVIDIA ecosystem including DeepStream GPUs and Trident to ensure data integrity and flow from the edge to the analytics platform.

Responsibilities:

Data Architecture & Storage

Design build and optimize high-throughput data pipelines using modern tools to ingest streaming data from various sources into our core data platform.
Serve as the subject matter expert for TimescaleDB managing schema design performance tuning compression policies and data retention strategies for petabytes of time-series data.
Architect and manage the tiered storage strategy leveraging cloud-based storage solutions (e.g. S3 GCS) for cold storage and archival of raw and processed data.
Ensure data models are scalable and optimized for both real-time operational queries and large-scale analytical processing.

Real-Time & Computer Vision Integration

Collaborate with ML and Computer Vision teams to integrate the data pipeline with NVIDIA DeepStreamapplications managing metadata and telemetry extracted from video streams.
Develop solutions that utilize NVIDIA GPUs effectively particularly concerning how derived data is ingested and processed immediately after the visual inferencing stage.
Familiarity with NVIDIA Trident storage orchestration is desirable for managing persistent volumes in Kubernetes environments hosting vision applications.
Implement data quality checks and validation processes to ensure the high integrity of timestamps and measurement data from the edge devices.

Engineering Excellence & Collaboration

Apply expert-level proficiency in a major programming language (Python or Scala preferred) for ETL/ELT pipeline development and tooling.
Drive the adoption of best practices including Infrastructure as Code (IaC) and comprehensive monitoring (e.g. Prometheus/Grafana) for the data platform components.
Provide technical guidance and mentorship to junior team members fostering a culture of high performance and technical rigor.

Qualifications :

Minimum of 5 years of professional experience in Data Engineering focusing on high-volume data platforms or distributed systems.
Expert proficiency with TimescaleDB (PostgreSQL) including experience managing production instances hypertable partitioning and continuous aggregates.
Demonstrated experience designing and managing large-scale data lakes or warehouses utilizing cloud-based object storage (AWS S3 Azure Blob Storage or GCP Cloud Storage).
Deep experience with streaming platforms (e.g. Apache Kafka Flink) and real-time data ingestion patterns.
Proficiency in modern programming languages (e.g. Python Scala or Go) for data processing and pipeline orchestration.

Preferred Skills & Domain Knowledge

Familiarity with the NVIDIA Computer Vision stack including concepts related to DeepStream NVIDIA GPUs or edge-to-cloud data flow.
Experience or strong understanding of the requirements for storing and retrieving high-dimensional time-series data (e.g. sensor data telemetry and machine learning metadata).
Experience with containerization and orchestration (Docker Kubernetes) in the context of data processing jobs.
Familiarity with database security protocols and compliance requirements for sensitive data.

Additional Information :

Perks you enjoy at KMS Mexico

Mexican law benefits
15 days of PTO (in year zero from the first year onwards it is 3 days per year).
5 days leave for the death of immediate family members negotiable.
Major Medical Expenses Insurance with coverage for immediate dependents (spouse and children).
Annual performance bonus (10% of annualized salary).
Annual salary adjustment.
Employee Referral Bonus.
Paid Certifications / Courses
Coursera License.
5% Savings Fund.
5% Grocery Vouchers.

Remote Work :

Employment Type :

Full-time

Were looking for a highly skilled Senior Data Engineer to join our team. This critical role involves managing massive volumes of high-velocity high-cardinality data generated by real-time processing systems primarily in the Computer Vision domain. The ideal candidate possesses deep expertise in Time...

Responsibilities:

Data Architecture & Storage

Design build and optimize high-throughput data pipelines using modern tools to ingest streaming data from various sources into our core data platform.
Serve as the subject matter expert for TimescaleDB managing schema design performance tuning compression policies and data retention strategies for petabytes of time-series data.
Architect and manage the tiered storage strategy leveraging cloud-based storage solutions (e.g. S3 GCS) for cold storage and archival of raw and processed data.
Ensure data models are scalable and optimized for both real-time operational queries and large-scale analytical processing.

Real-Time & Computer Vision Integration

Collaborate with ML and Computer Vision teams to integrate the data pipeline with NVIDIA DeepStreamapplications managing metadata and telemetry extracted from video streams.
Develop solutions that utilize NVIDIA GPUs effectively particularly concerning how derived data is ingested and processed immediately after the visual inferencing stage.
Familiarity with NVIDIA Trident storage orchestration is desirable for managing persistent volumes in Kubernetes environments hosting vision applications.
Implement data quality checks and validation processes to ensure the high integrity of timestamps and measurement data from the edge devices.

Engineering Excellence & Collaboration

Apply expert-level proficiency in a major programming language (Python or Scala preferred) for ETL/ELT pipeline development and tooling.
Drive the adoption of best practices including Infrastructure as Code (IaC) and comprehensive monitoring (e.g. Prometheus/Grafana) for the data platform components.
Provide technical guidance and mentorship to junior team members fostering a culture of high performance and technical rigor.

Qualifications :

Minimum of 5 years of professional experience in Data Engineering focusing on high-volume data platforms or distributed systems.
Expert proficiency with TimescaleDB (PostgreSQL) including experience managing production instances hypertable partitioning and continuous aggregates.
Demonstrated experience designing and managing large-scale data lakes or warehouses utilizing cloud-based object storage (AWS S3 Azure Blob Storage or GCP Cloud Storage).
Deep experience with streaming platforms (e.g. Apache Kafka Flink) and real-time data ingestion patterns.
Proficiency in modern programming languages (e.g. Python Scala or Go) for data processing and pipeline orchestration.

Preferred Skills & Domain Knowledge

Familiarity with the NVIDIA Computer Vision stack including concepts related to DeepStream NVIDIA GPUs or edge-to-cloud data flow.
Experience or strong understanding of the requirements for storing and retrieving high-dimensional time-series data (e.g. sensor data telemetry and machine learning metadata).
Experience with containerization and orchestration (Docker Kubernetes) in the context of data processing jobs.
Familiarity with database security protocols and compliance requirements for sensitive data.

Additional Information :

Perks you enjoy at KMS Mexico

Mexican law benefits
15 days of PTO (in year zero from the first year onwards it is 3 days per year).
5 days leave for the death of immediate family members negotiable.
Major Medical Expenses Insurance with coverage for immediate dependents (spouse and children).
Annual performance bonus (10% of annualized salary).
Annual salary adjustment.
Employee Referral Bonus.
Paid Certifications / Courses
Coursera License.
5% Savings Fund.
5% Grocery Vouchers.

Remote Work :

Employment Type :

Full-time

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

KMS Technology

KMS Technology was established in 2009 as a U.S.-based software services company. With development centers in Vietnam and Mexico, we have been trusted globally for the superlative quality of our software consulting & development services, technology solutions, and engineers' expertise ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click