Data Engineer

WorkNovas LLC

Job Location:

Bentonville, AR - USA

Monthly Salary: Not Disclosed

Posted on: 7 days ago

Vacancies: 1 Vacancy

Job Summary

Job Title: Data Engineer

Location: Sunnyvale CA & Bentonville AR - (On-Site)

Job Description:-

Worknovas is looking for a Data Engineer with Spark & Streaming skills builds real-time scalable data pipelines using tools like Spark Kafka and cloud services (GCP) to ingest transform and deliver data for analytics and ML.

Responsibilities:

As a Senior Data Engineer you will.

Design develop and maintain ETL/ELT data pipelines for batch and real-time data ingestion transformation and loading using Spark (PySpark/Scala) and streaming technologies (Kafka Flink).

Build and optimize scalable data architectures including data lakes data warehouses (BigQuery) and streaming platforms.

Performance Tuning: Optimize Spark jobs SQL queries and data processing workflows for speed efficiency and cost-effectiveness

Data Quality: Implement data quality checks monitoring and alerting systems to ensure data accuracy and consistency.

Programming:

Strong proficiency in Python SQL and potentially Scala/Java.

Big Data: Expertise in Apache Spark (Spark SQL DataFrames Streaming).

Streaming: Experience with messaging queues like Apache Kafka or Pub/Sub.

Cloud: Familiarity with GCP Azure data services.

Databases: Knowledge of data warehousing (Snowflake Redshift) and NoSQL databases.

Tools:

Experience with Airflow Databricks Docker Kubernetes is a plus..

Job Title: Data Engineer Location: Sunnyvale CA & Bentonville AR - (On-Site) Job Description:- Worknovas is looking for a Data Engineer with Spark & Streaming skills builds real-time scalable data pipelines using tools like Spark Kafka and cloud services (GCP) to ingest transform and deliv...