Junior Data Engineer

Hyderabad - India

Monthly Salary: Not Disclosed

Posted on: 20 hours ago

Vacancies: 1 Vacancy

Job Summary

Who we are

Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to Customers.

Job Description:

Job Title : Junior Data Engineer

Job Type : Full-Time

Experience: 1 - 2 years

Location : Hyderabad Telangana

Responsibilities:

Design and develop data ingestion pipelines to load CSV files business flat files and SAP Fusion data into Google BigQuery for analytics and reporting.
Implement real-time data ingestion pipelines using Google Cloud Dataflow and Pub/Sub and support streaming architectures using Delta Live Tables and Structured Streaming.
Develop and manage batch ETL pipelines using Cloud Dataflow Dataproc and Apache Spark ensuring scalable and reliable data processing.
Build low-code ETL workflows using Google Cloud Data Fusion and integrate data using DataFactory and connector-based ingestion frameworks.
Perform Spark-based data processing using Cloud Dataproc and Databricks Runtime (Photon) to handle large-scale data transformations.
Design and maintain data storage and ingestion layers using BigQuery and Delta Lake for high-performance analytics workloads.
Write and optimize complex SQL queries views and stored procedures to support transformation and reporting needs.
Perform data cleansing transformation and validation using SQL and Spark to ensure data quality and integrity.
Manage and maintain databases using DBeaver and SSMS ensuring smooth database operations.
Optimize BigQuery query performance partitioning and clustering for efficient data analytics.
Troubleshoot and resolve data ingestion issues ETL pipeline failures and database performance problems.
Ensure data accuracy integrity and consistency across ingestion and transformation pipelines.
Work closely with analytics reporting and business teams to deliver reliable datasets and support analytical use cases.
Support data modeling and schema design for efficient storage and query performance within Google BigQuery and Delta Lake environments.

Qualification:

Bachelors degree or equivalent combination of education and experience

Who we are Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to Customers. Job Description: Job Title : Junior Data Engineer Job Type : Full-Time Experience: 1 - 2 years Location : Hyderabad Telangana Responsi...