Regular Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
7 hours ago
Vacancies:
1 Vacancy
Job Summary
The role involves sourcing data from multiple systems optimizing workflows and collaborating with architects and analysts to deliver clean well-structured datasets.
Required Skills & Experience:
3 years of experience in data engineering across hybrid environments (on-premise and cloud).
Proficiency in SQL and Python or Java/Scala.
Hands-on experience with ETL/ELT tools and frameworks.
Good understanding of GCP data services: BigQuery Dataproc Dataflow Cloud Storage.
Familiarity with data modeling schema design and metadata management.
Knowledge of data governance security and compliance best practices.
Nice To Have:
GCP certification (e.g. Professional Data Engineer) is a major plus.
Experience with Big Data technologies.
Key Responsibilities:
Solution Design: Architect data pipelines down to the low-level elements ensuring clarity and precision in implementation.
Data Sourcing: Extract data from diverse repositories including relational
databases (Oracle PostgreSQL) NoSQL stores file systems and other
structured/unstructured sources.
Data Transformation: Design and implement ETL/ELT workflows to standardize and cleanse data using best practices in data engineering.
Pipeline Development: Build scalable fault-tolerant data pipelines that support batch and streaming use cases.
Cloud data processing: Load transformed data into GCP destinations such as BigQuery or Cloud Storage using tools like Dataproc Dataflow and other GCPnative services.
Workflow Orchestration: Design and manage workflows using orchestration tools such as Apache Airflow or Cloud Composer.
Data Format Expertise: Work with various data formats including JSON AVRO Parquet CSV and others.
Optimization & Monitoring: Ensure performance reliability and cost-efficiency of data pipelines through continuous monitoring and tuning.
Collaboration: Work closely with data architects analysts and business
stakeholders to understand data requirements and deliver high-quality solutions.
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Since 1998 weve been active in the Human Resources consulting market providing regional coverage across four key areas of expertise: recruitment and selection personnel leasing assessment centers and consultancy. As leaders in Transylvania weve expanded our reach to embrace a culture ... View more
