Nearshore Sector | Databricks Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
5 days ago
Vacancies:
1 Vacancy
Job Summary
We are seeking a talented and passionate Databricks Data Engineer to join our expanding Data & AI team Databricks Partner in our Nearshore Team at Devoteam Portugal. In this role you will be a crucial player in designing building and maintaining cutting-edge data solutions on the Databricks platform. You will have the opportunity to work with a variety of clients across different industries collaborating with both internal teams and client stakeholders to deliver high-quality data pipelines and enable data-driven decision-making:
- Data Pipeline Engineering: Design develop and deploy robust and scalable data pipelines using Databricks incorporating data extraction from diverse sources (databases APIs streaming platforms) transformation and cleansing using Spark and loading into target systems (data lakes data warehouses etc.).
- Databricks Ecosystem Expertise: Utilize the full capabilities of the Databricks platform including Databricks SQL Delta Lake Databricks Runtime and Databricks Workflows to orchestrate complex data workflows and ensure data quality and pipeline reliability.
- Cloud Integration: Seamlessly integrate Databricks with cloud services (Azure AWS or GCP) for storage compute and security leveraging services like Azure Data Lake Storage AWS S3 Google Cloud Storage and more.
- Data Architecture and Modeling: Contribute to the design and implementation of robust data models and architectures for data lakes and data warehouses ensuring data consistency integrity and accessibility for various use cases.
- Data Quality Assurance: Implement rigorous data quality checks and validation procedures throughout the data pipeline to maintain high accuracy and reliability. Adhere to data governance policies and best practices.
Qualifications :
- Hold a University Degree in Computer Science (Business) Engineering or a specialized masters in Machine Learning or AI.
- At least 3 years of experience as a Data Engineer with demonstrated expertise in Databricks.
- Hands-on experience with at least one major cloud platform (Azure AWS or GCP) with a good understanding of cloud data services and infrastructure.
- Strong programming skills in Python or Scala with experience in data manipulation libraries (e.g. PySpark Spark SQL).
- Solid understanding of data warehousing principles ETL processes data modeling techniques and database systems.
- Advanced SQL skills for data querying transformation and analysis.
- Excellent communication and interpersonal skills with the ability to effectively collaborate with technical and non-technical stakeholders.
- Strong analytical and problem-solving abilities with a proactive approach to identifying and resolving data-related challenges.
- Fluent in English.
Additional Information :
The Devoteam Group works for equal opportunities promoting its employees based on merit and actively fights against all forms of discrimination. We are convinced that diversity contributes to the creativity dynamism and excellence of our organization. All of our vacancies are open to people with disabilities.
Remote Work :
No
Employment Type :
Full-time
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Devoteam is a AI-driven tech consulting firm specialised in cloud platforms, cyber, data, and sustainability. Tech native for almost 30 years, Devoteam guides businesses through sustainable digital transformation to deliver value. With over 11,000 tech architects in more than 25 co ... View more
