Data Engineer (DATABRICKS PYSPARK)
Job Summary
- Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
- Work with Databricks PySpark and SQL to transform validate and optimize data
- Support implementation of Data Lakehouse solutions following engineering standards and best practices
- Build test and automate ETL processes and data workflows
- Improve data quality organization and reliability across platforms
- Collaborate with engineers analysts and stakeholders to deliver scalable solutions
- Troubleshoot optimize and monitor data pipelines
- Stay up to date with modern data engineering technologies
- Ensure deliverables meet quality standards and project requirements
Qualifications :
- 2 years of experience in Big Data development and database design
- Strong hands-on experience with SQL
- Strong Python development skills
- Hands-on experience with PySpark
- Experience developing and maintaining data pipelines in Databricks
- Experience with Delta Lake
- Experience with Medallion Architecture (Bronze Silver Gold layers)
- Strong analytical and problem-solving skills
- Intermediate or higher English level
WILL BE A PLUS
- Experience with AWS services including S3
Remote Work :
Yes
Employment Type :
Full-time
About Company
At Sigma Software, we are involved with the clients team to contribute to the design and development of a technical solution for their tokenized domain reservation platform. We started by assigning a software architect to design the smart contracts and integrate blockchain into the s ... View more