Data Engineer R
Share
Job Location:
Saint Louis, MI - USA
Monthly Salary:
$ 70 - 80
Posted on:
13 hours ago
Vacancies:
1 Vacancy
Job Summary
Senior Software Development Engineer
Primary Skills
- Spark - Pyspark SQL SQL (Basic Advanced) Python Hive Data Modelling Fundamentals
Specialization
- Data Analysis: Data Engineer
Job requirements
- Experience Range: 2 - 4 years of experience in software development with hands-on expertise in data analysis and engineering
- 1. Design and develop scalable data processing solutions using Spark and PySpark to support advanced analytics initiatives
- 2. Implement robust data models and optimize SQL queries for efficient data retrieval and transformation
- 3. Collaborate with data analysts and business stakeholders to translate requirements into technical solutions
- 4. Build and maintain ETL pipelines leveraging Python Hive and SQL for large-scale data integration
- 5. Conduct thorough code reviews performance tuning and debugging to ensure high-quality deliverables
- 6. Monitor troubleshoot and resolve issues in production data workflows ensuring data accuracy and reliability
- 7. Document technical processes data models and workflow architectures to facilitate knowledge sharing
- 8. Stay updated with industry trends in big data technologies and proactively recommend improvements
- 1. Advanced proficiency in Spark and PySpark
- 2. Strong knowledge of SQL (basic and advanced)
- 3. Expertise in Python programming for data processing
- 4. Experience with Hive for data warehousing solutions
- 5. Solid understanding of data modelling fundamentals
- 6. Ability to design and optimize ETL pipelines
- 7. Hands-on experience with large-scale data processing
- 8. Proficient in performance tuning of SQL and Spark jobs
- 9. Familiarity with distributed computing concepts
- 10. Competence in debugging and troubleshooting data workflows
- 1. Experience with cloud-based data platforms such as AWS or Azure
- 2. Knowledge of Airflow or similar workflow orchestration tools
- 3. Familiarity with Scala for Spark development
- 4. Understanding of data governance and security best practices
- 5. Exposure to machine learning frameworks in Python
- 6. Experience with continuous integration and deployment for data applications
- 1. Bachelors degree in Computer Science Information Technology Engineering or a related field 2. Relevant certifications in big data technologies or data engineering (such as Spark Python or SQL) are advantageous
Key Responsibilities:
Required Skills:
Preferred Skills:
Desired Qualifications:
$70 - $80 an hour
We may use artificial intelligence (AI) tools to support parts of the hiring process such as reviewing applications analyzing resumes or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed please contact us.
Required Experience:
IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Brillio is a global leader in Enterprise Digital Transformation Solutions, providing strategic consulting services and solutions using emerging technologies.
