Data Engineer
Share
Job Location:
Pittsburgh, PA - USA
Monthly Salary:
Not Disclosed
Posted on:
12 days ago
Vacancies:
1 Vacancy
Department:
Job Summary
About the Company
Bloomfield is a Carnegie Mellon spinout creating a global platform shaping the future of precision agriculture. We scan farms and vineyards at scale (capturing high-resolution imagery and other sensor data) analyze every plant and derive actionable intelligence to help growers be more efficient and productive. We collaborate with various movement platform companies from ATV and tractor manufacturers to robotics startups.
About the Role
Our rapidly growing engineering team is searching for a Data Engineer to help build and maintain our data and image processing pipelines. You will be responsible for designing implementing and maintaining the various components of our pipelines which need to support processing more than 75k images per day. You will be involved in all parts of the data engineering lifecycle including data generation ingestion transformation storage and serving of our data.
To be successful in this role you should have a strong background in software engineering and be proficient in Python. Familiarity with AWS technologies such as Lambdas Step Functions Redshift S3 Athena and Glue is a big plus.
As a Data Engineer you will be a part of our engineering team working to ensure that our data ecosystem is efficient scalable and able to support our rapidly growing data needs. You will be responsible for ensuring that the data is cleaned processed and transformed appropriately to support our data analytics and machine learning efforts.
Responsibilities
- Designing implementing and maintaining our data pipeline including data generation ingestion transformation and serving
- Ensuring the reliability security and performance of the data pipelines
- Troubleshooting and debugging any issues with our data pipelines
- Working with other engineers to understand their data needs and ensure that our data ecosystem can support their requirements
- Optimizing the data pipeline for efficiency scalability and reliability
- Developing and maintaining documentation for the data pipeline
Qualifications
- Bachelors or Masters degree in Computer Science Engineering or a related field
- 5 years of experience as a Data Engineer or a similar role
- Strong proficiency in Python Pandas and SQL
- Experience with AWS technologies such as Lambdas Step Functions Redshift S3 Athena Glue and Kinesis
- Familiarity with data cleaning processing and analytics of image data
- Strong problem-solving and troubleshooting skills
- Excellent communication and collaboration abilities.
What We Offer
In addition to the opportunity to apply and develop your skills toward key business objectives we offer an excellent compensation package including:
- Competitive base salary
- Medical dental and vision insurance
- 401(k) retirement plan with company match
- Unlimited PTO
- Parental Leave
- Incentive stock options
Bloomfield is an equal opportunity employer. We consider qualified applicants without regard to race color religion sex national origin sexual orientation disability gender identity protected veteran status or other protected classes.
In compliance with federal law all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire.
Required Experience:
Manager
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Using cutting edge robotic technology to provide plant breeders, growers, and scientists with the high-resolution crop data they need to grow better plants, faster.
