Data Access Engineer
Share
Job Location:
Chapel Hill, NC - USA
Monthly Salary:
$ 51000 - 100000
Posted on:
4 days ago
Vacancies:
1 Vacancy
Job Summary
Position Summary
The Argus Array will be the largest optical telescope array ever assembled with a collecting area comparable to the largest monolithic telescopes in the world. The Array will push our observations of the universe into a new regime scanning the sky 100000x faster than current nightly-cadence sky surveys. Argus will capture a continuous multi-color 55-gigapixel movie of the night sky shared with the entire astronomical community in real-time through public transient alerts images and light curves with millions of epochs for hundreds of millions of stars. Joining our local team of astronomers telescope-instrumentalists and engineers the Data Access Engineer will build software systems to get Argus data products into the hands of astronomers around the world. Three years of prototyping efforts have demonstrated the core pipeline architecture and built an archive of representative data; we are now working to build pipelines and platforms for broad accessibility that scale to the full array. This scale up involves integration with cloud services existing distributed storage networks and the Arrays high-performance GPU accelerated collaboration with a worldwide network of real-time data release and processing centers the Data Access Engineer will take the alert distribution system to production bringing streaming notifications and images of new and changing phenomena in the night sky to both professional and amateur astronomers alike. As Data Access Engineer you will also oversee periodic data releases of our trillion-point light curves establishing best practices for data versioning integrity and accessibility. Other responsibilities will include contributing to collaborative development of intuitive API -first user interfaces for both internal quality assurance and public data access. Our project management philosophy emphasizes a small and local team; flat management structure; and a highly collaborative working environment. We routinely produce and test prototypes and complex hardware in-house. While you will lead the data access systems team members frequently collaborate across boundaries and contribute hands-on to various Argus hardware and software subsystems.
Required Qualifications Competencies And Experience
We are searching for an engineer with deep Python expertise (5 years) and demonstrated success in designing and optimizing high-throughput distributed message systems (3 years). Experience with large software projects including proficiency across the software development lifecycle (version control documentation and testing) is required.
Preferred Qualifications Competencies And Experience
Experience with both cloud-based relational databases (PostgreSQL and Apache Kafka or equivalents) and time-series databases optimized for astronomical data storage and retrieval is preferred. Prior experience translating scientific requirements into technical specifications and building researcher-friendly interfaces for complex datasets.
The Argus Array will be the largest optical telescope array ever assembled with a collecting area comparable to the largest monolithic telescopes in the world. The Array will push our observations of the universe into a new regime scanning the sky 100000x faster than current nightly-cadence sky surveys. Argus will capture a continuous multi-color 55-gigapixel movie of the night sky shared with the entire astronomical community in real-time through public transient alerts images and light curves with millions of epochs for hundreds of millions of stars. Joining our local team of astronomers telescope-instrumentalists and engineers the Data Access Engineer will build software systems to get Argus data products into the hands of astronomers around the world. Three years of prototyping efforts have demonstrated the core pipeline architecture and built an archive of representative data; we are now working to build pipelines and platforms for broad accessibility that scale to the full array. This scale up involves integration with cloud services existing distributed storage networks and the Arrays high-performance GPU accelerated collaboration with a worldwide network of real-time data release and processing centers the Data Access Engineer will take the alert distribution system to production bringing streaming notifications and images of new and changing phenomena in the night sky to both professional and amateur astronomers alike. As Data Access Engineer you will also oversee periodic data releases of our trillion-point light curves establishing best practices for data versioning integrity and accessibility. Other responsibilities will include contributing to collaborative development of intuitive API -first user interfaces for both internal quality assurance and public data access. Our project management philosophy emphasizes a small and local team; flat management structure; and a highly collaborative working environment. We routinely produce and test prototypes and complex hardware in-house. While you will lead the data access systems team members frequently collaborate across boundaries and contribute hands-on to various Argus hardware and software subsystems.
Required Qualifications Competencies And Experience
We are searching for an engineer with deep Python expertise (5 years) and demonstrated success in designing and optimizing high-throughput distributed message systems (3 years). Experience with large software projects including proficiency across the software development lifecycle (version control documentation and testing) is required.
Preferred Qualifications Competencies And Experience
Experience with both cloud-based relational databases (PostgreSQL and Apache Kafka or equivalents) and time-series databases optimized for astronomical data storage and retrieval is preferred. Prior experience translating scientific requirements into technical specifications and building researcher-friendly interfaces for complex datasets.
Required Experience:
Staff IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
The University of North Carolina at Chapel Hill, the nation’s first public university, is a global higher education leader known for innovative teaching, research and public service. A member of the prestigious Association of American Universities, Carolina regularly ranks as the best ... View more
