Senior Data Engineer

Black Duck Software Inc. helps organizations build secure high-quality software minimizing risks while maximizing speed and productivity. Black Duck a recognized pioneer in application security provides SAST SCA and DAST solutions that enable teams to quickly find and fix vulnerabilities and defects in proprietary code open source components and application behavior. With a combination of industry-leading tools services and expertise only Black Duck helps organizations maximize security and quality in DevSecOps and throughout the software development life cycle.

Whatyoulldo

• Design and build data pipelines and datasets supporting one or more product areas within an established governed data platform.
• Implement ingestion patterns for batch streaming and event data onboarding new data sources using existing frameworks and standards.
• Build and maintain reliable data pipelines with appropriate data quality checks basic lineage monitoring and alerting.
• Develop curated datasets and views that are secure discoverable and ready for analytics reporting and downstream consumption.
• Contribute to customerfacing data products such as datasets metrics exports or feeds ensuring accuracy and clear documentation.
• Implement data models for both operational databases and analytics stores following agreed modelling conventions and best practices.
• Support ML and analytics use cases by producing trusted training datasets and featureready outputs with consistent definitions.
• Optimize pipeline performance cost and reliability within defined architectural patterns.
• Collaborate with product managers engineers analysts and security partners to deliver data solutions aligned to roadmap needs.
• Participate in oncall rotations and operational support for data services owned by the team.
• Contribute to technical discussions code reviews and continuous improvement of data engineering practices.

• Solid experience building and operating production data pipelines and datasets in a cloud environment.
• Strong SQL skills and working Python experience for building data pipelines transformations and automation.
• Handson experience with cloud platforms (AWS or Google Cloud) including core services such as compute storage IAM and networking.
• Experience working with both operational databases (e.g. RDSstyle systems) and analytics or columnar data stores.

Black Duck considers all applicants for employment without regard to race color religion sex gender preference national origin age disability or status as a Covered Veteran in accordance with federal addition Black Duck complies with applicable state and local laws prohibiting discrimination in employment in every jurisdiction in which it maintains facilities. Black Duck also provides reasonable accommodation to individuals with a disability in accordance with applicable laws.