Principal Data Engineer

Black Duck Software Inc. helps organizations build secure high-quality software minimizing risks while maximizing speed and productivity. Black Duck a recognized pioneer in application security provides SAST SCA and DAST solutions that enable teams to quickly find and fix vulnerabilities and defects in proprietary code open source components and application behavior. With a combination of industry-leading tools services and expertise only Black Duck helps organizations maximize security and quality in DevSecOps and throughout the software development life cycle.

What youll do

• Lead the design and build-out of cross-product data services for multiple product lines from one governed data plane.
• Define the customer data plane model: canonical customer identifiers shared dimensions and consistent facts used across products.
• Build and operationalize ingestion patterns for batch streaming and event data with repeatable onboarding for new sources.
• Own the operational playbook for data reliability: data contracts quality checks lineage monitoring and incident response.
• Implement and run access methods that make data usable: curated datasets secure query interfaces and product-ready data APIs where needed.
• Productize customer-facing data products (datasets metrics exports and feeds) with versioning documentation and clear ownership.
• Design data models that fit both operational systems (RDS) and analytics stores (columnar/OLAP) including performance and cost tuning.
• Ensure data products also power ML workflows: trusted training datasets feature-ready outputs and consistent definitions for decision-making.
• Enable AI automation by delivering reliable low-latency governed data products that can be used safely in automated workflows.
• Partner closely with product engineering and security stakeholders to align data products to roadmap priorities and customer outcomes.
• Raise the technical bar through architecture reviews standards and mentoringwhile staying hands-on in key systems.

Required

• Significant experience building and operating production data platforms at scale including on-call and operational ownership.
• Strong SQL skills and strong Python skills used to build pipelines services and automation.
• Hands-on experience running cloud systems on AWS and Google Cloud (IaaS level: compute storage networking IAM).
• Practical experience with both operational databases (RDS-style) and analytics stores (columnar/OLAP) including performance tuning.
• Strong data modeling ability including schema evolution conformed dimensions and one source of truth metric definitions.
• Track record of delivering data products that other teams or customers depend on with clear contracts and reliability expectations.
• Ability to make sound engineering tradeoffs across latency accuracy cost and security without creating brittle complexity.
• Experience with lakehouse patterns and open table formats (or similar) including governance and table maintenance.
• Experience with orchestration and streaming systems used in production (batch real-time) and managing backfills safely.
• Familiarity with ML data needs (training/serving splits feature-ready datasets evaluation datasets) and AI-adjacent workflows.

Preferred

• Experience building self-service data platforms (catalog discoverability access controls) used by multiple teams.
• Experience in regulated or security-sensitive environments including retention auditing and data access controls.

Work model location & travel

• Location: Belfast UK
• Reports to: VP of Data Engineering
• Work model: Hybrid (details TBD)
• Collaboration hours: Flexible; overlap with UK and US time zones
• Travel: Minimal

Black Duck considers all applicants for employment without regard to race color religion sex gender preference national origin age disability or status as a Covered Veteran in accordance with federal addition Black Duck complies with applicable state and local laws prohibiting discrimination in employment in every jurisdiction in which it maintains facilities. Black Duck also provides reasonable accommodation to individuals with a disability in accordance with applicable laws.