NDA Databricks-native platform | Integration Engineer

On behalf of NDA Databricks-native platform SD Solutions is looking for a talented Integration Engineer to build and maintain third-party data integrations for our Master Data Management platform built natively on the Databricks Data Intelligence this role you will own the end-to-end development of integrations with enterprise data providers such as Dun & Bradstreet ZoomInfo and Smarty delivering reliable connectors that enrich customer data at scale.

This is a hybrid role spanning full-stack development and data engineering. You will build Spark Structured Streaming pipelines to ingest and process third-party data develop the backend services and APIs that orchestrate these integrations and create the frontend configuration experiences that let customers set them up. Most integrations follow a standard architectural pattern your job is to execute that pattern consistently reliably and at a steady cadence while adapting to the quirks of each provider.

Success in this role depends heavily on communication and partnership. You will work directly with third-party vendors to understand their APIs authentication models data schemas rate limits and delivery mechanisms then translate those requirements into working integrations. Youll also collaborate closely with Product Platform and Delivery teams to ensure each integration meets customer needs.

This role suits someone who enjoys shipping well-scoped repeatable work end-to-end takes ownership of external relationships and values consistency and reliability over novelty.

SD Solutions is a staffing company operating globally. Contact us to get more details about the benefits we offer.

Responsibilities:

• Own integrations end-to-end: Design build test and maintain third-party data provider integrations from API contract through streaming pipeline to UI configuration following LakeFusions standard integration pattern.
• Build streaming data pipelines: Develop PySpark Structured Streaming pipelines on Databricks to ingest transform and land enriched data from third-party providers.
• Develop backend services: Build Python/FastAPI services that orchestrate API calls to third-party providers handle authentication manage rate limiting and retries with exponential backoff and expose integration configuration endpoints.
• Build frontend configuration UIs: Develop interfaces that let customers configure monitor and manage their third-party integrations within the LakeFusion product.
• Partner with third-party vendors: Act as the primary technical point of contact with data providers. Read API documentation carefully ask the right questions run test transactions and validate that integrations meet both vendor and LakeFusion requirements.
• Translate requirements into specs: Take vendor API documentation and customer requirements and produce clear implementable integration specs that follow LakeFusions standard patterns.
• Ensure reliability: Build integrations that handle provider-side failures gracefully rate limits (429s) auth expiration schema drift partial outages with observable logging alerting and idempotent retry behavior.
• Maintain and extend existing integrations: Monitor production integrations respond to provider API changes and iterate on the shared integration framework as patterns evolve.

Requirements:

• 5 years of hands-on experience across both application development and data engineering with demonstrated ability to ship features end-to-end.
• Strong Python skills with experience building production services using FastAPI or similar frameworks.
• Practical PySpark experience including Structured Streaming Delta Lake and common patterns like CDC/CDF and checkpointing. Databricks experience strongly preferred.
• and modern JavaScript/TypeScript experience with working knowledge of a frontend framework (React preferred).
• Proven track record integrating with third-party REST APIs including handling authentication (OAuth API keys token refresh) pagination rate limiting webhooks and bulk/batch endpoints.
• Excellent written and verbal communication skills in English. This role requires directly emailing meeting with and problem-solving alongside third-party vendor technical teams the ability to ask precise questions document answers clearly and follow through is essential.
• Strong reading comprehension for technical documentation. You can work through a 200-page API spec identify the relevant sections and pull out what matters for the integration.
• Solid SQL skills and comfort with relational data modeling.
• Ownership mindset: you treat each integration as yours from vendor kickoff through production monitoring and you dont drop things between handoffs.
  

Advantages:

• Experience integrating with enterprise data enrichment providers (Dun & Bradstreet ZoomInfo Smarty Experian LexisNexis or similar).
• Experience with Master Data Management entity resolution or data quality platforms.
• Familiarity with Databricks Unity Catalog Workflows and Jobs.
• Experience with Azureor AWS.
• Exposure to CI/CD pipelines Docker and infrastructure-as-code (Terraform).
• Experience building integrations as a reusable framework rather than one-off implementations.
  

About the company:

NDA is a Databricks-native platform that unifies master data product data and relationships into a single AI-ready foundation.

One platform. Three sources of trust:
MDM (Trusted Entities): Customers suppliers and accounts mastered and governed across systems
Graph (Trusted Networks): Relationships and hierarchies connected for intelligence and AI reasoning
PIM (Trusted Products): SKUs and catalogs enriched and ready for commerce


Built entirely on the Databricks NDA eliminates data duplication and brings governance directly to where your data lives.

Powered by LLMs we deliver explainable entity resolution automated stewardship and trusted golden records at scale.

By applying for this position you agree to the terms outlined in our Privacy Policy. Please take a moment to review our Privacy Policy and make sure you understand its contents. If you have any questions or concerns regarding our Privacy Policy please feel free to contact us.