ETL Lead

Job Tittle: ETL Lead Insurance Domain

Job Location: Green Ville SC (Onsite)

Duration : 6 Months contract

Visa: USC and GC

Years of Exp: 12

Key Responsibilities

Production Support Operations & Reliability

The ETL Team Lead owns end-to-end operational support for Canals existing data stack:

Ensure success of day-to-day production Operations.

Monitor daily ETL loads across SQL jobs DHIC (GW Data Hub and Info Center) and legacy SSIS packages.

Work with AMS team where necessary to troubleshoot pipeline failures performance issues schema mismatches permissions issues and cloud resource failures.

Work with AMS team where necessary to perform root-cause analysis and to implement permanent fixes.

Ensure SLA adherence and on-time delivery of critical reporting datasets for scheduled ETL jobs .

Provide direction for both AMS and ETL Developers for Legacy & Current ETL maintenance.

Refactor or retire outdated or redundant ETL processes.

Maintain and improve existing pipelines that utilize the following technologies:

Microsoft SQL Server Database programming

T-SQL Scripting

SQL Server Integration Services

Microsoft PowerShell

Guidewire Data Hub and Info Center

Oracle Database programming

Oracle PL-SQL Scripting

SAP BODS (SAP BusinessObjects Data Services)

PostgreSQL Scripting

Operational Excellence

Work with AMS team to assist with the creation and/or enhancement of operational runbooks SOPs monitoring dashboards and incident response workflows.

Partner with other IT operational segments business SMEs and AMS team to minimize downtime and to ensure that business SLAs are met.

Improve upon existing as well as implement new proactive for daily processing.

Business Continuity

Work with AMS team to ensure development support coverage for critical data pipelines (rotation-based).

Support month-end and quarter-end financial reporting cycles.

Coordinate production releases and validate deployments.

The ETL Tech Lead will become the steady-state technical owner of the entire data operations layer during the Canal modernization journey.

Technical Leadership & Collaboration

Serve as technical lead guiding onshore/offshore developers.

Review code enforce best practices and mentor junior engineers.

Partner with Scrum Masters Project Mangers Enterprise Architecture QA Automation Change Management and AMS support teams.

Data Ingestion ETL/ELT Development & Optimization

Develop reusable ingestion patterns for Guidewire DataHub and InfoCenter HubSpot Telematics and other facets of the business.

Work with Canal Architects to modernize existing ETL workloads using Delta Lake Medallion Architecture and Fabric Lakehouse.

Build scalable data ingestion pipelines using potential upcoming technology (Azure Data Factory MS Fabric Databricks Synapse Pipelines etc.).

Work to bring in Internal and External integrations data into the platform.

Real-Time Streaming & Event-Driven Engineering

Proven experience designing and implementing real-time data pipelines using Event Hub Fabric Real-Time Analytics Databricks Structured Streaming and KQL-based event processing.

Develop and enable real-time operational insights and automation capabilitiesincluding Telematics alerting FNOL automation and fraud/VNOS/VNOP detectionthrough event-driven architectures and streaming analytics.

Modern Azure Data Stack Leadership

Lead the strategy design and engineering of Canals modern Azure data ecosystem using next-generation tools and Medallion Architecture including:

Implementing Medallion Architecture (Bronze/Silver/Gold) across Fabric Lakehouse Warehouse Eventhouse and KQL Database.

Leveraging Delta tables with schema enforcement ACID compliance and versioning.

Building ingestion and transformation pipelines using Dataflows Gen2 Fabric Pipelines and Notebook-based ETL.

Enabling real-time analytics using RTA EventStreams and Event Hub.

Driving notebook-based transformations and advanced engineering using Databricks and Synapse runtimes.

Data Modeling Curation & Governance

Develop curated analytics-ready datasets to support Power BI operational reporting and advanced analytics use cases.

Assist Canal architect with implementation of Data Governance tools.

Establish robust data quality validation alerting and observability frameworks to maintain accuracy consistency and trust in enterprise data.

AI/ML Data Enablement (Optional)

Prepare ML-ready datasets for pricing risk fraud detection underwriting claims leakage and predictive insights