Lead Data Warehouse Engineer Scientific Computing & Data
Share
Job Location:
New York City, NY - USA
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
The Scientific Computing & Data group at the Icahn School of Medicine at Mount Sinai (ISMMS) partners with scientists to accelerate scientific discovery. To achieve these aims we support a cutting-edge high-performance computing (HPC) and research data ecosystem with MD/PhD-level support for researchers.Our research data ecosystem includes a data commons repository and two research clinical data warehouses: one for ISMMSs research community and a second for the Kidney Precision Medicine Project (KPMP) a multi-institutional research consortium () funded by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK).
The Lead Data Warehouse Engineer is a senior technical specialist responsible for leading the development maintenance and ongoing operations of Scientific Computing & Datas research data warehouses.In this position the Lead Data Warehouse Engineer will collaborate with other members of the research data warehouse team and with research stakeholders of ISMMS and KPMP to expand functionality and integrate new data sources into the data warehouses. Both research data warehouses are built on the Microsoft SQL Server technology stack and use the OMOP Common Data Model published by the Observational Health Data Sciences and Informatics (OHDSI) collaborative () for multi-institutional data sharing and interoperability. Data transformations are performed in-database using Transact-SQL stored procedures with SSIS used only for job orchestration.
Responsibilities
- Design databases and pipelines that balance functionality performance cost and development time; evaluate technical options with the product manager.
- Design build test and maintain ETL/ELT processes using T-SQL stored procedures SSIS and SQL Agent; apply metadata-driven design for extensibility.
- Serve as a team leader; contribute to project planning work breakdown dependency sequencing and release management.
- Develop and promote standards conventions design patterns DevOps/SDLC best practices and operational procedures for pipelines and warehouse maintenance.
- Mentor junior engineers in data warehousing data engineering skills and operational support.
- Design build and maintain data management processes including loading flat files (csv tsv pipe-delimited JSON).
- Lead design sessions code walkthroughs peer reviews and produce technical documentation.
- Tune database objects stored procedures and pipelines to optimize performance and minimize compute and storage costs.
- Monitor database and pipeline operations; lead troubleshooting and remediation of failures; provide occasional after-hours on-call support.
- Collaborate with DBAs and system administrators on backups performance tuning statistics/index maintenance and patching.
- Provide high-quality customer service to researchers clinicians and internal partners; maintain a sciencedriven customer-focused approach.
- Ensure patient privacy and data security in compliance with IRB & cybersecurity policies HIPAA 42 CFR Part 2 NYS Article 27-F and other regulations.
- Stay current with emerging technologies to improve capabilities efficiency quality or cost.
- Identify improvements in procedures technology compliance and data privacy/security.
- Periodically assist DBAs with user provisioning backups restorations capacity planning and performance monitoring.
- Perform related duties as assigned.
Qualifications
- Bachelors degree in a technical discipline; Masters degree preferred
- 12-15 years preferred of related experience including 7 years of experience with the design development and maintenance of relational databases data pipelines and dimensional/OLAP data warehouses.
Preferred
- Expert knowledge of data warehousing: 3NF & dimensional modeling (fact table types SCDs) change data capture incremental loads data lineage source-to-target mappings pattern-based & parameter-driven development.
- Expert-level experience with Microsoft SQL Server technologies: T-SQL indexing stored procedures UDFs sequences dynamic SQL Linked Servers SSIS Visual Studio SSDT and SQL Agent.
- Experience with DevOps/SDLC best practices; Agile (Scrum Kanban) with JIRA and Confluence; version control with git.
- Strong communication and customer service skills for working with researchers clinicians administrators and IT staff.
- Excellent critical thinking problem-solving multitasking and collaboration skills; ability to work independently in a fast-paced environment.
- Preferred experience with healthcare data (EHR billing/claims cost accounting) Epic Clarity/Caboodle data models (OMOP i2b2 PCORnet).
- Preferred experience with Azure Synapse Azure Data Factory Oracle PL/SQL PostgreSQL PL/pgSQL.
- Experience with SQL Server administration: configuration performance tuning partitioning materialized views permissions backups & restorations.
- Preferred experience with scripting in Windows & Linux (PowerShell Python or similar); HL7; web services/REST APIs; reporting tools like SSRS Power BI Tableau.
Required Experience:
IC
Key Skills
About Company
Strength through Unity and Inclusion The Mount Sinai Health System is committed to fostering an environment where everyone can contribute to excellence. We share a common dedication to delivering outstanding patient care. When you join us, you become part of Mount Sinai’s unparalleled ... View more
