Senior Data Engineer

Frisco, TX - USA

Monthly Salary: Not Disclosed

Posted on: 14 hours ago

Vacancies: 1 Vacancy

Job Summary

Who We Are

At Streamline we are experts in Enterprise Mobility Product Engineering and IT Transformation. We help organizations navigate the constantly evolving landscape of IT. Our sole focus is ensuring that our clients organization is armed with the strategies products and solutions that are transformative to their business. Streamline works closely with our clients takes pride in developing genuine relationships and embraces open communication and collaboration. Our team is comprised of world-class strategists architects engineers and developers.

Overview
This is a fully remote position. We are seeking a highly experienced and motivated Senior Data Engineer to design develop and optimize our data infrastructure and pipelines. The ideal candidate will have extensive experience across the Microsoft Azure data ecosystem and advanced proficiency in key data processing and analysis technologies. You will be instrumental in transforming raw data into actionable insights for the business.

Key Responsibilities

Design and Development: Design build and maintain robust scalable and efficient ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks (using Python).
Data Modeling & Storage: Develop and optimize data models in various storage solutions including Azure Data Explorer (ADX) and No-SQL databases.
Performance Optimization: Tune and improve the performance of data pipelines and data storage solutions for optimal data retrieval and processing speeds.
Collaboration: Work closely with data scientists analysts and business stakeholders to understand data requirements and deliver solutions that meet business needs.
Data Visualization Support: Ensure data is accurately structured and prepared for visualization tools primarily Power BI.
Quality & Governance: Implement best practices for data quality security and governance across all data platforms.

Qualifications and Experience

Bachelors or Masters degree in Computer Science Engineering or a related quantitative field.
Minimum 10 years of professional hands-on experience across the following technologies:

Python: Expert-level proficiency in Python for data manipulation scripting and pipeline development.
Azure: Deep knowledge and experience with the Azure cloud platform and its core data services.
Azure Data Factory (ADF): Extensive experience in building and managing complex orchestrated data pipelines.
Azure Data Explorer (ADX): Proven experience with ingesting querying and optimizing data within ADX (Kusto).
No-SQL: Strong understanding and practical experience with No-SQL databases (e.g. Cosmos DB MongoDB Cassandra).
Databricks: Hands-on experience with Databricks environments Spark and developing scalable data processing solutions.
Power BI: Experience preparing data specifically for and a strong understanding of how data is consumed by Power BI reports and dashboards.
Experience with stream processing technologies (e.g. Azure Event Hubs Kafka).
Azure certification (e.g. Azure Data Engineer Associate - DP-203).

Technical Skills

- Solid understanding of modern data warehousing concepts including Kimball and Inmon methodologies.
- Experience with version control systems (e.g. Git).
- Familiarity with CI/CD processes for data infrastructure deployment.

What We Offer

A challenging and rewarding role in a dynamic and international environment.
Opportunity to be part of a growing company with a strong commitment to innovation and excellence.
A supportive and collaborative team culture that values personal growth and development.
Competitive compensation and benefits package.

Required Experience:

Senior IC

Who We AreAt Streamline we are experts in Enterprise Mobility Product Engineering and IT Transformation. We help organizations navigate the constantly evolving landscape of IT. Our sole focus is ensuring that our clients organization is armed with the strategies products and solutions that are trans...