Software EngineerETL Engineer

Dallas, IA - USA

Monthly Salary: Not Disclosed

Posted on: 11 days ago

Vacancies: 1 Vacancy

Job Summary

Develop/Code/Configure

Design build and optimize scalable data pipelines for batch and real-time processing.
Develop and maintain data integration workflows between internal systems and external platforms.
Implement data quality checks validation rules and exception handling within pipelines.
Collaborate with business analysts and stakeholders to translate data requirements into technical solutions.
Create and maintain documentation for data flows transformations and system interfaces.
Monitor and troubleshoot data pipeline performance and reliability issues.
Ensure compliance with data governance security and privacy standards.
Work with large datasets and optimize SQL queries for performance and scalability.
Support and enhance existing data warehouse solutions including schema changes and performance tuning.
Demonstrate understanding of dimensional modeling concepts including dimensions conformed dimensions facts and aggregated facts.
Apply Kimball methodology and star schema design principles in data warehouse development.
- Knowledge of Medallion architecture is preferred but not required

Code Review

Participate in code review with other developers and designers to confirm what is developed is in alignment with the design.

Unit Test

Design and Write Unit Tests
- Create test cases for individual components of data pipelines and integration logic.
- Ensure coverage for both typical and edge-case scenarios.
Validate Data Transformations
- Test that data is correctly transformed according to business rules.
- Verify mappings and calculations within the pipeline.
Document Test Cases
- Clearly describe the purpose and expected outcome of each test.
- Make tests readable and maintainable for other developers.
Handle Exceptions and Errors
- Test error handling and logging mechanisms to ensure robustness under failure conditions.

Data Architecture Design While design is not a core responsibility for this role prior experience in design is valued and considered beneficial.

Perform Data Analysis / Data Modeling & Warehousing
- Review requirements & design and analyze data sources
- Review/query data sources (various systems) must have familiarity with query language SQL/T-SQL
- Create Data Mappings (source target and calculations needed) for source to Data Warehouse as well as Source to Source (example: system to system integrations)
- Review and interpret both logical and physical data models (ERDs).

Develop/Code/Configure Design build and optimize scalable data pipelines for batch and real-time processing. Develop and maintain data integration workflows between internal systems and external platforms. Implement data quality checks validation rules and exception handling within pipel...

Develop/Code/Configure

Design build and optimize scalable data pipelines for batch and real-time processing.
Develop and maintain data integration workflows between internal systems and external platforms.
Implement data quality checks validation rules and exception handling within pipelines.
Collaborate with business analysts and stakeholders to translate data requirements into technical solutions.
Create and maintain documentation for data flows transformations and system interfaces.
Monitor and troubleshoot data pipeline performance and reliability issues.
Ensure compliance with data governance security and privacy standards.
Work with large datasets and optimize SQL queries for performance and scalability.
Support and enhance existing data warehouse solutions including schema changes and performance tuning.
Demonstrate understanding of dimensional modeling concepts including dimensions conformed dimensions facts and aggregated facts.
Apply Kimball methodology and star schema design principles in data warehouse development.
- Knowledge of Medallion architecture is preferred but not required

Code Review

Participate in code review with other developers and designers to confirm what is developed is in alignment with the design.

Unit Test

Design and Write Unit Tests
- Create test cases for individual components of data pipelines and integration logic.
- Ensure coverage for both typical and edge-case scenarios.
Validate Data Transformations
- Test that data is correctly transformed according to business rules.
- Verify mappings and calculations within the pipeline.
Document Test Cases
- Clearly describe the purpose and expected outcome of each test.
- Make tests readable and maintainable for other developers.
Handle Exceptions and Errors
- Test error handling and logging mechanisms to ensure robustness under failure conditions.

Data Architecture Design While design is not a core responsibility for this role prior experience in design is valued and considered beneficial.

Perform Data Analysis / Data Modeling & Warehousing
- Review requirements & design and analyze data sources
- Review/query data sources (various systems) must have familiarity with query language SQL/T-SQL
- Create Data Mappings (source target and calculations needed) for source to Data Warehouse as well as Source to Source (example: system to system integrations)
- Review and interpret both logical and physical data models (ERDs).