Responsibilities:
- Design develop test and implement:
- data pipelines using Python and AWS services (Glue Step Functions Lambda)
- complex data transformation procedures
- data models for the efficient storing of data in relational databases as well as in unstructured data repositories
- reuseable classes and modules as to enhance the maintainability and ability to deliver solutions quickly.
- Review existing code base and assist other more junior team members ensuring a consistent level of quality.
- Assess new business requirements and propose adequate technical solutions (using onprem and oncloud resources)
- Monitor automatic of the various data loads and proactively address issues such as data processing errors or performance degradations.
General Development Experience (Python)
- Serverside scripting
- Working with XLSX CSV JSON files relational databases structured and unstructured data
- Data engineering and processes automation
- Linux experience
AWS Cloud Experience
- Experience using AWS Services such as Glue Step Functions Lambda S3 or equivalent
- Experience with cloud data warehousing and analytics (AWS Redshift or equivalent)
Data warehouse
- Data modeling (relational & dimensional) advanced SQL
- Extract/Transform/Load data
- Data reporting/visualization
Software Development Life Cycle
- Full SDLC from requirements gathering design implementation testing to deployment and production support
- Familiar with project management (agile/scrum and waterfall)
- Change and Incident management
General skills
- Communication presentation and negotiation skills
- Consulting problemsolving and decisionmaking skills
- Public sector experience
Requirements
MUST HAVES:
General Development Experience (Python)
- Serverside scripting
- Working with XLSX CSV JSON files relational databases structured and unstructured data
- Data engineering and processes automation
AWS Cloud Experience
- Experience with cloud data warehousing and analytics (AWS Redshift or equivalent)
- Experience using AWS Services such as Glue Step Functions Lambda S3 or equivalent
Data Warehouse
- Extract/Transform/Load data (ideally Informatica experience on Cloud)