Data Engineer

• Job DescriptionIn serving data back make accessible to analysts and PowerBI reporting. Can build dashboards on top of this area.
• Real time and batch data large volume will be batch. Doesn t change as often.
• Requirement to consume Kafka topic that will be real time in case a new consumer comes along and needs a seed file. One or two real time consumers.
• Senior needs to build out data pipelines from scratch. Stand up data domains in each line of business or specialized area. There is a lot coming down architecture center of excellence in terms of structure guidelines of what you should do. This resource will find common pattern or standard being used and adopt that. Someone calling out an area of improvement. Scratch would be ideal.
• Senior resource will be capable of outlining the patterns and creating the Jira and tickets that next line of engineers will be responsible for. Architects and senior leadership will create the process and hand off smaller tasks. Engineers need to be familiar with it but not necessarily the experts.
• ML won t be tackled in this space right away. Plenty of time

jira,azure,architects,kafka,pyspark,data,sql,python,word,microsoft azure,databricks