Data Engineer 3

MW Partners is currently seeking a Data Engineer to work for our client who is a global leader in multimedia and creativity software products.

Responsibilities and duties:

Data Asset Inventory and Assessment

• Inventory of existing data assets; tables dashboards and pipelines
• Follow requirements to pull required data to measure the current state of these assets set up usage metrics for internal and external stakeholders.

Table Metadata

• to improve documentation coverage for tables including table descriptions column definitions and data lineage.
• Implement a centralized metadata management system to maintain and access asset documentation.
• Ensure that all existing and new data assets are properly documented according to established standards.
• Pipeline Cleanup and Consolidation
• Consolidate and streamline pipelines by eliminating redundancies and unnecessary elements according to the set of provided rules.
• Clean up and restructure data tables ensuring consistent naming conventions data types and schema definitions.
• Retire or archive obsolete dashboards and workflows.
• Implement monitoring and alerting mechanisms for critical workflows to ensure timely issue detection and resolution.

Set up a foundation for scalable Data Model for the Stock Business

• Implement and build performant data models to solve common analytics usecases at Stock.
• Knowledge Transfer and Documentation
• Thoroughly document the work performed including methodologies decisions and any scripts or tools developed.
• Provide comprehensive knowledge transfer to the data team ensuring a smooth transition and the ability to maintain the optimized data environment.

Requirements:

• 4 7 Years of Experience
• Proven experience in data engineering and data asset management.
• Proficiency in SQL Python and other relevant data processing languages and tools.
• Expertise in data modeling ETL processes and workflow orchestration (e.g. Airflow Databricks).
• Strong analytical and problemsolving skills.
• Excellent communication and documentation abilities.
• Familiarity with cloud data platforms (e.g. Azure AWS GCP) is a plus.

For a confidential discussion or to find out more contact Nitin Kuntiya on (909)or apply now.