Data Designer (PythonPyspark)
Share
Job Location:
Monthly Salary:
Posted on:
2 hours ago
Vacancies:
1 Vacancy
Job Summary
The Data Designer will be responsible for designing and documenting data solutions ensuring data quality and effective data flow across systems. This role requires a blend of technical expertise in data tools and strong communication skills to collaborate with stakeholders and translate business requirements into technical specifications.
Required Skills & Qualifications:
- Technical Proficiency:
- Strong proficiency in SQL for data manipulation and querying.
- Experience with PySpark and Python for data processing and scripting.
- Hands-on experience with Azure Data Bricks for building and managing data pipelines.
- Data Design & Documentation:
- Proven experience with data profiling and data mapping.
- Ability to create detailed SRS documentation including data flows.
- Soft Skills:
- Excellent communication and interpersonal skills to effectively interact with technical and non-technical stakeholders.
- Strong analytical and problem-solving abilities.
- Ability to work independently and collaboratively within a team.
- Methodology:
- Familiarity with Agile development methodologies.
Key Responsibilities:
- Data Solution Design:
Design and develop data models data pipelines and data integration strategies within the Azure ecosystem leveraging Azure Databricks.
- Data Profiling & Mapping:
Conduct thorough data profiling to understand data characteristics and perform data mapping from source to target systems.
- Documentation:
Create comprehensive System Requirements Specification (SRS) documents including detailed data flow diagrams data dictionaries and transformation rules.
- Technical Implementation:
Develop and implement data transformations and processing logic using SQL PySpark and Python within Azure Databricks.
- Collaboration & Communication:
Work closely with business users data engineers and other stakeholders to gather requirements communicate design decisions and ensure alignment on data solutions.
- Agile Participation:
Actively participate in Agile ceremonies providing input on sprint planning backlog grooming and daily stand-ups.
- Data Quality & Governance:
Contribute to establishing and maintaining data quality standards and assist in implementing data governance policies.
Key Skills
- APIs
- Docker
- Jenkins
- REST
- Python
- AWS
- NoSQL
- MySQL
- JavaScript
- Postgresql
- Django
- GIT
