Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Wissen Technology is now hiring Data Engineer
We are seeking a skilled Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering including data ingestion data aggregation and experience with modern data platforms. You will be responsible for updating the ER model cataloging data and supporting the migration of local data platforms to a Common Data Platform (CDP). This role requires a mix of technical expertise problemsolving skills and the ability to interface with various stakeholders.
Experience 9 Years
Location: Chennai
Key Responsibilities:
Data Engineering Skills:
- Data Handling: Manage and optimize data within dimension and fact tables.
- Data Pipeline Development: Create and maintain data pipelines for integrating and ingesting data from various sources.
- Data Cleaning and Aggregation: Clean enrich and aggregate data within the Common Data Platform leveraging the Bronze Silver and Gold layers.
- Consumer Data Pipelines: Build and manage data pipelines to ensure the smooth flow of data to consumers.
- ER Model and Data Cataloging: Update the EntityRelationship (ER) model and catalog data to maintain data integrity and accessibility.
- Data Platform Migration: Lead the migration of local data platforms to the Common Data Platform (CDP) using tools like Talend or Informatica.
- ETL Transformation Replication: Replicate existing ETL transformations from local sources to the CDP.
- Quality Assurance: Support both Technical QA and Business QA processes to ensure the accuracy and reliability of data.
- Data Handling: Manage and optimize data within dimension and fact tables.
Technical Skills:
- Must Have:
- PySpark & SQL: Advanced proficiency in PySpark and SQL for data processing and querying.
- PySpark & SQL: Advanced proficiency in PySpark and SQL for data processing and querying.
- Should Have:
- Scripting (Python & Bash): Proficiency in scripting with Python and Bash for automation and data manipulation.
- AWS Knowledge: Strong understanding of AWS services and cloud architecture.
- CI/CD (Github Actions): Experience with Continuous Integration/Continuous Deployment (CI/CD) pipelines using Github Actions.
- Scripting (Python & Bash): Proficiency in scripting with Python and Bash for automation and data manipulation.
- Could Have:
- Terraform: Familiarity with Terraform for infrastructure as code.
- Airflow: Knowledge of Airflow for orchestrating complex data workflows.
- Neptune or Neo4j: Experience with graph databases like Neptune or Neo4j.
- Medallion Architecture: Understanding of Medallion Architecture principles.
- Atlassian or Unity Catalog: Experience with data governance tools like Atlassian or Unity Catalog.
- Terraform: Familiarity with Terraform for infrastructure as code.
- Must Have:
Soft Skills:
- Communication: Collaborate effectively with data source technical teams to gather requirements and manage data.
- Collaboration: Work with data consumer teams to resolve queries related to data handling and usage.
- Change Management: Assist in the deprecation of legacy systems and onboard business users to the new data environment.
- Communication: Collaborate effectively with data source technical teams to gather requirements and manage data.
Qualifications:
- Bachelors degree in Computer Science Information Technology or a related field.
- Insert years of experience in data engineering data warehousing or a similar role.
- Strong problemsolving abilities and experience working in a dynamic environment.
- Excellent communication and collaboration skills.
Preferred Qualifications:
- Handson experience with data platform migration projects.
- Familiarity with Talend or Informatica for ETL processes.
- Knowledge of modern data architectures and frameworks.
About Wissen Technology:
The Wissen Group was founded in the year 2000. Wissen Technology a part of Wissen Group was established in the year 2015. Wissen Technology is a specialized technology company that delivers highend consulting for organizations in the Banking & Finance Telecom and Healthcare domains. We help clients build world class products.
We offer an array of services including Core Business Application Development Artificial Intelligence & Machine Learning Big Data & Analytics Visualization & Business Intelligence Robotic Process Automation Cloud Adoption Mobility Digital Adoption Agile & DevOps Quality Assurance & Test Automation.
Over the years Wissen Group has successfully delivered $1 billion worth of projects for more than 20 of the Fortune 500 companies. Wissen Technology provides exceptional value in mission critical projects for its clients through thought leadership ownership and assured ontime deliveries that are always first time right.
The technology and thought leadership that the company commands in the industry is the direct result of the kind of people Wissen has been able to attract. Wissen is committed to providing them with the best possible opportunities and careers which extends to providing the best possible experience and value to our clients.
We have been certified as a Great Place to Work company for two consecutive yearsand voted as the Top 20 AI/ML vendor by CIO Insider. Great Place to Work Certification is recognized world over by employees and employers alike and is considered the Gold Standard. Wissen Technology has created a Great Place to Work by excelling in all dimensions HighTrust HighPerformance Culture Credibility Respect Fairness Pride and Camaraderie.
LinkedIn: https:// Leadership: Live: Thought Leadership: Speak:
Employment Type
Full-Time
