Senior Big Data

Responsibilities:-

• Design, develop and build real-time data pipelines from a variety of sources (streaming data, APIs, data warehouse, messages, etc.)
• Leverage the understanding of software architecture and software design patterns to write scalable, maintainable, well-designed, and future-proof software
• Manage existing pipelines and create new pipelines from a variety of sources (relational, XML, etc.)
• Actively apply best practices within CI/CD
• Propose and implement solutions for data pipeline stabilization and data quality checks
• Coordination with other teams to design optimal patterns for data ingest and egress, as well as lead and coordinate data quality initiatives and troubleshooting
• Design and build solutions to track data quality, stabilize data pipeline, etc. to ensure reliable operations
• Ensure best practices are followed across architecture, codebase and configuration