Senior Data Engineer

Supercharge Your Career as a QA/Senior QA Engineer at Technoidentity!

The Tax Compliance Data Engineering team at Technoidentity is at the heart of how our global tech platform client manages and scales its finance and compliance data infrastructure. This team sits within the Biztech Data Engineering organization and handles critical data related to Tax Payments and Legal Compliance regulations. Joining this team means working with cross-functional stakeholders designing scalable solutions and contributing to a world-class data engineering environment with a strong emphasis on quality scalability and robust engineering practices.

What Will You Be Doing

Our team charter is centered on enabling our stakeholders to comply with Tax Payments and Legal regulations so that platform participants can continue operating in regulated geographies. The products built by this team ingest process validate and deliver large datasets to government authorities often in partnership with tax bodies which means data accuracy and quality are non-negotiable. As you build and maintain key components of the critical compliance data ecosystem you will have the opportunity to contribute to defining standards and best practices for Data Engineering at scale. Your work on solving complex business challenges will directly shape the tools processes and standards used by the broader data community.

A Typical Day

• Design build and maintain robust and efficient data pipelines that collect process and store data from various sources including user interaction data PII data listing details and external data feeds
  
• Develop data models that enable efficient analysis; ensure data quality consistency and accuracy across all pipelines
  
• Build high quality data assets for product use cases by partnering with cross-functional teams for compliance and reporting purposes
  
• Build scalable data pipelines using SparkSQL and Scala leveraging Airflow as the scheduler and executor framework
  
• Collaborate with Data Scientists Product Managers and Software Engineers to define data requirements and deliver solutions that drive compliance and operational improvements
  
• Contribute to the broader Data Engineering community to influence tooling and standards improving culture and productivity
  
• Improve code and data quality by leveraging and contributing to internal tools designed to automatically detect and mitigate issues
  

Requirements

What Makes You the Perfect Fit

8 years of relevant industry experience

BE/ in Computer Science or a relevant technical discipline

Hands-on experience in DSA coding (Data Structures and Algorithms)

Extensive experience designing building and operating robust distributed data platforms (Spark Kafka Flink HBase) with demonstrated ability to handle data at petabyte scale

Strong understanding of Fintech Ecommerce or Booking Platform operating models

Strong proficiency in Scala or Python with deep expertise in data processing technologies and SQL query authoring

Demonstrated ability to analyze large datasets to identify gaps and inconsistencies surface insights and drive effective data product solutions

Expertise with ETL schedulers such as Apache Airflow Luigi Oozie AWS Glue or similar frameworks

Solid understanding of data warehousing concepts and hands-on experience with relational databases (PostgreSQL MySQL) and columnar databases (Redshift BigQuery HBase ClickHouse)

Excellent written and verbal communication skills