Data Engineer (Scalable data pipeline development)

Bengaluru - India

Monthly Salary: Not Disclosed

Posted on: 4 hours ago

Vacancies: 1 Vacancy

Job Summary

Data Engineer role for Asia Pacific Region is based out of Bangalore. We are looking for an expert Data Engineer (Analyst level) with deep expertise in big data/data warehousing and can build large-scale data processing systems by using the latest database and data processing technologies. This is a Pan-regional position and plays a critical role in enabling the data platforms through which Data Scientists Analysts and BI Users drive solutions for our Visa clients.

The Data Engineer takes responsibility for building and running data pipelines designing our local data warehouse data marts and data frameworks and catering for different data presentation techniques. We are looking for a talented technical proactive energetic and passionate person who embraces challenges and is a proven problem solver.

He/she will bring in deep expertise in data pipeline design and development with a strong background in developing and enhancing data Pipelines at scale to unlock business value.

Principal Responsibilities:

Create and maintain optimal data pipelines data marts and architecture(s) based on our Global Technology Stack
Collaborate with the global data engineering team to adopt global engineering standards in AP region
Develop custom-built packages to support the needs of Data Scientists across the region
Provide technical and business support for development of new platforms tools for data science using both on-prem and cloud technologies
Identify design and implement internal process improvements to provide greater scalability to our existing client solutions
Work with broader business stakeholders to assist clients and consultants with their data and infrastructure needs
Partner with Technology on quarterly planning cycle and support management with relevant metrics to evaluate performance stability and reliability of various tools
Continuous focus on improving Infrastructure efficiency by analyzing logs of queries tuning settings translating queries if required
Review scripts for best practices educating user base and building training assets for beginner and intermediate users
Increase consistency of tool usage by developing guidance for specific data science applications and sharing with user community

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications :

Required Skills & Experience (Must Have):

5 years experience plus a bachelors degree in computer science information science mathematics or similar/related streams. Graduate masters degree is a plus.
Skill Level: Hands on experience in a data engineering with good knowledge of distributed data architecture commonly used BI tools and approaches/packages used in machine learning build.
Strong understanding of Hadoop architecture and its core components including HDFS YARN and MapReduce.
Database Knowledge: Solid understanding of SQL and experience working with various database systems.
Expertise in creating production software/systems in Python and/or Scala and a proven track record of identifying and resolving performance bottlenecks in production systems.
Experience with complex high volume multi-dimensional data as well as machine learning models based on unstructured structured and streaming datasets.
Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.
Demonstrated ability to incorporate new techniques to solve business problems.
Good communication and presentation skills with ability to interact with different cross-functional team members at varying level
Self-motivated results oriented individual with ability to handle multiple projects concurrently.

(Good to Have)

Knowledge of Cloud Services such as AWS/Azure.
Understanding of Payments or Banking Industry will be a plus.
Understanding GenAI Tools.
Certification in Hadoop (Cloudera or Hortonworks) and Apache Spark preferred.
Understanding of Data Visualization Tools like Tableau / Power BI.

Additional Information :

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race color religion sex national origin sexual orientation gender identity disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Remote Work :

Employment Type :

Full-time

He/she will bring in deep expertise in data pipeline design and development with a strong background in developing and enhancing data Pipelines at scale to unlock business value.

Principal Responsibilities:

Create and maintain optimal data pipelines data marts and architecture(s) based on our Global Technology Stack
Collaborate with the global data engineering team to adopt global engineering standards in AP region
Develop custom-built packages to support the needs of Data Scientists across the region
Provide technical and business support for development of new platforms tools for data science using both on-prem and cloud technologies
Identify design and implement internal process improvements to provide greater scalability to our existing client solutions
Work with broader business stakeholders to assist clients and consultants with their data and infrastructure needs
Partner with Technology on quarterly planning cycle and support management with relevant metrics to evaluate performance stability and reliability of various tools
Continuous focus on improving Infrastructure efficiency by analyzing logs of queries tuning settings translating queries if required
Review scripts for best practices educating user base and building training assets for beginner and intermediate users
Increase consistency of tool usage by developing guidance for specific data science applications and sharing with user community

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications :

Required Skills & Experience (Must Have):

5 years experience plus a bachelors degree in computer science information science mathematics or similar/related streams. Graduate masters degree is a plus.
Skill Level: Hands on experience in a data engineering with good knowledge of distributed data architecture commonly used BI tools and approaches/packages used in machine learning build.
Strong understanding of Hadoop architecture and its core components including HDFS YARN and MapReduce.
Database Knowledge: Solid understanding of SQL and experience working with various database systems.
Expertise in creating production software/systems in Python and/or Scala and a proven track record of identifying and resolving performance bottlenecks in production systems.
Experience with complex high volume multi-dimensional data as well as machine learning models based on unstructured structured and streaming datasets.
Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.
Demonstrated ability to incorporate new techniques to solve business problems.
Good communication and presentation skills with ability to interact with different cross-functional team members at varying level
Self-motivated results oriented individual with ability to handle multiple projects concurrently.

(Good to Have)

Knowledge of Cloud Services such as AWS/Azure.
Understanding of Payments or Banking Industry will be a plus.
Understanding GenAI Tools.
Certification in Hadoop (Cloudera or Hortonworks) and Apache Spark preferred.
Understanding of Data Visualization Tools like Tableau / Power BI.

Additional Information :

Remote Work :

Employment Type :

Full-time

Key Skills

Apache Hive
S3
Hadoop
Redshift
Spark
AWS
Apache Pig
NoSQL
Big Data
Data Warehouse
Kafka
Scala

Apply Now

About Company

Visa

Visa (NYSE: V) is a world leader in digital payments, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories. Our purpose is to uplift everyone, everywhere by being the best way to pay and b ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click