Senior Data Engineer- JavaPythonSQLSpring-boot 4yrs

The Natural Language Query (NLQ) platform is an innovative initiative designed to transform the way users interact with data. Our platform leverages advanced natural language processing (NLP) to convert user queries in plain language into executable SQL queries enabling seamless data retrieval and analysis without the need for SQL expertise. The NLQ platform will be powered by the Enterprise Data Catalog ensuring comprehensive and accurate metadata definitions for all table and column information. This platform empowers different business units to build externalfacing conversational BI chatbots to handle customer requests while also significantly reducing data exploration efforts for internal data analysts by more than 90%. 

Key Responsibilities:

       Provide PlatformasaService offerings that are easy to consume scalable secure and reliable using open sourcebased solutions.

       Develop and enhance the NLQ Copilot platform ensuring it meets the evolving needs of multitenant environments.

       Implement Context builder algorithms by leveraging different prompt engineering techniques to generate 100% accurate SQL as per the customer needs.

       Collaborate with downstream clients to integrate business requirements adding robust guardrails to prevent unauthorized query generation.

       Work closely with data scientists engineers and product managers to optimize the performance of the NLQ platform.

       Utilize cuttingedge NLP / LLM and machine learning techniques to improve the accuracy and efficiency of query transformations.

       Ensure the platforms scalability and reliability through rigorous testing and continuous improvement.

       Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.

       Spend 80% of the time writing code in different languages frameworks and technology stacks.

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications :

Basic Qualification
4 yrs of experience in architecture design and development of largescale
data management platforms and data application with simple solutions
Bachelors or masters degree in Computer Science or related technical
discipline required
Must have extensive handson coding and designing skills on
Java/Python for backend
MVC (modelviewcontroller) for endtoend development
SQL/NoSQL technology. Familiar with Databases like Oracle DB2 SQL Server
etc.
Web Services (REST/ SOAP/gRPC)
React/Angular for frontend (UI frontend nice to have)
Expertise in design and management of complex data structures and data
processes
Expertise in efficiently leveraging the power of distributed big data systems
including but not limited to Hadoop Hive Spark Kafka streaming etc.
Strong service architecture and development experience with high
performance and scalability
Strong on driving for results and selfmotivated strong learning mindset with
good understanding of related advanced/new technology. Keep up with the
technology development in the related areas in the industry which could be
leveraged to enhance current architectures and build durable new ones.
Strong leadership and team player.
Strong skills on mentoring/growing junior people
Payment industry experience is a plus.

Preferred Qualification
Deep knowledge and hands on experience on big data and cloud computing
technologies.
Experience with LLM / GenAI tools / applications and Prompt Engineering
Experience with ETL/ELT tools / applications
Experience with Apache NiFi and Apache Spark for processing large data sets
Experience on Elastic Search
Knowledge on Data Catalog tools
Experience in building Data Pipeline development tools
Experience with Data Governance and Data Quality tools

Additional Information :

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race color religion sex national origin sexual orientation gender identity disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Remote Work :

No

Employment Type :

Fulltime