Generative AI- Data Engineer Senior Associate
Share
Job Location:
Monthly Salary:
Posted on:
12 hours ago
Vacancies:
1 Vacancy
Job Summary
Industry/Sector
Not ApplicableSpecialism
Data Analytics & AIManagement Level
Senior AssociateJob Description & Summary
At PwC our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals.In data analysis at PwC you will focus on utilising advanced analytical techniques to extract insights from large datasets and drive data-driven decision-making. You will leverage skills in data manipulation visualisation and statistical modelling to support clients in solving complex business problems.
Exp 4 yrs
PwC US Acceleration Center isseekinga highly skilled and versatile GenAI Data Engineer at the Senior Associate level. This role focuses on advanced data engineering to deliver robust scalable GenAI solutions. The ideal candidate will have a strong background in data engineering hands-on experience with modern data technologies and deepexpertisein cloud platforms and streaming data systems.
Responsibilities:
- Design develop andmaintainscalable data pipelines and ETL processes for GenAI and ML projects.
- Implement andoptimizereal-time data streaming solutions using platforms such as Kafka Spark Streaming or similar.
- Develop and manage data lakes for large-scale structured and unstructured data.
- Integrate andoperationalizeLLM frameworks (LangChain Semantic KernelLlamaIndex) for advanced language processing.
- Developand deploy solutions on multiple cloud platforms (Azure AWS GCP Databricks)leveragingcloud-native services and containerization (Kubernetes Docker).
- Collaborate with cross-functional teams to translate business requirements into technical solutions.
- Ensure best practices in modularity reusability and security (including session management and vulnerability prevention).
- Implement CI/CD pipelines and version control (Git) for efficient development and deployment.
- Monitor troubleshoot andoptimizedata infrastructure for reliability and performance.
- Stay current with GenAI and data engineering trends; recommend and implement innovative solutions.
- Document engineering processes methodologies and best practices.
Requirements:
- 3 years of experience in data engineering.
- Proficiencyin Python and SQL (must have) with hands-on experience in Scala Java and Shell scripting.
- Experience with streaming platforms (Kafka Spark Streaming or similar).
- Experience with Spark and/or Hadoop for distributed data processing.
- Strong background in building scalable decoupled systems for GenAI/ML use cases.
- Familiarity with Python web frameworks (FlaskFastAPI).
- Experience with cloud platforms: Azure AWS GCP Databricks.
- Hands-on experience with containerization (Kubernetes Docker) and cloud-native development (REST APIs microservices serverless).
- Strong Gitproficiencyand experience with CI/CD practices.
- Excellent collaboration and communication skills.
Preferred Qualifications:
- BE / MCA / ME / Degree / MBA / Any degree from reputed institute.
- Relevant certifications inDatabricks Cloud Data Engineering related
- Having finance background or experience working with Finance or Banking domain
Nice to Have Skills:
- Experience in setting up data pipelines for model training and real-time inference.
- Exposure to LLM frameworks and tools for interacting with large language models.
- Experience developing and deploying machine learning applications in production environments.
- Understanding data privacy and compliance regulations.
- Practical knowledge of ML/DL frameworks such as TensorFlowPyTorch and scikit-learn.
- Proficient in object-oriented programming with languages such as Java C or C#.
Educational Background:
- BE / MCA / Degree/MBA/ Any degree
Travel Requirements
Not SpecifiedJob Posting End Date
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
At PwC, our purpose is to build trust in society and solve important problems. We’re a network of firms in 155 countries with over 284,000 people who are committed to delivering quality in assurance, advisory and tax services. Find out more and tell us what matters to you by vis ... View more
