Bigdata Developer

PRINCIPAL ACCOUNTABILITIES
Should be able to understand the application requirement.
Responsible for application design and development in Azure Databricks Big data technologies(Scala Spark Hive).
Handle complete unit testing of the component or application.
Support testing and UAT phases and resolve issues identified.
Implement the application in production and complete post production checks.

KNOWLEDGE AND EXPERIENCE
Responsibilities : 4-8 years of development experience in object oriented applications with 2-4 years of experience in scala.
depth understanding/knowledge of Hadoop & spark Architecture and its components such as HDFS Job Tracker Task Tracker executor cores and memory params.
in Hadoop development working experience on SPARK SCALA is mandatory and Database exposure is must
on experience in Spark and Spark Streaming creating RDDs applying operations -Transformation and Actions.
in code optimize to fine tune the applications.
in writing Hadoop/spark Jobs for analyzing data using Spark Scala Hive Kafka and Python
with streaming work flow operations.
with developing large-scale distributed applications and developing solutions to analyze large data sets efficiently.
with Hadoop/HDFS Real-Time Systems Data Warehouses and Analytics solutions. Experience in Data Warehousing and ETL processes.
database SQL ETL and data analysis skills.
in Maintaining the Log and Audit information in Hive/SQL tables Experienced in providing Logging Error Handling
large amounts of data sets to determine optimal way to aggregate and report on it.
Spark Streaming to receive real time data from the Apache Kafka and store the stream data to HDFS using Scala.
Scheduling jobs using Airflow/ESP.
in developing scripts for doing transformations using Scala.
in developing Shell scripts to orchestrate execution of all other scripts and move the data files within and outside of HDFS.
Kafka on publish-subscribe messaging as a distributed commit log have experienced in its fast scalable and durability.
Kerberos Security Authentication protocol for existing cluster
be able to understand existing business logic & implement the business changes
with business and peers to define estimate and deliver functionality.
have gone through entire development life cycle including system integration testing
Production Support for any issues raised by the Business and experience on Production/Integration release management
with GIT Jenkins test frameworks.

soft skills:
communication skills to be able to work with Business.
logical reasoning analytical abilities. Recommend changes to client software based on information such as client feedback language changes and new technology.
working with remote teams.
to become the trusted partner of the Architects by regular engagement and proposals on best development practices which can be implemented and would bring change to the process
Performance and Technical Skills Attend required training and support peers by sharing knowledge and mentoring.
up training programs within your team to ensure understanding of BDF.

Agile skills:
in Agile development methodology.
the required technical tasks in backlog and update status regularly.
in all Agile ceremony meetings and update task status and adher to timelines and quality standards for the task.

TECHNICAL SKILLS:
- Azure Data bricks
- Hadoop/Big Data
- HDFS
- Scala
- Spark
- Hive
- Python
- Kafka
- Zookeeper
- Shell Scripting
- Java experience is a plus
- GIT Version control tool knowledge Jenkins and Scheduling tools Airflow & ESP

IQVIA is a leading global provider of clinical research services commercial insights and healthcare intelligence to the life sciences and healthcare industries. We create intelligent connections to accelerate the development and commercialization of innovative medical treatments to help improve patient outcomes and population health worldwide. Learn more at