Senior Big Data Application Engineer Remote
Senior Big Data Application Engineer Remote
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Description Summary:
IQVIAs Global Technology and Operations (GTO) groups responsibility is expanding. To effectively navigate this increased scope were looking for a Senior Big Data Application Engineer to join the team.
Position Overview:
Join our Data Office as a Senior Big Data Application Engineer where youll be a key player in optimizing our Big Data Factory operations. Our fastpaced team manages over 40 petabytes of data across thousands of workloads across public and private Hadoop CDP platforms Snowflake Data Cloud and Databricks. Your role will be critical in driving optimizations with minimal supervision ensuring our big data processes are as efficient and costeffective as possible.
Key Responsibilities:
Develop advanced Sparkbased optimizations to ensure optimal performance and costefficiency of our onpremises and cloud data platform infrastructure.
Utilize deep expertise in Scala Python SQL and TEZ to drive significant cost savings and performance enhancements.
Lead initiatives for automating data transfers and implementing data reusability and archival strategies significantly reducing storage and processing costs.
Develop and enforce best practices and standards for Spark application development and deployment ensuring consistency and reliability across our big data ecosystem.
Conduct thorough performance analysis of Spark applications optimizing code and queries to achieve peak efficiency.
Optimize the utilization of our BI platforms tailoring data models and dashboards to the specific needs of big data processing and analysis.
Create and deliver training materials focused on Spark optimization techniques and best practices for our business unit resources.
Qualifications:
Bachelors degree in Computer Science Engineering or a related field; Masters degree preferred; or equivalent work experience.
At least 8 years of deep data engineering experience focusing on Spark and SQL application development.
Demonstrated expertise in big data technologies including Apache Spark Scala Python SQL and TEZ.
Proven track record of driving significant cost savings and performance improvements in a big data environment.
Experience optimizing Spark code Impala queries and Hive partitioning strategy for better scalability reliability and performance.
Experience with Hive performance optimizations such as using distributed cache for small datasets data partitions bucketing in Hive and Map Side joins.
Proficiency in Snowflake Databricks Oracle and Cloudera for data management and optimization.
Exceptional analytical and problemsolving abilities capable of conducting detailed technical analysis to uncover optimization opportunities.
Excellent communication skills with the ability to lead crossfunctional teams and articulate complex technical concepts to nontechnical stakeholders.
#LIDNI
IQVIA is a leading global provider of clinical research services commercial insights and healthcare intelligence to the life sciences and healthcare industries. We create intelligent connections to accelerate the development and commercialization of innovative medical treatments to help improve patient outcomes and population health worldwide. Learn more at
We are committed to providing equal employment opportunities for all including veterans and candidates with disabilities. ability to operate and provide certain services to customers and partners necessitates IQVIA and its employees meet specific requirements regarding COVID19 vaccination status. potential base pay range for this role when annualized is $106900.00 $198500.00. The actual base pay offered may vary based on a number of factors including jobrelated qualifications such as knowledge skills education and experience; location; and/or schedule (full or parttime). Dependent on the position offered incentive plans bonuses and/or other forms of compensation may be offered in addition to a range of health and welfare and/or other benefits.
Required Experience:
Senior IC
Employment Type
Full-Time
