Senior Hadoop Software Engineer
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
At eBay were more than a global ecommerce leader were changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. Were committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.
Our customers are our compass authenticity thrives bold ideas are welcome and everyone can bring their unique selves to work every day. Were in this together sustaining the future of our customers our company and our planet.
Join a team of passionate thinkers innovators and dreamers and help us connect people and build communities to create economic opportunity for all.
About the team and the role:
The team builds and evolves eBays next-generation lakehouse platform powering large-scale analytics across the business. With a focus on Spark stability usability and performance the team enhances Iceberg capabilities and develops internal storage and metadata services that form the foundation of eBays big data ecosystem.
As a Software Engineer you will help design optimize and operate distributed data systems at global scale. You will partner with engineers across regions to improve platform reliability address production challenges and strengthen the performance of critical analytics infrastructure. This role follows eBays hybrid work model with three days per week in the office.
What you will accomplish:
Improve Spark stability usability and performance to support reliable large-scale data processing.
Extend and enhance Iceberg capabilities to enable scalable and efficient lakehouse operations.
Design and build internal storage and metadata services that power eBays lakehouse architecture.
Troubleshoot and resolve production issues reducing downtime and improving system resilience.
Enhance monitoring logging and alerting to proactively identify and address system and performance risks.
Collaborate with global engineering teams to solve complex distributed systems challenges and optimize platform performance.
Contribute to the evolution of eBays Hadoop and big data ecosystem supporting analytics at global scale.
What you will bring:
5 years of software engineering experience with strong programming skills in Java or Scala.
Deep knowledge of Apache Spark internals including Catalyst optimizer Tungsten execution engine shuffle mechanics Adaptive Query Execution (AQE) and memory management.
Strong understanding of file formats such as Apache Parquet and table formats including Apache Iceberg Delta or Apache Hudi including snapshots manifests partition specifications and table metadata.
Experience working with Linux/Unix systems including shell scripting and system commands and an understanding of networking principles in distributed environments.
Proven ability to diagnose and resolve complex defects and performance issues in distributed systems using strong analytical and problem-solving skills.
Experience designing and implementing distributed systems is a plus.
Familiarity with the Hadoop ecosystem including HDFS HBase or YARN and contributions to open-source projects such as Apache Spark or Apache Iceberg are advantageous.
#LI-CH2
Additional Details
eBay is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race color religion national origin sex sexual orientation gender identity veteran status and disability or other legally protected you have a need that requires accommodation please contact us at. We will make every effort to respond to your request for accommodation as soon as possible. View our accessibility statement to learn more about eBays commitment to ensuring digital accessibility for people with disabilities.
We use cookies to enhance your experience and may use AI tools for administrative tasks in the hiring process. To learn how we handle your personal data and use AI responsibly please visit ourTalent Privacy Notice Privacy Center and AI Hiring Guidelines.
Required Experience:
Senior IC
Key Skills
About Company
Founded in 1995 in San Jose, Calif., eBay (NASDAQ: EBAY) is where the world goes to shop, sell and give. Whether you’re buying new or used, common or luxurious, trendy or rare – if it exists in the world, it’s probably for sale on eBay. Our great value and unique selection help every ... View more
