Databrick Developer: Java Spark
Databrick Developer: Java Spark
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Databricks Developer: Java Spark
Clearance requirement: Active IRS MBI Clearance Required. IRS issued a laptop strongly preferred.
Location: Remote
Work Authorization: US Citizen or Greencard (No H1 visa holder)
Summary
This role supports a key modernization initiative within the Internal Revenue Service (IRS) focused on transforming legacy tax processing systems. The effort is critical to the mission of the IRS IT Application Development (AD) and Enterprise Services (ES) organizations and aligns with the broader objectives of the IRS IT Modernization Business Plan. The initiative aims to modernize IRS systems and processes enhance the taxpayer experience promote voluntary compliance and reduce dependency on outdated costly technologies.
Task Description:
The Databricks Developer will be responsible for designing developing and maintaining scalable data processing solutions on the Databricks platform with a focus on integrating and transforming IRS datasets such as the Information Returns Master File (IRMF) Business Master File (BMF) and Individual Master File (IMF). This role requires advanced proficiency in Java and Apache Spark and a deep understanding of big data processing performance optimization and secure data handling in a federal environment.
Required skills/Level of Experience:
We are seeking a Databricks Developer with deep expertise in Java and Apache Spark along with hands-on experience working with IRS data systems such as IRMF BMF or IMF. The ideal candidate will be responsible for designing developing and optimizing big data pipelines and analytics solutions on the Databricks platform. This role requires a deep understanding of distributed data processing performance tuning and scalable architecture.
Key Responsibilities:
-
Design develop and maintain scalable data pipelines using Apache Spark on Databricks
-
Implement data processing logic in Java 8 leveraging functional programming and OOP best practices
-
Integrate with IRS data systems including IRMF BMF or IMF
-
Optimize Spark jobs for performance reliability and cost-efficiency
-
Collaborate with cross-functional teams to gather requirements and deliver data solutions
-
Ensure compliance with data security privacy and governance standards
-
Troubleshoot and debug production issues in distributed data environments
Required Skills & Qualifications:
-
Active IRS MBI Clearance Required. IRS issued laptop strongly preferred. Please provide a copy of the candidates active MBI letter.
-
Bachelors degree in Computer Science Information Systems or a related field.
-
8 years of professional experience demonstrating the required technical skills and responsibilities listed:
IRS Data Systems Experience
-
Hands-on experience working with IRS IRMF BMF or IMF datasets
-
Understanding of IRS data structures compliance and security protocols
-
Programming Language Proficiency
-
Strong expertise in Java 8 or higher
-
Experience with functional programming (Streams API Lambdas)
-
Familiarity with object-oriented design patterns and best practices
Apache Spark
-
Proficient in Spark Core Spark SQL and DataFrame/Dataset APIs
-
Understanding of RDDs and when to use them
-
Experience with Spark Streaming or Structured Streaming
-
Skilled in performance tuning and Spark job optimization
-
Ability to use Spark UI for troubleshooting stages and tasks
Big Data Ecosystem
-
Familiarity with HDFS Hive or HBase
-
Experience integrating with Kafka S3 or Azure Data Lake
-
Comfort with Parquet Avro or ORC file formats
Data Processing and ETL
-
Strong understanding of batch and real-time data processing paradigms
-
Experience building ETL pipelines with Spark
-
Proficient in data cleansing transformation and enrichment
DevOps / Deployment
-
Experience with YARN Kubernetes or EMR for Spark deployment
-
Familiarity with CI/CD tools like Jenkins or GitHub Actions
-
Monitoring experience with Grafana Prometheus Datadog or Spark UI logs
Version Control & Build Tools
-
Proficient in Git
-
Experience with Maven or Gradle
Testing
-
Unit testing with JUnit or TestNG
-
Experience with Mockito or similar mocking frameworks
-
Data validation and regression testing for Spark jobs
Soft Skills / Engineering Practices
-
Experience working in Agile/Scrum environments
-
Strong documentation skills (Markdown Confluence etc.)
-
Ability to debug and troubleshoot production issues effectively
-
Preferred Qualifications:
-
Experience with Scala or Python in Spark environments
-
Familiarity with Databricks or Google DataProc
-
Knowledge of Delta Lake or Apache Iceberg
-
Experience with data modeling and performance design for big data systems
Nice to have skills:
-
Experience with Scala or Python in Spark environments
-
Familiarity with Databricks or Google DataProc
-
Knowledge of Delta Lake or Apache Iceberg
-
Data modeling and performance design for big data systems.
Employment Type
Full-time
