Middle Data Engineer Apache Spark

AgileEngine is a topranking provider of software solutions to Fortune 500 Global 500 and Future 50 companies. Listed on Inc. 5000 among the fastestgrowing US companies we are always open to talented software UX and data experts in the Americas Europe and Asia.

If you like a challenging environment where you re working with the best and are encouraged to learn and experiment daily there s no better place guaranteed! :)

What you will do

• Maintain an evergrowing Datalake with a diverse set of data sources and data consumers;
  

• Help implement data governance initiative;
  

• Maintenance deprecation and eventual migration of legacy ETL jobs into a LakeHouse solution.
  

Must haves

• Airflow experience;
  

• Worked with Spark/Glue;
  

• IaC experience; CDK or Terraform;
  

• Familiar with Datalake/Datawarehouse/Lakehouse;
  

• Worked with Iceberg/Delta;
  

• 3 years of experience with application development;
  

• Experience with AWS tools;
  

• Experience with data visualization tools;
  

• Experience designing developing operationalizing and maintaining complex data applications at an enterprise scale;
  

• Experience with creating software for retrieving parsing and processing structured and unstructured data;
  

• Experience with building scalable ETL and ELT workflows for reporting and analytics;
  

• Experience with creating solutions in a collaborative crossfunctional team environment;
  

• Ability to develop scripts and programs for converting various data types into usable formats support project team to scale and monitor and operate data platforms;
  

• Ability to obtain a security clearance;
  

• Bachelor s degree.
  

Nice to haves

• Airflow experience;
  

• Worked with Spark/Glue;
  

• IaC experience; CDK or Terraform;
  

• Familiar with Datalake/Datawarehouse/Lakehouse;
  

• Worked with Iceberg/Delta;
  

• 3 years of experience with application development;
  

• Experience with AWS tools;
  

• Experience with data visualization tools;
  

• Experience designing developing operationalizing and maintaining complex data applications at an enterprise scale;
  

• Experience with creating software for retrieving parsing and processing structured and unstructured data;
  

• Experience with building scalable ETL and ELT workflows for reporting and analytics;
  

• Experience with creating solutions in a collaborative crossfunctional team environment;
  

• Ability to develop scripts and programs for converting various data types into usable formats support project team to scale and monitor and operate data platforms;
  

• Ability to obtain a security clearance;
  

• Bachelor s degree.
  

The benefits of joining us

• Professional growth
  

Accelerate your professional journey with mentorship TechTalks and personalized growth roadmaps.

• Competitive compensation
  

We match your evergrowing skills talent and contributions with competitive USDbased compensation and budgets for education fitness and team activities.

• A selection of exciting projects
  

Join projects with modern solutions development and toptier clients that include Fortune 500 enterprises and leading product brands.

• Flextime
  

Tailor your schedule for an optimal worklife balance by having the options of working from home and going to the office whatever makes you the happiest and most productive.