Lead Data Engineer

AgileEngine is a topranking provider of software solutions to Fortune 500 Global 500 and Future 50 companies. Listed on Inc. 5000 among the fastestgrowing US companies we are always open to talented software UX and data experts in the Americas Europe and Asia.

If you like a challenging environment where you re working with the best and are encouraged to learn and experiment daily there s no better place guaranteed! :)

What you will do

• Design and build parts of our data pipeline architecture for extraction transformation and loading of data from a wide variety of data sources using the latest Big Data technologies.
  

• Identify design and implement internal process improvements: automating manual processes optimizing data delivery redesigning infrastructure for greater scalability etc.
  

• Work with stakeholders including the Executive Product Data and Design teams to assist with datarelated technical issues and support their data infrastructure needs.
  

• Work with machine learning data and analytics experts to drive innovation accuracy and greater functionality in our data system.
  

Must haves

• Bachelor s degree in Engineering Computer Science or relevant field.
  

• 5 years of experience in a Data Engineer role.
  

• 3 years experience with Apache Spark and solid understanding of the fundamentals.
  

• Deep understanding of Big Data concepts and distributed systems.
  

• Strong coding skills with Scala Python Java and/or other languages and the ability to quickly switch between them with ease.
  

• Advanced working SQL knowledge and experience working with a variety of relational databases such as Postgres and/or MySQL.
  

• Cloud Experience with DataBricks
  

• Experience working with data stored in many formats including Delta Tables Parquet CSV and JSON.
  

• Comfortable working in a linux shell environment and writing scripts as needed.
  

• Comfortable working in an Agile environment
  

• Machine Learning knowledge is a plus.
  

• Must be capable of working independently and delivering stable efficient and reliable software.
  

• Excellent written and verbal communication skills in English.
  

• Experience supporting and working with crossfunctional teams in a dynamic environment.
  

The benefits of joining us

• Professional growth
  

Accelerate your professional journey with mentorship TechTalks and personalized growth roadmaps.

• Competitive compensation
  

We match your evergrowing skills talent and contributions with competitive USDbased compensation and budgets for education fitness and team activities.

• A selection of exciting projects
  

Join projects with modern solutions development and toptier clients that include Fortune 500 enterprises and leading product brands.

• Flextime
  

Tailor your schedule for an optimal worklife balance by having the options of working from home and going to the office whatever makes you the happiest and most productive.