Seior Data Engineer Apache Spark
Seior Data Engineer Apache Spark
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
AgileEngine is a topranking provider of software solutions to Fortune 500 Global 500 and Future 50 companies. Listed on Inc. 5000 among the fastestgrowing US companies we are always open to talented software UX and data experts in the Americas Europe and Asia.
If you like a challenging environment where you re working with the best and are encouraged to learn and experiment daily there s no better place guaranteed! :)
What you will do
Maintain an evergrowing Datalake with a diverse set of data sources and data consumers;
Help implement data governance initiatives;
Maintenance deprecation and eventual migration of legacy ETL jobs into a LakeHouse solution.
Must haves
Airflow experience;
Worked with Spark/Glue;
IaC experience; CDK or Terraform;
Familiar with Datalake/Datawarehouse/Lakehouse;
Worked with Iceberg/Delta;
3 years of experience with application development;
Experience with AWS tools;
Experience with data visualization tools;
Experience designing developing operationalizing and maintaining complex data applications at an enterprise scale;
Experience with creating software for retrieving parsing and processing structured and unstructured data;
Experience with building scalable ETL and ELT workflows for reporting and analytics;
Experience with creating solutions in a collaborative crossfunctional team environment;
Ability to develop scripts and programs for converting various data types into usable formats support project team to scale and monitor and operate data platforms;
Ability to obtain a security clearance;
Bachelor s degree.
Nice to haves
Lake Formation experience;
Worked with big data;
DynamoDB/DMS experience;
Experience with Python SQL Scala or Java;
Experience with UNIX and Linux including basic commands and Shell scripting;
Experience with AWS tools including Quicksight and AWS Glue;
Experience with distributed data and computing tools including Spark Databricks Hadoop Hive AWS EMR or Kafka;
Experience with working on realtime data and streaming applications;
Experience with NoSQL implementation including MongoDB or Cassandra;
Experience with data warehousing using AWS Redshift MySQL or Snowflake;
Experience with Agile engineering practices.
The benefits of joining us
Professional growth
Accelerate your professional journey with mentorship TechTalks and personalized growth roadmaps.
Competitive compensation
We match your evergrowing skills talent and contributions with competitive USDbased compensation and budgets for education fitness and team activities.
A selection of exciting projects
Join projects with modern solutions development and toptier clients that include Fortune 500 enterprises and leading product brands.
Flextime
Tailor your schedule for an optimal worklife balance by having the options of working from home and going to the office whatever makes you the happiest and most productive.
Airflow experience; Worked with Spark/Glue; IaC experience; CDK or Terraform; Familiar with Datalake/Datawarehouse/Lakehouse; Worked with Iceberg/Delta; 3+ years of experience with application development; Experience with AWS tools; Experience with data visualization tools; Experience designing, developing, operationalizing, and maintaining complex data applications at an enterprise scale; Experience with creating software for retrieving, parsing, and processing structured and unstructured data; Experience with building scalable ETL and ELT workflows for reporting and analytics; Experience with creating solutions in a collaborative, cross-functional team environment; Ability to develop scripts and programs for converting various data types into usable formats, support project team to scale, and monitor and operate data platforms; Ability to obtain a security clearance; Bachelor s degree.
Employment Type
Full Time
