Lead Data Engineer

This is a remote position.

Join the squad of experienced software experts and pump up your skills by building off-the-wall software solutions for Fortune 500 and Future 50 brands. We have opportunities in 90+ projects you can contribute to.

What you will do

• Analyze system requirements and design responsive algorithms and solutions
  

• Use big data and cloud technologies to produce production quality code
  

• Engage in performance tuning and scalability engineering
  

• Work with team, peers and management to identify objectives and set priorities
  

• Perform related SDLC engineering activities like sprint planning and estimation
  

• Work effectively in small agile teams
  

• Provide creative solutions to problems
  

• Identify opportunities for improvement and execute
  

Must haves

• Bachelor s degree in computer science, information systems, math, engineering, or other technical field, or equivalent experience
  

• 6 years of experience with Python or Java
  

• 4+ years of experience in building data lake, cloud data platform leveraging cloud (GCP/AWS) cloud native architecture, ETL/ELT, and data integration
  

• Three years of development experience with cloud services ( AWS,GCP,AZURE) utilizing various support tools (e.g. GCS, Dataproc, Cloud Data flow, Airflow(Composer), Kafka , Cloud Pub/Sub)
  

• Expertise in developing distributed data processing and Streaming frameworks and architectures (Apache Spark, Apache Beam, Apache Flink, )
  

• In-depth knowledge of NoSQL database technologies (e.g. MongoDB, BigTable, DynamoDB)
  

• Expertise in build and deployment tools (Visual Studio, PyCharm, Git/Bitbucket/Bamboo, Maven, Jenkins, Nexus, )
  

• 5 years of experience and expertise in database design techniques and philosophies (e.g. RDBMS, Document, Star Schema, Kimball Model)
  

• 5 years of experience with integration and service frameworks (e.g API Gateways, Apache Camel, Swagger API, Zookeeper, Kafka, messaging tools, microservices)
  

• Expertise with containerized Microservices and REST/GraphQL based API development
  

• Experience leveraging continuous integration/development tools (e.g. Jenkins, Docker, Containers, OpenShift, Kubernetes, and container automation) in a Ci/CD pipeline
  

• Advanced understanding of software development and research tools
  

The benefits of joining us

• Professional growth
  

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps

• Competitive compensation
  

We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities

• A selection of exciting projects
  

Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands

• Flextime
  

Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office whatever makes you the happiest and most productive.

About the project

Our team of data experts and engineers has helped create a supply chain management platform used by Forbes 500 enterprises and featured by Gartner. We ve accompanied the platform from an early prototype to a full-fledged solution based on big data, AI, and machine learning. Data quality is integral to our current engagement with this project as we re expanding the platform functionality.

AgileEngine has also been working on data and ML solutions in multiple industries and verticals like telecom, FinTech, energy efficiency, and more. Our work with our clients data has contributed to patented software products, Nasdaq-grade fraud detection, and products used by Deloitte and T-Mobile.