Data Engineer - Astegic

Job Responsibilities:

• Create and maintain optimal data pipeline architecture.
• Assemble large complex data sets that meet functional / nonfunctional business requirements.
• Identify design and implement internal process improvements: automating manual processes optimizing data delivery redesigning infrastructure for greater scalability etc.
• Build the infrastructure required for optimal extraction transformation and loading of data from a wide variety of data sources using Spark Hadoop and AWS big data technologies.(EC2 EMR S3 Athena Dynamodb Lambda).
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive Product Data and Design teams to assist with datarelated technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

Job Requirement:

• Advanced working SQL knowledge and experience working with relational databases query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing big data data pipelines architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation data structures metadata dependency and workload management.
• A successful history of manipulating processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing stream processing and highly scalable big data data stores.
• Strong project management and organizational skills.
• Experience supporting and working with crossfunctional teams in a dynamic environment
• Experience with big data tools: Hadoop Spark Pig Vetica etc.
• Experience with AWS cloud services: EC2 EMR S3 Athena Dynamodb Lambda
• Experience with Linux
• Experience with objectoriented/object function scripting languages: Python Java Shell Scala etc.

Education & Experience:

• Graduate degree in Computer Science Statistics Informatics Information Systems or another quantitative field.
• 5 years of experience in a Data Engineer role.