Data Engineer [Multiple Positions Available]
Data Engineer [Multiple Positions Available]
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
DESCRIPTION:
Duties: Identify analyze and interpret trends/patterns in complex data sets acquired from various data sources and extract transform and load data. Enforce guidelines to ensure consistency quality and completeness of data assets. Work on the migration of infrastructure data and applications out of legacy data centers into cloud and hybrid environments. Design and implement data pipelines. Develop and maintain data processing and transformation scripts. Build and maintain data warehouses and data lakes. Deliver work products on time and on budget while applying quality assurances best practices.
QUALIFICATIONS:
Minimum education and experience required: Masters degree in Management Information Systems Computer Science Data Analytics Business Analytics or related field of study plus 3 years of experience in the job offered or as Data Engineer Systems Engineer Software Engineer Data Analyst or related occupation.
Skills Required: This position requires experience with the following: data lifecycle to understand the data creation its movement between different stages and archival and purging based on the defined rule; data management functions including processing of data between various data layers using tools services and techniques including AWS services Spark SQL using EMR and various ide for development; real- time streaming data process to implement data pipelines; Relational and Non-Relational database querying skills to manipulate the data as defined by the business rule; Python and UNIX shell scripting to manage the data flow in the data pipeline; Big Data technology including Hadoop Hive and Cloud including AWS for creating data pipeline frameworks using programming languages including Python SQL and Spark; understanding and implementing the data lineage; creating verification and entitlement engine to make sure data is masked and protected; CI/CD including Jira and Jenkins; AWS infra provisioning using the TFE.
Job Location: 8181 Communications Parkway Plano TX 75024.
Full-Time.
Employment Type
Full-Time
