Lead Data Engineer
Lead Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Adtalem is a data driven organization. The Data Engineering team builds data solutions that powers strategic and tactical business decisions and supports the Analytics and Artificial Intelligence operations. By implementing the data platform data pipelines and data governance policies this team provides the basis for decisionmaking in Adtalem. Adtalem is looking for a Lead Data Engineer who is a technical powerhouse who can scale our data platform to meet ever growing business needs.
Responsibilities
- Collaborate with various business IT Analytics and Data Science groups to lead business requirement discussion and translate them into scalable high performance data solutions
- Design for all things Data Platform from data modeling to data ingestion pipelines transformation processes the storage optimization delivery and discoverability of high quality datasets
- Collaborate in producing data platform architecture development standards and best practices documenting and socializing with wider group in Data Engineering
- Develop data ingestion processes using available tools and technologies complex SQLs/programs to transform to target data model build test plan execute testing and deploy across environments
- Optimize data and processing pipelines for scalability reliability and faulttolerance by using data volume velocity and variety metrics
- Make decisions in complex and multifaceted data engineering situations requiring understanding of software package options and programming language and compliance requirements that influence and lead Technology to meet deliverables and drive organizational change
- Design CI/CD pipelines to ensure continuous deployment and delivery of our data products.
- Ensure quality of critical data elements prepare data quality remediation plans and collaborate with business and system owners to fix the quality issues at its root.
- Create design documents Source to Target Mapping documents and any supporting documents needed for deployment/migration. Capture store and socialize data lineage and operational metadata
- Ensure that the data pipelines are stable secure maintainable and highly available. Support team during troubleshooting & debugging defects & bug fixes business requests environment migrations & other ad hoc requests
- Work with business and technology stakeholders to communicate EDW incidents/problems and manage their expectations
- Develop REST APIs to expose data to other teams within the company.
- Mentor and provide technical guidance to junior data engineers promoting knowledge sharing across the team
- Performs other duties as assigned
- Complies with all policies and standards
Qualifications :
- Bachelors Degree Computer Science Computer Engineering Software Engineering or other related technical field.
- Masters Degree Computer Science Computer Engineering Software Engineering or other related technical field.
- 10 years experience in data engineering solutions such as data platforms ingestion data management or publication/analytics.
- 4 years experience in Google cloud with services like BigQuery Composer GCS DataStream Dataflows.
- Should have expert level understanding of Data Architecture Data Warehousing Cloud Data Platform ETL (FiveTran) Data Quality Data Mapping etc.
- Hands on experience working in GCP services like Big Query Cloud Composer Cloud Storage (GCS) cloud function cloud dataflow Pub/sub Cloud Shell GSUTIL Big Query Data Proc
- Experience working with Airflow as workflow management tools and build operators to connect extract and ingest data as needed
- Expert knowledge on SQL and Python programming. Experience in tuning queries for performance and scalability
- Experience in Real Time Data ingestion using GCP PubSub Kafka Spark or similar
- Should be selfsufficient and able to independently design and develop the data model data processing pipelines and CI/CD processes based on the established standards and methodology
- Should have working experience on any DevOps tools like GitHub Jenkins etc.
- Have proven experience leading incremental following the agile methodology and sprint planning
- Excellent problemsolving and criticalthinking skills to recognize and comprehend complex data issues affecting the business environment
- Think strategically. Excellent organizational prioritization and analytical abilities
Additional Information :
In support of the pay transparency laws enacted across the country the expected salary range for this position is between $96404.1 and $169021.85. Actual pay will be adjusted based on jobrelated factors permitted by law such as experience and training; geographic location; licensure and certifications; market factors; departmental budgets; and responsibility. Our Talent Acquisition Team will be happy to answer any questions you may have and we look forward to learning more about your salary requirements. The position qualifies for the below benefits.
Adtalem offers a robust suite of benefits including:
- Health dental vision life and disability insurance
- 401k Retirement Program 6 employer match
- Participation in Adtalems Flexible Time Off (FTO) Policy
- 12 Paid Holidays
For more information related to our benefits please visit: are also eligible to participate in an annual incentive program subject to the rules governing the program whereby an award if any depends on various factors including without limitation individual and organizational performance.
Equal Opportunity Minority / Female / Disability / V / Gender Identity / Sexual Orientation
Remote Work :
No
Employment Type :
Fulltime
Employment Type
Full-time
