Senior Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
Yesterday
Vacancies:
1 Vacancy
Job Summary
About This Role
Passionate about your field Looking for a work environment where you can meet challenges innovate use state-of-the-art tools and collaborate with experts from various sectors Want to join a worldclass company and industry leader and be proud to contribute to our projects and mission Come join our team and share your passion with us!
Key Responsibilities:
We are seeking a highly skilled and motivated Data Platform Engineer to join our team. This role involves administering and enhancing our enterprise data platform including Data Warehouses (DWH) and Data Lakes ensuring optimal performance and scalability. The ideal candidate will have extensive experience with Terraform and a strong background in various data technologies.
Responsibilities:
Administer and enhance the enterprise data platform including Data Warehouses (DWH) and Data Lakes ensuring optimal performance and scalability.
Design implement and maintain robust and reliable data pipeline architectureto meet business needs utilizing SQL and big data technologies such as Hadoop Spark and Kafka.
Establish and oversee infrastructure for efficient extraction transformation and loading (ETL) of datafrom diverse sources ensuring adherence to best practices and standards.
Implement DevOps practices for continuous integration and continuous deployment (CICD) of data systems enhancing agility and efficiency.
Integrate and manage data from multiple sources including external and IoT data ensuring data is a valuable enterprise asset.
Create and maintain backend data solutions for analysts and data scientists facilitating insightful data analysis and decision-making.
Identify and implement process enhancements such as automating manual tasks and optimizing data delivery to improve efficiency and scalability.
Partner with stakeholders including product teams data experts and architects to resolve technical issues and support data infrastructure needs.
Ensure compliance with data architecture and security standards maintaining data protection and privacy.
Set up observability and monitoring systemsto track data pipeline reliability and provide prompt operational support in case of incidents.
Work with executive CX design and development teamsto support data-related technical needs and propose innovative ideas for data and AI products.
Inspire and motivate colleaguesby sharing knowledge contributing to a collaborative work environment and promoting an Agile mindset.
Stay connected with industry trends and advancements suggesting innovative ideas and improvements for data strategies and delivery processes.
Qualifications:
Bachelors degree in Computer Science Engineering or related field. A Masters is a plus.
A minimum of 6 years industry experience working with data coding and scripting (Python Java Scala SQL JavaScript Bash) design and testing.
Solid knowledge of CS fundamentals in algorithms and data structures.
Strong experience with Terraform.
Experience supporting and working with cross-functional teams in a dynamic environment.
Experience with big data tools: Hadoop Spark Kafka.
Experience with relational SQL and NoSQL databases including SQL Server PostgreSQL and CosmosDB.
Experience with automated data pipeline and workflow management tools: DevOps ARM Azure Data Factory Airflow.
Experience with Microsoft cloud services: Azure Databricks Fabric Purview
Experience with stream-processing systems: Storm Spark-Streaming.
Familiarity with Kubernetes for container orchestration.
Proficiency in Power BI for data visualization and reporting.
Position Type
RegularCAE thanks all applicants for their interest. However only those whose background and experience match the requirements of the role will be contacted.
Equal Opportunity Employer
CAE is an equal opportunity employer committed to providing equal employment opportunities to all applicants and employees without regard to race nationality colour religion sex gender indentity and expression sexual orientation disability neurodiversity veteran status age or other characteristics protected by local laws.
If you dont see yourself fully reflected in every job requirement listed in the job posting we still encourage you to reach out and apply. At CAE everyone is welcome to contribute to our success. Applicants needing reasonable accommodations should contact their recruiter at any point in the recruitment process. If you need assistance to submit your application because of incompatible assistive technology or a disability please contact us at
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
CAE's Defense and Security business unit focuses on helping prepare military customers to develop and maintain the highest levels of mission readiness. With nearly 5,000 employees, CAE USA Inc. is the largest segment within CAE's Defense and Security business unit. The organization le ... View more
