Data Engineer
Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Job Title: Data Engineer
Location: Plano TX
WESTLEY RESOURCE
--- Job Summary: Westley Resource is seeking a skilled Data Engineer to design build and maintain robust data pipelines and scalable data infrastructure supporting mission-critical AI and personalization services. As a key member of our data engineering team you will leverage your expertise in Python Spark PySpark SQL and Databricks to enable real-time data processing and actionable insights across enterprise systems.
--- Key Responsibilities:
- Develop optimize and maintain scalable ETL pipelines for ingesting transforming and delivering large-scale structured and unstructured data.
- Implement real-time data streaming solutions using Spark Structured Streaming and PySpark.
- Collaborate with data scientists architects and application teams to support data-driven initiatives ensuring reliable data access and performance.
- Build and automate data workflows using Databricks integrating various data sources and APIs. - Design and manage data models and storage solutions for analytics and machine learning use cases.
- Ensure high data quality consistency and integrity through monitoring validation and automated data quality checks. - Develop RESTful APIs for data access integration and delivery (as required).
- Troubleshoot and resolve data pipeline issues optimizing for performance and scalability.
- Document data flows architecture and best practices for cross-functional collaboration.
--- Required Skills:
- Proficiency in Python for data engineering scripting and API development.
- Hands-on experience with Apache Spark and PySpark including Structured Streaming for real-time data processing. - Strong knowledge of SQL for data manipulation transformation and query optimization.
- Experience with Databricks for building and managing collaborative data workflows.
- Experience with RESTful API development using Python (required for specific positions).
- Understanding of data architecture ETL concepts and best practices for large-scale distributed data systems.
- Experience integrating with cloud-based data platforms (Azure Data Factory Synapse or similar is a plus).
- Strong analytical troubleshooting and problem-solving skills.
--- Preferred Qualifications:
- Bachelor s or Master s degree in Computer Science Engineering or related field.
- Experience with additional programming languages (Java Scala) for data ingestion.
- Familiarity with cloud platforms (Azure preferred) and DevOps tools.
- Experience working in Agile or Kanban environments.
- Excellent communication and collaboration skills.
--- Why Westley Resource
- Work in a dynamic technology-driven environment on innovative data and AI solutions.
- Be part of a collaborative team enabling AI-powered personalization and next-generation customer experiences.
- Access to cutting-edge tools career growth opportunities and competitive benefits.
--- To Apply: Submit your resume and a cover letter detailing your experience with the technologies listed above. Join us at Westley Resource to shape the future of data engineering!
--- Westley Resource is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race color religion sex sexual orientation gender identity national origin or protected veteran status and will not be discriminated against on the basis of disability.
Employment Type
Full-time
