Senior Data Engineer
Senior Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Not Disclosed
Salary Not Disclosed
1 Vacancy
Job Description
SUMMARY:
Design develop and maintain data pipelines that ingest transform and load data from various sources into data lakes data warehouses and data marts using Python Spark and other cloud services to serve analytics needs and machine learning model requirements. Contribute to the detailed design and architecture of the data platform ensuring consistency efficiency and reusability of data components and processes. Perform data cleansing and validation using SQL/PySpark and other tools and frameworks to remove or correct erroneous incomplete or inconsistent data. Apply data transformations and implement business logic using SOL Python and Spark to enhance enrich or standardize data. Handle large-scale and complex data sets using distributed systems and parallel computing to improve performance of the pipelines. Monitor troubleshoot and optimize the performance and cost of data pipelines and data cleansing and transformation solutions using diagnostic tools and metrics such as CloudWatch Datadog. Research and evaluate tools and technologies to improve the data platform capabilities. Implement security and compliance standards and policies across data pipelines and data sets using techniques like data masking encryption tokenization and other techniques to hide personally identifiable information (PII) data and be compliant with CCPA GDPR. Collaborate with data architects data analysts data scientists and business stakeholders. Deploy cloud infrastructure applicable for data pipelines using terraform or cloud formation templates. Remote work permitted.
QUALIFICATIONS:
The position requires a Masters degree in Computer Science Computer Engineering or a related field of study and three (3) years of experience in the job offered or in a software and/or data engineering role. Employer will accept a Bachelors degree in Computer Science Computer Engineering or a related field of study and five (5) years of experience in the job offered or in a software and/or data engineering role. The position requires a minimum of three (3) years of experience with the following skills and technologies: Python and SOL; building data models and data warehousing with Redshift/Snowflake/Big Query; optimizing queries and data pipelines; working with distributed systems using Apache Spark/AWS Glue/Databricks; working with public cloud offerings like AWS Azure or GCP and their data/analytical services; deploying cloud resources such as virtual machines storage networks databases; experience with terraform/CloudFormation templates. The position requires experience with the following skills: data warehousing/data platform principles such as SCDs data partitioning data retention data lineage; implementing automated data quality checks for tracking and reporting data anomalies; building real time pipelines using message streams like Kinesis/Kafka/Data streams; working with message queues like SOS/Java MQs; implementing data security and compliance standards (CCPA/GDPR) data encryption/obfuscation.
Remote work permitted.
Hours: Monday Friday 8:00 am to 5:00 pm 40 Hours/ Week
Salary: $188870 / year
Salary and Benefits:
RealPage provides a competitive salary package along with a comprehensive benefit plan that includes:
- Health dental and vision insurance.
- Retirement savings plan with company match.
- Paid time off and holidays.
- Professional development opportunities.
- Performance-based bonus based on position.
Compensation may vary depending on your location qualifications including job-related education training experience licensure and certification that could result at a level outside of these ranges. Certain roles are eligible for additional rewards including annual bonus and sales incentives depending on the terms of the applicable plan and role as well as individual performance.
Equal Opportunity Employer: RealPage Company is an equal opportunity employer and committed to creating an inclusive environment for all employees.
Required Experience:
Senior IC
Employment Type
Full-Time
