Senior Data Engineer
Share
Job Location:
San Francisco, CA - USA
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Earnest LLC Oakland CA
Title: Senior Data Engineer
Duties:
- The Senior Data Engineer (Multiple Positions Open) at Earnest LLC in Oakland CA will design and implement tooling and abstraction patterns for orchestrating transformations in Extract Load and Transform (ELT) processes by utilizing technologies such as Airflow and DBT.
- Develop CI/CD pipelines troubleshoot and identify and resolve defects through standard techniques such as debugging profiling logging or log analysis.
- Design and implement integrations with third-party RESTful APIs to retrieve data and ingest it into the data warehouse thereby enabling downstream analytical workflows and business intelligence initiatives.
- Develop ETL/ELT pipelines for ingesting data from internal and external services into the Data Lake and Data Warehouse along with setting up ELT monitoring and alerting solutions to ensure system reliability and performance.
- Establish and maintain data infrastructure using Ansible and Terraform provide technical leadership by guiding the team in selecting appropriate technologies tools and languages to address both technical andbusiness challenges.
- Collaborate with analytics engineers data analysts and data scientists to support building and optimizing data pipelines for analytics and reporting.
- Cooperate with senior leadership to strategize and implement sustainable data architecture initiatives as well as facilitating cross-functional requirements gathering sessions with engineering product management analytics and data science teams to understand and fulfill data needs.
- Create and maintain detailed documentation for data engineering processes including data ingestion transformation and storage to ensure clarity and transparency within the team and among stakeholders.
- Document break down and estimate project planning in JIRA provide estimates for project completion and communicate acceptance criteria for project completion.
- Train other data engineers on ETL/ELT tooling coach engineers on best practices for data exposure and storage in production and provide feedback on coding and architecture via GitHub or through pair programming.
Position is 100% remote.
Requirements:
- Bachelors degree in Computer Engineering Information Systems or a closely
related field plus 3 years of software development experience. - The 3 years of experience must include 3 years of experience with each of the following:
- (1) building and managing data pipelines data warehouses and backend services;
- (2) Python;
- (3) SQL;
- (4) OLTP databases including Postgresql or MySQL;
- (5) OLAP databases including Snowflake or BigQuery or Redshift;
- (6) AWS cloud technologies;
- (7) orchestration tools including Airflow Dagster or Prefect;
- (8) container technologies including Docker or Kubernetes;
- (9) server-side concepts;
- (10) building and maintaining REST APIs or Graphql for data transfer;
- (11) real-time data streaming using Kafka or Spark;
- (12) automating tasks with bash scripting;
- (13) data infrastructure provisioning using Terraform; and
- (14) code version control using Git.
This notice is subject to Earnest LLCs employee referral program.
Interested candidates can apply online at or send a resume to
and reference job code 063.
Required Experience:
Senior IC
Key Skills
About Company
Earnest offers low-interest loans based on your unique financial profile, including student loan refinancing, private student loans, and personal loans.
