Sr. SWE- ETL Developer (Pyspark) Contract Skillman
Share
Job Location:
New York City, NY - USA
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Role: Sr. SWE- ETL Developer (Pyspark) - Contract Skillman
Location: Hybrid NYC NY
MOI: Video
Make sure the template is filled out and the resume is under 4 pages. Also the education details must be on the resume. Which school(s) attended dates attended degree(s) achieved and major(s). The full address is needed not just the city and state as well as full date of birth with year included.
KEYS ARE:
-PYTHON: DEEP EXPERTISE IN TYPICAL DATA ENGINEERING USES NOT SWE USES
-ABILITY TO BUILD/DESIGN SCALABLE PIPELINE SOLUTIONS TO INCLUDE INGESTION AND MODELING
-ADVANCED SQL
-AIRFLOW OR SIMILAR SCHEDULING TOOL
-HADOOP OR SIMILAR
Description & Requirements:
The Risk Engineering team is looking for an experienced engineer to join our growing and diverse group. We build tools that help the company discover classify and keep track of data-especially sensitive and personal information. Our work helps protect this data supports privacy regulations like GDPR and CCPA and makes it easier for teams across the company to understand and use data addition to enabling data protection and privacy we are also building data inventory and risk analytics capabilities that provide deep visibility into data and support informed risk-aware decisions. We focus on building reliable systems that bring transparency and trust to how data is managed.
A key objective of this role is for you to help to build and support enterprise level data analytics programs leveraging traditional warehouse technologies Pyspark MPP databases and Hadoop.
Well trust you to:
- Join a fast-paced team of dedicated engineers who are committed to building risk analytics to drive enterprise-wide data governance and privacy.
- Shape the strategic and technological direction of the team
- Empower your career growth through exposure to cutting-edge tools processes and challenges
- Work closely with engineering and risk partners to deliver enterprise-scale impact
- Translate business needs into robust engineering solutions
In order to be successful:
- You should have a working knowledge of industry standard Data Infrastructure (e.g. Warehouse BI Analytics Big-Data etc.) tools with the goal of providing end users with analytics at the speed of thought.
- You should be proficient at developing architecting standardizing and supporting technology platforms using Industry leading ETL solutions.
- You should thrive in building scalable and high throughput systems
- You should have experience with agile BI & ETL practices to assist with the interim Data preparation for Data Discovery & self-service needs.
- You must have strong communication presentation problem-solving and trouble-shooting skills.
- You should be highly motivated to drive innovations company-wide.
Youll need to have:
- 5 years of experience in designing and developing ETL pipelines leveraging pyspark/python.
- Experience on python database libraries like SQL Alchemy psycopg2 .etc.
- Strong understanding of data warehousing methodologies ETL processing and dimensional data modeling.
- Advanced SQL capabilities are required. Knowledge of database design techniques and experience working with extremely large data volumes is a plus.
- Demonstrated experience and ability to work with business users to gather requirements and manage scope.
- Experience in workflow tools such as oozie or Airflow or Tidal.
- Experience working in a big data environment with technologies such as Greenplum Hadoop and HIVE.
- BA BS MS PhD in Computer Science Engineering or related technology field.
Wed love to see:
- Experience with large database and DW Implementation (20 TBs).
- Understanding of VLDB performance aspects such as table partitioning sharding table distribution and optimization techniques.
- Knowledge of reporting tools such as Qlik Sense Tableau Cognos.
AGENCY INTAKE FORM (Candidates needs to fill the below form)
- What are they currently working on Comments and the below.
- What type of development have they been working with If mixture provide % breakdown.
- Embedded
- What type of development have they been working with If mixture provide % breakdown.
-
-
- System / Machine / Kernel level
-
-
-
- Application (Backend)
-
-
-
- Application (Middle)
-
-
-
- Application (Front-end)
-
-
- Programming Languages
- What language are they currently using
- Programming Languages
-
-
- Preferred/Strongest Programming language
-
-
-
- Other languages & technologies that candidate feels confident in
-
- Are they focused on new development or maintenance
- How much exposure to databases do they have Which databases are they working with and at what level
- Whats motivating them to make a move
- Are they interested working on something finance related
- Are they active on the market now Any other interviews scheduled or offers
- What are they specifically looking to work on (More details than difficult problems...)
Key Skills
- SQL
- Pentaho
- PL/SQL
- Microsoft SQL Server
- SSIS
- Informatica
- Shell Scripting
- T Sql
- Teradata
- Data Modeling
- Data Warehouse
- Oracle
