Senior Data Engineer
Share
Job Location:
New York City, NY - USA
Monthly Salary:
Not Disclosed
Posted on:
18 days ago
Vacancies:
1 Vacancy
Job Summary
Consensys is the leading blockchain and web3 software company founded by Joe Lubin CEO of Consensys and Co-Founder of Ethereum. Since 2014 Consensys has been at the forefront of innovation pioneering technological developments within the web3 ecosystem.
Through our product suite including the MetaMask platform Infura Linea Diligence and our NFT toolkit Phosphor we have become the trusted collaborator for users creators and developers on their path to build and belong in the world they want to see.
Whether building a dapp an NFT collection a portfolio or a better future the instinct to build is universal. Consensys inspires and champions the builder instinct in everyone by making web3 universally easy to use and develop on.
Our mission is to unlock the collaborative power of communities by making the decentralized web universally easy to access use and build on.
Youll get to work on the tools infrastructure and apps that scale these platforms to onboard one billion participants and 5 million developers. Youll be constantly exposed to new concepts ideas and frameworks from your peers and as you work on different projects challenging you to stay at the top of your game. Youll join a network of builders that reaches the edge of our ecosystem. Consensys alumni have moved on to become tech entrepreneurs CEOs and team leads at tech companies.
About the Data Team
The Data team sits within the Office of the COO and builds the infrastructure that powers analysis across the organisation. We design run and optimise pipelines that bring together product marketing and financial data at scale. Our role is to ensure data is reliable accessible and ready for usehelping Consensys move fast while maintaining quality and trust.
What youll do
- Design build and maintain robust data pipelines that integrate sources across the business.
- Collaborate closely with analysts business stakeholders and other engineering teams to gather requirements align timelines discuss architecture and ensure successful delivery.
- Document data pipelines best practices and processes to support onboarding and knowledge sharing within the team.
- Develop and optimize data models to deliver trusted structured and business-ready data.
- Ensure data quality security and governance are embedded in all pipelines and systems.
- Orchestrate and monitor pipeline execution to ensure reliability and scalability.
- Deploy and manage infrastructure as code.
- Build and tune big data pipelines using SQL Python and distributed processing frameworks.
- Work with cloud data warehouses to enable insights and analytics.
- Maintain and update reporting solutions and user dashboards.
- Automate workflows and improve CI/CD pipelines to reduce manual processes.
Would be great if you brought this to the role
- Over 6 years of experience as a Data Engineer
- Use Trusted Execution Environments (TEEs) to securely process sensitive user data ensuring it remains protected and only aggregated insights are released.
- Strong SQL skills and experience with cloud warehouses (e.g.: Snowflake BigQuery Redshift).
- Hands-on experience with transformation and orchestration tools (e.g.: dbt Airflow Dagster).
- Comfort with Python or other scripting languages for ETL and automation.
- Familiarity with data governance and metadata management (e.g.: DataHub).
- Experience deploying and managing infrastructure as code (e.g.: Terraform Pulumi).
- Exposure to data integration and ingestion tools (e.g.: Airbyte Segment).
- Experience with big data and distributed processing (e.g.: Apache Spark AWS EMR S3).
- Exposure to open source projects
- Experience maintaining and improving reporting solutions and dashboards (e.g.: Preset/Superset ).
- Familiarity with CI/CD practices and automation (e.g.: GitHub Actions).
- A collaborative mindset and eagerness to work with both technical and non-technical colleagues.
Dont meet all the requirements Dont sweat it. Were passionate about building a diverse team of humans and as such if you think youve got what it takes for our chaotic-but-fun remote-friendly start-up environmentapply anyway detailing your relevant transferable skills in your cover letter. While we have a pretty good idea of what we need were ready for you to challenge our thinking on who needs to be in this role.
It is a requirement of employment in this position that applicants will be required to submit to background checks including but not limited to employment education and criminal record checks. Further details will be provided to applicants that successfully meet the criteria for the position as determined by the company in its sole discretion. By submitting an application for employment you are acknowledging and consenting to this requirement.
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
Build next-generation apps, launch blockchain-based financial infrastructure, and access the decentralized web with ConsenSys' Ethereum product suite.
