Data Engineer (Weather Data)
Share
Job Location:
Mexico City - Mexico
Monthly Salary:
Not Disclosed
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Who We Are
Build. Scale. Sustain.
PALO IT is a global technology consultancy that crafts tech as a force for good. We design develop and scale digital and sustainable products and services to unlock value across the triple bottom line: people planet profit. We do the right thing and we do it right. Were proud to be a World Economic Forum New Champion and a B Corp-certified company.
We are small enough to care locally big enough to deliver globally (5 continents 18 offices 650 experts from 50 nationalities)
We are robust and resilient (100% independent and 0 debt)
We are entrepreneurs and passionate experts: we invest in what we believe genuinely and work as a collective intelligence
We are positive courageous caring doers and committed to excellence
About Gen-e2
While the market is still largely AI-augmenting delivery we have reinvented the SDLC to be AI First. Our approach is a game-changer in productivity and quality with a strong collaboration between generative AI and our best Talents:
We now generate 95% of the entire product code documentation infrastructure as code and even design with GitHub Copilot.
The quality consistently exceeds the output of our best traditional engineering teams.
A product repository houses all product artefacts giving AI full project context for higher-quality generation.
A library of rules and prompts defines coding standards design principles and security guidelines ensuring enterprise-grade quality and scalability.
With Gen-e2 we deliver end-to-end products 23 faster than traditional approaches while raising the bar for engineering excellence.
Your Role
Were looking for a Data Engineer to support our partner in managing the influx of weather-related datasets that are critical for their business transformation in the insurance domain. Youll be the go-to expert for ingesting organizing and preparing weather-based data pipelines that drive insights and automation.
Key Responsibilities:
- Design develop and manage scalable and secure data pipelines to ingest and process large volumes of weather data from various external sources.
- Leverage APIs batch and streaming ingestion mechanisms for real-time and historical weather datasets.
- Collaborate with data scientists solution architects and engineering teams to integrate weather data into products and machine learning workflows.
- Work with stakeholders to define data models and ensure weather data is cleaned structured and usable for downstream applications.
- Implement data quality checks validation routines and anomaly detection specific to weather information (e.g. temperature precipitation wind patterns).
- Optimize storage and compute costs for high-volume weather datasets ensuring data pipelines are cost-effective and resilient.
- Maintain documentation and metadata to provide transparency and governance over data usage.
Who You Are
- Proven experience working as a Data Engineer ideally within insurance climate tech or industries where weather data plays a pivotal role.
- Strong proficiency in Python SQL and at least one major cloud platform (AWS preferred; GCP or Azure also welcome).
- Experience working with weather data APIs such as NOAA OpenWeatherMap Weather Company Meteomatics or similar.
- Proficient in ETL tools data orchestration frameworks (Airflow Prefect Dagster) and streaming platforms (Kafka Kinesis).
- Familiar with geospatial data formats and processing tools (GeoJSON raster data GDAL PostGIS).
- Solid understanding of data lakes data warehouses (Snowflake BigQuery Redshift) and data modeling best practices.
- Strong problem-solving skills with attention to detail and a proactive mindset.
- B2-C1 level in English
More About PALO IT
Were eager to adapt to change learn from our experiences and move to meet our planets urgent needs. We are continuously taking action to:
Become a climate net-zero company
Deliver projects with a positive impact
Train 100% of our workforce on impact
Achieve B Corp certification among all our offices across the globe
Continuously measure & improve employee happiness
Our clients are amongst the worlds most successful companies. We innovate with both established Fortune 1000s SMEs and start-ups who aim to make an impact become global leaders and address the worlds most complex challenges.
What We Offer
Stimulating working environments
Unique career path
International mobility
Internal R&D projects
Knowledge sharing
Personalized training
Entrepreneurship & intrapreneurship opportunities
For more on our team culture and benefits
Check out our careers page
And our PALOCast with direct testimonies from our Palowans!
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
PALO IT is a global innovation consultancy and Agile software development company dedicated to helping organisations embrace tech as a force for good.
