Data Engineer
Share
Job Location:
Monthly Salary:
Posted on:
30+ days ago
Vacancies:
1 Vacancy
Job Summary
Job Summary for SparkCognition - Data Engineer:
- Design build and maintain scalable data ETL and MLOps pipelines for AI-driven SaaS platforms in the renewable energy sector.
- Collect and ingest large volumes of data from diverse sources such as wind solar and storage sites into cloud-based systems.
- Engineer test and optimize big data architectures and data management systems to support asset performance management and prescriptive analytics.
- Collaborate with multidisciplinary teams to deliver insights and solutions for renewable energy power plants.
- Utilize Python and big data technologies (e.g. Hadoop Spark Kafka) to process and manage large disconnected datasets.
- Work with relational databases (e.g. Postgres) and leverage Elasticsearch as needed.
- Preferably apply experience in workflow orchestration (Airflow Prefect) and cloud platforms (AWS GCP).
- Support and drive clean energy initiatives through innovative data engineering solutions.
- Design build and maintain scalable data ETL and MLOps pipelines for AI-driven SaaS platforms in the renewable energy sector.
- Collect and ingest large volumes of data from diverse sources such as wind solar and storage sites into cloud-based systems.
- Engineer test and optimize big data architectures and data management systems to support asset performance management and prescriptive analytics.
- Collaborate with multidisciplinary teams to deliver insights and solutions for renewable energy power plants.
- Utilize Python and big data technologies (e.g. Hadoop Spark Kafka) to process and manage large disconnected datasets.
- Work with relational databases (e.g. Postgres) and leverage Elasticsearch as needed.
- Preferably apply experience in workflow orchestration (Airflow Prefect) and cloud platforms (AWS GCP).
- Support and drive clean energy initiatives through innovative data engineering solutions.
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
