Java Tech lead ( Nifi, Airflow)
Share
Job Location:
Phoenix, NM - USA
Monthly Salary:
Not Disclosed
Posted on:
2 hours ago
Vacancies:
1 Vacancy
Job Summary
ROLE: Java Tech lead ( Nifi Airflow)
Description:
Strong NiFi Expertise: Deep understanding of Apache NiFi architecture processors controllers and data flow management capabilities.
Programming Languages: Proficiency in Java Spring boot & API Microservices.
Database and SQL: Familiarity with databases SQL and data manipulation techniques.
Big Data Technologies: Kafka ( Event driven & streaming architecture)
Apachae Airflow - Workflow orchestration
Design and Implementation:
o Creating and implementing data flow solutions using NiFi including defining data sources destinations and transformation processes.
Ingestion and Transformation:
o Managing data ingestion from various sources (e.g. files databases APIs) and transforming data into desired formats for downstream consumption.
Routing and Enrichment:
o Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g. timestamps location).
Scalability and Performance:
o Ensuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Management:
o Maintaining and improving NiFi clusters including configuration optimization and troubleshooting.
Microservices Integration:
o Developing and integrating microservices to support data conditioning format validation and transformation processes.
Troubleshooting and Issue Resolution:
o Identifying and resolving issues related to data flows NiFi configurations and data integration processes
Airflow Expertise:
o Possessing deep knowledge of Airflows architecture including schedulers executors (Celery Kubernetes) and plugin development.
Workflow Design and Development:
o Designing and developing complex modular and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines.
Performance Optimization:
o Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Description:
Strong NiFi Expertise: Deep understanding of Apache NiFi architecture processors controllers and data flow management capabilities.
Programming Languages: Proficiency in Java Spring boot & API Microservices.
Database and SQL: Familiarity with databases SQL and data manipulation techniques.
Big Data Technologies: Kafka ( Event driven & streaming architecture)
Apachae Airflow - Workflow orchestration
Design and Implementation:
o Creating and implementing data flow solutions using NiFi including defining data sources destinations and transformation processes.
Ingestion and Transformation:
o Managing data ingestion from various sources (e.g. files databases APIs) and transforming data into desired formats for downstream consumption.
Routing and Enrichment:
o Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g. timestamps location).
Scalability and Performance:
o Ensuring data flows are scalable and performant to handle varying data volumes and velocity.
NiFi Cluster Management:
o Maintaining and improving NiFi clusters including configuration optimization and troubleshooting.
Microservices Integration:
o Developing and integrating microservices to support data conditioning format validation and transformation processes.
Troubleshooting and Issue Resolution:
o Identifying and resolving issues related to data flows NiFi configurations and data integration processes
Airflow Expertise:
o Possessing deep knowledge of Airflows architecture including schedulers executors (Celery Kubernetes) and plugin development.
Workflow Design and Development:
o Designing and developing complex modular and reusable DAGs (Directed Acyclic Graphs) to automate data pipelines.
Performance Optimization:
o Identifying and addressing performance bottlenecks in Airflow environments and implementing best practices for orchestration and scheduling
Key Skills
- Corporate Accounting
- Facilities Management
- Account Development
- Accountancy
- Jdbc
- ITI
