Role: Senior Data Engineer
Location: Chicago IL - Remote
Mandatory skills:
Python PySpark AWS
11 years of experienced
Responsibilities:
- Proficient developer in multiple languages Python is a must with the ability to quickly learn new ones.
- Expertise in SQL (complex queries relational databases preferably PostgreSQL and NoSQL database - Redis and Elasticsearch).
- Extensive big data experience including EMR Spark Kafka/ Kinesis and optimizing data pipelines architectures and datasets.
- AWS expert with hands-on experience in Lambda Glue Athena Kinesis IAM EMR/PySpark Docker
- Proficient in CI/CD development using Git Terraform and agile methodologies.
- Comfortable with stream-processing systems (Storm Spark-Streaming) and workflow management tools (Airflow).
- Exposure to knowledge graph technologies (Graph DB OWL SPARQL) is a plus. .