Senior Data Engineer (Go)
Share
Job Location:
Monthly Salary:
Posted on:
21 hours ago
Vacancies:
1 Vacancy
Job Summary
We are a leading European AI company developing large language models and generative platforms for enterprise and government clients.
Our products combine high-performance technologies data security and transparency fully aligned with European regulatory and ethical standards.
As a Data Engineer you will design build and maintain high-performance backend services and data pipelines enabling our teams to deliver scalable reliable and production-ready systems.
Responsibilities:
- ETL Data Pipelines (Batch & Streaming)
- Develop and operate ETL pipelines to extract transform and load data from multiple sources
- Support both batch workloads (large-scale periodic data processing) and streaming workloads (real-time or near-real-time data flows)
- Optimize performance scalability and reliability of data processing pipelines
- Collaborate with data engineers and analysts to ensure high-quality clean and accessible datasets
- ML Data Pipelines using Temporal
- Design implement and maintain robust ML data pipelines for training validation and inference of machine learning models
- Use Temporal for workflow orchestration ensuring reliability retries and state management across complex ML workflows
- Collaborate closely with ML engineers and researchers to automate and scale model pipelines
- Ensure pipelines are reproducible maintainable and observable
- Backend Services & Infrastructure
- Design build and maintain backend services in Go
- Work with PostgreSQL and object storage (S3) to store and manage structured and unstructured data
- Deploy and manage services using Kubernetes (K8s) and Helm
- Implement best practices in CI/CD using GitHub Actions
- Apply system design and data modeling principles handling concurrency and performance optimization
Requirements:
- 3 years of commercial experience in Go (most of which in a production setup with real customers)
- Strong knowledge of ETL pipeline development (batch and streaming workloads)
- Experience with Temporal or other asynchronous workflow orchestration tools
- Experience with PostgreSQL and object storage (S3)
- Familiarity with Kubernetes (K8s) and Helm
- Understanding of concurrency patterns and performance optimization in Go
- Experience building and operating ML data pipelines is highly desirable
- Strong collaboration skills and attention to detail
Nice to Have:
- Experience designing APIs/SDKs
- Experience with complex migrations or data model changes
- Knowledge of TDD DDD or other development best practices
- Familiarity with resiliency patterns (retries circuit breakers)
- Experience integrating backend systems with ML models
- Experience with OpenFGA or similar tools
- Familiarity with stakeholder management and brief concise communication
Technology Stack:
- Core Backend: Python (FastAPI) Go
- Data Storage: PostgreSQL S3 / object storage
- Workflow Orchestration: Temporal (for ML pipelines)
- ETL: Batch and streaming pipelines
- DevOps & Infrastructure: Kubernetes (K8s) Helm GitHub Actions
- Internal Tools: TypeScript / Nx for CLI automation (if applicable)
We offer*:
- Flexible working format - remote office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program tech talks and trainings centers of excellence and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Required Experience:
Senior IC
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
About Company
N-iX is a global software development company that helps world’s leading organizations achieve lasting business value using advanced technology.
