Data Engineer Mid Level

We are seeking a skilled Mid level Data Engineer responsible for designing developing and maintaining scalable data pipelines and data platforms. The role focuses on building reliable data ingestion and transformation processes using modern data engineering tools and frameworks. The ideal candidate will have strong experience with Python-based data engineering SQL and the Databricks platform along with a solid understanding of data architecture and data modeling. This role will collaborate closely with analytics platform and business teams to ensure high-quality well-structured data is available for reporting analytics and advanced data use cases.

Key Responsibilities

• Design develop and maintain scalable data pipelines for ingesting transforming and processing large volumes of data.
• Build and manage ELT/ETL pipelines using Python and modern data engineering frameworks.
• Utilize the Databricks platform to process and manage distributed data workloads.
• Develop and optimize Python-based data transformation logic using libraries such as Pandas Polars and Delta Live Tables (DLT).
• Implement and maintain data transformations using dbt and SQL.
• Design and manage data schemas and data models to support analytical and operational workloads.
• Build and maintain data ingestion pipelines integrating with various ELT tools such as cData Hevo MuleSoft or Fivetran.
• Monitor troubleshoot and resolve issues in data pipelines and workflows to ensure reliability and performance.
• Apply data architecture principles including data lake structures and medallion architecture (Bronze Silver Gold layers).
• Optimize data processing and transformation logic for performance scalability and maintainability.
• Document data pipelines transformations and data models to ensure transparency and knowledge sharing.
• Collaborate with cross-functional teams to understand data requirements and deliver reliable data solutions.
• Use version control systems (Git) and follow engineering best practices for collaborative development.
• Participate in Agile/Scrum ceremonies and contribute to continuous improvement of engineering processes.

Essential Qualifications

• Strong experience with the Databricks platform for building and managing data workloads.
• Proficiency in Python for data engineering including libraries such as Pandas Polars and DLT.
• Strong understanding of ELT/ETL concepts and experience building end-to-end data ingestion pipelines.
• Hands-on experience building Python-based ELT pipelines.
• Strong knowledge of data schema design and data modeling concepts.
• Experience writing efficient and scalable data transformation logic.
• Ability to debug and troubleshoot data pipelines effectively.
• Good understanding of distributed computing and storage concepts.
• Experience using dbt for data transformations.
• Knowledge of data lake architecture and medallion architecture.
• Experience with pipeline orchestration monitoring and troubleshooting.
• Familiarity with Git-based version control systems.
• Experience working in Agile development environments.

Nice to Have Skills

• Experience with ELT tools such as cData Hevo MuleSoft Fivetran or similar tools.
• Exposure to Salesforce or NetSuite data schemas.
• Knowledge of vector databases or vector storage platforms such as Weaviate.
• Understanding of embeddings and vector-based data concepts.
• Experience with LLM tooling such as Ollama or OpenAI APIs.
• Familiarity with data validation frameworks such as Great Expectations.
• Agile or Scrum Master certification.

Soft Skills

• Strong analytical and problem-solving skills.
• Ability to work collaboratively with cross-functional teams.
• Good communication and documentation skills.
• Strong attention to detail and focus on data quality.
• Ability to manage multiple priorities in a fast-paced environment.
• Proactive mindset with a focus on continuous improvement and learning.

Benefits

• Comprehensive benefits package including health insurance paid time off and professional development opportunities.

About Potentiam

Potentiam is a global provider of highly qualified professionals to European SMEs from our offices in Romania South Africa and works with clients in finance energy leisure marketing business services and technology industries providing technical professional multi- lingual highly motivated staff most of whom have had experience of working for international companies. Staff cover a wide range of roles from accounting marketing data management HR sales/account management engineering technology and operations. Potentiam manages our staffs career development and personal development training all infrastructure HR and payroll with our clients directly managing day-to-day staff responsibilities and role training and development.

If interested please apply here if you have any questions regarding the role please feel free to write to

Data Privacy Notice

The personal information you provide during the application and recruitment process will be used solely for recruitment purposes in accordance with our data protection policies.

For any questions regarding data processing related to HR activities please contactat

All data shared with third parties complies with applicable confidentiality and retention requirements.

Required Experience:

Manager