Senior Data Engineer

Your responsibilities: 

• Design and implement data integration procedures and pipelines that extract data from various sources transform it into the desired format and load it into the appropriate modern analytics data storage and management systems. Integrates data from-to different internal and external sources (batch incremental streaming). 

• Adoption and drive of active metadata usage in data integration processes with high level of automation and simplicity. You will be responsible for using innovative and modern tools techniques and architectures to automate the most-common repeatable and tedious data preparation and integration tasks partially or completely. 

• Collaborates with analytics owners (business analysts project finance analysts domain owners and SMEs) to optimize data products in domain of data and business intelligence responsibility. 

• Improving data quality and governance with business data owners. 

• Educate and train counterparts such in these data pipelining and preparation techniques which make it easier for them to integrate and consume the data they need for their own use cases. 

• Ensures data consistency and integrity during the integration process identifying root cause of quality issues address them and work with technical system owners to identify and implement optimal solution.  

• Optimizes data pipelines and data processing workflows for performance scalability and efficiency. 

• Monitors and tunes data analytics systems identifies and resolves performance bottlenecks and implements caching and indexing strategies to enhance query performance. 

• Implements data quality checks and validations (business rules) within data pipelines to ensure the accuracy consistency and completeness of data. 

• Takes authority responsibility and accountability for exploiting the value of enterprise information assets and of the analytics used to render insights for decision making automated decisions and augmentation of human performance. 

• Establishes the governance of data and algorithms used for analysis analytical applications and automated decision making. 

Qualifications :

Skills 

• Strong experience with various Data Management architectures like data warehouse data lake LakeHouse architecture Data Fabric vs Data Mesh concepts and the supporting processes like data Integration MPP engines governance metadata management. 

• Intermediate experience in Apache technologies such as Spark Kafka and Airflow to build scalable and efficient data pipelines.  

• Strong experience to design build and deploy data solutions that capture explore transform and utilize data to create data products and support data informed initiatives. Proficiency in ETL/ELT data replication/CDC message-oriented data movement API design and access and upcoming data ingestion and integration technologies such as stream data integration and data virtualization. 

• Basic knowledge and ability in data science languages/tools such as R Python TensorFlow Databricks Dataiku Knime SAS or others. 

• Proficiency in the design and implementation of modern data architectures and concepts such as cloud services (i.e. AWS OCI Azure GCP) and modern data warehouse tools (Snowflake Databricks etc) 

• Strong experience with database technologies such as SQL NoSQL PostgreSQL Oracle Hadoop Teradata etc. 

• Intermediate experience working with popular data discovery analytics and BI software tools like PowerBI Tableau Qlik Sense Looker ThoughtSpot MicroStrategy or others for semantic-layer-based data discovery is advantage. 

• Expert problem-solving skills including debugging skills allowing the determination of sources of issues in unfamiliar code or systems and the ability to recognize and solve repetitive problems. 

Soft skills and characteristics 

• Strong experience supporting and working with cross-functional teams in a dynamic business environment. 

• An ideal candidate would be expected to collaborate with both the business and IT teams to define the business problem refine the requirements and design and develop data deliverables accordingly. The successful candidate will also be required to have regular discussions with data consumers on optimally refining the data pipelines developed in nonproduction environments and deploying them in production. 

• Ideal candidate is a confident energetic self-starter with strong interpersonal skills. 

• Has good judgment a sense of urgency and has demonstrated commitment to high standards of ethics regulatory compliance customer service and business integrity. 

• Good business acumen and interpersonal skills; able to work across business lines at a senior level to influence and effect change to achieve common goals. 

• Ability to describe business use cases/outcomes data sources and management concepts and analytical approaches/options. 

• Willingness to learn and grow. 

• Advanced in English (both spoken and written). 

Additional Information :

We prioritize diversity equity and inclusion by creating an equal opportunities workplace and a human-centric environment where people of all cultural backgrounds genders and ages can contribute and grow.  

To succeed we must work together with a human first approach. Why because our people are our greatest strength leading to our continued success on improving the lives of those around us. 

We offer: 

• Training and career development opportunities internally  
• Strong emphasis on personal and professional growth 
• Friendly supportive working environment 
• Opportunity to work with colleagues based all over the world with English as the company language 

Our core values are key to how we operate and if you feel they resonate with you then PrimeVigilance could be a great company to join!  

• Quality 
• Integrity & Trust  
• Drive & Passion  
• Agility & Responsiveness  
• Belonging 
• Collaborative Partnerships  

We look forward to welcoming your application. 

Remote Work :

No

Employment Type :

Full-time