Data processing is an essential part of national security in the modern world. As an engineer in our Big Data Platform team you will build and own a solid data platform as the foundation to support critical national security missions. You will push the limits of technology to ensure that the whole big data pipeline runs optimally and smoothly from ingestion to processing to persistence. You are encouraged to suggest the best way to deliver value and the technologies used. Join the awesome CSIT family and use cutting-edge technologies to protect the nation.
Responsibilities
Design implement and maintain real-time high-volume data processing querying and persistence solutions using Big Data technologies
Design and implement highly available resilient scalable and sustainable Big Data platforms
Administer (configuration backup and recovery) Big Data platform services running across multiple clusters
Deploy and manage the cluster efficiently using DevOps and automation tools
Build performance and health monitoring tools to ensure optimal system performance and availability
Work closely with different teams to ensure smooth onboarding and operations of the Big Data platform
Conduct resource planning and scaling exercises to support increasing workload
Conduct research and evaluations of new technologies
Innovate and improve the current system
Requirements (Minimum Qualifications)
Bachelors degree in Computer Engineering Computer Science or equivalent experience
Able to communicate clearly
Positive attitude and excellent teamwork
Passionate about big data technologies
Love for open source technologies
Additional qualifications for senior position:
At least 5 years of experience in big data domain
Deep expertise in big data platform
Experience in leading engineering teams
Technical leadership and mentoring/coaching experience
Nice-to-haves
Experience in distributed systems
Experience in supporting search workloads in multi-terabytes and billions of records
Experience in managing physical and virtual infrastructures
Experience in using and managing Hadoop cluster services (Spark Kafka HBase Hive Hadoop HDFS Nifi Kudu Solr ElasticSearch)
Experience in troubleshooting and resolving issues of the Big Data stack ranging from hardware networking OS to software
Experience with cluster management and DevOps tools (e.g. Ansible Kubernetes)
Experience working with application or product teams
Experience working in an agile and innovative environment
Why join us
The work is purposeful and meaningful
You will work with the best engineers
We work with modern technologies and tech stacks
We have excellent engineering culture and work-life balance
We aspire to engineering and operational excellence
We empower to innovate
We grow together as a family
As CSIT is an agency under the Ministry of Defence (Singapore) only Singapore Citizens will be considered.
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.