Over 9 years of professional IT experience with expertise in Hadoop and Big data ecosystem related technologies.
Over 9 years of professional IT experience with expertise in Hadoop and Big data ecosystem related technologies.
• 6 years of exclusive experience in Big Data technologies and Hadoop ecosystem components like Spark, MapReduce, Hive, YARN, HDFS, NoSQL (Cassandra, Redis, MongoDB, HBase, ArrangoDB), Elasticsearch (ELK Stack), Kafka, NiFi, Oozie, Sqoop, Flume.
• Strong Knowledge on Architecture of Distributed systems and Parallel processing frameworks.
• In-depth understanding of MapReduce Framework and Spark execution model.
• Experience on creating and optimizing ETL workflows.
• Strong experience working with both batch and real-time processing using Spark framework.
• Hands on experience in installing, configuring and deploying Hadoop distribution (Cloudera).
• Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive to optimize performance.
• Worked with Sqoop to move (import/export) data from a relational database into Hadoop.
• Experience in handling different file formats parquet, ORC, Avro, Sequence files and Flat text files.
• Hands on experience in Linux, Redhat and Debian based distros, Shell Scripting and some AWS services.
Data Lake Design and Implementation based on Lambda Architecture: we create batch and streaming layer in our solution using HDFS and Cassandra as a Data Store Layer and Spark framework as a Processing Layer.
Using Cloudera Manager for provisioning, managing and monitoring Hadoop Cluster.
Data Pipelines: Designed and developed complex data pipelines and maintained the data quality to support a rapidly growing business.
Create Data Pipelines and the ETL workflows: getting data from some our data sources (Crawler - Nutch, RDBMS, Log files and some Rest services) and process it to generate metadata by calling some web services, transform to appropriate format and clean bad records in a chain of data services using Apache Kafak and Apache NiFi.
Designing appropriate Data Models for inserting data in HDFS and NoSQL databases.
Using HDFS+ YARN + Apache Spark for data analytics and ad-hoc queries.
Installation, Configuration and Performance tuning of big data tools: The Hadoop Stack (HDFS + Yarn + Hive + Spark), Apache Kafka, Apache NiFi, Apache Cassandra, Apache Solr and Apache Zookeeper.
Tuning Cassandra cluster for heavy write situations: create some benchmarks and test our Cassandra cluster to tune it based on our requirement.
Data Lake Design and Implementation based on Lambda Architecture
Using Cloudera Manager for provisioning, managing and monitoring Hadoop Cluster
Data Pipelines: Designed and developed complex data pipelines
Create Data Pipelines and the ETL workflows
Designing appropriate Data Models for inserting data in HDFS and NoSQL databases
Installation, Configuration and Performance tuning of big data tools
Tuning Cassandra cluster for heavy write situations
Design, Modeling Data, Deployment and Tuning of the Big Data Infrastructure
Deployment, Configuration and Tune MongoDb and Redis cluster
Deploying a large web farm with HAProxy as a load balancer
Create Hadoop cluster with 100 nodes - HDFS + Yarn + Mapreduce + Spark
Built scalable, cost-effective Hadoop cluster using Openstack Sahara
Developing Vector Space Model with Mapreduce framework: developing TFIDF and Inverted Index algorithms using Mapreduce framework and execute it on News data
Implementing a genetic scheduling algorithm with Mapreduce framework
Implementing a triangle counting algorithm using Spark to calculate clustering coefficient metric in huge graph data
Developing Spark SQL applications to correlate IP logs: create Spark cluster with 60 nodes, Kafka cluster with 10 nodes and Cassandra cluster with 20 nodes
Design, Modeling Data, Deployment and Tuning of the Big Data Infrastructure
Deployment, Configuration and Tune MongoDb and Redis cluster
Deploying a large web farm with HAProxy as a load balancer
Big Data courses for organizations
Big Data Fundamental: training Apache Hadoop ecosystem (HDFS, Yarn, Mapreduce, Sqoop, Hive)
NoSQL Databases and Modern Data Management Systems: training Apache Cassandra, Redis, Neo4j, Elasticsearch
Apache Spark Programming: training Spark core, SQL, MLib and Streaming
Streaming Data Processing: Spark Streaming, Kafka, NiFi and Cassandra
The AI-first Recruitment Operating System that unifies and automates the entire hiring lifecycle for modern enterprises worldwide.
Dr Job FZ LLC. 2026 © All Rights Reserved
