Sreelakshmi Cheeti

• Over 7+ years of IT experience encompassing Analysis, Design, and Development of Big Data solutions using technologies such as Hadoop, AWS, Python, DataStage, Scala, and Data Lake.
• Proficient in creating applications with Spark and Python, along with expertise in analyzing data using Hive, Pig, and custom MR programs in Java.
• Expertise in Informatica cloud apps including Data Synchronization, Data Replication, and Real-Time apps like process design and process development.
• Experienced in writing Map Reduce programs & UDFs for Pig & Hive in Java, extracting data from log files using Flume, and integrating Hive and HBase for optimal operations.
• Extensive experience with GCP Cloud services such as Cloud Dataflow, Big Query, and Cloud Storage.
• Expertise in data preprocessing using Python libraries like Pandas for data analysis and numerical computations.
• Developed Spark applications using Spark Core, Streaming, SQL, Data Frames, Datasets & Spark-ML.
• Hands-on experience with AWS Databases like RDS(Aurora), Redshift, DynamoDB, and Elastic Cache.
• Exclusive focus on Big Data Ecosystem using Hadoop framework and related technologies like HDFS, MapReduce, HIVE, PIG, HBASE, STORM, YARN, OOZIE, SQOOP, airflow, Spark Core, Spark SQL, Spark Streaming, Scala, and Kafka.
• Proficient in distributed computing architectures including AWS products, Hadoop, Snowflake Database, Python, Spark, and effective use of map-reduce and SQL.
• Experience in Apache Spark cluster and stream processing using Spark Streaming, and developing MapReduce jobs in Python for Data cleaning and preprocessing.
• Expert in configuring, deploying, and automating major Cloud environments (AWS, Azure & GCP).
• Highly capable of processing large sets of Structured, Semi-structured, and Unstructured datasets for supporting Big Data and Machine learning applications.
• Experience loading data into HDFS using Sqoop, working with databases like Teradata, Oracle, and writing complex SQL, PL/SQL.
• Proficient in version control tools like Bit-Bucket, GIT, SVN, Data warehousing and ETL tools like Informatica.
• Worked on data ingestion using Sqoop from various sources and extensive experience with Spark tools like RDD transformations and Spark SQL.
• Experienced in Data Modelling & Data Analysis using Dimensional Data Modelling, Relational Data Modelling, Star Schema/Snowflake Modelling, FACT & Dimensions tables, and Physical & Logical Data Modelling.
• Developed Unix shell scripts for high-l

Collaborated on a Scala code base related to Apache Spark, executing Actions and Transformations on RDDs, Data Frames, and Datasets using Spark SQL and Spark Streaming Contexts.
Conducted Data blending and preparation using Alteryx and SQL for Tableau consumption, publishing data sources to the Tableau Server.
Developed Spark applications with Pyspark and Spark-SQL for data extraction, transformation, and aggregation across multiple file formats.
Converted the Manual Report system to a fully automated CI/CD Data Pipeline, ingesting data from various Marketing platforms into the AWS S3 data lake.
Implemented Apache Airflow for authoring, scheduling, and monitoring Data Pipelines.
Contributed to the development of a test environment on Docker containers and configured Docker containers using Kubernetes.
Created action filters, parameters, and calculated sets for preparing Power BI dashboards and worksheets.
Developed a data pipeline using Spark, Scala, and Apache Kafka to ingest data from the CSL source and store it in an HDFS-protected folder.
Developed views and templates with Python and Django for a user-friendly website interface.
Utilized Python programming for data processing and handling Data integration between On-prem and Cloud DB or Datawarehouse.
Set up Infrastructure as a code (IaaS) in the AWS cloud platform using CloudFormation templates and integrating appropriate AWS services as per business requirements.
Ensured the development process followed SDLC (Software Development Life Cycle) guidelines.
Developed ETLs using PySpark, employing both Data frame API and Spark SQL API.
Optimized workflows, scheduled ETL jobs, and utilized Apache Airflow components like Pool, Executors, and multi-node functionality.
Worked on continuous integration using tools like Jenkins and SVN as the version control tool.
Effectively handled data errors during the modification of existing reports and manual creation of new reports using Tableau.
Proficient in deploying machine learning models to production environments using platforms like Docker, Kubernetes, or cloud services.
Experience in Amazon AWS services such as EMR, EC2, S3, CloudFormation, and RedShift for fast and efficient processing of Big Data.
Created and maintained a data lake across AWS using S3, lambda, glue, DynamoDB, Elasticsearch, CloudWatch, and Athena.
Used Informatica Power Centre to Extract, Transform, and Load data into Netezza Data Warehouse from various sources like Oracle and flat files.
Implemented GitHub actions workflows to deploy Terraform templates into AWS.
Involved in developing a test environment on Docker containers and configuring Dockers and containers using Kubernetes.
Exported and imported data from file formats and Excel Spreadsheets by creating SSIS Packages and participated in the development process following the Agile methodology.
Implemented solutions for ingesting data from various sources and processing data at rest using Big Data technologies such as Hadoop, Map Reduce Frameworks, HBase, and Hive.

Developed and deployed Spark application using Pyspark to compute popularity score for all the contents using an algorithm and load the data into Elastic Search for the App content management team to consume.
Using Python and shell scripts to automate the process of running the model on new data as required and save the results to final Phoenix tables.
Worked extensively on building and automating data ingestion pipelines and moving terabytes of data from existing data warehouses to the cloud.
Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data to uncover insights into customer usage patterns.
Developed PySpark-based pipelines using spark data frame operations to load data to EDL using EMR for jobs execution and AWS S3 as a storage layer.
Database development experience using SQL, SPARK, or Big Query and experience with a variety of relational, NoSQL-oriented databases like Hadoop, MongoDB and Cassandra.
Integrated data from multiple sources into AWS data lake, performed validation and ETL to load data into Redshift.
Utilize Power BI and SSRS to produce parameter-driven, matrix, sub-reports, drill-down, drill-through, dashboards, and integrated report hyperlink functionality to access external applications and make dashboards available in Web clients and mobile apps.
Scheduled Airflow DAGs to run multiple Hive and Pig jobs, which independently run with time and data availability.
Created and maintained data warehouse, databases, tables, SQL queries, and ingestion pipelines to power reports (Tableau), dashboards, predictive models, and downstream analysis.
Responsible for maintaining and tuning existing cubes using SSAS and Power BI.
Worked on the full spectrum of data engineering pipelines like data ingestion, data transformations and data analysis/consumption.
Developed AWS lambdas using Python and Step functions to orchestrate data pipelines.
Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Yarn, Hive, Pig, Impala, Spark, Kafka.)
Created an AWS Lambda pipeline to migrate microservices from MuleSoft API Gateway to AWS API Gateway
Worked on extracting the data from the Oracle financials and the Redshift database and Created Glue jobs in AWS and loaded incremental data to the S3 staging area and persistence area.
Migrated in-house database to AWS Cloud and designed, built, and deployed a multitude of applications utilizing the AWS stack (Including S3, EC2, RDS, Redshift, Athena) by focusing on high availability and auto-scaling.
Used Apache Kafka to aggregate web log data from multiple servers and make them available in Downstream systems for analysis and used Kafka Streams to Configure Spark streaming to get information and then store it in HDFS.
Created monitors, alarms, notifications and logs for Lambda functions, Glue Jobs, EC2 hosts using CloudWatch and used AWS Glue for the data transformation, validation and data cleansing.
Worked on Improving the performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frame, Pair RDDs, and YARN.
Participate in data modelling discussion and influence the data architecture to ensure the best performance for solutions required by various teams.
Implemented data ingestion and handling clusters in real-time processing using Kafka.
Experience with Spark applications using Spark-SQL in EMR for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data to uncover insights into the customer usage patterns.
Involved in all the steps and scope of the project reference data approach to MDM, have created a Data Dictionary and Mapping from Sources to the Target in MDM Data.

Experience writing Hive queries for data analysis to meet the business requirements.
Engineered a robust Data pipeline using Spark, Hive, and HBase for seamless data ingestion into the Hadoop cluster, ensuring efficient analysis.
Developed PIG Latin scripts to extract and load data from web server output files into HDFS, enhancing data processing capabilities.
Designed and implemented reliable data transformations for various purposes such as reporting, growth analysis, and multi-dimensional analysis.
Successfully converted Hive/SQL queries into Spark transformations using Spark RDDs and Scala, contributing to improved performance.
Implemented performance optimization techniques, including distributed cache usage, partitioning, and bucketing in Hive, and map-side joins.
Hands-on experience with Big Data technologies on AWS (S3/Glue/EMR/RedShift), contributing to effective data handling.
Expertise in designing, developing, and maintaining software solutions within the Hadoop cluster.
Implemented Zookeeper for concurrent access to Hive Tables with shared and exclusive locking.
Utilized Sqoop for importing/exporting data from Oracle and PostgreSQL into HDFS for analysis purposes.
Conducted quantitative and qualitative data analysis, generating dashboards, reports, and assessments for informed, data-driven decision-making.
Leveraged Spark Data Frame API on the Cloudera platform for analytics on Hive data.
Worked on Role-Based Access Controls across Interface, Logic, and Data Tier.
Deployed Big Data Hadoop applications on Talend Cloud AWS and Microsoft Azure.
Designed batch processing jobs using Apache Spark, achieving a ten-fold increase in speed compared to MR jobs.
Implemented Kafka for real-time data ingestion, creating distinct topics for reading data.
Converted HQL into Spark transformations using Spark RDD with support from Python and Scala.
Transferred data from S3 bucket to Snowflake Data Warehouse for report generation.
Proficient in handling various file formats, including delimited text files, clickstream log files, Apache log files, Parquet files, Avro files, JSON files, XML files, and others.
Developed PySpark Streaming by consuming static and streaming data from different sources.
Designed and implemented high-performance data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks.
Extensive experience working on AWS, utilizing EMR for operations, EC2 instances, S3 storage, RDS, and Redshift for analytical operations, and writing data normalization jobs for new data ingested into Redshift, managing multi-terabytes of data.