Akamsha Reddy

Akamsha Reddy

Data Engineer
United States of America

About Me

6+ years in Data Engineering and implementing Hadoop, Spark, and cloud data warehousing solutions. Expertise in providing ETL solutions for diverse business models, ensuring data integration and transformation. Proficien…

Experience

Data Engineer

Macy's
Jan 2022 - Present · 4 years 5 months

 Developed data pipelines for data gathering, transformation, and loading into data lakes or warehouses, a common practice in retail for data analysis.
 Created efficient Hive tables with static and dynamic partitions to manage and analyze retail data.
 Implemented ETL tasks to extract and load data from multiple sources, a fundamental process for retail data integration and analytics.
 Developed Spark applications for data extraction, transformation, and aggregation from various file formats commonly used in retail for big data processing.
 Built real-time monitoring dashboards using AWS Lambdas to track sales, inventory, and customer behavior in real time.
 Worked on data visualization using Python and Tableau for presenting and interpreting retail data insights.
 Proficient in data ingestion techniques, including real-time and batch data ingestion, crucial for managing the constant flow of data.

Data Engineer

Macy’s – Atlanta, GA
Jan 2022 - Present · 4 years 6 months

Developed data pipelines for data gathering, transformation, and loading into data lakes or warehouses.
Created efficient Hive tables with static and dynamic partitions to manage and analyze retail data.
Implemented ETL tasks to extract and load data from multiple sources.
Developed custom Kafka Connect connectors in Java or Python to integrate Kafka with proprietary or legacy systems.
Developed Spark applications for data extraction, transformation, and aggregation from various file formats.
Built real-time monitoring dashboards using AWS Lambdas to track sales, inventory, and customer behavior in real time.
Worked on data visualization using Python and Tableau for presenting and interpreting retail data insights.
Proficient in data ingestion techniques, including real-time and batch data ingestion.
Implemented data caching strategies to reduce query response times.
Developed Map Reduce programs with Apache Hadoop for handling large volumes of data.
Managed Hadoop infrastructure with Cloudera Manager to ensure the stability and scalability of data processing.
Worked on job migration from Tidal to Control-M, involving scheduling retail-specific tasks and automating data processes.
Created Python wrapper scripts for data extraction using Sqoop.
Created and maintained technical documentation for launching Hadoop clusters and executing Hive queries.
Designed and implemented large-scale distributed solutions in AWS and GCP clouds.
Worked with Salesforce integration.
Conducted data extraction, transformation, comparison, and loading from DB2 into a data warehouse using Teradata Client Utilities.
Conducted testing, data validation, and data comparison between source and target applications to ensure data accuracy.
Implemented Kafka Streams API using Java or Scala to perform complex transformations on incoming data streams.
Optimized SQL queries and data processing workflows to enhance the performance and efficiency of data operations.
Worked on data modeling, including creating and maintaining data schemas, to support advanced analytics and reporting.
Developed complex streaming pipelines involving joins, aggregations, and windowed operations on Kafka topics using Data Frame and Dataset APIs in Spark.
Conducted performance tuning and optimization of data pipelines and processes.
Identify and prioritize critical business rules governing retail data, ensuring they are consistently validated in unit tests.
Collaborated with cross-functional teams to gather, analyze, and prioritize business requirements.
Conducted data profiling and analysis to identify trends and patterns in retail data.

Data Engineer

Chevron – Houston, TX
Jan 2020 - Dec 2021 · 1 year 11 months

Built a reporting data warehouse from the ERP system.
Developed data models and deployed data pipelines to meet Chevron's business requirements.
Utilized Big Data and Hadoop Ecosystem components within Azure as a Data Engineer.
Worked with Neo4j, a NoSQL graph database, to create data models and relationships.
Collaborated with reporting developers to implement report and universe designs.
Assisted data scientists and architects in creating and maintaining data models using Databricks.
Developed data pipelines using tools like Flume, Sqoop, and Pig to extract and store data from weblogs.
Utilized PowerShell and UNIX scripts for file transfer, email communication, and other file-related tasks.
Participated in the deployment process from development to production.
Worked with Informatica Cloud for data integration between Salesforce, RightNow, Eloqua, and web services applications.
Monitored and optimized query performance within Big Query.
Leveraged Teradata SQL and Client Utilities for data migration to data warehouses.
Monitored and fine-tuned query performance within Azure's Big Query.
Utilized sensor data from oil wells and drilling equipment to predict and prevent equipment failures.
Integrated data from various sources, including seismic data, well logs, and drilling reports.
Worked with data scientists to develop predictive models for oil and gas production forecasts.
Demonstrated expertise in ETL processes using Spark, Spark SQL, and Spark Streaming.
Developed customized Airflow plugins within Azure to enhance data workflow management.
Collaborated on a migration project involving web methods code to Informatica Cloud.
Worked as a Data Engineer using Big Data and Hadoop Ecosystem components to build highly scalable data pipelines.
Automated data extraction from warehouses and weblogs within Azure using Oozie and NiFi.
Worked closely with business users to identify and address data gaps.
Developed custom Airflow plugins to enhance the capabilities of Airflow for managing data workflows.
Vizient, Dallas, TX
Worked as a Data Engineer using Big Data and Hadoop Ecosystem components to build highly scalable data pipelines for petroleum-related products.

Data Engineer

Vizient, Dallas, TX
Aug 2017 - Dec 2019 · 2 years 4 months

Designed and implemented a healthcare-specific Data Warehouse, aggregating data from various sources including claims data, EHR, and medical billing data.
Utilized data technologies and tools like Hadoop, Spark, and Hive for efficient data processing and analysis.
Developed and maintained ETL processes for accurate extraction and transformation of healthcare data.
Collaborated on the refinement and creation of BI reports and dashboards, integrating AWS QuickSight for enhanced visualization and analytics capabilities.
Administered user accounts and access in tools like PowerCenter for secure data management.
Monitored and managed Hadoop clusters to maintain data availability and system performance.
Managed user accounts and secure data access using AWS-managed services like IAM.
Employed a suite of data technologies such as Hadoop, Spark, and Hive within AWS infrastructure.
Demonstrated the impact of work through quantifiable results, such as reduced processing time and improved data accuracy.
Implemented data quality checks and validation procedures to ensure the accuracy and reliability of healthcare data.
Integrated data with healthcare systems to facilitate improved patient care.
Established an automated CI/CD pipeline using Git, Jenkins, and custom tools for efficient data processing.
Collaborated with data scientists and analysts to create predictive models and machine learning algorithms for healthcare analytics.
Developed data pipelines for the ingestion of streaming healthcare data.
Created complex SQL, PL/SQL, and Spark SQL queries for data analysis and reporting.
Worked on optimizing data storage and retrieval for large healthcare datasets.
Implemented data governance practices to maintain data integrity and enforce data security and privacy regulations.
Conducted data profiling and data lineage analysis to track the flow of healthcare data and identify potential issues.
Leveraged cloud services, including AWS, for scalable and cost-effective data solutions.
Automated data cleansing and transformation tasks to streamline the ETL process and reduce manual intervention.
Conducted performance tuning of SQL and Spark queries to optimize query execution times and resource utilization.
Assisted in disaster recovery planning and data backup strategies to ensure data availability and continuity of healthcare services.
Implemented data encryption and access control mechanisms to protect sensitive healthcare information.
Created and maintained data documentation, including data dictionaries and metadata, for data transparency and understanding.
Collaborated with third-party healthcare data providers to acquire and integrate external data sources for analysis.
Monitored data pipelines for anomalies and implemented automated alerting and notification systems.
Stayed current with healthcare industry trends and emerging data technologies to drive innovation and efficiency in data engineering processes.
Investigated and promptly resolved data discrepancies, providing swift resolution to address data-related challenges within AWS environments.
Implement robust data security measures to protect sensitive healthcare information from unauthorized access and breaches.
Implement real-time data streaming solutions for monitoring patient data, enabling immediate interventions when necessary.

Skills

Big Data Data Repository Hadoop Scala Spark Apache Hive Apache Pig Amazon Web Services (AWS) Kafka NoSQL Redshift S3 EC2 RDS IAM Glue DynamoDB Data Modeling Databricks HDFS MapReduce Informatica PowerCenter Talend Python Azure Data Lake Azure Data Factory SQL Database SQL Data Warehouse PL/SQL AWS AppSync GraphQL Aurora Postgres EMR OLAP Operational Data Store (ODS) BigQuery ETL Informatica Cloud Ab Initio SQL Business Objects Tableau Shell Control-M Hive Kafka Connect Java Spark SQL Spark Streaming Cloudera Manager Tidal Sqoop Salesforce Teradata GCP AWS QuickSight
Report this Profile?