About Me
Data Engineer with experience in data migration, reporting, and analytics workflows. Skilled in Python, SQL, Spark, PySpark, Hive, HDFS, Cloudera, AWS, GCP, Tableau, and machine learning/data science techniques including…
Data Engineer with experience in data migration, reporting, and analytics workflows. Skilled in Python, SQL, Spark, PySpark, Hive, HDFS, Cloudera, AWS, GCP, Tableau, and machine learning/data science techniques including NLP, CNNs, and text preprocessing.
Experience
Data Engineer
<p>Performed data migration from on premise Hadoop cluster to GCP BigQuery using automated tools and scripts.<br>
Created development and postproduction QA reports for the implemented workflows.<br>
Created base views and snapshot tables and pointed them based on full refresh and incremental refresh tables.<br>
Implemented strategic and tactical migration of tables from on premise cluster to GCP BigQuery.<br>
Created dashboards in Tableau and performed visualization of migrated tables.</p>
Software Engineering Intern
<p>Performed data analysis and generated and ran CNN algorithms on different shapes of data.<br>
Performed text preprocessing and utilized the text-davinci-003 model to generate text completions based on given prompts.<br>
Implemented scripts that process the datasets and classify the images.<br>
Used the OpenAI API to create text embeddings using the text-embedding-ada-002 model.<br>
Used Pinecone to store embeddings and enabled efficient vector search for retrieving relevant items.<br>
Developed text generation pipeline using Hugging Face Transformers library and stored the resulting pipeline in the local LLM.<br>
Built classification models based on Logistic Regression, Decision Trees, Random Forest, and Support Vector Machine.<br>
Preprocessed the data by removing all missing values and performing Tokenization, Lemmatization, and POS Tagging.<br>
Implemented sentiment analysis on movie review dataset to calculate polarity of positive and negative reviews.<br>
Performed triplet extraction on the abstracts by using Python on multiple text files.</p>