VLM Engineer

NewGen Technologies

Job Location:

Herndon, VA - USA

Monthly Salary: Not Disclosed

Posted on: 9 days ago

Vacancies: 1 Vacancy

Job Summary

This is role is anticipating to end at the end of July with expectation of mod/additional funding through 2027.

Responsibilities

Design and execute fine-tuning pipelines for Vision-Language Models (VLMs) on domain-specific imagery datasets including data preprocessing training orchestration and hyperparameter optimization
Develop and implement evaluation frameworks for multimodal model performance including task-specific metrics for image understanding visual question answering and spatial reasoning
Build scalable training infrastructure on AWS (SageMaker EC2 GPU instances) for distributed fine-tuning of large multimodal models
Engineer data pipelines for curating annotating and transforming geospatial imagery datasets into model-ready formats for supervised and instruction-tuning workflows
Collaborate with applies scientists and solutions architects to iterate on model architectures adapter strategies (LoRA/QLoRA) and inference optimization techniques

Requirements

TS/SCI CI Poly Clearance with current NGA eligibility and SBU/SecNet/COE accounts
Must be willing to work in SCIF daily or as needed
5 years of professional machine learning engineering experience with a focus on deep learning
4 years of advanced Python development for ML workloads
3 years of experience with computer vision or multimodal models
3 years of experience with AWS ML infrastructure
SageMaker Training jobs Processing jobs and endpoint deployment
GPU instance selection multi-node training and cost optimization on EC2 (P4/P5/G5/G6e)
2 years of experience building ML evaluation pipelines
Automated benchmarking metric computation and result analysis
Experience with both quantitative metrics and qualitative/human evaluation approaches
1 years of hands-on experience fine-tuning large foundation models (LLMs or VLMs)
Experience with parameter-efficient fine-tuning methods (LoRA QLoRA adapters)
Familiarity with supervised fine-tuning instruction tuning and RLHF/DPO alignment techniques
Strong proficiency with PyTorch and the HuggingFace ecosystem (Transformers PEFT Datasets Accelerate)
Experience with distributed training frameworks (DeepSpeed FSDP or Megatron)
Understanding of vision transformer architectures (ViT CLIP LLaVA-family models or similar)
Experience processing and augmenting image datasets at scale
Strong software engineering fundamentals (version control CI/CD for ML workflows testing)

Desired Skills

2 years of experience with geospatial or remote sensing imagery
2 years of experience with Authority to Operate (ATO) processes in government environments
Familiarity with electro-optical and SAR satellite imagery formats and characteristics
Understanding of geospatial metadata coordinate systems and imagery preprocessing
Experience with model quanitzation and inference optimization (vLLM TensorRT ONNX)
Experience with MLOps and experiment tracking tools (MLflow Weights & Biases SageMaker Experiements)
Familiarity with data annotation platforms and active learning workflows for imagery
Experience with containerized ML workflows (Docker ECR ES/EKS)
Implementation of NIST 800-53 controls and security compliance for ML systems
Experience deploying models in air-gapped or disconnected environments
Familiarity with multimodal evaluation benchmarks (MMMU MMBench GQA or domain-specific equivalents)
Publications or demonstrated contributions in computer vision VLMs or multimodal AI
Experience with synthetic data generation for training data augmentation

About Us
For more than 20 years NewGen Technologies has solved our clients toughest IT challenges with integrity security and outstanding service by delivering both technology and talent. We have helped secure borders have used artificial intelligence (AI) to fight terror aided the identification of criminals and have helped to prevent crime through the introduction of team of Highly Cleared Specialists have hard-to-find skills and expertise in a wide spectrum of technologies to provide solutions that transform business processes and solve problems of national significance. #CJ

Required Experience:

This is role is anticipating to end at the end of July with expectation of mod/additional funding through 2027.ResponsibilitiesDesign and execute fine-tuning pipelines for Vision-Language Models (VLMs) on domain-specific imagery datasets including data preprocessing training orchestration and hyperp...

This is role is anticipating to end at the end of July with expectation of mod/additional funding through 2027.

Responsibilities

Design and execute fine-tuning pipelines for Vision-Language Models (VLMs) on domain-specific imagery datasets including data preprocessing training orchestration and hyperparameter optimization
Develop and implement evaluation frameworks for multimodal model performance including task-specific metrics for image understanding visual question answering and spatial reasoning
Build scalable training infrastructure on AWS (SageMaker EC2 GPU instances) for distributed fine-tuning of large multimodal models
Engineer data pipelines for curating annotating and transforming geospatial imagery datasets into model-ready formats for supervised and instruction-tuning workflows
Collaborate with applies scientists and solutions architects to iterate on model architectures adapter strategies (LoRA/QLoRA) and inference optimization techniques

Requirements

TS/SCI CI Poly Clearance with current NGA eligibility and SBU/SecNet/COE accounts
Must be willing to work in SCIF daily or as needed
5 years of professional machine learning engineering experience with a focus on deep learning
4 years of advanced Python development for ML workloads
3 years of experience with computer vision or multimodal models
3 years of experience with AWS ML infrastructure
SageMaker Training jobs Processing jobs and endpoint deployment
GPU instance selection multi-node training and cost optimization on EC2 (P4/P5/G5/G6e)
2 years of experience building ML evaluation pipelines
Automated benchmarking metric computation and result analysis
Experience with both quantitative metrics and qualitative/human evaluation approaches
1 years of hands-on experience fine-tuning large foundation models (LLMs or VLMs)
Experience with parameter-efficient fine-tuning methods (LoRA QLoRA adapters)
Familiarity with supervised fine-tuning instruction tuning and RLHF/DPO alignment techniques
Strong proficiency with PyTorch and the HuggingFace ecosystem (Transformers PEFT Datasets Accelerate)
Experience with distributed training frameworks (DeepSpeed FSDP or Megatron)
Understanding of vision transformer architectures (ViT CLIP LLaVA-family models or similar)
Experience processing and augmenting image datasets at scale
Strong software engineering fundamentals (version control CI/CD for ML workflows testing)

Desired Skills

2 years of experience with geospatial or remote sensing imagery
2 years of experience with Authority to Operate (ATO) processes in government environments
Familiarity with electro-optical and SAR satellite imagery formats and characteristics
Understanding of geospatial metadata coordinate systems and imagery preprocessing
Experience with model quanitzation and inference optimization (vLLM TensorRT ONNX)
Experience with MLOps and experiment tracking tools (MLflow Weights & Biases SageMaker Experiements)
Familiarity with data annotation platforms and active learning workflows for imagery
Experience with containerized ML workflows (Docker ECR ES/EKS)
Implementation of NIST 800-53 controls and security compliance for ML systems
Experience deploying models in air-gapped or disconnected environments
Familiarity with multimodal evaluation benchmarks (MMMU MMBench GQA or domain-specific equivalents)
Publications or demonstrated contributions in computer vision VLMs or multimodal AI
Experience with synthetic data generation for training data augmentation

Required Experience:

Apply Now

About Company

NewGen Technologies

After a couple of years without an operating business, NewGen Technologies (formerly Bongiovi Entertainment) is once again revving its engines following a reverse merger with alternative fuel developer ReFuel America (which now functions as its operating subsidiary). The company hopes ... View more

View Profile View Profile

AI AutoApply

Apply to 100+ jobs with one click