AI Inference Engineer

TheAI Inference Engineerplays a critical role in the AI lifecycle by bridging the gap between high-performance model development and optimized deployment environments. This position focuses onoptimizingLarge Language Models (LLMs)for inference serving diverse environmentsfrom GPU-rich data centers to resource-constrained edge deviceswith a strong emphasis on maximizing throughput minimizing latency andmaintainingmodel accuracy.

This role is pivotal in advancing F5s AI capabilities ensuring enterprise-grade reliability byleveraginghardware acceleration designing scalable infrastructure andmonitoringsystem performance.

Key Responsibilities

High-Performance AI Serving

• Build andmaintainrobust inference engines using tools likevLLMTGI (Text Generation Inference) andNVIDIA Triton ensuring high performance at scale.

• Handle deployment optimizations to deliver low-latency AI serving solutions for multiple business applications.

Hardware Acceleration and Optimization

• Profile andoptimizemodels for specialized hardware backends includingNVIDIA GPUs(CUDA/TensorRT)Apple Silicon (CoreML) and AI accelerators likeTPUsandLPUs.

• Collaborate with hardware teams to maximizeutilizationand performance across various computational environments.

Inference Orchestration and Scalability

• Design and implementauto-scaling architecturesfor online (real-time) and batch inference pipelinesleveragingKubernetesfor inference routing and orchestration.

• Ensure software solutions areoptimizedfor peak performance during traffic spikesmaintainingreliability and scalability.

Performance Monitoring and Observability

• Establish robust observability frameworks tomonitorTime to First Token (TTFT) tokens per second and memory bandwidthutilizationagainst service-level agreements (SLAs).

• Build and executeperformance and load testing suitestoidentifybottlenecks and ensure consistent reliability at scale.

Technical Requirements

Required Skills:

• Programming Languages:Proficiencyin programming languages such asPythonCRust orGolangspecifically for high-performance AI workflows.

• Inference Tools:Proven hands-on experience with tools likevLLMTensorRT andOllamafor inference development and optimization.

• Infrastructure Expertise:Strong familiarity with infrastructure technologies includingDockerKubernetes and cloud platforms such asAWSGCP andAzure.

• Hardware Optimization Expertise:Comprehensive understanding of GPU and AI hardware including techniques for profiling andoptimizingperformance for accelerators like NVIDIA GPUs and TPUs.

Preferred Experience:

• Prior experience deployingLarge Language Models (LLMs)with advanced techniques likeSpeculative DecodingorPagedAttention.

• Contributions toopen-source inference librariesor hardware-level kernel development (e.g. CUDA Triton kernels).

• Background inMLOpsorSREroles focused onhigh-performance AI endpointsand reliability during demand surges.

• Proficiencyin designing scalable solutions for high-throughput inference environmentsoptimizedfor traffic bursts.

Success Metrics (KPIs):

• Latency Reduction:Continuously improve inference latency metrics ensuring minimalTime to First Token (TTFT)andmaximumtokens per second.

• Cost Efficiency:Achieve lower Cost per 1K Tokens through better resourceutilizationand hardware optimization.

• Scalability:Maintainsystem stability and reliability duringtraffic spikes ensuring performance consistency across environments.

• Throughput Maximization:Deploy modelsoptimizedfor peak hardware usage and maximized process throughput.

Why Join F5

F5 empowers you to push boundaries inAI optimizationandhigh-performance engineering. Joining our team means:

• Collaborating withcutting-edgetechnologies and hardware solutions to support real-time AI applications.

• Advancing your career in a fast-paced multidisciplinary environment focused on innovation scalability and problem-solving.

• Driving transformative projects that deliver real-time AI reliability to global customers whilemaintainingcost and efficiency standards.

• Working on advancedMLOpssolutionsthat seamlessly scale enterprise AI systems and shape the future of intelligent deployment.

What Success Looks Like:

As anAI Inference Engineerat F5 success is measured by your ability to:

• Combine technicalexpertiseand problem-solving skills to deliver low-latency scalable and high-performing AI prediction systems.

• Collaborate efficiently across cross-functional teamsparticipatingin knowledge sharing and system refinement.

• Demonstrate initiative by driving optimizations across hardware tools and orchestration processes balancing immediate solutions with long-term architectural goals.

• Translatecomplex AI and inference workflows into practical solutions that align with F5s strategicobjectives.

#LI-AK1