Software Engineer at Google · Bengaluru

I make ML models
run fast on real hardware.

I'm Fenil Soni, working on ML infrastructure and performance at Google. Before that I shipped on-device LLMs and hand-tuned ARM kernels at Samsung Research, and built real-time vision systems for drones.

Connect on LinkedIn GitHub Email me
Fenil Soni
⚡ARM NEON · SME
🧠On-device LLMs
5+ yrsbuilding ML & embedded systems
2–3×operator speedups with ARM NEON
90%less CPU time on GEMM/Conv with SME
+50%tracking FPS on NVIDIA Jetson
// What I do

Where models meet hardware

Memory layout, operator coverage, quantization, kernels, and the APIs that hide all of it from the people building on top.

ML infrastructure

Making large-scale ML workloads run efficiently, from runtime APIs down to how tensors move through memory.

Kernels & performance

Hand-written ARM NEON/SME intrinsics, OpenCL and CUDA kernels, and profiling until the numbers move.

Edge AI & autonomy

Quantized models on NPUs and Jetson, real-time video pipelines, and vision for autonomous drones.

// Experience

Where I've worked

Software Engineer III @ Google

Jul 2025 — Present
Bengaluru, India
  • ML infrastructure and performance optimization.

Associate Staff Engineer @ Samsung Research

Aug 2024 — Jul 2025
Bengaluru, India
  • Architected an API layer over the Exynos Neural Network (ENN) framework to run sharded LLMs on-device, hiding execution graphs, tensor bindings and memory management from app developers.
  • Built a shared tensor-buffer pool linking inputs and outputs across shards, cutting allocation overhead and improving data locality.
  • Added missing operators (GridSample, LayerNorm, GroupNorm, DepthwiseConv) and optimized them with ARM NEON for 2–3× speedups on NPU-powered devices.
  • Enabled GEMM and Conv2D with ARM SME on ARMv9, cutting CPU execution time by up to 90%; brought new ops to GPU with OpenCL.
C++ARM NEONARM SMEOpenCLLLMsNPU

Technical Lead @ HCLTech

Aug 2021 — Aug 2024
Bengaluru, India · Senior Software Engineer → Technical Lead
  • Led the team building onboard compute for UAVs: system architecture, release management and test planning.
  • Moved object tracking to a NanoTrack Siamese network in a GStreamer/DeepStream plugin on Jetson for +50% tracking FPS, and worked on its CUDA path.
  • Trained DetectNet v2 on VisDrone (70% mAP), then INT8-quantized and deployed it via ONNX and TensorRT on the Jetson DLA.
  • Led an autonomous landing module: AprilTag pose estimation (85% accuracy from 20 m) fused with LIDAR and GPS, validated in ROS/Gazebo and flight tests.
  • Built a GStreamer video stabilization plugin (optical flow + Kalman filter) cutting jitter by 65%, and a custom Yocto Linux image 70% smaller for the i.MX 8M Plus.
CUDATensorRTDeepStreamGStreamerOpenCVROSYocto
// Selected work

Things I've built

A few highlights. The code for these belongs to the companies I built them for.

Samsung Research

On-device LLM runtime

Middleware that runs sharded LLMs on mobile NPUs behind a minimal API, with one pre-allocated buffer pool shared across shards.

1 APIno graphs, bindings or memory management for app developers
Samsung Research

ARM NEON & SME kernels

Hand-written intrinsics for normalization, sampling and matrix-heavy operators on ARMv9 CPUs.

−90%CPU execution time on targeted workloads
HCLTech

UAV vision stack

Detection, tracking, stabilization and RTSP streaming pipelines running on NVIDIA Jetson and i.MX 8M Plus.

+50%tracking FPS with NanoTrack + DeepStream
HCLTech

Autonomous landing assist

Fiducial-marker pose estimation fused with LIDAR and GPS so drones can land on moving vehicles and ships.

85%pose accuracy from 20 m altitude
// Toolbox

Skills

Languages

C++PythonCUDAOpenCLMATLAB

ML & inference

TensorFlowKerasONNXTensorRTDeepStreamINT8 quantization

Performance

ARM NEONARM SMEGPU kernelsMemory planning

Systems

LinuxYoctoGStreamerROSOpenCVDocker
// Education

Education

M.Tech, Aerospace Engineering

Indian Institute of Technology Bombay

2019 — 2021

B.E., Aeronautical Engineering

Sathyabama Institute of Science & Technology

2015 — 2019

Let's talk

Open to conversations about ML systems, performance engineering and on-device AI.

LinkedIn Email GitHub