Kabilan KB

Kabilan KB

Roboticist building open-source robot learning pipelines that start in an NVIDIA digital twin and end on real edge hardware.

Teleop rigs that collect data, reinforcement learning that trains thousands of robots at once, and vision-language-action and world models running on Jetson AGX Thor and GB10.

Currently
Engineer, Autonomous Tech, General Motors
Based in
Bangalore, India
Focus
Robot learning, simulation, edge AI
Community
NVIDIA DLI and Omniverse Ambassador, Jetson AI Lab Research Group core contributor, Dell Ambassador
Elsewhere
GitHub, LinkedIn, X
Latest
reBot Duo: Sim-to-Policy, October 2026

Selected work

Twelve open-source projects, newest first, each with measured results. Most follow one pipeline: simulate and teleoperate, train with RL, learn and benchmark policies, then deploy on NVIDIA edge hardware.

  1. reBot Duo: Sim-to-Policy

    October 2026 · Bimanual VLA, hardware in the loop

    Two Seeed reBot B601 arms put two pens and an eraser into a pencil cup. The whole pipeline is built in simulation: an Isaac Lab scene, scripted demonstrations, SmolVLA fine-tuned on a Dell Pro Max, and serving through vla-edge. Then hardware in the loop: Isaac Sim runs on the Dell while every action chunk is computed on a Jetson AGX Thor. It ships with an open course.

    • 52 demonstrations, 70,884 frames; SmolVLA fine-tuned for 20k steps on one RTX PRO 5000 Blackwell laptop GPU
    • SmolVLA inference ≈ 278 ms per 50-step chunk on Jetson Thor, ≈ 160 ms on the Dell
    • On 20 unseen layouts it grasps and carries objects but rarely releases them: 1 of 20 episodes completes the task, the same failure on both machines

    Isaac Lab 3 · Isaac Sim 6 · LeRobot SmolVLA · vla-edge · Jetson AGX Thor · Dell Pro Max 16 Plus

    Course · Code on GitHub

    Isaac Sim on the Dell, policy on Jetson Thor, 5× speed

  2. ASPIRE × YAM gear assembly in Isaac Sim

    October 2026 · Agentic code-as-policies

    Runs NVIDIA ASPIRE on bimanual YAM arms in Isaac Lab. An AI agent writes the robot's control program in Python, runs it, reads the failures and fixes the code, then turns lessons that hold up into verified, reusable skills. The task: pick up a factory gear and seat it on the middle shaft.

    • A bridge serves the Isaac Lab task to ASPIRE: arm commands, RGB-D cameras, seeded resets and task success
    • Round 1 seats the gear on 6 of 10 clean held-out seeds; a separate verifier agent confirmed all 5 new skills
    • Round 2's tilted reach solves 3 of 9 far-shaft seeds, where round 1 solved none

    NVIDIA ASPIRE · Isaac Sim 6 · Isaac Lab 3.0 · YAM bimanual · SAM3 · mink IK

    Code on GitHub

    Isaac Lab, 8× speed

  3. newton-claw-thor

    October 2026 · On-device training + LLM agent

    Trains and runs robot policies in Newton on a single Jetson AGX Thor, driven by a local LLM agent that turns a request into one command.

    • Go2 velocity tracking: 4,096 envs × 600 iterations in ~10 min 43 s on Thor
    • Trained Go2 policy fell 10 times in evaluation vs 18 for the reference policy
    • Local Gemma 4 agent routed 9 of 9 test prompts to the correct command

    Jetson AGX Thor · Newton 1.6 · rsl_rl PPO · Gemma 4 12B · Ollama

    Code on GitHub

  4. YAMLab on Isaac Sim 6 with SO-101 leader-arm teleoperation

    October 2026 · Simulation + teleoperation

    Ported a bimanual YAM robot framework to Isaac Lab 3.0 and drove both simulated arms with real SO-101 leader arms.

    • Two leaders drive both arms, or one leader switches between them
    • Teleop sessions record to HDF5 and replay into LeRobot datasets with 3 camera views
    • PutPotOnCooktop pick-and-place replays successfully at ~40 env-steps/s on 4 RGB envs

    Isaac Sim 6.0.1 · Isaac Lab 3.0 · PyTorch 2.10 · LeRobot

    Code on GitHub

    YAMLab on Isaac Sim 6 with SO-101 leader-arm teleoperation screenshot

  5. Cosmos3-Edge SO-101 on Jetson Thor

    September 2026 · Edge deployment · hardware-in-the-loop · Featured on Jetson AI Lab

    Runs a ~2B Cosmos3-Edge action policy on Jetson Thor and closes the control loop with an Isaac Lab digital twin on a laptop.

    • Policy returns 32-step × 6-joint action chunks, streamed to a live web dashboard
    • torch.compile + cuDNN attention cut chunk time from 4.3 s to ~1.5–2.1 s
    • Split architecture: sim and UI on a laptop, inference on Thor over SSH + msgpack

    Jetson AGX Thor · JetPack 7 · Cosmos3-Edge · Isaac Lab 3.0 · WebSocket

    Code on GitHub

    Cosmos3-Edge SO-101 on Jetson Thor screenshot

  6. isaaclab_microduck

    August 2026 · Reinforcement learning

    Isaac Lab 3.0 RL environments for the Microduck bipedal robot, built on Newton's MuJoCo-Warp solver.

    • Five tasks: locomotion, running, ball kicking, a two-robot rally and side-by-side running
    • A fixed 61-D observation lets ONNX policies be hot-swapped at runtime
    • BallKick and BallRally trained and measured; 149 CPU-only config tests

    Isaac Lab 3.0 · Newton (MuJoCo-Warp) · PyTorch · ONNX

    Code on GitHub

    isaaclab_microduck screenshot

  7. gb10_cosmos_policy_server

    August 2026 · Policy serving

    A Docker setup that serves the Cosmos3-Nano action policy against the RoboLab benchmark on NVIDIA GB10 hardware.

    • Containerized policy server; the RoboLab client can run any of 120 registered tasks
    • Patched for GB10: PyTorch attention fallback and a PyAV video decoder
    • Fine-tuning pipeline: DCP conversion, dry-run-validated SFT configs, export for serving

    NVIDIA GB10 (Grace Blackwell) · CUDA 13.0 · Docker · Cosmos3-Nano · RoboLab

    Code on GitHub

  8. rebot-isaacsim-bridge

    August 2026 · Real-to-sim bridge

    A real-time bridge that mirrors a real arm's LeRobot teleop session into Isaac Sim, without touching real-arm control.

    • Streams follower joint state every loop iteration over a simple UDP JSON protocol
    • Per-joint sign flips and gripper scaling match any simulated asset's conventions
    • Supports Seeed reBotArm B601 follower and 102 leader; pip-installable sender

    LeRobot · Isaac Sim · UDP / JSON · Seeed reBotArm B601

    Code on GitHub

    rebot-isaacsim-bridge screenshot

  9. SO-101 Bench × GR00T-N1.7

    July 2026 · VLA fine-tuning + benchmarking

    Fine-tune and evaluate GR00T-N1.7 on SO-101 Bench in an Isaac Lab digital twin, with no physical robot required.

    • Language-conditioned tabletop suite over 56 household objects
    • Loss 1.114 → 0.065 over 20k steps in ~90 min on a single RTX PRO 6000
    • Automatic episode scoring with failure tags such as failed_grasp and wrong_object

    Isaac Sim / Isaac Lab 5.1 · GR00T-N1.7-3B · LeRobot · ZMQ

    Code on GitHub

  10. Cosmos3-Nano on NVIDIA GB10

    June 2026 · World foundation model · GB10 + Jetson Thor

    Runs NVIDIA's 16B-parameter Cosmos3-Nano world model on Blackwell aarch64 devices, behind a Gradio generator UI.

    • Text-to-video and image-to-video: 57 frames at 480p in ~3 min on GB10
    • Text-to-image in ~30 s, with output up to 1280×720
    • ~33 GB model fits in GB10's 128 GB unified memory and loads in ~5 s

    GB10 · Jetson Thor · CUDA 13.0 · PyTorch 2.12 · Diffusers · Gradio · Docker

    Code on GitHub

  11. SO-101 VR Control

    May 2026 · VR teleoperation

    Open-source, browser-based VR teleoperation for the SO-101 arm with a Meta Quest: one command, no app install.

    • Hold grip and the end effector follows your hand; trigger closes the gripper
    • PyBullet inverse kinematics drives six STS3215 servos through LeRobot
    • Safety: 200°/s velocity limit, 10° per-command cap, 5° joint margins

    Meta Quest · WebXR + A-Frame · WebSocket (WSS) · PyBullet IK · LeRobot

    Code on GitHub

    SO-101 VR Control screenshot

  12. XLeRobot IsaacLab

    March 2026 · Mobile manipulation + teleoperation

    A standalone Isaac Lab extension for the XLeRobot dual-arm mobile manipulator, with keyboard and gamepad teleoperation.

    • Three environments: ground plane, cube lifting and a loft scene
    • Drive the base at three speeds and control both arms and grippers from keyboard or gamepad
    • Runs up to 4 parallel environments; URDF and USD assets ship with the repo

    Isaac Sim 4.5+ · Isaac Lab · URDF / USD · Keyboard + Xbox gamepad

    Code on GitHub

    XLeRobot IsaacLab screenshot

Press & recognition

Two features on the NVIDIA Blog, a featured Jetson AI Lab community project, and ambassador roles with NVIDIA and Dell.

About

Simulation & learning
Isaac Sim, Isaac Lab, Newton, LeRobot, GR00T, Cosmos, ASPIRE
Compute
Jetson AGX Thor (JetPack 7), NVIDIA GB10 Grace Blackwell, RTX PRO 6000
Robots
SO-101, YAM, reBotArm B601, XLeRobot, Go2, Microduck

I work as an engineer in autonomous tech at General Motors, based in Bangalore.

I started with an autonomous wheelchair on a Jetson Nano, built as an undergraduate at Karunya Institute of Technology and Sciences, which NVIDIA featured on its blog in 2023.

Since then my work has moved into robot learning. I bring up robots in Isaac Sim and Isaac Lab, collect demonstrations with leader arms and VR, train policies with massively parallel RL, fine-tune and benchmark VLA models without a physical robot, and deploy policies and world models on Jetson AGX Thor and GB10. Every project is open source, with the numbers to show how it performs.

As an NVIDIA Deep Learning Institute and Omniverse Ambassador I teach Isaac Sim and ROS.

Contact

Let's build robots that learn.

Open to collaborations, talks and roles in robot learning, simulation and edge AI. The fastest way to reach me is a message on LinkedIn or X.