Alexander Schwarz

How Robots Learn to be Robots

How do you safely teach a robot to handle dangerous tasks? By letting it fail a million times in a digital world first.

How Robots Learn to be Robots
#1about 3 minutes

The evolution from digital agents to physical AI

Physical AI extends digital agentic systems into the physical world, enabling robots to understand and act based on sensory input and instructions.

#2about 4 minutes

The three-computer solution for robotics challenges

The high cost of data and physical testing is solved by a three-computer workflow using simulation for data generation, a separate system for training, and simulation again for validation.

#3about 3 minutes

Understanding reinforcement and imitation learning for robots

Robots learn skills through reinforcement learning by trial and error with rewards, or through imitation learning by mimicking expert demonstrations.

#4about 2 minutes

Building digital twins with Omniverse and Isaac Sim

NVIDIA Omniverse is a development platform based on OpenUSD for creating digital twins, with Isaac Sim providing a dedicated application for robot simulation.

#5about 2 minutes

Scaling training data with simulated teleoperation

A small number of human demonstrations collected via teleoperation in Isaac Sim can be algorithmically scaled into a large, diverse, and photorealistic synthetic dataset.

#6about 3 minutes

Generating photorealistic data with Cosmos foundation models

Cosmos is a platform of world foundation models, including Cosmos Transfer, which uses control nets to transform basic simulation outputs into photorealistic videos.

#7about 3 minutes

Generating novel robot scenarios with Isaac GR00T Dreams

The Isaac GR00T Dreams workflow uses a post-trained Cosmos Predict model to generate new robot behaviors from a single image, which are then filtered and labeled to create training data.

#8about 2 minutes

A hybrid data strategy for robust foundation models

Robust robot foundation models like Isaac GR00T are trained by combining vast internet video data, high-quality human demonstrations, and large-scale synthetic data.

#9about 1 minute

Testing and deploying robots in large-scale simulations

Before real-world deployment, trained robot models can be rigorously tested at scale within digital twins of complex environments like factories using OpenUSD.

Related jobs
Jobs that call for the skills explored in this talk.

Job ad Dark

SabIna compys

SabIna compys
Vienna, Austria

Remote
€20-100K
Intermediate
JavaScript
.NET
+1

Matching moments

An overview of NVIDIA Omniverse and Isaac Sim
02:47 MIN

An overview of NVIDIA Omniverse and Isaac Sim

Enhancing AI-based Robotics with Simulation Workflows

Real-world examples of simulation-trained robots
02:50 MIN

Real-world examples of simulation-trained robots

Enhancing AI-based Robotics with Simulation Workflows

AI is moving from the screen to the physical world
02:26 MIN

AI is moving from the screen to the physical world

Robots 2.0: When artificial intelligence meets steel

Core requirements for developing AI-powered robots
02:59 MIN

Core requirements for developing AI-powered robots

Enhancing AI-based Robotics with Simulation Workflows

Introducing the concept of an immersive AI copilot
03:06 MIN

Introducing the concept of an immersive AI copilot

Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!

Using synthetic data generation for AI training
02:33 MIN

Using synthetic data generation for AI training

Enhancing AI-based Robotics with Simulation Workflows

Why robotic capabilities are advancing so rapidly
02:07 MIN

Why robotic capabilities are advancing so rapidly

Robots 2.0: When artificial intelligence meets steel

Implementing an AI-in-the-loop continuous learning cycle
09:45 MIN

Implementing an AI-in-the-loop continuous learning cycle

A solution to embed container technologies into automotive environments

Featured Partners

Related Videos

See all videos
Enhancing AI-based Robotics with Simulation Workflows30:57

Enhancing AI-based Robotics with Simulation Workflows

Teresa Conceicao

Robots 2.0: When artificial intelligence meets steel28:19

Robots 2.0: When artificial intelligence meets steel

Thomas Tomow

The shadows of reasoning – new design paradigms for a gen AI world25:14

The shadows of reasoning – new design paradigms for a gen AI world

Jonas Andrulis

From Code to Motion: Building an Autonomous Hat-Hunting Robot with Kubernetes & ML30:58

From Code to Motion: Building an Autonomous Hat-Hunting Robot with Kubernetes & ML

Daniel Brintzinger

Robots are coming into the wild! Full-Stack Robotics Engineers, be ready!29:34

Robots are coming into the wild! Full-Stack Robotics Engineers, be ready!

Falk-Moritz Schaefer

AI & Ethics28:27

AI & Ethics

PJ Hagerty

How AI Models Get Smarter35:16

How AI Models Get Smarter

Ankit Patel

AI'll Be Back: Generative AI in Image, Video, and Audio Production29:54

AI'll Be Back: Generative AI in Image, Video, and Audio Production

Fabian Pottbäcker, Thomas Endres & Martin Foertsch

Related Articles

View all articles
DC
Daniel Cranney
Stephan Gillich - Bringing AI Everywhere
In the ever-evolving world of technology, AI continues to be the frontier for innovation and transformation. Stephan Gillich, from the AI Center of Excellence at Intel, dove into the subject in a recent session titled "Bringing AI Everywhere," sheddi...
Stephan Gillich - Bringing AI Everywhere
CH
Chris Heilmann
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
Join Scott Hanselman at WWC24 to explore AI's role as a superhero or supervillain. Scott shares his 32 years of experience in software engineering, discusses AI myths, ethical dilemmas, and tech advancements. Engage with his live demos and insights o...
WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?
BB
Benedikt Bischof
How we Build The Software of Tomorrow
Welcome to this issue of the WeAreDevelopers Live Talk series. This article recaps an interesting talk by Thomas Dohmke who introduced us to the future of AI – coding.This is how Thomas describes himself:I am the CEO of GitHub and drive the company’s...
How we Build The Software of Tomorrow

From learning to earning

Jobs that call for the skills explored in this talk.