Zaid Zaim
Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
#1about 3 minutes
Introducing the concept of an immersive AI copilot
The talk introduces building an interactive AI character inspired by Disney's WALL-E using immersive technologies and generative AI.
#2about 3 minutes
Using Microsoft Mesh for collaborative XR experiences
Microsoft Mesh integrates into Microsoft Teams to provide a platform for immersive meetings, trainings, and social events.
#3about 1 minute
The core technologies for building an immersive copilot
The project is built using three main components: the Unity game engine, the Mesh Toolkit for deployment, and Azure OpenAI for intelligence.
#4about 5 minutes
Integrating Azure OpenAI service with Unity via REST API
A C# script in Unity calls the Azure OpenAI REST API to handle communication, parse the JSON response, and manage text-to-speech output.
#5about 2 minutes
Using the Mesh Toolkit for cross-platform deployment
The Mesh Toolkit provides shaders, visual scripting, and an optimizer to prepare and deploy the Unity environment to the Mesh platform.
#6about 6 minutes
Live demo of the AI copilot on a HoloLens device
A live demonstration shows interaction with the AI character "Anna" using voice commands on a Microsoft HoloLens.
#7about 6 minutes
Live demo in a shared Microsoft Mesh environment
The AI copilot is demonstrated within a shared, multi-user Microsoft Mesh space, showing collaborative interaction with the character.
#8about 6 minutes
Integrating immersive spaces directly into Microsoft Teams
The final demo showcases how users can join a Microsoft Mesh immersive space directly from a standard Microsoft Teams meeting.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
03:03 MIN
Live demo of the immersive AI assistant Anna
How Mixed Reality, Azure AI and Drones turned me into a Magician?
03:32 MIN
Fusing mixed reality, AI, and drones for magical experiences
How Mixed Reality, Azure AI and Drones turned me into a Magician?
01:51 MIN
Exploring the core technology stack for the project
How Mixed Reality, Azure AI and Drones turned me into a Magician?
04:24 MIN
A practical walkthrough of the Azure AI Foundry playground
How Mixed Reality, Azure AI and Drones turned me into a Magician?
02:06 MIN
Creating an intelligent cockpit experience with AI and AR
Software defines the vehicle: Why customers and developers will love cars even more
01:27 MIN
Getting started with the Azure AI Foundry platform
How Mixed Reality, Azure AI and Drones turned me into a Magician?
03:35 MIN
Live demo of a cross-platform XR interaction toolkit
Apple Vision Pro: Proven Development Methods Meet the Latest Technology
02:41 MIN
Exploring the tools for drone and MR development
Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Featured Partners
Related Videos
35:26How Mixed Reality, Azure AI and Drones turned me into a Magician?
Zaid Zaim
38:28Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Zaid Zaim
23:33Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact
Oliver Will
24:24Best practices: Building Enterprise Applications that leverage GenAI
Damir
31:39Livecoding with AI
Rainer Stropek
1:12:04WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney
1:09:49Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More
Simi Olabisi
26:35Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure
Ricardo
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
webLyzard
Vienna, Austria
DevOps
Docker
PostgreSQL
Kubernetes
Elasticsearch
+2
Crossover
Ulm, Germany
Remote
€100K
Senior
Node.js
Crossover
Dortmund, Germany
Remote
€100K
Senior
Node.js
Tether
Barcelona, Spain
Remote
Tensorflow
Machine Learning
Crossover
Köln, Germany
Remote
€100K
Senior
Node.js