Okay, this latest Microsoft patent filing is a bit of a trip but let’s try to break it down.

Our homes and workplaces are increasingly filled with IoT devices (Internet Of Things) which are used to connect vehicles, homes, appliances, or any other type of electronic device to the Internet. One example of an IoT device is a smart thermostat that automatically monitors and controls the temperatures of our homes.

In this filing, Microsoft wants to use VR headsets (or AR devices) to visualise the sensor data being picked up by all of these IoT devices in our local environment.

The main problem Microsoft is trying to solve is helping people easily track where IoT devices are located and what conditions they are monitoring. For example, the temperature conditions of a room can be monitored, but we may not know which room is specifically being monitored. Given how portable, small and ubiquitous IoT devices are becoming, Microsoft foresees a problem of managing all of these devices.

For example, in the above image, there is a temperature sensor that is monitoring conditions in the kitchen. In the headset, the temperature data of the device is visualised to show that the area close to the pot is hotter than the rest of the environment.

I think this filing is interesting in that it describes augmenting human senses by leveraging all of the sensors and data captured by IoT devices. For example, imagine if you walk into a building, your headset device could essentially allow you to ‘see’ another room that you’re not in by connecting to the temperature sensors in that room.

Where is this useful? If there was a fire in a building, perhaps soon we’ll be sending in robots that are remotely controlled by humans. Using data from IoT sensors, the robot could show the human controller where the fire source is located and if there are any humans in the building. Using a VR headset, the human controller could then navigate the robot to rescue anyone in danger or put out the fire.

This is of course speculative, but the idea of VR headsets / AR glasses syncing up with sensor data from IoT devices to augment our senses is something that will undoubtedly happen.