Imagine a fire at its peak. There is smoke everywhere and high flames that impair the vision. In this scenario, the actions of first responders are very compromised. But imagine again that this reality is not all they see. Through their glasses, they are able to know the position of all the team members, get information about the terrain and communicate with the control centre. This way they can make fast choices and act to tackle the emergency in a short time. Although you might think this scene belongs to the domain of fiction, it’s more than real: it’s Augmented Reality.
Augmented Reality is a technology used in devices to complement our vision with a virtual layer of information. Although its use in disasters is relatively new, the technology has been successfully applied to other fields since its development in the late 1960s. One of the first uses were attributed to the Air Force allowing military to control virtually guided operations. After, it was applied to advertising, tourism, industry and to the video games sector, with the recent exponent of Pokemon GO.
“Since Augmented Reality enhances our senses, its use in disaster situations can be crucial: during emergencies, reality is not enough” comments Giorgia Sassi Communication Manager at JoinPad, an Italian company specialised in this technology. As part of the I-REACT project, JoinPad is currently applying Augmented Reality to create smart glasses applications for first responders. “Either through an object recognition algorithm or through geolocalisation, the technology will able to identify objects in the responders’ field of vision and add important data to aid operations”, explains Sassi.
These glasses will condense all the information from the field and show it to the first responders. Agnese Ragucci, Interaction designer at JoinPad, explains that this way “they will be able to know the position of other teams, the missions assigned to each of them, alerts or warnings in the surroundings or even get help to navigate to assets or points of interest”. In addition, thanks to this technology, responders will be able to report from the ground without using their hands through videos, voice and text. “In I-REACT one of important tasks is to investigate speech-to-text and text-to-speech algorithms to enrich the user experience” adds Ragucci.
The project’s smart glasses offer another radical advantage by enabling the formulation of precise instructions from the control centres to the responders. “One of the biggest limitations in disaster response is when the operator has to describe the scenario remotely through a mobile phone and then be able to interpret the instructions in a very short timeframe” explains Ragucci. However, the I-REACT glasses will overcome this limitation as they are coupled to a camera to send videos to the control centre. “With our smart assistance system, the first responders will become the eyes of the decision-makers, which will greatly improve the operation efficiency and speed up the reaction times” she adds.
Whereas big efforts are being made in the I-REACT consortium to collect and process information from multiple sources, the role of JoinPad is to ensure that this information is provided to the end-users in the most effective way. “Our main challenge lays in the user experience”, explains Ragucci, “we have to be careful to provide the best information, at the right time and in clear and simple way”. “The information will be provided in the responders’ field of vision so it has to be shown in a non-invasive way, not to compromise the operations” she states.
The company is expecting to have a prototype of the Augmented Reality interface in two months after which it will be implemented on the device. A key issue in this process will be to provide continuity and interaction with the other applications in the project, such the mobile app or the wearables, that are also going to be used by responders. Finally, the technologies will be tested in several in-field demonstration to ensure a good performance in a realistic scenario.
“Our project will represent a breakthrough in the application of Augmented Reality to disaster management”, concludes Sassi. Indeed, the I-REACT smart glasses application will allow for a faster and more coordinated response, which might be vital to save lives during emergencies. This will be in a year’s time and, until then, we will have to content ourselves with the real world.
