Augmented reality allows us to enrich reality with virtual elements. Information is superimposed, and 3D objects complete the reality to obtain an augmented version. This technology has applications in many different fields, such as video games, museum visits and industrial processes.
The communication between electric vehicles and charging stations is part of Trialog’s expertise, and we are now trying to apply augmented reality to the field of electric mobility. Thus, we have created a proof of concept to facilitate the use of the charging stations with the assistance of augmented reality: the Augmented Reality for Electric Vehicles (AR4EV) application.
Presentation of the application
In contrast to the gas pump, the charging station is a new essential tool for the use of an electric vehicle. Before a user buys an electric car, he/she may not be familiar with new technologies and may not know how to use a charging station.
Through our AR4EV application, the user can be guided during his or her use of a charging station. With a few clicks, the application can detect the charging station due to augmented reality and display the guides and information associated with the station. The application is a more interactive user manual and provides support for the user in case of an issue. The AR4EV application can generate a diagnosis of the issues encountered and provides feedback to improve the user experience of the charging station.
Figure 1: State diagram of the application
Use cases
Charging
There are many types of charging stations, which can be complicated to operate. Users can employ our application when they encounter a new charging station for the first time. A user must only aim their smartphone at the charging station and the application will recognise it and display the guides and types of cables that can be used.
Figure 2: First use case – charging the car
Diagnosis
When a user encounters issues with a station, they do not always know what the error codes mean or who to contact for help. Through our application, the user can indicate the brand of the station and the error code on the screen. Possible solutions will then be displayed and contact with a technician will be proposed if the error is not listed.
Figures 3 and 4: Second use case – sending an error report to the technician
Visualisation
If the user has never owned a charging station, our application can offer a preview of it. The user will be able to see what the installation of a charging station looks like at home and will become familiar with the use of this station in advance.
Figure 5: Visualisation of the charging station
Technologies used
This application uses several technologies. Additionally, we used various tools to develop the application.
Figure 6: Connections between the tools (green boxes indicate the tools used)
3D scan/photogrammetry
The use of model recognition requires 3D models of the charging stations, which we do not possess. Thus, we must model the stations. The modelling of an object can be achieved using a 3D scanner, which can provide more detail but is also extremely expensive.
With the development of image processing techniques, we now have an alternative: photogrammetry. This method offers a solution to model an object using photographs of it. All that is required is photographs of the object from various angles, which are then processed using software, for example, Meshroom[1] (open-source photogrammetry software).
Figure 7: The photogrammetry[2] process
After obtaining the initial model, it must still be processed because it is often not usable directly after photogrammetry. The retopology is achieved using PC software, such as Blender[3] or Catia[4]. We used Blender, which is open source and is very powerful[5]. It can create linear or non-linear animations and videos. Moreover, it can handle lighting by ambient occlusion and radiosity. It can even simulate realistic fluids, smoke and fire. For example, the movies Spider-Man 2 and Captain America were created using Blender. Furthermore, NASA and Ubisoft have chosen Blender for their 3D models.
Figures 8 and 9: Models of our charging stations
Augmented reality
Augmented reality (AR) is the basis of our application. It is used to recognise charging stations, position instructions, and provide navigation.
There are many APIs available to realise AR, such as ARCore[6] by Google, ARKit[7] by Apple, and Vuforia[8] by PTC. Among the APIs, ARCore and ARKit are primarily used to develop mobile games such as Pokémon GO, and Vuforia is often used in the industry. Thus, we chose Vuforia for our application.
Figure 10: APIs of augmented reality
To facilitate the development of the application, we used a graphic engine. There are two main graphics engines on the market: Unity3D[9] and Unreal Engine[10]. Both engines are similar in terms of functionality, i.e., they can create video games, applications, or movies. However, Unreal Engine is heavier (the size of an empty project is approximately2 GB) due to its strong image processing capacity, which we have no use for. Thus, we chose Unity3D (empty project 100 MB).
Figure 11: Graphic engines
Conclusion
The AR4EV application is an exploratory work in the field of augmented reality. Here, we present the initial version of our application. Several features have been realised:
- Visualisation of the charging station
- Recognition of the charging station
- Charging process for an electric vehicle
- Sending of the error report
This prototype shows satisfactory operation. Thus, we can think about applications in other fields, such as energy or health.
In the future, the capabilities of this tool can be extended. For example, a specific version of the application could be created for technicians, so that they can have more details on a terminal that they intervene on. We will also attempt to apply digital twins in our application, which would allow us to connect with a charging station and inspect the status of that station.
References
[1] “AliceVision | Photogrammetric Computer Vision Framework”. https://alicevision.org/.
[2] “(PDF) Production, exploration and analysis techniques of virtual ….”, 5 juil.. 2022. https://www.researchgate.net/publication/320409287_Production_exploration_and_analysis_techniques_of_virtual_archaeological_environments.
[3] “blender.org – Home of the Blender project – Free and Open 3D ….”. https://www.blender.org/.
[4] “CATIA – Dassault Systèmes”. https://www.3ds.com/fr/produits-et-services/catia/.
[5] “Blender – Wikipédia.” https://fr.wikipedia.org/wiki/Blender
[6] “ARCore – Google Developers”. https://developers.google.com/ar.
[7] “Augmented Reality – Apple Developer”. https://developer.apple.com/augmented-reality/.
[8] “Logiciel de Réalité Augmentée, Vuforia, RA industrie | PTC”. https://www.ptc.com/fr/products/vuforia.
[9] “Unity Real-Time Development Platform | 3D, 2D VR & AR Engine”. https://unity.com/.
[10] “Unreal Engine: The most powerful real-time 3D creation tool”. https://www.unrealengine.com/.
