Augmented Reality Game for Microsoft HoloLens: Gaming App Powered by Unity and Windows Mixed Reality
A startup company.
The client planned to hit the market with an AR-powered PokemonGO-like recommendation app for venues, events, and fun activities. The client’s initial business goals were to launch a PoC iOS app, test it in London, and raise the investment for a fully-fledged product.
The customer wanted to hire an experienced team of both iOS developers and AR engineers who can quickly research the new ARKit technology and build the prototype app within 2 months — to demo it to the investors at the WebSummit Lisbon.
The app in focus is a recommendation service with AR capabilities. The user can recommend a venue or some activity to his friends by placing the clickable marker, Pin, on the map (via Apple Maps) and adding the commentaries to it through the attached images/video fragments.
To transform an old-fashioned social recommendation service model into an interactive AR experience, Softeq’s team introduced a combination of the brand-new iOS framework ARKit, analytical algorithms and Apple Maps/Google Places services. While the user opens AR Mode — he sees not only a plain map with the 2D icons of the recommended venues, but the graphically rich AR Pins with the audio and video content, overlaid on the real-world places by means of ARKit framework.
Major app’s functionality:
1) Upon the registration the user adjusts various filters — preferences, personality, interests. Before each session, he adds information about the current budget, distance, time and mood. The application processes personal user data with the analytical algorithms to sort out places or events that might interest the person.
2) Leveraging the location-based AR and integrated map services, the app tracks the user’s geo position and shows him the Pins of suitable events/places both on the map and in the AR Mode.
3) The user can make photo/video commentaries about the place or event and attach it to the Pin on the map — to explain his friends why they should visit the venue. Both image and video content are optimized, so the recurrent graphical processing doesn’t affect the app’s performance.
4) The app has an embedded messenger, which allows users to discuss upcoming events without leaving the app.
5) The app is integrated with Facebook — to analyze user’s interests and extract his friends’ profiles from the contact list.
6) The app involves a gamification element — each user has a dynamic rating, depending on the popularity of his Pins. If people like an activity the user recommended, they rate it. The higher rating the user has — the more bonuses from the app and the venues he gets.
GPS, as the core technology which underlies a location-based AR, places some limitations on the Pins positioning accuracy — the maximum achievable GPS’s positioning accuracy is about 10-30 meters.
To increase the tracking accuracy, Softeq had to construct a virtual coordinate system and map it to the real-world positioning, taking into account the location, orientation and motion of the mobile device. The coordinate system was designed by means of ARKit, while the real-world location tracking system was implemented through the combination of the MapKit and CoreLocation iOS frameworks. MapKit allows to integrate into the app the Apple Maps and Google Places services. The CoreLocation utilizes Wi-Fi, GPS, magnetometer, barometer, and cellular hardware to gather data, which helps to position the AR Pins as accurately as possible — both in the Map and AR Modes.
It took Softeq’s team less than 2 months to develop the POC app and the client presented it at the WebSummit as planned.
The next major step is to transform the prototype into a fully-fledged product, which includes the backend development and integration of the neural network based analytical algorithms.