Back to Blog

A HEROW'S TALE: THE JOURNEY OF BUILDING A CONTEXT RECOGNITION SDK

Lucas Brechot

Written by Lucas Brechot , Senior Product Marketing Manager, January 7, 2019.

In 2018 the HEROW (formerly Connecthings) team embarked on a journey to develop a mobile SDK that would recognize the physical context of mobile users in real-time and with deep behavioral insights such as user’s recurrent places, home and work locations and commute recognition. Launching this shiny new product in October 2018, the journey has been a roller coaster full of product iterations to deliver the highest efficiency of location-based context recognition. For anyone interested in mobile apps leveraging the physical context of their users or perhaps looking to recognize the context of their own mobile users using location, we hope that our experience can provide interesting insights into the challenges that this development journey involved.

Goal 1: Homogeneous context recognition for all (devices)!

Photo by Hal Gatewood on Unsplash

The single most important goal in the development of the HEROW SDK is to effectively infer the context of mobile users. The method for user context recognition relies first and foremost on user location, and more specifically background location (by background location we mean when a mobile app accesses the device location while the user is not active on the app: app closed or in the background). While this obviously requires irreproachable privacy and transparency measures, when done correctly, background location enables developers to fulfill such a promising potential of mobile apps: to adapt their features, their content, their engagement, their whole service based on user context. User needs are contextual, so app services should be as well.

Challenge 1: Collection location when the app is in the background

The first challenge is that background tasks from apps consume battery. If mobile apps were able to perform any task from a background state, the battery of the phone would deplete. Therefore operating systems must put restrictions on apps using background access to location data or perform any other tasks. While this concern is true across operating systems, it becomes most challenging on Android because each manufacturer has some freedom to adapt their device’s behavior and impose different restrictions. In fact, some manufacturers choose to impose stricter restrictions on background tasks in order to preserve the device’s battery and in turn purchase cheaper batteries.

The challenge of recognizing user context is thus first and foremost to circumvent restrictions imposed on background tasks and to do so in a homogeneous way across devices.

Challenge 2: Recognizing context in a homogeneous way

This need for homogeneous data is a crucial part of HEROW’s mission to deliver valuable insights to apps on their user’s behavior, and it proved to be one of the most challenging aspects of the product. In fact, to analyze truly the real-world behavior of users and most importantly to compare them together, the volume, the precision and the reliability of the data collected needs to be the most possible uniform across users.

An illustration of this is that if an SDK detects user visits at retail stores more precisely on device A than device B, the app might draw the conclusion that users with device A are savvier shoppers, while in reality, it might not be the case.

Data homogeneity is key because a flawed dataset = flawed insights = flawed decisions.

The problem is that location services are not homogeneous across operating systems. For example, a key difference in the native geofence capabilities on iOS and Android is that while iOS geofencing detects zone entries in a precise way, the Android geofencing is slower (especially on Android 7 and above). Therefore, while in iOS the user is detected right at zone entry, in Android the user is detected later, then closer to the center of the geofence. Moreover on top of disparities between operating systems, recognizing context means adapting to the different specificities of mobile manufacturers. The most extreme case being Huawei which quite simply kills all mobile applications that work in the background, under some default device settings. While Huawei is an extreme case, in order to have homogeneous data across devices and users, mobile apps have to adapt to all kinds of device specificities. For example, some devices have underperforming GPS/Wifi/GSM chips, which may affect the precision of location collection.

The HEROW SDK highly improves the native geofencing code in order to level precision and frequency across devices and operating systems.

Challenge 3: Keeping the recognition tool consistent throughout OS versions

While linked to the need for homogeneity, a third challenge that we faced is the need to adapt these procedures every year, at Apple’s and Google’s releases.

The trend among operating systems has been towards battery savings (for reasons mentioned above). Manufacturers and operating systems alike, regulate more and more what impacts the battery of the user. For example, as mentioned before, Android versions above Android 7 started to restrict more severely what can be done when the app is in the background.

What this means is that mobile apps using location, need to be prepared to adapt their code every time that there is a new operating system release. Even if a developer has optimized background location collection for devices that represent 80% of its user base, every year there will still be the need to consequently modify the code to adapt to new operating system versions.

The HEROW SDK offers a solution to this problem by ensuring the highest background efficiency, maintains it across devices and operating systems, and constantly updates it for new releases.

Goal 2: Ensuring the highest level of user privacy

Photo by Matthew Henry on Unsplash

The second goal is to offer highly relevant contextual intelligence, without compromising on the privacy of mobile users. Too many tools focus on user intelligence without ensuring the proper respect of user privacy, and it, unfortunately, linked the location industry with a lack of transparency and accountability.

At HEROW, we believe that the value of contextual intelligence goes way beyond advertising and can be an opportunity for mobile apps to provide a unique experience to each user. It is therefore absolutely essential for the SDK to comply very early on to the highest standards of privacy regulations and opt-in permissions methods.

Challenge 1: Optimizing data privacy processes

User location is considered by privacy regulation, such as GDPR or CCPA, as personal user information. In consequence, a mobile app that leverages the location of its users will need to put in place the proper architecture in order to comply with the privacy measures imposed by such regulation.

Collecting user location in a GDPR-compliant way requires apps to collect active opt-ins from users, explaining what information is used and for what purpose it is used. It also means creating the infrastructure for users who opted-in to request access to the data collected on them, to modify it, to delete it and to stop data collection at any point.

The HEROW SDK not only complied with these measures well before the regulation was in effect but it was also able to mutualize learnings across different client apps to provide guidance and best practices to apps looking to collect location in a privacy respectful way.

Challenge 2: Optimizing permission requests

At HEROW, we also have the objective to ensure that permission requests comply with all different operating system versions and does so in an efficient way.

For example, the release of iOS 13 meant that the SDK had to integrate the new permission pop-ups (more detail here) and change analytics logs to reflect new permission states like the “Only Once” option. As mentioned in the first section of the article the trend in operating systems is to impose more sophisticated methods for mobile apps to request permission for background actions. While this ultimately leads to greater transparency and trust from users, it also means that mobile apps using location need to adapt there code at every new operating system release.

Much like compliance to privacy regulation, the HEROW team automatically updates the SDK code to comply with new operating system releases. It also uses its expertise across app clients to provide true guidance on how to request permission to access user location, maximizing opt-in and user satisfaction.

Goal 3: Achieve all this with minimal battery consumption

Photo by Alvaro Serrano on Unsplash

Finally, the third goal in developing the HEROW SDK is to ensure that the SDK’s context recognition has minimal impact on the device’s battery, creating a seamless experience for the user. Battery performance is a key specification that consumers look for in their mobile phones, and thus the impact of context recognition on battery depletion needed to be minimal.

Challenge 1: Dynamically adapt location statements precision

Mobile devices have essentially three ways to retrieve the location of users: GPS, position triangulation from Wifi signals, and triangulation from cell towers. While GPS is the most precise technique, it is also the most battery depleting. On the other side, cell positioning consumes less battery but is also less precise. Mobile apps that want to ask the location of their users will need to ask the device for the location through an API and specify which level of precision they require.

Note: It is worth noting, that mobile apps are not able to ask for a specific location source. They can only ask for a desired level of precision (low, medium or high precision), although location source is highly inferable from the level of precision requested (low precision will usually be a mix between cell and wifi triangulation)

The challenge is to dynamically modulate the precision based on different user moments. After many iterations, this amounted to an SDK that collects with high precision user locations while keeping battery consumption to a minimum.

Challenge 2: Circumventing the operating system geofence limits

Geofences tracked by a mobile app are an effective way to flag locations where there is a higher need for precision. A mobile app that wishes to track activity at supermarkets can use geofences to ensure high precision around those locations.

The challenge is that in iOS the maximum number of geofences that can be tracked simultaneously is 20, and on Android this number is 100. Therefore an app that wishes to track the location of their users at supermarkets would be limited to 20 supermarkets tracked globally, which obviously is not sufficient coverage.

HEROW offers a solution to this challenge with a system of rolling zones that dynamically downloads and replaces tracked geofences based on the user’s context. This took quite a long time to make it work in all possible conditions, particularly when network coverage is poor at a given place.

Challenge 3: Orchestrating the whole process

Something to consider is that once the location is collected, the SDK still needs to send this information to its servers, which requires access to the internet and therefore consumes battery. It is therefore not sufficient for an app to efficiently track the user location, it also needs to communicate it efficiently. This involves grouping data into batches and sending them at the most opportune moment.

This is really part of a bigger challenge which is the orchestration of tasks needed to efficiently track location with minimum impact on battery consumption. The SDK updates geofences, modulate the frequency and precision of location statements, group data, decide whether to send to a server or not and start all over again. This orchestration of tasks is not trivial. What happens when the user is in a busy urban area and there are 17,000 locations to download?

This problem has kept the HEROW development team busy for many nights and eventually led us to develop a proprietary task manager that ensures the continuous and most efficient flow of these background tasks. Moreover, we have also introduced a caching system (local memory space in the smartphone), to increase notifications displays, even if we don’t have network access all the time.

Fun bonus - All of this needs to be tested!

A mobile app that wishes to access location in the background, in addition to developing all these techniques, also needs to test the performance of its code at each iteration. Because performance is based on location precision and battery consumption, the testing process obligatorily involves someone actually walking around with a device, and accounting for all the different devices and operating system versions possible, which is extremely time-consuming. At HEROW, we have partnered with a subcontractor who makes at least 2 hours of tests each day, with tenth of different devices. We are in a continuous testing process, to always optimize our performance and battery consumption.

And that does it! Once these challenges faced and resolved, mobile apps can effectively leverage mobile user context in their app experiences: engaging users at key locations, segmenting them better, adapting app content and features according to different user contexts. If you have any questions regarding this development journey, we will be happy to answer any questions and provide additional details for the more curious readers. In our next article, we will detail the challenges that follow the development of a context SDK: storing and processing this contextual data within a SaaS platform. Because it’s not all to collect information, the key goal is to take action and to create contextual app experiences based on this information.

We hope you enjoyed the read and wish you the best of luck on your journey towards superior app engagement.

Read it on Medium.

More from the Blog