When it comes to mobile, how does your organization determine what to test?
What devices are in your lab? Are they the right ones?
How do you tell what the right ones are? Are they a mix of devices your test team had on hand, or does your device pool consist of the top devices your users actually use?
What are the critical changes you could make today to your mobile testing strategy to ensure your team is developing a future five-star application?
In the mobile world, the feedback from poor user experience is almost immediate. This can come in various forms. Everything from a website review to a blog to the ever important app store review. Other users see this feedback and use it in their decision to download your app. The main problem with this review system is that the feedback can be very generic or anecdotal. This type of feedback is not helpful in resolving the actual issues with your applications. Development and QA teams need to be able to make actionable decisions based on what their users are seeing with the application.
Applying analytics to your mobile application is the key to translating that generic app store review that stated, “the app is slow,” into a defect your developers can actually fix. Let’s look at an example of how analytics can be applied to help find an issue in an application. For this example, the analytics application we will use is HPE AppPulse Mobile.
HPE AppPulse Mobile allows analytics to be applied quickly and without having to instrument the entire application. Developers can simply wrap the application using the HPE AppPulse Mobile SDK. From there, they can view their user experience in the HPE AppPulse Mobile Dashboard.
In this example, we can see the aggregate data of users using our Android application. Overall, the response times look pretty good. We do see an outlier operating system (OS) version when it comes to performance.
It looks like we have a slowdown in version 4.4.2. Let’s take a closer look to see what we can find.
As we drill into the data, we can see some slower responses on older devices. This insight can be helpful to determine if you need to make changes to the application to support these devices or if they should no longer be supported.
Let’s take a different view and see what transactions are having the problem within the application.
It looks like the “Get stock quotes” action is taking a long time for many of the app’s users. Double-clicking on the action will bring up individual calls to the action for analysis.
It looks like the application is making a series of calls to populate the page. Each of these calls takes about a second, but they are being called serially. If these calls were made in parallel, the page could load in just over a second, instead of 13! This is an actionable defect that could be sent to development, leading to happier users of your application.
It isn’t just your development team who can benefit from analytics. Take this user behavior model that is generated automatically when your users interact with your application.
This view shows how your real users are flowing through your application—screen by screen, including when they exit the app. Do your regression tests mimic this user flow? Why not?
Analytics is the key to the continuous improvement of your mobile application. Without the valuable information that it provides, your company is at the mercy of the next wave of app reviews. Why not stay ahead of the issues and make sure the reviews are all five stars!