Uncovering breaking changes behind UI on mobile applications

Uncovering breaking changes
behind UI on mobile applications
HeadSpin
Senior Software Engineer, Device Automation

Hi✋
• Kazuaki Matsuo
• from ", GitHub: KazuCocoa, Twitter: @Kazu_cocoa
• HeadSpin
• Senior Software Engineer now, former was a test/QA engineer
• Maintain Appium project

Topics
• Case study:
• Combination of capturing HTTP requests over Appium scenarios
• Tips:
• Some tips for Appium

App: Long running app
• Many historical code
• Multiple teams developed one app
• Automated test environment was not matured enough
• Both Android and iOS

Situation: Collecting user ﬂow
• Tracking user behaviour is important
• evaluate efforts by features, campaigns, etc
• Should be reliable
• The data was wrong case happened after
releasing a new version
• Unreliable data
• Monitoring it, get rid of them from the
analytics

User events, ﬂow
Open the app
Click the button
Go to the view

Events to track actions
{
“events”: [
{
“name”: “OpenAnAppFromLauncher”,
“time”: ”2019-06-09 13:39:04 +0900” # time series to track user actions
},
{ …. }
]
}

Store the log on the app
Store the log
{
“events”: [
{
“name”: “OpenAnAppFromLauncher”,
“time”: ”2019-06-09 13:39:04 +0900”
} 
]
}

Send the log to the server
Store the log
Send to the stored log
to server

UI and non-UI
• UI layer
• User actions
• non-UI layer
• Store the data into local
• Send it to servers
UI
no UI

“behind UI”
• Behind UI in this title means such non-
UI layer
• Event logs
• HTTP requests
• Too many event logs/few event logs
unexpectedly

Might break
non-UI layer cases

Android
• Android has two main component to build UIs
• Activity
• Base view
• Fragment
• Re-useable view component

Lifecycle: Activity & Fragment
Activity Fragment

Before they are active
Activity Fragment

Where we set logs?
Activity Fragment

Easy to mistake
• “Open”, from where?
• Open the app from icon
• Open the app from app history
• Open the app from notification bar
• Back button and go to the app view from another
app
• They would like to define user flow
• We must define them correctly

From App Icon?
app

From App History?
app

From App Notiﬁcation Bar?
app

From Back Button?
app

Where we set logs, again
Activity Fragment

Fragment
• Fragment is re-usable component
• Unexpected usage:
• A developer put a log into a fragment, for view A
• Another developer uses the fragment in another view, for view B,
while a log deﬁned in the fragment should not be reused on view B
• What happen if a user open the view A and B?
• The log event happens on both view A and view B
• The data will break since the log should be called only on view A
View A
View B

Change their implemented languages
• Developers change their implementation languages
• Objective-C -> Swift
• Re-write to Swift, the behaviour should not change

Prototype, Implement more
• prototype/minimal implementation
• Evaluate the value
• It is worth
• Re-write/refactor/add new features
• Do not change logs to make the data
comparable

Various path to move views
View A
View B
View D
View C

User ﬂow
View A
View B
View D
View C

Wrap up the issue behind UI
• Logging user ﬂows
• Android has view lifecycle which is complicated
• New programming language has been published
• Developers rewrite/refactor their product code
• Long running code has history
• It is not easy to introduce new testable architecture

Solution
• Capture HTTP requests during test running by automation
• Proxy server tools: WireMock, http_proxy
• Can we prevent such breaks only human communication to make
sure the behaviour?
• Probably, sometimes we can, but we cannot almost time
• Against release modules (or the same conﬁguration)

HTTP proxy server
• Need a proxy server to capture the requests
• We have some kind of ways to capture it

How to capture the network?
WiFi
Proxy serverNo proxy settings
on the device
Change Proxy Setting
on the device
Reverse
proxy
No proxy server
Application
Servers
Change app endpoint and connect directly

Where?
WiFi
on the device
on the device
Reverse
proxy
Application
Servers
No proxy server

Which way?
• We can choose various points to capture the request
• Which way was the most ﬁt for us?

Some points
• Wanted to handle test environment in the test script
• Would have like to start from small implementation
• Less conﬁguration in server side

Direct connect
WiFi
on the device
on Android device
Reverse
proxy
No proxy server
Application
Servers

How to connect to the proxy server?
Proxy server
Reverse
proxy
Application
Servers
• Change the endpoint with:
1.Build conﬁguration (need rebuild)
2.Environment variables, process
arguments, intent arguments
3.Change the preference via helper
app like io.appium.settings

2nd way
Proxy server
Direct connect
Reverse
proxy
Application
Servers
• Change the endpoint with:
1.Build conﬁguration (need rebuild)
2.Environment variables, process
arguments, intent arguments
3.Change the preference via helper
app like io.appium.settings

Test Communication ﬂow
between the Appium client to servers
Proxy
server
Server
Driver
App
Test
target

Which parts are handled by Ruby
Proxy
server
Server
Driver
App
Test
target

Capture the http requests in proxy server
Proxy
server
Server
Driver
App
Test
target
Capture data

With Appium
• Appium can control building test environment outside the app with the
same script
• Appium’s test cycle is before running the app under test
• Leverage the system to iOS and Android
• Espresso is after onCreate, XCTest is after launching the app
• XCTest was not released when I started to try this issue (2016)
• Espresso did not work on our app then (test not found happened
frequently, then)

Basic Appium ﬂow
Conﬁgure the device/the test app Run tests on the test device Remain apps/delete apps

Appium’s test lifecycle
def setup()
// call create session
end
def test_some_cases()
// processes
end
def teardown()
// quit
end
Can conducts tests
Keeping the session

Espresso/XCTest
Setup Test cases Teardown
Runner install the test
target before test scripts
Espresso
XCTest

XCTest/Espresso
• Below setup and test cases are called after ﬁnishing to install the test
app
class appiumConfTest: XCTestCase {
override func setUp() {
// simulator is already running
XCUIApplication().launch()
}
func testExample() {
}
}
@RunWith(AndroidJUnit4::class)
class ExampleMainActivityTest {
private val testWatcher = MyTestWatcher()
private val scenarioRule =
ActivityScenarioRule<MainActivity>(
MainActivity::class.java)
// AndroidTestRunner launch onCreate (for example)
@get:Rule val ruleChain =
RuleChain.outerRule(testWatcher)
.around(scenarioRule)
@Test
fun test() {
//...
}
}
XCTest Espresso

HTTP proxies
• WireMock: https://github.com/tomakehurst/wiremock
• Java
• http_proxy: https://github.com/KazuCocoa/http_proxy
• Elixir

WireMock/http_proxy
• Can control responses/requests as JSON format
• Very handy to launch them as standalone
{
“request”: {
“path”: “/request/path”,
“port”: 8080,
“method”: “GET”
},
“response”: {
“body”: “<html>hello world</html>“,
“cookies”: {},
“headers”: {
“Content-Type”: “text/html; charset=UTF-8",
“Server”: “GFE/2.0"
},
“status_code”: 200
}
}

Integrate with Appium
• Launch proxy server before launching the test app
• Can conﬁgure test environment from test scripts
• Test lifecycle is greater than XCTest/Espresso
• Before installing test apps ~ after uninstalling them

Test suite
Conﬁgure the device
the test app
Run tests
on the test device
Remain apps
delete apps
def setup()
// call create session
end
def test_some_cases()
// processes
end
def teardown()
// quit
end
Start Stop
Analyse
the data
Summarise
test results
Proxy

Assert the captured data
• Count the number of HTTP requests
• Parse the JSON body and make sure the logs
• If something happens, the Ruby script raises failure

Summary
• Test lifecycle on Appium
• Getting HTTP requests via a proxy server
• How I implemented the capturing data behind UI

Process arguments
• XCTest framework provides
environment arguments for
xcodebuild argument
• An app can handle it as
env[“PROCESS_VALUE"] in the
app
• The environment works only
for the process
{
platformName: :ios,
automationName: 'XCUITest',
...
processArguments: {
env: {
"PROCESS_VALUE": "process value"
}
}
}

Packages, instrument args
• Android case
• We can make the release
app debug mode with:
• Launch an activity with
the arguments (adb
command)
• Install a particular
package name
{
platformName: :android,
automationName: 'uiautomator2',
...
optionalIntentArguments: '--ez key_name true’,
}
Package name:
com.example.randome_package_names

Simulating push notiﬁcation, launch via history
• When we would like to emulate background to foreground
• Launch the all without auto launch
• Launch the app with broadcast receiver
• partially, we can emulate
• suspend/back
• Start activity with process arguments

One more
• HTTP request is not the only thing which happens behind the UI
• CPU power etc
• Recently, we can see “performance measurement feature” by Google and Apple to capture them in particular
period
• https://developers.google.com/web/fundamentals/performance/rail
• This is mainly for Web, but this kind of performance should be same as mobile
• XCTMetric, benchmark etc
• https://github.com/KazuCocoa/droid-monitor
• Easy to collect data by Android via add/linux commands
• Nowadays, HeadSpin can do more rich thing tho…
• HeadSpin (company, service)

Takeaways
• Happening behind UI is not easy to uncover, but they are also
important
• The beneﬁt of Appium to conﬁgure test environment
• Some tips to control the test app condition via Appium capability

Thanks
email: kazu@headspin.io
Twitter: @Kazu_cocoa

Uncovering breaking changes behind UI on mobile applications

More Related Content

What's hot (20)

Similar to Uncovering breaking changes behind UI on mobile applications (20)

More from Kazuaki Matsuo (20)

Recently uploaded (20)

Uncovering breaking changes behind UI on mobile applications