Software Development Principles and Practices

Testing
Week 7

If you’re not failing, you’re not trying hard enough.

— Martin Fowler
Objectives
● Understand the importance of software testing
● Understand various types of testing
Content
● Definition and importance of testing
● Types of testing
● Testing for our project
○ Unit testing
○ Test doubles
What is Testing?
● Process of evaluating software to gain confidence that it
does what it is supposed to do

● Simple test
Controlled Environment

Single behavior Observable

Specific input Expected output
to test output
 Simple Test Example
// Verifies a Calculator class can handle negative results.

public void main(String[] args) {

Calculator calculator = new Calculator();

Expected Output
int expectedResult = -3; Specific Input

int actualResult = calculator.subtract(2, 5);

Observed Output
// Given 2, Subtracts 5.
Behavior To Test
assert(expectedResult == actualResult);

}
Why Test?
● Improves the design of your system
○ Testable code is good
○ Good code is testable
● Serves as executable, up-to-date documentation
● Enables fast, high-quality releases
● Caveats
○ Edsger W. Dijkstra: “Program testing can be used to show the
presence of bugs, but never to show their absence!”
Types of Testing: Purpose
● Functional testing
○ Does my app do what it's supposed to?
● Performance testing
○ Does it do it quickly and efficiently? Does it handle high workloads?
● Usability testing
○ Can users easily accomplish their objectives with the software?
● Acceptance testing
○ Does it satisfy all the contracted requirements from the user?
● Compatibility testing
○ Does it work well on every device and API level?
Types of Testing: Scope
● Unit testing
○ Checks functionality of one method/class
● Integrated testing
○ Checks that multiple components interface correctly
● End-to-end testing
○ Checks that scenario specifications are met
Types of Testing: Environments
● Local tests
○ Tested in a development machine or server
● Instrumented tests
○ Run on actual devices (e.g.: physical Android devices)
Types of Testing: Prior Knowledge
● White box testing
○ Exploits structure within the program
○ Tests how the program creates output
○ Tests the logical structure of the program
● Black box testing
○ Checks whether the application functions as expected by users
○ Creates test cases based on requirements and specifications
○ Tests what the program does
○ We don’t care how the application does it as long as it outputs
the correct answer
Types of Testing: Subject
● Manual test
○ A human performs the tests step by step, without test scripts
○ Easy to perform and allows a degree of flexibility
○ Slow, subjective, and hard to scale
● Automated test
○ Tests are executed automatically via test automation frameworks,
along with other tools and software.
○ Fast, objective, and scalable
○ Increased confidence in changes
Test Strategies
● Fault injection
○ Inject exceptions, simulate failures to check if the app works in the
presence of bad inputs, bad returns from libraries
● Fuzz testing
○ Multiple random inputs are thrown at your code
○ Tests app the way it wasn’t meant to be used
● Mutation testing
○ When a deliberate error is introduced in code, does some test
break?
Unit Tests
● Narrow scoped tests to reduce bugs
● Tests a single function, class, or method
● Run very often and must be fast
● Does not well reflect user perspective of the app
● Not affected by external systems
○ External dependencies must be replaced with test doubles
How to Create Unit Tests?
● Unit tests must focus on success, error and edge cases
● Edge cases are uncommon scenarios that human testers
and larger tests are unlikely to catch
○ Math operations using boundary conditions, induce overflow
○ All the possible network connection errors
○ Corrupted data, such as malformed JSON
○ Simulating full storage when saving to a file
○ Object recreated in the middle of a process (such as an activity
when the device is rotated)
Test Double
● An object or function that can replace a real
implementation in a test to remove dependencies
○ Like a stunt double for an actor in an action movie

● Test doubles create controlled environment for unit tests to

○ Increase speed
○ Avoid undesired side effects during unit testing
○ Remove non-deterministic behavior
 Test Double

Source: https://martinfowler.com/bliki/UnitTest.html
 Test Double Example (1/3)
● Situation: Implementing an e-commerce to process credit card
services. We want to test that our method behaves correctly
when the credit card is expired.
class PaymentProcessor {
private CreditCardService creditCardService;
boolean makePayment(CreditCard creditCard, Money amount) {
if (creditCard.isExpired()) { return false; }
boolean success = creditCardService.chargeCreditCard(creditCard, amount);
return success;
}}

● Can’t use an actual card to test makePayment → use test double!

Source: https://abseil.io/resources/swe-book/html/ch13.html#example_onethree-onedot_a_credit_card_s
 Test Double Example (2/3)
● Create test double class

class TestDoubleCreditCardService implements CreditCardService {

@Override
public boolean chargeCreditCard(CreditCard creditCard, Money amount) {
return true;
}
}
 Test Double Example (3/3)
class PaymentProcessor {
private CreditCardService creditCardService;
PaymentProcessor(CreditCardService creditCardService) {
this.creditCardService = creditCardService; Dependency Injection
}
...
}
public class TestDoubleTest { Use the created test double class
PaymentProcessor paymentProcessor =
new PaymentProcessor(new TestDoubleCreditCardService());

@Test public void cardIsExpired_returnFalse() {

boolean success = paymentProcessor.makePayment(EXPIRED_CARD, AMOUNT);
assertThat(success).isFalse();
} Internally uses the test double instead
} of actual CreditCardService
Test Double Types
● Dummy
● Fake
● Stub
● Spy
● Mock
Dummy
● Most primitive type of test double
● Empty implementation that does not perform any action
● Used when we need an instance to pass
Dummy: Example
public interface Logger {
void log();
}

public class LoggerDummy implements Logger {

@Override
public void log() {

}
}
 Dummy: Example
@RunWith(MockitoJUnitRunner.class)
public class ExampleUnitTest {
public class CustomerReader { // Dummy logger
private MyDataBase database; LoggerDummy dummyLogger = new
private final Logger logger; LoggerDummy();
public CustomerReader(Logger log) { // Class to be tested
logger = log; private CustomerReader customerReader =
} new CustomerReader(dummyLogger);
// ... // ...

} @Test
public void happyPathScenario(){
// ...
}
}
Fake
● A much simplified but working implementation of the
original function
● For instance, in-memory data structure instead of file
system to store information
● Use when the productivity significantly improves
 Fake: Example
// A fake file system.
public class FakeFileSystem implements FileSystem {
// Files are stored in an in-memory map, not on disk, to avoid disk I/O in tests.
private Map<String, String> files = new HashMap<>();
@Override
public void writeFile(String fileName, String contents) {
files.add(fileName, contents);
}
@Override
public String readFile(String fileName) {
String contents = files.get(fileName);
// The real impl. will throw this exception if the file isn’t found, so the fake must throw it too.
if (contents == null) { throw new FileNotFoundException(fileName); }
return contents;
}
}

Source: https://abseil.io/resources/swe-book/html/ch13.html#example_onethree-oneonedot_a_fake_file
Stub
● Fake class with no logic that comes with pre-programmed
return values
● Use when you need a function to return a specific value to
reach a certain system state
 Stub: Example (1/3)
public class CustomerReader { External dependency
private MyDataBase database;
public String findFullName(Long customerID) {
// … code here …
Customer customer = database.find(customerID);
return customer.getFirstName() +" "+customer.getLastName();
}
}
Stub: Example (2/3)
public class CustomerReaderTest {
● Naïve approach: Pre-fill a // Class to be tested
private CustomerReader customerReader = new

database with customers CustomerReader();

// Dependency
● Creates a hard dependency private MyDataBase database = new MyDataBase();

on a database (not isolated @Test

public void happyPathScenario()
// Prefill the database
as unit tests should be) Customer sampleCustomer = new Customer();
sampleCustomer.setFirstName("Susan");
sampleCustomer.setLastName("Ivanova");
customerReader.setDataBase(database);

// Add to database
database.add(sampleCustomer);

// Test the function

String fullName = customerReader.findFullName(1);
assertEquals("Susan Ivanova",fullName);
}
}
Stub: Example (3/3)
public class CustomerReaderTest {
● Completely remove the // Class to be tested
private CustomerReader customerReader = new CustomerReader();
database dependency & // Dependency --> mocked
private MyDataBase database = mock(MyDataBase.class);

stub the database @Test

public void happyPathScenario(){
connection instead // Prefill the database
Customer sampleCustomer = new Customer();
sampleCustomer.setFirstName("Susan");
● Interactions unaffected: sampleCustomer.setLastName("Ivanova");
customerReader.setDataBase(database);
database.find() returns a // Stub the find() function to return the sampleCustomer
when(database.find(1))
.thenReturn(sampleCustomer);
predetermined value // Test the function
String fullName = customerReader.findFullName(1);
assertEquals("Susan Ivanova",fullName);
}
}
Spy
● Stubs that record some information based on how they
were called
● Example: an email service that records how many
messages were sent
Spy: Example
public class LoggerSpy implements Logger {
private int numberOfCalls = 0;
@Override
public void log() {
numberOfCalls++;
}

public int getNumberOfCalls() {

return numberOfCalls;
}
}
Mock
● Creates a fake object to check if the code behaves as
expected
● Mock tests interactions/behaviours whereas stub tests
states
● Example usage: to check the number or order of function
calls
 Mock: Example (1/3)
public class LateInvoiceNotifier {
private final EmailSender emailSender;
private final InvoiceStorage invoiceStorage;

public LateInvoiceNotifier(final EmailSender emailSender, final InvoiceStorage

invoiceStorage) {
this.emailSender = emailSender;
this.invoiceStorage = invoiceStorage;
}

public void notifyIfLate(Customer customer) {

if(invoiceStorage.hasOutstandingInvoice(customer)) {
emailSender.sendEmail(customer);
}
}
}

We want to test notifyIfLate() that does not return anything. How do we test it?
Mock: Example (2/3)
Check emailSender.sendEmail is correctly invoked depending on outstanding invoice
public class LateInvoiceNotifierTest {
//Class to be tested
private LateInvoiceNotifier lateInvoiceNotifier;
//Dependencies (will be mocked)
private EmailSender emailSender = mock(EmailSender.class);
private InvoiceStorage invoiceStorage = mock(InvoiceStorage.class);
//Test data
private Customer sampleCustomer;

@Before
public void setup(){
lateInvoiceNotifier = new LateInvoiceNotifier(emailSender,invoiceStorage);
sampleCustomer = new Customer();
sampleCustomer.setFirstName("Susan");
sampleCustomer.setLastName("Ivanova");
}
 Mock: Example (3/3)
public void notifyIfLate(Customer customer) {
if(invoiceStorage.hasOutstandingInvoice(customer)) {
emailSender.sendEmail(customer);
}
@Test }
public void lateInvoice() {
when(invoiceStorage.hasOutstandingInvoice(sampleCustomer)).thenReturn(true);
lateInvoiceNotifier.notifyIfLate(sampleCustomer);
verify(emailSender).sendEmail(sampleCustomer);
}
Verifies expected behavior happened
@Test
public void noLateInvoicePresent() {
when(invoiceStorage.hasOutstandingInvoice(sampleCustomer)).thenReturn(false);
lateInvoiceNotifier.notifyIfLate(sampleCustomer);
verify(emailSender, times(0)).sendEmail(sampleCustomer);
}
}s
 Test Double Spectrum

Source: https://hudi.blog/test-double/
Sources
- Cornell Univ 18, 19장
- Software Engineering at Google book (16,17,18)
- ESaaS chapter 8 youtube playlist
- ESaaS slide chapter 8
Thank You.
Any Questions?