Testing Web Applications Through User Interface Constraints (CASCON 2015 Talk)
0 likes1,127 views
We shall explore the concept of web application testing through the verification of declarative constraints on the contents of its user interface --that is, DOM, CSS and behavioural properties. It turns out that a lot of non GUI (e.g. functional, security) problems can also be verified by observing the application's GUI.
![$(document).mouseUp(function(event) {
var e = arguments.callee;
if ($(event.target).text() === "Go") {
var current_values = [];
$(".value").each(
current_values.push($(this).text());
);
if (e.lastValues !== undefined) {
var found = false;
for (var v in current_values) {
if ($.inArray(v, e.lastValues)) {
found = true;
break;
}
}
}
if (!found)
console.log("Error");
e.lastValues = current_values;}
};);
The same in jQuery](https://image.slidesharecdn.com/cascon-2015-151102142813-lva1-app6891/85/Testing-Web-Applications-Through-User-Interface-Constraints-CASCON-2015-Talk-40-320.jpg)
More Related Content
Viewers also liked (20)
Similar to Testing Web Applications Through User Interface Constraints (CASCON 2015 Talk) (20)
More from Sylvain Hallé (20)
Recently uploaded (20)
Testing Web Applications Through User Interface Constraints (CASCON 2015 Talk)
- 1. Testing Web Applications Through User Interface Constraints Sylvain Hallé Université du Québec à Chicoutimi CRSNG NSERC November 3rd, 2015
- 8. Box has correct size, if user not logged in
- 9. Exhibit B Elements extending outside their container
- 14. Layout bugs don't impact the application's functionality... Wait a minute...
- 15. Exhibit D Tone on tone text
- 17. Exhibit E Too much escaping
- 21. Can't scroll to the left part of the popup
- 22. Can't scroll to get the missing buttons
- 24. Box pre-filled with 6 characters, but has maxlength="5"
- 25. Textbox disappears if window below specific width (scrolling won't help)
- 26. Layout bugs can't probe the application's state... Wait a minute...
- 27. Close button has no effect... stuck in the pop-up!
- 31. Layout bugs are limited to a single snapshot of a page... Wait a minute...
- 32. Incorrect search results https://youtu.be/zCg537MZ6Ww Crazy right click https://youtu.be/iw7x3H51TFo Incorrect button state https://youtu.be/NGbl4vP350s Summ
- 33. Experience reveals that many bugs manifest themselves through their UI They can be detected by observing elements of the UI... ...in a single snapshot, or across a sequence of snapshots
- 34. A language for expressing "high"-level oracles over web UIs - DOM + CSS attributes - First-order quantification - Temporal operators 1.
- 35. For each $x in $(css-selector) something about $x. There exists $x in $(css-selector) such that something about $x.
- 36. $x's prop is greater than $y's prop $x's height $x's text $x's color $x's prop matches "a string" ...and most DOM/CSS attributes ...and most common functions
- 37. something And something else Always something Eventually something something Until something else The next time something, then something else ...in other words, propositional logic + LTL
- 38. We say that pattern When something. Example: We say that $x is wider than $y When $x's width is greater than $y's width. User-defined predicates to improve legibility and create "mini-DSLs"
- 39. We say that I click on Go when ( There exists $b in $(button) such that ( ($b’s text is "Go") And ($b’s event is "mouseup"))). Always ( If (I click on Go) Then ( There exists $x in $(.value) such that ( The next time (I click on Go) Then ( There exists $y in $(.value) such that ( $x’s text equals $y’s text ))))).
- 40. $(document).mouseUp(function(event) { var e = arguments.callee; if ($(event.target).text() === "Go") { var current_values = []; $(".value").each( current_values.push($(this).text()); ); if (e.lastValues !== undefined) { var found = false; for (var v in current_values) { if ($.inArray(v, e.lastValues)) { found = true; break; } } } if (!found) console.log("Error"); e.lastValues = current_values;} };); The same in jQuery
- 41. An architecture to instrument a web app and report relevant UI attributes to an interpreter 2.
- 42. Server Application under test <script ... "getprobe?uid=39f2ac8"> Probe creationRequest for probe code Probe interaction Probe code Probe (JS object) Probe status report Probe instructions (optional) Probe log Probe repository Probe retrieval Developer Probe ID Dashboard (analytics) Developer 1 234 5 6 7 8 9 12 10 11 13 16 14 15
- 43. ... $(ul li) ... ... 's width ... ... $(p#abc) ... ... 's color ... The JavaScript probe is selective: it only sends back the attributes mentioned in the property only for the elements matching one of the selectors
- 44. A mechanism for explaining the violation of a property when found 3.
- 47. Producing a witness for a violation does not require to be coded in any way It is just a by-product of evaluating the logical expression It works for temporal formulas too!
- 48. See it in action https://www.youtu.be/90YitGRRx2w BEST TOOL Hallé, Bergeron, Guérin, Le Breton. Testing Web Applications Through Layout Constraints. Proc ICST 2015. https://bitbucket.org/sylvainhalle/cornipickle
- 49. cIcI My 2 cents... "Most browsers" { {Desktop market share Non-desktop market share 75% 25%
- 50. cIcI My 2 cents... In UI-based bugs, the browser matters Don't take design choices that limit you from the start (Selenium, any browser plugin) If the app runs in browser X, so should your testing tool This may mean a lot of work ahead!
- 51. cIcI My 2 cents... For each $x in $(selector) blabla... There exists $y in $(selector) blabla... CSS selectors are fragile. Simple changes to the layout may break the property
- 52. cIcI My 2 cents... ...in search of a carpenter Hammer and nails...
- 53. We still have to write the properties, so what's the point?* We're too busy to learn another tool!* Using this requires a higher level of education. We'll have to increase the wages!* *Really happened We can already do all this (with a bunch of home-brewed Python scripts)!*
- 54. Turn specifications into sets of test cases Run Cornipickle in tandem with a crawler Hub for sharing specifications (à la Greasemonkey)