![2 N. Kratzke
university and college programming courses are steadily increasing. Even mas-
sive open online courses [14] are used more and more systematically to convey
necessary programming capabilities to students of different disciplines [19]. The
coursework consists of programming assignments that need to be assessed. Since
the submitted assignments are executable programs with a formal structure,
they are highly suited to be assessed automatically. In consequence, plenty of
automated programming assignment assessment systems (APAAS) evolved. We
refer to [1–3,5,9,17] for an overview of such tools.
Previous research [11] showed how astonishingly simple it is for students to
trick automated programming assignment assessment systems. It is often over-
looked that APAAS solutions are systems that execute injected code (student
submissions) and code injection is known as a severe threat from a security point
of view [20]. We refer to [7,15], and [6] for an overview of such kind of attacks.
Of course, such code injection vulnerabilities are considered by current solu-
tions. However, in previous research [11], it was astonishing to see that current
APAAS solutions sometimes overlook the cheating cleverness of students. On the
one hand, APAAS solutions protect the host system via sandbox mechanisms,
and APAAS systems put much effort in sophisticated plagiarism detection and
authorship control of student submissions [13,16]. On the other, the grading com-
ponent can be cheated in various – sometimes ridiculously simple – ways making
these solutions highly suspect for (semi-)automated and unattended program-
ming examinations that contribute to certificate a certain level of programming
expertise. Previous research [11] identified at least four simple cheat patterns:
– Overfitting
– Evasion
– Redirection
– Injection
Moreover, it strived to raise a general problem awareness but did not focus
on solutions to mitigate these patterns. To propose solutions on how to mitigate
these identified cheat patterns is the primary intent and contribution of this
Chapter. We propose to use the following three techniques:
– Randomization of test cases
– Pragmatic code inspections
– Separation of student and evaluation logic
These techniques mitigate the presented patterns and demonstrate the suit-
ability for the APAAS solution Moodle/VPL and the programming language
Java. Nevertheless, the principles are transferable to other APAAS solutions
and programming languages and therefore of broader interest and not limited to
Moodle/VPL and Java.
Consequently, the remainder of this paper is outlined as follows. Section 2
presents the methodology that has been used to identify and categorize stu-
dent cheat-patterns and to validate appropriate “countermeasures”. Section 3
will explain the identified cheat-patterns. Section 4 presents a mitigation analysis](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-22-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 3
and shows that the identified cheat patterns can be addressed by three “coun-
termeasures” that should be considered by every APAAS solution. Section 5
explains how these insights have been considered in the development of a unit
testing framework (JEdUnit) that focuses intentionally on educational aspects
and considers the aspects mentioned above consequently. We will discuss our
results of JEdUnit in Sect. 6 and provide some guidelines on the generalizability
and limitations of this research. Finally, we conclude our findings in Sect. 7.
Fig. 1. Research methodology.
2 Methodology
Figure 1 presents the overall research approach that compromised two phases:
Problem systematization and solution proposal validation.
2.1 Problem Systematization
For the initial problem systematization, two first semester programming Java
courses in the winter semester 2018/19 (see Table 1) have been systematically
evaluated. Both courses were used to search for student submissions that inten-
tionally trick the grading component of APAAS solutions. Table 1 provides a
summarized overview of the course design. All assignments were automatically
evaluated by the VPL Moodle plugin (version 3.3.3) following the general rec-
ommendations described by [21]. For more details, the reader is referred to [11].
Figure 2 shows an exemplifying VPL screenshot from a students perspective.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-23-320.jpg)
![4 N. Kratzke
Fig. 2. VPL screenshot (on the right evaluation report presented to students) [11].
Table 1. Courses used for problem systematization and solution validation.
Systematization Validation
Prog I (CS) Prog I (ITD) Prog II (CS)
Students 113 79 73
Assignments (total) 29 20 8
Number of bunches 11 6 7
Assignments per bunch (avg) 3 3 1
Time for a bunch (weeks) 1 2 2
Groups 7 6 4
Students per group (avg) 18 12 18
Student/advisor ratio (avg) 6 6 6
To minimize Hawthorne and Experimenter effects [4] neither the students
nor the advisers in the practical programming courses were aware that student
submissions were analyzed to deduce cheating patterns. Even if cheating was
detected, this had no consequences for the students. It was not even communi-
cated to the student or the advisers.
Furthermore, students were not aware that the version history of their sub-
missions and therefore even intermediate cheating experiments (that did not
make it to the final submission) were logged. However, not every submission
was inspected for understandable effort reasons. Therefore, only significant sub-
mission samples (see Table 2) were investigated to search systematically for
cheat-patterns.
Table 3 summarizes the results quantitatively. Within these six samples,
cheat and trick patterns were identified mainly by a manual but a script-
supported observation. VPL submissions were downloaded from Moodle and
analyzed weekly. We developed a Jupyter-based [10] quantitative analysis and](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-24-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 5
submission data model for this dataset. Each student submission was represented
as an object containing its version and grading history that references its student
submitter and its corresponding study programme. The analytical script and
data model made use of the well known Python libraries statistics, NumPy [12],
matplotlib [8], and the Javaparser library [18]. It was used to identify the num-
ber of submissions and evaluations, points per submission versions, timestamps
per submission version, occurrences of unusual terms, and more. Based on this
quantitative data, the mentioned samples (S1–S5) were selected automatically
(or randomly in case of S6). Additionally, the source codes of the sample sub-
missions were exported weekly as an archived PDF document. However, the
scanning for cheat-patterns was done manually within these documents.
Table 2. Weekly analyzed samples (systematization phase).
Description Rationale
S1 TOP 10 of submissions with many
triggered evaluations
Parameter optimization could
cause plenty of evaluations
S2 TOP 10 of submission with many
versions
Cheating experiments could cause
plenty of versions
S3 TOP 10 of submissions with
astonishingly less average points
across all triggered evaluations but
full points in the final submission
Cheating could cause such point
boosts
S4 Submissions with unusual (above
95% percentile) many condition
related terms like if, return,
switch, case, and so on
Parameter optimization could
cause unuasal many condition
related terms
S5 Submissions with unusual terms
like System.exit,
System.getProperties, := that
would stop program execution or
have a special meaning in VPL or
Java but are unlikely to
contribute to a problem solution
APAAS attacks could cause such
unusual terms
S6 Ten random submissions To cover unintended observation
aspects
2.2 Solution Proposal Validation
Based on these elicited cheat patterns, corresponding mitigation options have
been derived. Three of them (randomization, code inspection, and submission/e-
valuation logic separation) have been implemented in a unit testing frame-
work called JEdUnit as a solution proposal to mitigate the identified problems.
JEdUnit has been validated using a Programming II course for computer science
students in the summer semester 2019. The course has been given analogously](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-25-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 7
Table 3. Detected cheats.
Fig. 3. Observed cheat-pattern frequency (without application of JEdUnit) [11].
Especially overfitting and evasion tricks are “poor-mans’ weapons” often used
by novice programmers as a last resort to solve a problem. Much more alarm-
ing redirection and injection cheats occurred only in rare cases (less than 10%).
However, how do these tricks and cheats look like? How severe are they? More-](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-27-320.jpg)
![8 N. Kratzke
over, what can be done against it? We will investigate these questions in the
following paragraphs.
3.1 Overfitting Tricks
Overfitted solutions strive to get a maximum of points for grading but do not
strive to solve the given problem in a general way. A notable example of an
overfitted solution would be Listing 1.2.
Listing 1.2. Overfitting solution.
int countChar(char c, String s) {
if (c == ’a’ s.equals(Abc)) return 1;
if (c == ’A’ s.equals(abc)) return 1;
if (c == ’x’ s.equals(ABC)) return 0;
if (c == ’!’ s.equals(!!!)) return 3;
// [...]
if (c == ’x’ s.equals(X)) return 1;
return 42;
}
This solution maps merely the example input parameters to the expected
output values. The solution is completely useless outside the scope of the test
cases.
3.2 Problem Evasion Tricks
Another trick pattern is to evade a given problem statement. According to our
experiences, this pattern occurs mainly in the context of more sophisticated
and formal programming techniques like recursive programming or functional
programming styles with lambda functions.
So, let us now assume that the assignment is still to implement a
countChar() method, but this method should be implemented recursively. A
reference solution might look like in Listing 1.3 (we do not consider performance
aspects due to tail recursion):
Listing 1.3. Recursive reference solution.
int countChar(char c, String s) {
s = s.toLowerCase ();
c = Character.toLowerCase(c);
if (s.isEmpty ()) return 0;
char head = s.charAt (0);
String rest = s.substring (1);
int n = head == c ? 1 : 0;
return n + countChar(c, rest );
}](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-28-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 9
However, sometimes, student submissions only pretend to be recursive with-
out being it. Listing 1.4 is a notable example.
Listing 1.4. Problem evasing solution.
int countChar(char c, String s) {
if (s.isEmpty ()) return 0;
return countChar(c, s, 0);
}
int countChar(char c, String s, int i){
for (char x : s.toCharArray ()) {
if (x == c) i++;
}
return i;
}
Although countChar() is calling (an overloaded version of) countChar() which
looks recursively, the overloaded version of countChar() makes use of a for-loop
and is therefore implemented in a fully imperative style.
The same pattern can be observed if an assignment requests functional pro-
gramming with lambda functions. A lambda-based solution could look like in
Listing 1.5.
Listing 1.5. Lambda reference solution.
(c, s) - Stream.of(s.toCharArray ())
.filter(x - x == c)
.count ();
However, students take refuge in familiar programming concepts like loops. Very
often, submissions like in Listing 1.6 are observable:
Listing 1.6. Lambda problem evasion.
(c, s) - {
int i = 0;
for (char x : s.toCharArray ()) {
if (x == c) i++;
}
return i;
};
The (c, s) - { [...] }; seems functional on the first-hand. But, if we look
at the implementation, it is only an imperative for loop embedded in a functional
looking context.
The problem here is, that evaluation components just looking on input-
output parameter correctness will not detect these kinds of programming style
evasions. The just recursive- or functional-looking solutions will generate the
correct results. Nevertheless, the intent of such kind of assignments is not just
to foster correct solutions but although to train specific styles of programming.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-29-320.jpg)
![12 N. Kratzke
The intent of such a submission is merely to inject a line like Grade :=
100’ into the output stream to let the grading component evaluate the sub-
mission with full points.
4 Mitigation Analysis of Cheat Patterns
So, in our problem systematization phase, we identified four patterns of tricking
or cheating that can be observed in regular programming classes. These tricks
work because students know that a dumb automat checks their submissions. In
the following Sects. 4.1, 4.2, 4.3, 4.4 and we will ask the question what can be
done to make APAAS more “intelligent” to prevent this kind of cheating?
4.1 What Can Be Done to Prevent Overfitting?
Randomized test data make overfitted submissions ineffective. Therefore, our
general recommendation is to give a substantial fraction of points for randomized
test cases. However, to provide some control over randomized tests, these tests
must be pattern based to trigger expectable problems (e.g., off-by-one errors,
boundary cases) in student submissions. We refer to [17] for further details. E.g.
for string-based data, we gained promising results to generate random strings
merely by applying regular expressions [22] inversely. Section 5.1 explains how
randomization is used by JEdUnit to tackle the overfitting problem.
4.2 What Can Be Done to Avoid Problem Evasion?
Problem evasion cannot be detected by comparing the equality of input-
output results (black-box testing). To mitigate problem evasion, we need auto-
mated code inspection approaches (white-box inspections). Submissions must
be scanned for unintended usage of language concepts like for and while loops.
However, this makes it necessary to apply parsers and makes the assignment
specific evaluation logic much more complicated and time-intensive to program.
To simplify this work, we propose a selector-based model that selects nodes from
an abstract syntax tree (AST) of a compilation unit to detect and annotate such
kind of violations in a practical way. The approach works similarly like CSS
selectors selecting parts of a DOM-tree in a web context (see Sect. 5.2).
4.3 What Can Be Done to Prevent Redirection?
Interestingly problem evasion and redirection can be solved by the same mit-
igation approach. Similar to evasion cheats, submissions can be scanned for
unintended usage of language concepts, e.g. calls to classes containing the refer-
ence logic that is used for testing. This white box inspection approach makes it](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-32-320.jpg)
![14 N. Kratzke
5 JEdUnit
The Sects. 4.1, 4.2, 4.3, and 4.4 showed that it is possible to mitigate identi-
fied cheat patterns using the strategies listed in Table 4. These insights flowed
into a unit testing framework called JEdUnit. JEdUnit has a specific focus on
educational aspects and strives to simplify automatic evaluation of (small) Java
programming assignments using Moodle and VPL. The framework has been
mainly developed for our purposes in programming classes at the Lübeck Uni-
versity of Applied Sciences. However, this framework might be helpful for other
programming instructors. Therefore, it is provided as open source.
Every JEdUnit evaluation is expressed in a Checks.java compilation unit
and usually relies on a reference implementation (which is by convention pro-
vided in a file called Solution.java) and a submission (which is by convention
provided in a file called Main.java). However, the conventions can be adapted
to assignment specific testing requirements.
Listing 1.10. Reference Solution expressed in JEdUnit.
public class Solution {
public int countChars(char c, String s) {
int n = 0;
for (char x : s.toLowerCase (). toCharArray ()) {
if (Character.toLowerCase(c) == x) n++;
}
return n;
}
}
Similar to JUnit, each test case is annotated with a @Test annotation (see
Listing 1.11). However, a JEdUnit @Test annotation takes two additional param-
eters:
– weight is a value between 0 and 1.0 that indicates how much this test case
contributes to the final grade of the assignment.
– description is a string briefly explaining the intent of the test case.
Listing 1.11. Test template expressed in JEdUnit.
public class Checks extends Constraints {
@Test(weight =0.25 , description=Example calls)
public void test_01_exampleCalls () { [...] }
@Test(weight =0.25 , description=Boundary tests)
public void test_02_boundary_cases () { [...] }
@Test(weight =0.5, description=Randomized tests)
public void test_03_randomized_cases () { [...] }
}](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-34-320.jpg)
![16 N. Kratzke
Table 5. Random generators provided by JEdUnit.
Method Description
c() Random character ([a-zA-Z])
c(String regexp) Random character from a regular
expression (first char)
s(String... regexps) Random string from a sequence of
regular expressions
s(int min, int max) Random string between a minimum
and maximum length
b() Random boolean value
i() Random integer value
i(int m) Random integer value [0, m]
i(int l, int u) Random integer value [l, u]
d() Random double value
d(double m) Random double value [0, m[
d(double l, double u) Random double value [l, u[
T ListT l(int l, SupplierT
g)
Random list with length l generated by
g
T ListT l(int l, int u,
SupplierT g)
Random list with length in the range
of [l, u] generated by g
The reader is referred to the Wiki1
for a more detailed introduction to
JEdUnit and further features like:
– Initializing assignments.
– Configuration of checkstyle (coding conventions).
– Making use of predefined code inspections.
– Checking complex object-oriented class structures automatically.
However, in the following Sects. 5.1, 5.2, and 5.3 we focus mainly on how
JEdUnit makes use of randomization, code inspection, and stream separation to
mitigate observed overfitting, problem evasion, redirection, and injection cheat-
ing problems (see Table 4).
5.1 How Does JEdUnit Support Randomization?
JEdUnit provides a set of random generators to mitigate overfitting problems.
These random generators enable to generate randomized test data in specified
ranges and according to patterns to test explicitly common problem cases like
boundary cases, off-by-one errors, empty data structures, and more. Because
1
https://github.com/nkratzke/JEdUnit/wiki.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-36-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 17
these generators are frequently used, they intentionally have short names (see
Table 5).
Using this set of random generators, we can quickly create test data – for
instance a random list of predefined terms.
Listing 1.13. Demonstration of a randomized list generator creating a list of strings
from a regular expression.
List String words = l(2, 5, () -
s(This|is|just|a|silly|example)
);
The randomly generated lists have a length between two and five entries.
Possible resulting lists would be:
– [This, a]
– [silly, is, This]
– [example, example, a, a]
– [This, is, just, a, example]
The generators shown in Table 5 are designed to work seamlessly with the
test().each() pattern introduced in Listing 1.12. Listing 1.14 exemplifies a
randomized test case for our continuous example. It works with random strings
but generates test data to check intentionally for cases where the to be counted
character is placed in front, in the middle or at the end of a string to cover
frequent programming errors like off-by-one errors.
Listing 1.14. Example for a randomized test case expressed in JEdUnit.
@Test(weight =0.5, description=Randomized tests)
public void test_03_randomized_cases () {
// Regexp to generate random strings
String r = [a-zA -Z]{5 ,17};
// Pick a random character to search
char c = c();
test(
t(c, s(c + {1,7}, r, r)), // First position
t(C, s(r, r, c + {1,7})), // Last position
t(c, s(r, c + {1,7}, r)), // Middle position
). each(check , explain , onError );
// check , explain , onError defined as in Listing 1.12
}
Such kinds of test cases are not prone to overfitting because the test data is
randomly generated for every evaluation. A possible generated feedback for the
test case shown in Listing 1.14 would look like so:](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-37-320.jpg)
![18 N. Kratzke
– [OK] countChars(‘j’, jUcEzCzODWWN) should return 1
– [FAILED] countChars(‘j’, zOdAqavJJkxjvrjj) should return 5
but returned 3
– [OK] countChars(‘j’, SPAqlORwxjjjjRHIKCCWS) should
return 4
Each evaluation will be run with random data but according to comparable
patterns. So, JEdUnit provides both: comparable evaluations and random test
cases that are not prone to overfitting.
5.2 How Does JEdUnit Support Code Inspection?
JEdUnit integrates the JavaParser [18] library to parse Java source code into
an abstract syntax tree. JEdUnit tests have full access to the JavaParser library
and can do arbitrary checks with this parser. However, this can become quickly
very complex. Therefore, JEdUnit tests can make use of a selector-based model
that selects nodes from an abstract syntax tree (AST) of a compilation unit
to detect and annotate such kind of violations in a practical way. The approach
works similarly like CSS selectors selecting parts of a DOM-tree in a web context.
The following examples demonstrate how to use this selector model on sub-
mission ASTs pragmatically for code inspections. The reader should inspect the
two evasion examples shown in Listing 1.14 and 1.6 that show typical evasion
patterns to avoid recursive or lambda/stream-based programming in Java.
To detect lambda evasion (see Listing 1.6), we can add the following inspec-
tion to a test case. It scans for lambda functions that make use of block state-
ments in their definition. To use blocks in lambda functions might indicate a
kind of “problem evasion” in a submission – a student may try to evade from
lambda programming into more simple statement based programming (which is
likely not the intent of the assignment).
Listing 1.15. Example for a lambda evasion check in JEdUnit.
@Inspection(description=Lambda evasion inspection)
public void assignmentInspections () {
penalize (25, No blocks in lambdas., () -
inspect(Main.java, ast - ast.select(LAMBDA)
.select(BLOCK)
.annotate(No blocks in lambdas.)
.exists ()
)
);
}
So, this inspection would effectively detect submissions like the one already
presented in Listing 1.6. To detect recursion evasion (see Listing 1.4) we simply
have to search for methods outside the main() method that make use of loop
statements.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-38-320.jpg)
![20 N. Kratzke
Listing 1.16. Example to detect recursion evasion.
inspect(cu , ast - ast.select(METHOD , [name != main])
.select(FOR , FOREACH , WHILE , DOWHILE)
.annotate(no loops allowed)
.exists ()
);
The same technique can be used to detect submissions that make use of
System.out.println() or System.exit() calls that might indicate injection
attacks.
Listing 1.17. Example to detect suspect method calls (used JEdUnit internally).
inspect(cu , ast - ast.select(METHODCALL)
.filter(c - c.toString (). startsWith(System.exit)))
.annotate(c - [CHEAT] Forbidden call: + c)
.exists ()
);
The reader may notice that these selector-based code inspections are quite
powerful and flexible to formalize and detect arbitrary violation patterns in
student code submissions. JEdUnit makes intensively use of this code inspection
technique and provides several predefined code inspections that can be activated
by config options. Table 6 lists some of these predefined inspections.
5.3 How Does JEdUnit Supports Separation of Evaluation
and Submission Logic?
The main problem of injection attacks is that the submission logic and the eval-
uation logic make use of the same console streams (stdout/stderr). The problem
is that the grading component interprets this console output and this output
could be compromised by student submission’s outputs (intentionally or unin-
tentionally). JEdUnit solves this problem simply be redirecting the stdout/stderr
console streams to other streams. Simple methods like this do this redirection.
Listing 1.18. Redirection of stdout console stream.
/**
* Creates a file called console.log that stores all
* console output generated by the submitted logic.
* Used to isolate the evaluation output from the
* submitted logic output to prevent injection attacks.
* @since 0.2.1
*/
public void redirectStdOut () throws Exception {
this.redirected = new PrintStream(
Config. STD_OUT_REDIRECTION
);
System.setOut(this.redirected );
}](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-40-320.jpg)
![22 N. Kratzke
done manually and without notice to the students. In the validation phase cheat
detection has been done automatically by JEdUnit and been reported immedi-
ately as JEdUnit evaluation feedback to the students. The consequence was that
cheating occurred only in the first weeks. What is more, the cheating detected
in the validation phase has been only observed in intermediate outcomes and
not in the final submissions (while the cheating reported in the systematization
phase is cheating that made it to the final submissions).
So, we can conclude that JEdUnit effectively detects the four identified pat-
terns of cheating (overfitting, evasion, redirection, and injection). Cheating was
not even applied in intermediate results and came entirely to an end in the
second half of the semester. It was no effective mean anymore from a students
perspective.
However, we should consider and discuss the following threats of internal and
external validity [4] that apply to our study, and that might limit its conclusions.
Selection Bias and Maturation. We should consider that the target audience
of the systematization and validation phase differed a bit. In the systematiza-
tion phase, we worked with first-semester novice programmers of a computer
science and information technology and design study programme. In the vali-
dation phase, we worked with second-semester computer science students that
made some programming progress during their first semester. This increase in
expertise might be one reason why a general decrease in cheating was observable
in the second semester. However, that explains not the fact that cheating comes
entirely to an end in the second half of the semester. This decrease might be
only explained by the fact that students learned that the grading automat “was
not so dumb” anymore.
Contextual Factors. This threat occurs due to specific conditions under which
research is conducted that might limit its generalisability. In this case, we were
bound to a Moodle-based APAAS solution. The study would not have been
possible outside this technical scope. We decided to work with VPL because it is
the only mature-enough open source solution for Moodle. Therefore, the study
should not be taken to conclude on all existing APAAS systems. Especially
JEdUnit is a Moodle/VPL specific outcome. However, it seems to be worth to
check existing APAAS solutions whether they are aware of the four identified
cheat-patterns (or attack vectors from a system security perspective) and what
can be transferred from JEdUnit into further APAAS solutions.
Hawthorne Effects. This threat occurs due to participants’ reactions to being
studied or tested. It alters their behaviour and therefore, the study results. We
can observe the Hawthorne effect quite obvious in Table 3. The reader should
compare the systematization phase (unaware of observation) and the validation
phase (noticing a more clever grading automat capable of detecting cheats). In
the validation phase cheating was drastically reduced, only occurred in inter-
mediate outcomes, and even comes entirely to an end in the second half of the](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-42-320.jpg)
![How Programming Students Trick and What JEdUnit Can Do Against It 23
semester. JEdUnit made the grading component more“clever” and changed the
behaviour of students not to cheat.
Because of this internal feedback loop, the study should not be taken to draw
any conclusions on the quantitative aspects of cheating. Furthermore, the reader
should additionally take all the threat to validity discussion of the systematiza-
tion phase [11] into account in order to avoid drawing wrong conclusions.
7 Conclusion
Students trick – at least 15% of them. Maybe because grading components of
automated programming assessment systems can be tricked very easily. Even
first-year students are clever enough to do this. We identified recurring pat-
terns like overfitting, evasion, redirection, and even injection tricks. Most
APAAS solutions provide sandbox mechanisms and code plagiarism detection to
protect the host against hostile code or to detect “copy-paste” cheating. How-
ever, these measures do not prevent submissions like
System.out.println(Grade := 100);
System.exit(0);
which would give full points (in a Moodle/VPL setting) regardless of the assign-
ment complexity. These two lines (or the injection idea behind) are like an
“atomic bomb” for a grading component that is unaware to many program-
ming instructors. Current APAAS solutions can little do against it. This study
aimed to systematize such kind of recurring programming student cheat patterns
and to search for mitigation options. To handle these kinds of cheats we do not
only need sandboxing nor code plagiarism detection (that almost all APAAS
solutions provide) but additionally means
1. to randomise test cases (contributes to 63% of all cheats, see Table 4),
2. to provide pragmatic code inspection techniques (contributes to 36% of
all cheats, see Table 4),
3. and to isolate the submission and the evaluation logic consequently in
separate processes (contributes to 1% of all cheats, see Table 4).
Therefore, this paper presented and evaluated the solution proposal JEdUnit
that handles these problems. According to our validation in three courses with
over 260 students and more than 3.600 submissions of programming assignments,
JEdUnit can make grading components much more “intelligent”. We showed
how to make overfitting inefficient, how to detect evasion and redirection, and
how to deny injection cheat patterns. When students learn that the grading
component is no dumb automat anymore, cheating decreases immediately. In
our case, cheating comes even entirely to an end.
However, as far as the author oversees the APAAS landscape, exactly these
mentioned features are only incompletely provided by current APAAS solutions.
Most APAAS solutions focus on sandboxing and code plagiarism detection but](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-43-320.jpg)
![Exploring the Affordances of SimReal
for Learning Mathematics in Teacher
Education: A Socio-Cultural Perspective
Said Hadjerrouit(B)
University of Agder, Kristiansand, Norway
Said.Hadjerrouit@uia.no
Abstract. SimReal is an innovative technology that emerged the last few years in
mathematics education. It provides new potentialities for mathematical learning
by means of dynamic and interactive visualizations of mathematical concepts. This
paper uses SimReal in the context of teacher education to explore the affordances
and constraints of the tool for the learning of mathematics. It presents a framework
that combines Vygotsky’s socio-cultural theory, the role of mediation, and the
notion of affordance at a technological, pedagogical, and socio-cultural level. The
aim of the article is to explore the extent to which SimReal as a mediating artifact
affords students’ mathematical learning in teacher education.
Keywords: Affordance · Agency · Mathematical learning · SimReal · Teacher
education · Visualization
1 Introduction
SimReal is a visualization tool that is used to teach a wide range of topics ranging
from school to undergraduate mathematics. While the suitability of the tool has been
evaluated in university mathematics courses, it is not the case in teacher education,
without an appropriate theoretical framework. Several theoretical approaches can be
used to address the use of digital tools in mathematics education such as instrumental or
documentational approach, theory of didactical situations, or anthropological theory of
didactics [1]. However, these approaches are not ready-made to assess the affordances
of SimReal to support the learning of mathematics and explore the extent to which
the tool is useful in teacher education [2]. The notion of affordance combined with
a socio-cultural perspective provides a powerful framework to address the impacts of
SimReal by considering the tool as a mediating artefact with affordances and constraints
at the technological, pedagogical, and socio-cultural level. This suits well with the way
mathematics is taught in the socio-cultural context of teacher education.
This paper is an extended version of the article that was published in the proceedings
of CSEDU 2019 [3]. This paper has a different title and structure. The sections that
are extended are Introduction, Literature Review, Theoretical Framework, Discussion,
and Conclusion and Future Work. The abstract has changed, and research questions
© Springer Nature Switzerland AG 2020
H. C. Lane et al. (Eds.): CSEDU 2019, CCIS 1220, pp. 26–50, 2020.
https://doi.org/10.1007/978-3-030-58459-7_2](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-46-320.jpg)
![Exploring the Affordances of SimReal for Learning Mathematics 27
are included. The result section contains more findings and information on affordances
and programming issues. Blocks of text are also partly rewritten. This new version is
structured as follows. Firstly, the visualization tool SimReal is presented. Secondly, the
theoretical framework is described. This is followed by the context of the study, the
research questions and methods. The results are then presented and discussed. Finally,
a summary of the results, and recommendations for future work conclude the article.
2 Literature Review
2.1 The Notion of Visualization
Mathematics educators have used visualizations for many years [4, 5]. Textbooks and
Web sites are filled with pictures, illustrations, animations, diagrams, and graphs. Like-
wise, graphing calculators and videos have become integral part of mathematics class-
room. According to Arcavi [6], visualization is defined as the ability to use and reflect
upon pictures, graphs, animations, images, and diagrams on paper or with digital tools
with the goal of communicating information, thinking about and advancing understand-
ings. Visualizations include both the process and product of creation, and reflection
upon pictures, images in the mind internally, and on paper or with digital tools exter-
nally [6]. Another use of the notion is visual representation, but sometimes no differ-
ence is made between mathematical visualizations (pictures, images, and diagrams) and
mathematical representations (verbal, graphical, and symbolical). Finally, Presmeg [5]
uses similar terms to define the notion of visualization. This includes processes of con-
struction and transformation of both visual mental images and inscriptions (or external
representations) of a spatial nature that may be implicated in doing mathematics.
2.2 The Visualization Tool SimReal
SimReal is a visualization tool for a range number of mathematical subjects ranging from
school to higher education. It uses a combination of graphic calculator, video lessons,
video live streaming, and interactive simulations [7]. An example of SimReal utilization
in mathematics education is given in Fig. 1.
SimReal has more than 5000 applications, exercises, and tasks [8]. The tool can be
divided into small subsets, while keeping the same basic structure and user interface.
Hogstad, Ghislain, and Vos [9] investigated a subset of SimReal called Sim2Bil, which
provides 4 windows for visualizations: simulation, graph, formula, and menu window.
Some studies on SimReal focus on mathematics at the undergraduate university
level [8, 10, 11], and aimed to report on students’ views of the tool as a supplement
to ordinary teaching and its usefulness in difficult and abstract mathematical areas.
Students’ views were mostly positive, except some technical difficulties, e.g., navigating
through the tool. In engineering education, Hogstad, Ghislain, and Vos [9] studied a
subset of SimReal called Sim2bil that aims at exploring how students use visualizations
in their mathematical activities. Furthermore, Hadjerrouit and Gautestad [13] used the
theory of instrumental orchestration and usability criteria to analyze teachers’ use of
SimReal in an engineering class. Another study addressed the use of SimReal in an upper](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-47-320.jpg)
![28 S. Hadjerrouit
Fig. 1. An example of SimReal utilization in mathematics education [3].
secondary school [12]. It reports on positive students’ attitudes towards the use of the tool
inclassroom. Somestudents didnot foundvisualizations useful, andtheintegrationof the
tool into the curriculum was not easy. Other studies were carried out in teacher education
in a technology-based course. Firstly, Hadjerrouit [14] evaluated the suitability of the
tool using usability criteria. Secondly, Hadjerrouit [15] addressed students’ perceptions
of SimReal impacts on learning using a survey and open-ended questions based on three
categories of affordances. Finally, Hadjerrouit [3] is a continuation of these two studies
and aimed at using SimReal to investigate the affordances of the tool in a more in-
depth way by means of specific mathematical tasks - Pythagoras’ and Square theorem -
and programming languages to create digital visualizations.
3 Theoretical Framework
3.1 Socio-Cultural Theory and Mediating Artifacts
An important element of Vygotsky’s socio-cultural theory [16] is the role of tools in edu-
cational settings, and the way they mediate teaching and learning activities. A crucial
issue is thus the role of mediating artifacts and how teachers and students take advan-
tage of artifacts, such as mathematical symbols, textbooks, blackboard, and in particular
digital artifacts, such as GeoGebra, Excel, and SimReal. These artifacts have both affor-
dances and constraints. Based on the writings of Vygotsky, the role of mediating artifacts
in mathematics education can be represented as follows (Fig. 2).
In this paper, the subject is the student, the object is the mathematical concept to be
learned, e.g., Pythagoras’ theorem, and the mediating artifact is SimReal. Among several
theoretical approaches that can be applied to explore the impacts of SimReal, such as the
instrumental approach [17], the notion of affordance combined with Vygotsky’s socio-
cultural stance provides the most appropriate framework to address the affordances and
constraints of SimReal in a teacher education context.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-48-320.jpg)
![Exploring the Affordances of SimReal for Learning Mathematics 29
SimReal as mediating artifact with affordances and constraints
Subject (Student) Object (mathemaƟcal learning)
Fig. 2. SimReal as mediating artifact with its affordances and constraints.
3.2 The Notion of Affordance
The notion of affordance was originally proposed by Gibson [18]. It refers to the relation-
ship between an object’s physical properties and the characteristics of a user that enables
particular interactions between user and object. More specifically, Gibson used the term
“affordance” to describe the action possibilities offered to an animal by the environment
with reference to the animal’s action capabilities [18, 19]. A typical example is a tall tree
that has the affordance of food for a giraffe because it has a long neck. Gibsons’ notion
of affordance implies a complementarity between the animal and the environment.
The notion of affordance was introduced to the computer world by Norman [20]. It
refers to the perceived and actual properties of the thing, primarily those fundamental
propertiesthatdeterminejusthowthethingcouldpossiblybeused.AccordingtoNorman
[20], an affordance is the design aspect of an object, which suggests how the object
should be used and a visual clue to its function and use. Examples of affordances are
user interface elements, which directly suggest suitable actions: Clickable geometrical
figures, draggable sliders, pressable buttons, selectable menus for figures or algebraic
calculations, etc.
Several research studies used Norman’s ideas to implement the notion of affordance
in various educational settings. For example, Turner and Turner [21] specified a three-
layer articulation of affordances: Perceived affordances, ergonomic affordances, and
cultural affordances. Likewise, Kirchner, Strijbos, Kreijns, and Beers [22] described
a three-layer definition of affordance: Technological affordances that cover usability
issues, educational affordances to facilitate teaching and learning, and, social affordances
to foster social interactions. In mathematics education, Chiappini [23] applied the notions
ofperceived,ergonomic,andculturalaffordancestoAlnuset,adigitaltoolforhighschool
algebra. Finally, Hadjerrouit [3, 15] presented a model with three levels of affordances
of digital tools in teacher education. The same model is used in this paper.
De Landa [24] emphasized that affordances are not intrinsic properties of the object.
Rather affordances become actualized in a specific context, e.g., the socio-cultural con-
text of the classroom. In other words, affordances emerge from the relationship between
the object, e.g., a digital tool, and the particular environment with which it is interacting.
From this perspective, the specific context of the mathematics classroom in teacher edu-
cation may include several artifacts with their affordances and constraints, which interact
with the teacher and students. The artifacts used in classroom may include paper-pencil
techniques, the blackboard, Interactive White Board (IWB), PowerPoint slides, and](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-49-320.jpg)
![30 S. Hadjerrouit
diverse digital tools, such as Smartphones, IPad, GeoGebra, SimReal, and mathematical
symbols, notations, and representations, etc. Hence, artifacts with their affordances and
constraints mediate between the subject (student) and the object (mathematics). This
view of digital tools is in line with Vygotsky’s socio-cultural stance.
3.3 SimReal Affordances
Based on the research literature described above, the specificities of mathematics edu-
cation, and the socio-cultural stance as an overreaching framework, this study proposes
three categories of affordances at six different levels (Fig. 3):
Fig. 3. Three categories of SimReal affordances at six different levels [3].
a) Technological affordances that describe the tool’s functionalities
b) Pedagogical affordances at four levels:
• Pedagogical affordances at the student level or mathematical task level
• Pedagogical affordances at the classroom or student-teacher interaction level
• Pedagogical affordances at the mathematics subject level
• Pedagogical affordances at the assessment level
c) Socio-cultural affordances that cover curricular, cultural, and ethical issues
The affordances that are highlighted in this section are only a subset of potential
affordances that could emerge in specific classroom contexts. These affordances are](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-50-320.jpg)
![Exploring the Affordances of SimReal for Learning Mathematics 31
some of the most important ones from the researcher’s perspective, but as De Landa [24]
pointed out affordances are not intrinsic properties of the object, of the visualization
tool SimReal in this study. Rather affordances of SimReal become actualized in specific
contexts, e.g., the socio-cultural context of the classroom in teacher education.
Regarding the affordance model presented in Fig. 3, two types of technological affor-
dances can be distinguished: Ergonomic and functional affordances. From the ergonomic
point of view, these are ease-of-use, ease-of-navigation, and accessibility of SimReal at
any time and place, accuracy and quick completion of mathematical activities. From the
functional point of view, SimReal provides support for performing calculations, drawing
graphs and functions, solving equations, constructing diagrams, measuring figures and
shapes, and presenting mathematical concepts visually and dynamically. Technological
affordances are a pre-requisite for any digital tool and provide the ground for pedagogical
affordances in educational settings.
There are several pedagogical affordances at the student level, e.g., using SimReal
to freely build and transform mathematical expressions that support conceptual under-
standing, such as collecting real data, creating a mathematical model, using a slider to
vary a parameter or drag a corner of a triangle, moving between symbolic, numerical,
and graphical representations, simulating mathematical concepts, or exploring regularity
and change [25]. At this level, the motivational factor is important to engage students
in mathematical problem solving. Furthermore, feedback of the tool in various forms to
students’ actions may foster mathematical thinking, e.g., whether an answer given by
the student is correct or not. Programming mathematical tasks also provide affordances
that may foster mathematical learning and understanding.
Likewise, several pedagogical affordances can be provided at the classroom level
[25]. Firstly, affordances that result in changes of interpersonal dimensions, such as
change of teachers’ and students’ role, less teacher-directed and more student-oriented
instruction. Secondly, affordances that foster more learner autonomy, resulting in stu-
dents taking greater control over their own learning, and using SimReal as a “new”
authority in assessing their own learning. Other affordances at this level are change
of social dynamics and more focus on collaborative learning and group work. These
affordances have an impact on the didactical contract and the role of the teacher as an
authority in classroom [26]. Variation in teaching and differentiation of tasks are other
affordances of digital tools at this level [27]. Other affordances at this level may result
in flipping the classroom, which is an alternative way of using SimReal out-of-class.
Furthermore, three types of pedagogical affordances can be provided at the math-
ematics subject level [25]. The first one is fostering mathematical fidelity according
to “ideal” mathematics, the congruence between machine mathematics and “ideal” and
paper-pencil mathematics and promoting faithfulness of machine mathematics [28]. The
second affordance is amplifying and reorganizing the mathematical subject. The former
is accepting teachers’ and curricular goals to achieve those goals better. Reorganizing the
mathematical subject means changing the goals by replacing some issues, adding and
reordering others. For example, in advanced topics such as calculus there might be less
focus on skills and more on mathematical concepts [25]. In geometry, there might be and](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-51-320.jpg)
![32 S. Hadjerrouit
emphasis on more abstract geometry, and away from facts, more argumentation and con-
jecturing [25]. Likewise, it may be useful to support tasks that encourage metacognition,
e.g., starting with real-world tasks, and using SimReal to generate results.
Affordances at the assessment level consist of summative and formative assessment.
Summative assessment is for testing, scoring and grading, and it can be provided in
the form of statistics that the tool can generate. Formative assessment is crucial for the
learning process. Feedback is an essential component of formative assessment, and it
can take many forms, e.g., immediate feedback to students’ actions, a combination of
conceptual, procedural, and corrective information to the students, or asking question
types, etc.
Finally, several socio-cultural affordances can emerge at this level. Firstly, SimReal
should provide opportunities to concretize the mathematics subject curriculum in teacher
education. Secondly, SimReal should be tied to teaching mathematics in schools, and
support the learning of mathematics at the primary, secondary, and upper secondary
school level. In other words, teachers using SimReal should consider the requirement of
adapted education. Finally, other socio-cultural affordances are associated with ethical,
gender, and multi-cultural issues, which are too complex to address in this paper.
4 The Study
4.1 Context of the Study and Participants
The study was conducted at the University of Agder in the context of a technology-based
course for teacher educators. The course covered various digital tools and their didactical
use in classroom, such as Excel, GeoGebra, Aplusix, Numbas, Smart Board, Campus
Inkrement, diverse pedagogical software, and SimReal.
Fifteen teacher students (N = 15) were enrolled in the course in 2016. The stu-
dents had very different knowledge background in mathematics ranging from primary
to upper secondary and university level. The students were categorized in four groups
according to their knowledge level in mathematics associated with their study pro-
grammes: Primary teacher education level 1–7, primary teacher education level 5–10,
advanced teacher education level 8–13, and mathematics master’s programme. The par-
ticipants had varied experience in using digital tools and resources inside or outside
classroom settings. In terms of mathematics, the basic requirement was the comple-
tion of a bachelor-degree in teacher or mathematics education, which presupposes at
least the acquisition of mathematics at the primary and middle school level. In terms of
digital tools, the recommended prerequisites were basic knowledge in information and
communication technologies and some experiences with standard digital tools like text
processing, spreadsheets, calculators and Internet. None of the students had any prior
experience with SimReal.
4.2 Research Questions
Based on the results achieved in the paper presented in Hadjerrouit [3], the purpose of
this work is to extend the investigation of affordances and constraints of SimReal and
their impact on students’ mathematical learning in teacher education. It addresses two
main research questions:](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-52-320.jpg)
![Exploring the Affordances of SimReal for Learning Mathematics 33
a) What are the students’ perceptions of SimReal’s affordances and constraints?
b) What are the impacts of SimReal on mathematical learning in terms of affordances
and constraints in teacher education?
The first research question is addressed in the result section, and the second in the
discussion.
4.3 Teaching Activities
A digital learning environment centered around SimReal was created over three weeks,
starting from 25 August to 8 September 2016. The SimReal environment included video
lectures, visualizations, and simulations of basic, elementary, and advanced mathemat-
ics, and diverse online teaching material. Basic mathematics focused on games, dices,
towerofHanoi,andprison.Elementarymathematicsconsistedofmultiplication,algebra,
Pythagoras’ and Square theorem, and reflection. The topics of advanced mathematics
were measurement, trigonometry, conic section, parameter, differentiation, Fourier, and
programming issues.
To assess students’ experiences on specific mathematical topics that are of consid-
erable interest for the participants, two specific tasks were chosen. The first one was
Pythagoras’ theorem, and the many ways of representing it. The theorem has also been
given numerous proofs, including both geometric and algebraic proofs, e.g., proofs by
dissection and rearrangement, Euclid’s proof, and algebraic proofs. Thus, Pythagoras’
theorem is more than just a way of calculating the lengths of a triangle. An example of
representing the theorem is given in the following figure (Fig. 4).
Fig. 4. An example of representation of Pythagoras’ theorem [3].
The second task was the Square theorem. Like Pythagoras’ theorem, there are many
ways of using and representing the theorem (Fig. 5).
4.4 Methods
This work is a single case study in teacher education. It aims at exploring the affor-
dances of SimReal for mathematical learning in teacher education. Both quantitative
and qualitative methods were used to collect and analyze the affordances and constraints
of SimReal and the students’ experiences with the tool. The following methods were
used:](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-53-320.jpg)
![34 S. Hadjerrouit
Fig. 5. An example of representation of the Square theorem [3].
a) A survey questionnaire with a five-point Likert scale from 1 to 5, and quantitative
analysis of the results
b) Students’ comments in their own words to each of the statements of the survey
c) Students’ written answers to open-ended questions
d) Qualitative analysis of students’ comments on point b and answers to open-ended
questions to point c
e) Task-based questions on Pythagoras’ and Square theorem, and programming issues
The design of the survey was guided by the theoretical framework and the research
questions. To measure students’ perceptions of SimReal, a survey questionnaire with a
five-point Likert scale from 1 to 5 was used, where 1 was coded as the highest and 5 as
the lowest (1 = “Strongly Agree”; 2 = “Agree”; 3 = “Neither Agree nor Disagree”; 4 =
“Disagree”; 5 = “Strongly Disagree”). The average score (MEAN) was calculated,
and the responses to open-ended questions were analyzed qualitatively. The survey
included 72 statements that were distributed as follows: Technological affordances (12),
pedagogical affordances at the student level (11), classroom level (19), mathematical
subject level (9), assessment level (10), and finally socio-cultural level (11). The students
were asked to respond to the survey using the five-point Likert scale and to comment
each of the statements in their own words. The students were also asked to provide
written answers in their own words to open-ended questions. The responses to students’
comments to the survey and open-ended questions were analyzed qualitatively.
Of particular importance are task-based questions on Pythagoras’ and Square theo-
rem to collect data on affordances of these two mathematical tasks. Additional questions
on programming issues were given to the students to assess the affordances of program-
ming languages for the learning of mathematics. Asking task-based questions provides
information on the affordances of SimReal. This method also provides more nuanced
and in-depth information about the students’ experiences with SimReal. The analysis of
the data was guided by the specified affordances of the theoretical framework presented
in Fig. 3, and open coding to bring to the fore information about affordances that was
not covered by the theoretical framework.](https://image.slidesharecdn.com/16355401-250514194249-e2d9909c/85/Computer-Supported-Education-1st-Ed-H-Chad-Lane-Susan-Zvacek-54-320.jpg)
Computer Supported Education 1st Ed H Chad Lane Susan Zvacek
- 1. Computer Supported Education 1st Ed H Chad Lane Susan Zvacek download https://ebookbell.com/product/computer-supported-education-1st- ed-h-chad-lane-susan-zvacek-32710802 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Computer Supported Education 12th International Conference Csedu 2020 Virtual Event May 24 2020 Revised Selected Papers Communications In Computer And Information Science 1st Ed 2021 H Chad Lane Editor https://ebookbell.com/product/computer-supported-education-12th- international-conference-csedu-2020-virtual-event-may-24-2020-revised- selected-papers-communications-in-computer-and-information- science-1st-ed-2021-h-chad-lane-editor-36429460 Computer Supported Education 6th International Conference Csedu 2014 Barcelona Spain April 13 2014 Revised Selected Papers 1st Edition Susan Zvacek https://ebookbell.com/product/computer-supported-education-6th- international-conference-csedu-2014-barcelona-spain- april-13-2014-revised-selected-papers-1st-edition-susan-zvacek-5236728 Computer Supported Education 7th International Conference Csedu 2015 Lisbon Portugal May 2325 2015 Revised Selected Papers 1st Edition Susan Zvacek https://ebookbell.com/product/computer-supported-education-7th- international-conference-csedu-2015-lisbon-portugal- may-2325-2015-revised-selected-papers-1st-edition-susan-zvacek-5483630 Computer Supported Education 10th International Conference Csedu 2018 Funchal Madeira Portugal March 1517 2018 Revised Selected Papers 1st Ed Bruce M Mclaren https://ebookbell.com/product/computer-supported-education-10th- international-conference-csedu-2018-funchal-madeira-portugal- march-1517-2018-revised-selected-papers-1st-ed-bruce-m- mclaren-10488598
- 3. What We Know About Cscl And Implementing It In Higher Education Computersupported Collaborative Learning Series 1st Edition Janwillem Strijbos https://ebookbell.com/product/what-we-know-about-cscl-and- implementing-it-in-higher-education-computersupported-collaborative- learning-series-1st-edition-janwillem-strijbos-2007300 Computersupported Collaborative Learning In Higher Education Tim S Roberts https://ebookbell.com/product/computersupported-collaborative- learning-in-higher-education-tim-s-roberts-4099328 Scripting Computersupported Collaborative Learning Cognitive Computational And Educational Perspectives Computersupported Collaborative Learning Series 1st Edition Frank Fischer https://ebookbell.com/product/scripting-computersupported- collaborative-learning-cognitive-computational-and-educational- perspectives-computersupported-collaborative-learning-series-1st- edition-frank-fischer-2110934 Dialogic Education And Technology Expanding The Space Of Learning Computersupported Collaborative Learning Series 1st Edition Rupert Wegerif https://ebookbell.com/product/dialogic-education-and-technology- expanding-the-space-of-learning-computersupported-collaborative- learning-series-1st-edition-rupert-wegerif-2110942 Computers Supported Education 8th International Conference Csedu 2016 Rome Italy April 2123 2016 Revised Selected Papers Costagliola https://ebookbell.com/product/computers-supported-education-8th- international-conference-csedu-2016-rome-italy- april-2123-2016-revised-selected-papers-costagliola-6753178
- 5. H. Chad Lane Susan Zvacek James Uhomoibhi (Eds.) 11th International Conference, CSEDU 2019 Heraklion, Crete, Greece, May 2–4, 2019 Revised Selected Papers Computer Supported Education Communications in Computer and Information Science 1220
- 6. Communications in Computer and Information Science 1220 Editorial Board Members Joaquim Filipe Polytechnic Institute of Setúbal, Setúbal, Portugal Ashish Ghosh Indian Statistical Institute, Kolkata, India Raquel Oliveira Prates Federal University of Minas Gerais (UFMG), Belo Horizonte, Brazil Lizhu Zhou Tsinghua University, Beijing, China
- 7. More information about this series at http://www.springer.com/series/7899
- 8. H. Chad Lane • Susan Zvacek • James Uhomoibhi (Eds.) Computer Supported Education 11th International Conference, CSEDU 2019 Heraklion, Crete, Greece, May 2–4, 2019 Revised Selected Papers 123
- 9. Editors H. Chad Lane University of Illinois at Urbana-Champaign Urbana, IL, USA Susan Zvacek Anderson Academic Commons University of Denver Denver, CO, USA James Uhomoibhi School of Engineering University of Ulster Newtownabbey, UK ISSN 1865-0929 ISSN 1865-0937 (electronic) Communications in Computer and Information Science ISBN 978-3-030-58458-0 ISBN 978-3-030-58459-7 (eBook) https://doi.org/10.1007/978-3-030-58459-7 © Springer Nature Switzerland AG 2020 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
- 10. Preface The present book includes extended and revised versions of selected papers from the 11th International Conference on Computer Supported Education (CSEDU 2019), held in Heraklion, Crete, Greece, during May 2–4, 2019. CSEDU 2019 received 202 paper submissions from 48 countries, of which 15% were included in this book. The papers were selected by the event chairs based on a number of criteria that include the classifications and comments provided by the Program Committee members, the session chairs’ assessment, and also the program chairs’ global view of all papers included in the technical program. The authors of selected papers were then invited to submit a revised and extended version of their papers, having at least 30% additional material. CSEDU 2019 is an annual meeting place for presenting and discussing new edu- cational tools and environments, best practices and case studies on innovative technology-based learning strategies, and institutional policies on computer supported education including open and distance education. CSEDU welcomes research and practice articles on current technologies, as well as emerging trends, and promotes discussion about the pedagogical potential of new educational technologies in the academic and corporate world. In this collection of extended papers, the breadth of research covered by the CSEDU community is immediately evident. Research presented here occurs in a number of important learning contexts, including collaborative learning, higher education, online learning, informal learning, and with learners of all ages. The articles also explore a rich and exciting set of technologies and approaches, such as adaptive educational technologies, learning analytics, game-based approaches, virtual and augmented real- ity, social technologies, and physiological sensing. Finally, readers will see that papers also touch on a wide range of target domains, including computer programming, computational thinking, electronics, population modeling, self-regulation, and more. We are very happy to share this exciting set of papers with the broader educational technology research community. We would like to thank all the authors for their contributions, and also to the reviewers who helped to ensure the quality of this publication. May 2019 H. Chad Lane Susan Zvacek James Uhomoibhi
- 11. Organization Conference Chair James Uhomoibhi Ulster University, UK Program Co-chairs H. Chad Lane University of Illinois at Urbana-Champaign, USA Susan Zvacek SMZTeaching.com, USA Program Committee Dor Abrahamson University of California, Berkeley, USA Mehdi Adda Université du Québec à Rimouski, Canada Nelma Albuquerque Concepts and Insights, Brazil Efthimios Alepis University of Piraeus, Greece Eleftheria Alexandri Hellenic Open University, Greece Richard Alterman Brandeis University, USA Fahriye Altinay Near East University, Cyprus António Andrade Universidade Católica Portuguesa, Portugal Francisco Arcega Universidad de Zaragoza, Spain Juan Ignacio Asensio University of Valladolid, Spain Breno Azevedo Instituto Federal de Educação, Ciência e Tecnologia Fluminense, Brazil Nilufar Baghaei Unitec, New Zealand Zoltan Balogh Constantine the Philosopher University in Nitra, Slovakia Adriano Baratè Università degli Studi di Milano, Italy João Barros Polytechnic Institute of Beja, Portugal Marcelo Barros Federal University of Campina Grande, Brazil Patrícia Bassani Universidade Feevale, Brazil Emad Bataineh Zayed University, UAE Kay Berkling Baden-Württemberg Cooperative State University, Germany Gilda Bernardino de Campos Pontifícia Universidade Católica do Rio de Janeiro, Brazil Pavol Bistak Slovak University of Technology, Slovakia Andreas Bollin Klagenfurt University, Austria Elmar-Laurent Borgmann Koblenz University of Applied Sciences, Germany Ivana Bosnic University of Zagreb, Croatia Federico Botella Miguel Hernandez University of Elche, Spain Rosa Bottino CNR, Italy
- 12. François Bouchet Laboratoire d’Informatique de Paris 6, France Patrice Bouvier SYKO Studio, France Tharrenos Bratitsis University of Western Macedonia, Greece Krysia Broda Imperial College London, UK Martin Bush London South Bank University, UK Egle Butkeviciene Kaunas University of Technology, Lithuania Santi Caballé Open University of Catalonia, Spain Renza Campagni Università di Firenze, Italy Pasquina Campanella University of Bari, Italy Chris Campbell Griffith University, Australia Thibault Carron Pierre and Marie Curie University, France Ana Carvalho University of Coimbra, Portugal Vítor Carvalho IPCA-EST, University of Minho, Portugal Cristian Cechinel Universidade Federal de Pelotas, Brazil Isabel Chagas Universidade de Lisboa, Portugal Chia-Hu Chang National Taiwan University, Taiwan, China Mohamed Chatti University of Duisburg-Essen, Germany Lukas Chrpa Czech Technical University in Prague, Czech Republic Maria Cinque LUMSA Università, Italy António Coelho University of Porto, Portugal Marc Conrad University of Bedfordshire, UK Ruth Cook DePaul University, USA Fernando Costa Universidade de Lisboa, Portugal Gennaro Costagliola Università degli Studi di Salerno, Italy Manuel Perez Cota University of Vigo, Spain John Cuthell Virtual Learning, UK Rogério da Silva University of Houston-Victoria, USA Ines Dabbebi LIUM, France Sergiu Dascalu University of Nevada, Reno, USA Angélica de Antonio Universidad Politécnica de Madrid, Spain Luis de-la-Fuente-Valentín Universidad Internacional de la Rioja, Spain Giuliana Dettori ITD-CNR, Italy Tania Di Mascio University of L’Aquila, Italy Yannis Dimitriadis University of Valladolid, Spain Amir Dirin Haaga-Helia UAS, Finland Danail Dochev Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Bulgaria Georg Dr. Schneider Trier University of Applied Sciences, Germany Toby Dragon Ithaca College, USA Benedict du Boulay University of Sussex, UK Amalia Duch Politechnical University of Catalonia, Spain Ishbel Duncan University of St Andrews, UK Nour El Mawas Université de Lille, France Larbi Esmahi Athabasca University, Canada João Esteves University of Minho, Portugal viii Organization
- 13. Ramon Fabregat Gesa Universitat de Girona, Spain Sanaz Fallahkhair University of Brighton, UK Si Fan University of Tasmania, Australia Michalis Feidakis University of West Attica (UniWA), Greece Richard Ferdig Kent State University, USA Rosa Fernandez-Alcala University of Jaen, Spain Rita Francese Università degli Studi di Salerno, Italy Jesús G. Boticario aDeNu Research Group, UNED, Spain Mikuláš Gangur University of West Bohemia, Czech Republic Francisco García Peñalvo Salamanca University, Spain Isabela Gasparini UDESC, Brazil Piotr Gawrysiak Warsaw University of Technology, Poland Sébastien George Le Mans University, France Biswadip Ghosh Metropolitan State University of Denver, USA Henrique Gil Escola Superior de Educação do Instituto Politécnico de Castelo Branco, Portugal Mercè Gisbert Universitat Rovira i Virgili, Spain Apostolos Gkamas University Ecclesiastical Academy of Vella, Greece Anabela Gomes Coimbra Polytechnic (ISEC), Portugal Cristina Gomes Instituto Politécnico de Viseu, Portugal Maria Gomes University of Minho, Portugal Nuno Gonçalves Polytechnic Institute of Setúbal, Portugal Carina Gonzalez Universidad de la Laguna, Spain Ana González Marcos Universidad de la Rioja, Spain Anandha Gopalan Imperial College London, UK Angela Guercio Kent State University, USA Christian Guetl Graz University of Technology, Austria Nathalie Guin Université Claude Bernard Lyon 1, France David Guralnick Kaleidoscope Learning, USA Roger Hadgraft University of Technology Sydney, Australia Yasunari Harada Waseda University, Japan Cecilia Haskins Norwegian University of Science and Technology, Norway Oriel Herrera Universidad Catolica de Temuco, Chile Antonio Hervás Jorge Universidad Politécnica de Valencia, Spain Janet Hughes The Open University, UK Nnenna Ibezim University of Nigeria Nsukka, Nigeria Tomayess Issa Curtin University, Australia Ivan Ivanov SUNY Empire State College, USA Marc Jansen University of Applied Sciences Ruhr West, Germany Jose Janssen Open Universiteit, The Netherlands Hannu-Matti Järvinen Tampere University, Finland Stéphanie Jean-Daubias Université Claude Bernard Lyon 1, LIRIS, France Michail Kalogiannakis University of Crete, Greece Atis Kapenieks Riga Technical University, Latvia Vaggelis Kapoulas Computer Technology Institute and Press, Greece Organization ix
- 14. Ilias Karasavvidis University of Thessaly, Greece Jerzy Karczmarczuk University of Caen, France David Kaufman Simon Fraser University, Canada Jalal Kawash University of Calgary, Canada Mizue Kayama Shinshu University, Japan Samer Khasawneh Walsh University, USA Rob Koper Open Universiteit, The Netherlands Maria Kordaki University of the Aegean, Greece Adamantios Koumpis Berner Fachhochschule, Switzerland Miroslav Kulich Czech Technical University in Prague, Czech Republic Lam-for Kwok City University of Hong Kong, Hong Kong, China Jean-Marc Labat Pierre and Marie Curie University, France José Lagarto Universidade Católica Portuguesa, Portugal Elicia Lanham Deakin University, Australia Rynson Lau City University of Hong Kong, Hong Kong, China Geoffrey Lautenbach University of Johannesburg, South Africa Borislav Lazarov Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Bulgaria José Leal University of Porto, Portugal Dominique Leclet UPJV, France Chien-Sing Lee Sunway University, Malaysia Mark Lee Charles Sturt University, Australia Newton Lee Newton Lee Laboratories LLC and Institute for Education Research and Scholarships, USA Marie Lefevre Université Claude Bernard Lyon 1, France José Alberto Lencastre University of Minho, Portugal Andrew Lian Suranaree University of Technology, Thailand Cheng-Min Lin Nan Kai University of Technology, Taiwan, China Andreas Lingnau Ruhr West University of Applied Sciences, Germany Martin Llamas-Nistal University of Vigo, Spain Chee-Kit Looi Nanyang Technological University, Singapore Luca Andrea Ludovico Università degli Studi di Milano, Italy Krystina Madej Georgia Tech, USA Maria Marcelino University of Coimbra, Portugal Massimo Marchiori University of Padua, Italy Jacek Marciniak Adam Mickiewicz University, Poland Ivana Marenzi Leibniz University Hannover, Germany José Marques FEUP, Portugal Lindsay Marshall Newcastle University, UK Alke Martens Universität of Rostock, Germany Scheila Martins University of Houston-Victoria, USA Demétrio Matos IPCA-ID+, Portugal Bruce Maxim University of Michigan-Dearborn, USA Madeth May Le Mans University, France Godfrey Mayende University of Agder, Norway Elvis Mazzoni University of Bologna, Italy x Organization
- 15. José Carlos Metrôlho Instituto Politécnico de Castelo Branco, Portugal Louise Mifsud Oslo Metropolitan University, Norway Bakhtiar Mikhak Harvard University, USA Peter Mikulecky University of Hradec Kralove, Czech Republic Andreea Molnar Swinburne University of Technology, Australia Gyöngyvér Molnár University of Szeged, Hungary António Moreira Universidade de Aveiro, Portugal Jerzy Moscinski Silesian University of Technology, Poland Maria Moundridou School of Pedagogical and Technological Education (ASPETE), Greece Antao Moura Federal University of Campina Grande (UFCG), Brazil Michal Munk Constantine the Philosopher University in Nitra, Slovakia Cristina Muntean National College of Ireland, Ireland Antoanela Naaji Vasile Goldis Western University of Arad, Romania Hiroyuki Nagataki Osaka Electro-Communication University, Japan Ryohei Nakatsu National University of Singapore, Singapore Minoru Nakayama Tokyo Institute of Technology, Japan Sotiris Nikolopoulos Technological Educational Institute of Larissa, Greece Fátima Nunes Universidade de São Paulo, Brazil Dade Nurjanah Telkom University, Indonesia Emma O’Brien Mary Immaculate College, Ireland Ebba Ossiannilsson The Swedish Association for Distance Education (SADE), Sweden Kuo-liang Ou National Tsinghua University, Taiwan, China Alessandro Pagano University of Bari, Italy José Palma Polytechnic Institute of Setúbal, Portugal Manuel Palomo Duarte Universidad de Cádiz, Spain Stamatios Papadakis Department of Preschool Education, Faculty of Education, University of Crete, Greece Iraklis Paraskakis South East European Research Centre, Greece Pramod Pathak National College of Ireland, Ireland Emanuel Peres University of Trás-os-Montes and Alto Douro, INESC TEC, Portugal Paula Peres ISCAP, Portugal Arnulfo Perez The Ohio State University, USA Isidoros Perikos University of Patras, Greece Donatella Persico CNR, Italy Alfredo Pina Public University of Navarra, Spain Niels Pinkwart Humboldt University, Germany Elvira Popescu University of Craiova, Romania Tamara Powell Kennesaw State University, USA Francesca Pozzi CNR, Italy Augustin Prodan Iuliu Hatieganu University, Romania Organization xi
- 16. Franz Puehretmair Competence Network Information Technology to Support the Integration of People with Disabilities (KI-I), Austria Clark Quinn Quinnovation, USA Muthu Ramachandran Leeds Beckett University, UK Altina Ramos University of Minho, Portugal Fernando Ramos University of Aveiro, Portugal Eliseo Reategui Universidade Federal do Rio Grande do Sul, Brazil Manuel Reis University of Trás-os-montes and Alto Douro, Portugal Fernando Ribeiro Instituto Politécnico de Castelo Branco, Portugal Sandro Rigo Unisinos, Brazil Razvan Rughinis University POLITECHNICA of Bucharest, Romania Rebecca Rutherfoord Kennesaw State University, USA Barbara Sabitzer Johannes Kepler University Linz, Austria Libuse Samkova University of South Bohemia, Czech Republic Demetrios Sampson University of Piraeus, Greece Juan M. Santos University of Vigo, Spain Manoj Saxena Central University of Himachal Pradesh, India Sabine Schlag University of Wuppertal, Germany Wolfgang Schreiner Johannes Kepler University Linz, Austria Ulrik Schroeder RWTH Aachen University, Germany Sabine Seufert University of St. Gallen, Switzerland Haya Shamir Waterford Research Institute, USA Ali Fawaz Shareef Cyryx College, Maldives Pei Siew Universiti Tunku Abdul Rahman, Malaysia Juarez Bento da Silva Universidade Federal de Santa Catarina, Brazil Jane Sinclair University of Warwick, UK Natalya Snytnikova Novosibirsk State University, Russia Filomena Soares University of Minho, Portugal Ellis Solaiman Newcastle University, UK Michael Sonntag Johannes Kepler University Linz, Austria Marcus Specht TU Delft, The Netherlands J. Spector University of North Texas, USA Claudia Steinberger University of Klagenfurt, Austria Jun-Ming Su National University of Tainan, Taiwan, China Masanori Sugimoto Hokkaido University, Japan Katsuaki Suzuki Kumamoto University, Japan Nestori Syynimaa University of Jyväskylä, Finland Qing Tan Athabasca University, Canada Steven Tanimoto University of Washington, USA Luca Tateo Aalborg University, Denmark Dirk Tempelaar Maastricht University, The Netherlands Marco Temperini Sapienza University of Rome, Italy Uwe Terton University of the Sunshine Coast, Australia Neena Thota University of Massachusetts Amherst, USA Tomas Trescak Western Sydney University, Australia xii Organization
- 17. Abdallah Tubaishat Zayed University, UAE Richard Van Eck University of North Dakota, USA Leo van Moergestel University of Applied Sciences Utrecht, The Netherlands Carlos Vaz de Carvalho ISEP, Portugal Andreas Veglis Aristotle University of Thessaloniki, Greece Ioanna Vekiri Independent Researcher, Greece J. Velázquez-Iturbide Universidad Rey Juan Carlos, Spain Giuliano Vivanet University of Cagliari, Italy Bahtijar Vogel Malmö University, Sweden Harald Vranken Open Universiteit, The Netherlands Alf Wang Norwegian University of Science and Technology, Norway Fangju Wang University of Guelph, Canada Edgar Weippl University of Vienna, SBA Research, Austria David Whittinghill Purdue University, USA Leandro Wives Universidade Federal do Rio Grande do Sul, Brazil Jie Yang National Central University, Taiwan, China Amel Yessad Laboratoire d’Informatique de Paris 6, Sorbonne Université, France Katarina Zakova Slovak University of Technology in Bratislava, Slovakia Diego Zapata-Rivera Educational Testing Service, USA Thomas Zarouchas Computer Technology Institute and Press, Greece Iveta Zolotova Technical University of Kosice, Slovakia Javier García Zubía Universidad de Deusto, Spain Additional Reviewers Sandra Elsom University of the Sunshine Coast, Australia Laura Freina CNR, Institute for Educational Technology, Italy Richard Ladner University of Washington, USA Andrew Munoz University of Nevada, Reno, USA Marcello Passarelli CNR, Italy Luis Pedro University of Aveiro, Portugal Connor Scully-Allison University of Nevada, Reno, USA Eiji Tomida Ehime University, Japan Yasushi Tsubota Kyoto University, Japan Invited Speakers Maria Roussou University of Athens, Greece Michael Baker CNRS, France Organization xiii
- 18. Contents How Programming Students Trick and What JEdUnit Can Do Against It. . . . 1 Nane Kratzke Exploring the Affordances of SimReal for Learning Mathematics in Teacher Education: A Socio-Cultural Perspective . . . . . . . . . . . . . . . . . . 26 Said Hadjerrouit Feedback Preferences of Students Learning in a Blended Environment: Worked Examples, Tutored and Untutored Problem-Solving. . . . . . . . . . . . . 51 Dirk T. Tempelaar, Bart Rienties, and Quan Nguyen Teaching Defence Against the Dark Arts Using Game-Based Learning: A Review of Learning Games for Cybersecurity Education . . . . . . . . . . . . . 71 Rene Roepke and Ulrik Schroeder A Collective Dynamic Indicator for Discussion Forums in Learning Management Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Malik Koné, Madeth May, Sébastien Iksal, and Souleymane Oumtanaga Immersion and Control in Learning Art Knowledge: An Example in Museum Visit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Morgane Burgues, Nathalie Huet, and Jean-Christophe Sakdavong Studying Relationships Between Network Structure in Educational Forums and Students’ Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 O. Ferreira-Pires, M. E. Sousa-Vieira, J. C. López-Ardao, and M. Fernández-Veiga Emotion Recognition from Physiological Sensor Data to Support Self-regulated Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Haeseon Yun, Albrecht Fortenbacher, René Helbig, Sven Geißler, and Niels Pinkwart Separating the Disciplinary, Application and Reasoning Dimensions of Learning: The Power of Technology-Based Assessment . . . . . . . . . . . . . . 174 Gyöngyvér Molnár and Benő Csapó Behind the Shoulders of Bebras Teams: Analyzing How They Interact with the Platform to Solve Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Carlo Bellettini, Violetta Lonati, Mattia Monga, and Anna Morpurgo
- 19. Computational Pedagogy: Block Programming as a General Learning Tool. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Stefano Federici, Elisabetta Sergi, Claudia Medas, Riccardo Lussu, Elisabetta Gola, and Andrea Zuncheddu RefacTutor: An Interactive Tutoring System for Software Refactoring . . . . . . 236 Thorsten Haendler, Gustaf Neumann, and Fiodor Smirnov User-Centered Design: An Effective Approach for Creating Online Educational Games for Seniors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 Louise Sauvé and David Kaufman Population Growth Modelling Simulations: Do They Affect the Scientific Reasoning Abilities of Students?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Kathy Lea Malone and Anita Schuchardt Novice Learner Experiences in Software Development: A Study of Freshman Undergraduates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 Catherine Higgins, Ciaran O’Leary, Claire McAvinia, and Barry Ryan Promoting Active Participation in Large Programming Classes . . . . . . . . . . . 331 Sebastian Mader and François Bry Agile Methods Make It to Non-vocational High Schools . . . . . . . . . . . . . . . 355 Ilenia Fronza, Claus Pahl, and Boris Sušanj Testing the Testing Effect in Electrical Science with Learning Approach as a Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 James Eustace and Pramod Pathak A Web Platform to Foster and Assess Tonal Harmony Awareness . . . . . . . . 398 Federico Avanzini, Adriano Baratè, Luca A. Ludovico, and Marcella Mandanici MATE-BOOSTER: Design of Tasks for Automatic Formative Assessment to Boost Mathematical Competence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 Alice Barana, Marina Marchisio, and Raffaella Miori Educational Practices in Computational Thinking: Assessment, Pedagogical Aspects, Limits, and Possibilities: A Systematic Mapping Study . . . . . . . . . . 442 Lúcia Helena Martins-Pacheco, Nathalia da Cruz Alves, and Christiane Gresse von Wangenheim Virtual Reality and Virtual Lab-Based Technology-Enhanced Learning in Primary School Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 Diana Bogusevschi and Gabriel-Miro Muntean xvi Contents
- 20. Increasing Parental Involvement in Computer Science Education Through the Design and Development of Family Creative Computing Workshops . . . . 479 Nina Bresnihan, Glenn Strong, Lorraine Fisher, Richard Millwood, and Áine Lynch Retention of University Teachers and Doctoral Students in UNIPS Pedagogical Online Courses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Samuli Laato, Heidi Salmento, Emilia Lipponen, Henna Vilppu, Mari Murtonen, and Erno Lehtinen Designing Culturally Inclusive MOOCs . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 Mana Taheri, Katharina Hölzle, and Christoph Meinel Evaluation of an Interactive Personalised Virtual Lab in Secondary Schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 Ioana Ghergulescu, Arghir-Nicolae Moldovan, Cristina Hava Muntean, and Gabriel-Miro Muntean Potential Benefits of Playing Location-Based Games: An Analysis of Game Mechanics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 Samuli Laato, Tarja Pietarinen, Sampsa Rauti, and Erkki Sutinen Resolving Efficiency Bottleneck of the Bellman Equation in Adaptive Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 Fangju Wang Ontology-Based Analysis and Design of Educational Games for Software Refactoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602 Thorsten Haendler and Gustaf Neumann in-Game Raw Data Collection and Visualization in the Context of the “ThimelEdu” Educational Game . . . . . . . . . . . . . . . . . . . . . . . . . . . 629 Nikolas Vidakis, Anastasios Kristofer Barianos, Apostolos Marios Trampas, Stamatios Papadakis, Michail Kalogiannakis, and Kostas Vassilakis Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647 Contents xvii
- 21. How Programming Students Trick and What JEdUnit Can Do Against It Nane Kratzke(B) Lübeck University of Applied Sciences, Mönkhofer Weg 239, 23562 Lübeck, Germany [email protected] Abstract. According to our data, about 15% of programming students trick if they are aware that only a “dumb” robot evaluates their pro- gramming assignments unattended by programming experts. Especially in large-scale formats like MOOCs, this might become a question because to trick current automated assignment assessment systems (APAAS) is astonishingly easy and the question arises whether unattended grading components grade the capability to program or to trick. This study ana- lyzed what kind of tricks students apply beyond the well-known “copy- paste” code plagiarism to derive possible mitigation options. Therefore, this study analyzed student cheat patterns that occurred in two program- ming courses and developed a unit testing framework JEdUnit as a solu- tion proposal that intentionally targets such tricky educational aspects of programming. The validation phase validated JEdUnit in another pro- gramming course. This study identified and analyzed four recurring cheat patterns (overfitting, evasion, redirection, and injection) that hardly occur in “normal” software development and are not aware to normal unit testing frameworks that are frequently used to test the correct- ness of student submissions. Therefore, the concept of well-known unit testing frameworks was extended by adding three “countermeasures”: randomization, code inspection, separation. The validation showed that JEdUnit detected these patterns and in consequence, reduced cheating entirely to zero. From a students perspective, JEdUnit makes the grad- ing component more intelligent, and cheating does not pay-off anymore. This Chapter explains the cheat patterns and what features of JEdUnit mitigate them by a continuous example. Keywords: Automatic · Assessment · Programming · Course · Education APAAS · MOOC · Moodle · VPL · Trick · Cheat · Pattern · Mitigation 1 Introduction In a digitized world, more and more experts are needed with at least some basic programming skills. Programming might even evolve into a foundational skill similar to reading, writing, and calculating. Therefore, the course sizes of c Springer Nature Switzerland AG 2020 H. C. Lane et al. (Eds.): CSEDU 2019, CCIS 1220, pp. 1–25, 2020. https://doi.org/10.1007/978-3-030-58459-7_1
- 22. 2 N. Kratzke university and college programming courses are steadily increasing. Even mas- sive open online courses [14] are used more and more systematically to convey necessary programming capabilities to students of different disciplines [19]. The coursework consists of programming assignments that need to be assessed. Since the submitted assignments are executable programs with a formal structure, they are highly suited to be assessed automatically. In consequence, plenty of automated programming assignment assessment systems (APAAS) evolved. We refer to [1–3,5,9,17] for an overview of such tools. Previous research [11] showed how astonishingly simple it is for students to trick automated programming assignment assessment systems. It is often over- looked that APAAS solutions are systems that execute injected code (student submissions) and code injection is known as a severe threat from a security point of view [20]. We refer to [7,15], and [6] for an overview of such kind of attacks. Of course, such code injection vulnerabilities are considered by current solu- tions. However, in previous research [11], it was astonishing to see that current APAAS solutions sometimes overlook the cheating cleverness of students. On the one hand, APAAS solutions protect the host system via sandbox mechanisms, and APAAS systems put much effort in sophisticated plagiarism detection and authorship control of student submissions [13,16]. On the other, the grading com- ponent can be cheated in various – sometimes ridiculously simple – ways making these solutions highly suspect for (semi-)automated and unattended program- ming examinations that contribute to certificate a certain level of programming expertise. Previous research [11] identified at least four simple cheat patterns: – Overfitting – Evasion – Redirection – Injection Moreover, it strived to raise a general problem awareness but did not focus on solutions to mitigate these patterns. To propose solutions on how to mitigate these identified cheat patterns is the primary intent and contribution of this Chapter. We propose to use the following three techniques: – Randomization of test cases – Pragmatic code inspections – Separation of student and evaluation logic These techniques mitigate the presented patterns and demonstrate the suit- ability for the APAAS solution Moodle/VPL and the programming language Java. Nevertheless, the principles are transferable to other APAAS solutions and programming languages and therefore of broader interest and not limited to Moodle/VPL and Java. Consequently, the remainder of this paper is outlined as follows. Section 2 presents the methodology that has been used to identify and categorize stu- dent cheat-patterns and to validate appropriate “countermeasures”. Section 3 will explain the identified cheat-patterns. Section 4 presents a mitigation analysis
- 23. How Programming Students Trick and What JEdUnit Can Do Against It 3 and shows that the identified cheat patterns can be addressed by three “coun- termeasures” that should be considered by every APAAS solution. Section 5 explains how these insights have been considered in the development of a unit testing framework (JEdUnit) that focuses intentionally on educational aspects and considers the aspects mentioned above consequently. We will discuss our results of JEdUnit in Sect. 6 and provide some guidelines on the generalizability and limitations of this research. Finally, we conclude our findings in Sect. 7. Fig. 1. Research methodology. 2 Methodology Figure 1 presents the overall research approach that compromised two phases: Problem systematization and solution proposal validation. 2.1 Problem Systematization For the initial problem systematization, two first semester programming Java courses in the winter semester 2018/19 (see Table 1) have been systematically evaluated. Both courses were used to search for student submissions that inten- tionally trick the grading component of APAAS solutions. Table 1 provides a summarized overview of the course design. All assignments were automatically evaluated by the VPL Moodle plugin (version 3.3.3) following the general rec- ommendations described by [21]. For more details, the reader is referred to [11]. Figure 2 shows an exemplifying VPL screenshot from a students perspective.
- 24. 4 N. Kratzke Fig. 2. VPL screenshot (on the right evaluation report presented to students) [11]. Table 1. Courses used for problem systematization and solution validation. Systematization Validation Prog I (CS) Prog I (ITD) Prog II (CS) Students 113 79 73 Assignments (total) 29 20 8 Number of bunches 11 6 7 Assignments per bunch (avg) 3 3 1 Time for a bunch (weeks) 1 2 2 Groups 7 6 4 Students per group (avg) 18 12 18 Student/advisor ratio (avg) 6 6 6 To minimize Hawthorne and Experimenter effects [4] neither the students nor the advisers in the practical programming courses were aware that student submissions were analyzed to deduce cheating patterns. Even if cheating was detected, this had no consequences for the students. It was not even communi- cated to the student or the advisers. Furthermore, students were not aware that the version history of their sub- missions and therefore even intermediate cheating experiments (that did not make it to the final submission) were logged. However, not every submission was inspected for understandable effort reasons. Therefore, only significant sub- mission samples (see Table 2) were investigated to search systematically for cheat-patterns. Table 3 summarizes the results quantitatively. Within these six samples, cheat and trick patterns were identified mainly by a manual but a script- supported observation. VPL submissions were downloaded from Moodle and analyzed weekly. We developed a Jupyter-based [10] quantitative analysis and
- 25. How Programming Students Trick and What JEdUnit Can Do Against It 5 submission data model for this dataset. Each student submission was represented as an object containing its version and grading history that references its student submitter and its corresponding study programme. The analytical script and data model made use of the well known Python libraries statistics, NumPy [12], matplotlib [8], and the Javaparser library [18]. It was used to identify the num- ber of submissions and evaluations, points per submission versions, timestamps per submission version, occurrences of unusual terms, and more. Based on this quantitative data, the mentioned samples (S1–S5) were selected automatically (or randomly in case of S6). Additionally, the source codes of the sample sub- missions were exported weekly as an archived PDF document. However, the scanning for cheat-patterns was done manually within these documents. Table 2. Weekly analyzed samples (systematization phase). Description Rationale S1 TOP 10 of submissions with many triggered evaluations Parameter optimization could cause plenty of evaluations S2 TOP 10 of submission with many versions Cheating experiments could cause plenty of versions S3 TOP 10 of submissions with astonishingly less average points across all triggered evaluations but full points in the final submission Cheating could cause such point boosts S4 Submissions with unusual (above 95% percentile) many condition related terms like if, return, switch, case, and so on Parameter optimization could cause unuasal many condition related terms S5 Submissions with unusual terms like System.exit, System.getProperties, := that would stop program execution or have a special meaning in VPL or Java but are unlikely to contribute to a problem solution APAAS attacks could cause such unusual terms S6 Ten random submissions To cover unintended observation aspects 2.2 Solution Proposal Validation Based on these elicited cheat patterns, corresponding mitigation options have been derived. Three of them (randomization, code inspection, and submission/e- valuation logic separation) have been implemented in a unit testing frame- work called JEdUnit as a solution proposal to mitigate the identified problems. JEdUnit has been validated using a Programming II course for computer science students in the summer semester 2019. The course has been given analogously
- 26. 6 N. Kratzke to the systematization phase except that JEdUnit has been applied. The search for cheats have been conducted similarly, except the fact, that we inspected every submission because of smaller course size and less (but more extensive and more complex) assignments. The mitigation options and JEdUnit, as well as the validation results, are presented in the following Sections. 3 Cheat-Patterns to Consider Some basic Java programming knowledge must be assumed throughout this paper. The continuous example assignment for this Chapter shall be the fol- lowing task. A method countChar() has to be programmed that counts the occurrence of a specific character c in a given String s (not case-sensitive). The following example calls are provided for a better understanding of the intended functionality. – countChar(‘a’, Abc) → 1 – countChar(‘A’, abc) → 1 – countChar(‘x’, ABC) → 0 – countChar(‘!’, !!!) → 3 A reference solution for our “count chars in a string” problem might be the following implementation of countChar(). Listing 1.1. Reference solution (continuous example). int countChar(char c, String s) { s = s.toLowerCase (); c = Character.toLowerCase(c); int i = 0; for (char x : s.toCharArray ()) { if (x == c) i++; } return i; } According to our data, most students strive to find a solution that fits the scope and intent of the assignment (see Table 3 and Fig. 3). However, in the systematization phase, a minority of students (approximately 15%) make use of the fact that a “dumb automat” grades. Accordingly, we observed the following cheating patterns that differ significantly from the intended reference solution above (see Fig. 3): – Overfitting solutions (63%) – Redirection to reference solutions (6%) – Problem evasion (30%) – Injection (1%)
- 27. How Programming Students Trick and What JEdUnit Can Do Against It 7 Table 3. Detected cheats. Fig. 3. Observed cheat-pattern frequency (without application of JEdUnit) [11]. Especially overfitting and evasion tricks are “poor-mans’ weapons” often used by novice programmers as a last resort to solve a problem. Much more alarm- ing redirection and injection cheats occurred only in rare cases (less than 10%). However, how do these tricks and cheats look like? How severe are they? More-
- 28. 8 N. Kratzke over, what can be done against it? We will investigate these questions in the following paragraphs. 3.1 Overfitting Tricks Overfitted solutions strive to get a maximum of points for grading but do not strive to solve the given problem in a general way. A notable example of an overfitted solution would be Listing 1.2. Listing 1.2. Overfitting solution. int countChar(char c, String s) { if (c == ’a’ s.equals(Abc)) return 1; if (c == ’A’ s.equals(abc)) return 1; if (c == ’x’ s.equals(ABC)) return 0; if (c == ’!’ s.equals(!!!)) return 3; // [...] if (c == ’x’ s.equals(X)) return 1; return 42; } This solution maps merely the example input parameters to the expected output values. The solution is completely useless outside the scope of the test cases. 3.2 Problem Evasion Tricks Another trick pattern is to evade a given problem statement. According to our experiences, this pattern occurs mainly in the context of more sophisticated and formal programming techniques like recursive programming or functional programming styles with lambda functions. So, let us now assume that the assignment is still to implement a countChar() method, but this method should be implemented recursively. A reference solution might look like in Listing 1.3 (we do not consider performance aspects due to tail recursion): Listing 1.3. Recursive reference solution. int countChar(char c, String s) { s = s.toLowerCase (); c = Character.toLowerCase(c); if (s.isEmpty ()) return 0; char head = s.charAt (0); String rest = s.substring (1); int n = head == c ? 1 : 0; return n + countChar(c, rest ); }
- 29. How Programming Students Trick and What JEdUnit Can Do Against It 9 However, sometimes, student submissions only pretend to be recursive with- out being it. Listing 1.4 is a notable example. Listing 1.4. Problem evasing solution. int countChar(char c, String s) { if (s.isEmpty ()) return 0; return countChar(c, s, 0); } int countChar(char c, String s, int i){ for (char x : s.toCharArray ()) { if (x == c) i++; } return i; } Although countChar() is calling (an overloaded version of) countChar() which looks recursively, the overloaded version of countChar() makes use of a for-loop and is therefore implemented in a fully imperative style. The same pattern can be observed if an assignment requests functional pro- gramming with lambda functions. A lambda-based solution could look like in Listing 1.5. Listing 1.5. Lambda reference solution. (c, s) - Stream.of(s.toCharArray ()) .filter(x - x == c) .count (); However, students take refuge in familiar programming concepts like loops. Very often, submissions like in Listing 1.6 are observable: Listing 1.6. Lambda problem evasion. (c, s) - { int i = 0; for (char x : s.toCharArray ()) { if (x == c) i++; } return i; }; The (c, s) - { [...] }; seems functional on the first-hand. But, if we look at the implementation, it is only an imperative for loop embedded in a functional looking context. The problem here is, that evaluation components just looking on input- output parameter correctness will not detect these kinds of programming style evasions. The just recursive- or functional-looking solutions will generate the correct results. Nevertheless, the intent of such kind of assignments is not just to foster correct solutions but although to train specific styles of programming.
- 30. 10 N. Kratzke 3.3 Redirection Cheats Another shortcoming of APAAS solutions can be compiler error messages that can reveal details of the evaluation logic. In the case of VPL, an evaluation is processed according to the following steps. 1. The submission is compiled and linked to the evaluation logic. 2. The compiled result is executed to run the checks. 3. The check results are printed in an APAAS specific notation on the console (standard-out). 4. This output is interpreted by the APAAS solution to run the automatic grad- ing and present a kind of feedback to the submitter. This process is straightforward and provides the benefit that evaluation com- ponents can handle almost all programming languages. If one of the steps fails, an error message is generated and returned to the submitter as feedback. This failing involves typically to return the compiler error message. That can be prob- lematic because these compiler error messages may provide unexpected cheating opportunities. Let us remember. The assignment was to program a method countChar(). Let us further assume that a student makes a small spelling error like to name the method instead of countChar() – so just a trailing s is added. That is a general programming error that happens fast (see Listing 1.7). Listing 1.7. A slightly misspelled submission. int countCharS(char c, String s) { int i = 0; for (char x : s.toCharArray ()) { if (x == c) i++; } return i; } If this submission would be submitted and evaluated by an APAAS solution, this submission would likely not pass the first compile step due to a simple spelling error. What is returned is a list of compiler error messages like this one here that shows the problem: Checks.java:40: error: cannot find symbol Submission.countChar(‘a’, Abc) == Solution.countChar(‘a’, Abc) The compiler message provides useful hints to correct a misspelt method name, but it also reveals that a method (Solution.countChar()) exists to check the student submission. The reference solution can be assumed to be correct. So, compiler error messages can reveal a reference solution method that could be called directly. A student can correct the naming and redirects the submitted
- 31. How Programming Students Trick and What JEdUnit Can Do Against It 11 method directly to the method that is used for grading. Doing that, the student would let the evaluation component evaluate itself, which will provide very likely full points. A notable example would be Listing 1.8. Listing 1.8. Redirection submission. int countChar(char c, String s) { return Solution.countChar(c, s); } This is categorized as a redirection cheat. Students can gain insights sys- tematically into the evaluation logic by submitting non-compilable submissions intentionally. 3.4 Point Injection Cheats All previous cheat-patterns focused the compile, or the execution step, and try to formulate a smart submission that tricks the evaluation component and its checks. Instead of this, injection cheats target intentionally the grading compo- nent. Injection cheats require in-depth knowledge about what specific APAAS solution (e.g., VPL) is used, and knowledge about the internals and details how the APAAS solutions generate intermediate outcomes to calculate a final grade. We explain this “attack” by the example of VPL. However, the attack can be easily adapted to other APAAS tools. VPL relies on an evaluation script that triggers the evaluation logic. The evaluation logic has to write results directly on the console (standard-out). The grading component parses and searches for lines that start with specific prefixes like – Grade := to give points – Comment := for hints and remarks that should be presented to the sub- mitter as feedback. VPL assumes that students are not aware of this knowledge. It is furthermore (somehow inherently) assumed that student submissions do not write to the console (just the evaluation logic should do that) – but it is possible for submis- sions to place arbitrary output on the console and is not prohibited by the Jails server. So, these assumptions are a fragile defence. A quick internet search with the search terms grade VPL will turn up the documentation of VPL explain- ing how the grading component is working under the hood. So, submissions like Listing 1.9 are possible and executable. Listing 1.9. Injection submission. int countChar(char c, String s) { System.out.print(Grade := 100); System.exit (0); return 0; // for compiler silence }
- 32. 12 N. Kratzke The intent of such a submission is merely to inject a line like Grade := 100’ into the output stream to let the grading component evaluate the sub- mission with full points. 4 Mitigation Analysis of Cheat Patterns So, in our problem systematization phase, we identified four patterns of tricking or cheating that can be observed in regular programming classes. These tricks work because students know that a dumb automat checks their submissions. In the following Sects. 4.1, 4.2, 4.3, 4.4 and we will ask the question what can be done to make APAAS more “intelligent” to prevent this kind of cheating? 4.1 What Can Be Done to Prevent Overfitting? Randomized test data make overfitted submissions ineffective. Therefore, our general recommendation is to give a substantial fraction of points for randomized test cases. However, to provide some control over randomized tests, these tests must be pattern based to trigger expectable problems (e.g., off-by-one errors, boundary cases) in student submissions. We refer to [17] for further details. E.g. for string-based data, we gained promising results to generate random strings merely by applying regular expressions [22] inversely. Section 5.1 explains how randomization is used by JEdUnit to tackle the overfitting problem. 4.2 What Can Be Done to Avoid Problem Evasion? Problem evasion cannot be detected by comparing the equality of input- output results (black-box testing). To mitigate problem evasion, we need auto- mated code inspection approaches (white-box inspections). Submissions must be scanned for unintended usage of language concepts like for and while loops. However, this makes it necessary to apply parsers and makes the assignment specific evaluation logic much more complicated and time-intensive to program. To simplify this work, we propose a selector-based model that selects nodes from an abstract syntax tree (AST) of a compilation unit to detect and annotate such kind of violations in a practical way. The approach works similarly like CSS selectors selecting parts of a DOM-tree in a web context (see Sect. 5.2). 4.3 What Can Be Done to Prevent Redirection? Interestingly problem evasion and redirection can be solved by the same mit- igation approach. Similar to evasion cheats, submissions can be scanned for unintended usage of language concepts, e.g. calls to classes containing the refer- ence logic that is used for testing. This white box inspection approach makes it
- 33. How Programming Students Trick and What JEdUnit Can Do Against It 13 possible to scan the submission with questionable calls like Solution.x() calls. Additionally, we deny to make use of getClass() calls and the import of the reflection package. Both would enable to formulate arbitrary indirections. The techniques necessary to deny specific method calls and to deny the import of packages will be explained in Sect. 5.2. 4.4 What Can Be Done to Avoid Injection Attacks? In a perfect world, the submission should be executed in a context that by design cannot access the grading logic. The student logic should be code that deserializes input parameters from stdin, passes them to the submitted function, and seri- alizes the output to stdout. The grading logic should serialize parameters, pipe them into the wrapped student code, deserialize the stdout, and compare it with the reference function’s output. However, this approach would deny making use of common unit testing frameworks for evaluation although it would effectively separate the submission logic and the evaluation logic in two different processes (which would make most of the attack vectors in this setting ineffective). How- ever, to the best of the author’s knowledge, no unit testing framework exists that separates the test logic from the to be tested logic in different processes. In the case of VPL, the shared use of the stdout (System.out) stream is given. APAAS systems that separate the submission logic stdout stream from the evaluation logic stdout stream might be not or less prone to injection attacks. However, even for VPL, there are several options to handle this problem. E.g., we can prohibit making use of the System.exit() call to assure that submissions could never stop the evaluation execution on their own. This prohibition can be realized using a Java SecurityManager – it is likely to be more complicated for other languages not providing a virtual machine built-in security concept. For these systems, parser-based solutions (see Sect. 3.2) would be a viable option (see Sect. 5.2). A very effective way to separate the stdout/stderr streams is to redirect these console streams to new (for the submission logic unaware) streams. This redirec- tion is an astonishingly simple solution for the most severe identified problem. It will be explained in Sect. 5.3. Table 4. Mapping of presented JEdUnit features to cheat patterns. Randomization Code Inspection Separation Overfitting (63%) ← Evasion (30%) ← Redirection (6%) ← Injection (1%) ←
- 34. 14 N. Kratzke 5 JEdUnit The Sects. 4.1, 4.2, 4.3, and 4.4 showed that it is possible to mitigate identi- fied cheat patterns using the strategies listed in Table 4. These insights flowed into a unit testing framework called JEdUnit. JEdUnit has a specific focus on educational aspects and strives to simplify automatic evaluation of (small) Java programming assignments using Moodle and VPL. The framework has been mainly developed for our purposes in programming classes at the Lübeck Uni- versity of Applied Sciences. However, this framework might be helpful for other programming instructors. Therefore, it is provided as open source. Every JEdUnit evaluation is expressed in a Checks.java compilation unit and usually relies on a reference implementation (which is by convention pro- vided in a file called Solution.java) and a submission (which is by convention provided in a file called Main.java). However, the conventions can be adapted to assignment specific testing requirements. Listing 1.10. Reference Solution expressed in JEdUnit. public class Solution { public int countChars(char c, String s) { int n = 0; for (char x : s.toLowerCase (). toCharArray ()) { if (Character.toLowerCase(c) == x) n++; } return n; } } Similar to JUnit, each test case is annotated with a @Test annotation (see Listing 1.11). However, a JEdUnit @Test annotation takes two additional param- eters: – weight is a value between 0 and 1.0 that indicates how much this test case contributes to the final grade of the assignment. – description is a string briefly explaining the intent of the test case. Listing 1.11. Test template expressed in JEdUnit. public class Checks extends Constraints { @Test(weight =0.25 , description=Example calls) public void test_01_exampleCalls () { [...] } @Test(weight =0.25 , description=Boundary tests) public void test_02_boundary_cases () { [...] } @Test(weight =0.5, description=Randomized tests) public void test_03_randomized_cases () { [...] } }
- 35. How Programming Students Trick and What JEdUnit Can Do Against It 15 A test case usually runs several test data tuples against the submitted solu- tion and compares the result with the reference solution. A test data tuple can be created using the t() method. Listing 1.12 shows this for the example calls of our continuous example. Listing 1.12. Example test case expressed in JEdUnit. @Test(weight =0.25 , description=Example calls) public void test_01_exampleCalls () { test( t(’a’, ABC), t(’A’, abc), t(’X’, abc), t(’x’, XxY) ). each( // check d - assertEquals( Solution.countChars(d._1 , d._2), Main.countChars(d._1 , d._2) ), // explain d - f(countChar (%s, %s) should return %s, d._1 , d._2 , Solution.countChars(d._1 , d._2) ), // on error d - f(but returned %s, Main.countChars(d._1 , d._2) ) ); } The each() method takes three parameters to run the evaluation on the test data provided as tuples in the test() method. – A Predicate that checks whether the submitted solution returns the same as the reference solution (indicated as correctness check in Listing 1.12). – A Function that explains the method call and reports the expected result (indicated as the expected behavior in Listing 1.12). – A Function that reports the actual result if the check predicate evaluates to false (indicated as in Listing 1.12). The functions are used to provide meaningful feedback for students. To make this straightforward JEdUnit provides the format f() method. f() is merely a convenience wrapper for the String.format() method providing additional formatting of feedback outputs that often confuses students. E.g., f() indicates non printable characters like spaces, tabulators, and carriage returns by visible representation characters like , , or . Additionally f() realizes visible map, and list representations (and more).
- 36. 16 N. Kratzke Table 5. Random generators provided by JEdUnit. Method Description c() Random character ([a-zA-Z]) c(String regexp) Random character from a regular expression (first char) s(String... regexps) Random string from a sequence of regular expressions s(int min, int max) Random string between a minimum and maximum length b() Random boolean value i() Random integer value i(int m) Random integer value [0, m] i(int l, int u) Random integer value [l, u] d() Random double value d(double m) Random double value [0, m[ d(double l, double u) Random double value [l, u[ T ListT l(int l, SupplierT g) Random list with length l generated by g T ListT l(int l, int u, SupplierT g) Random list with length in the range of [l, u] generated by g The reader is referred to the Wiki1 for a more detailed introduction to JEdUnit and further features like: – Initializing assignments. – Configuration of checkstyle (coding conventions). – Making use of predefined code inspections. – Checking complex object-oriented class structures automatically. However, in the following Sects. 5.1, 5.2, and 5.3 we focus mainly on how JEdUnit makes use of randomization, code inspection, and stream separation to mitigate observed overfitting, problem evasion, redirection, and injection cheat- ing problems (see Table 4). 5.1 How Does JEdUnit Support Randomization? JEdUnit provides a set of random generators to mitigate overfitting problems. These random generators enable to generate randomized test data in specified ranges and according to patterns to test explicitly common problem cases like boundary cases, off-by-one errors, empty data structures, and more. Because 1 https://github.com/nkratzke/JEdUnit/wiki.
- 37. How Programming Students Trick and What JEdUnit Can Do Against It 17 these generators are frequently used, they intentionally have short names (see Table 5). Using this set of random generators, we can quickly create test data – for instance a random list of predefined terms. Listing 1.13. Demonstration of a randomized list generator creating a list of strings from a regular expression. List String words = l(2, 5, () - s(This|is|just|a|silly|example) ); The randomly generated lists have a length between two and five entries. Possible resulting lists would be: – [This, a] – [silly, is, This] – [example, example, a, a] – [This, is, just, a, example] The generators shown in Table 5 are designed to work seamlessly with the test().each() pattern introduced in Listing 1.12. Listing 1.14 exemplifies a randomized test case for our continuous example. It works with random strings but generates test data to check intentionally for cases where the to be counted character is placed in front, in the middle or at the end of a string to cover frequent programming errors like off-by-one errors. Listing 1.14. Example for a randomized test case expressed in JEdUnit. @Test(weight =0.5, description=Randomized tests) public void test_03_randomized_cases () { // Regexp to generate random strings String r = [a-zA -Z]{5 ,17}; // Pick a random character to search char c = c(); test( t(c, s(c + {1,7}, r, r)), // First position t(C, s(r, r, c + {1,7})), // Last position t(c, s(r, c + {1,7}, r)), // Middle position ). each(check , explain , onError ); // check , explain , onError defined as in Listing 1.12 } Such kinds of test cases are not prone to overfitting because the test data is randomly generated for every evaluation. A possible generated feedback for the test case shown in Listing 1.14 would look like so:
- 38. 18 N. Kratzke – [OK] countChars(‘j’, jUcEzCzODWWN) should return 1 – [FAILED] countChars(‘j’, zOdAqavJJkxjvrjj) should return 5 but returned 3 – [OK] countChars(‘j’, SPAqlORwxjjjjRHIKCCWS) should return 4 Each evaluation will be run with random data but according to comparable patterns. So, JEdUnit provides both: comparable evaluations and random test cases that are not prone to overfitting. 5.2 How Does JEdUnit Support Code Inspection? JEdUnit integrates the JavaParser [18] library to parse Java source code into an abstract syntax tree. JEdUnit tests have full access to the JavaParser library and can do arbitrary checks with this parser. However, this can become quickly very complex. Therefore, JEdUnit tests can make use of a selector-based model that selects nodes from an abstract syntax tree (AST) of a compilation unit to detect and annotate such kind of violations in a practical way. The approach works similarly like CSS selectors selecting parts of a DOM-tree in a web context. The following examples demonstrate how to use this selector model on sub- mission ASTs pragmatically for code inspections. The reader should inspect the two evasion examples shown in Listing 1.14 and 1.6 that show typical evasion patterns to avoid recursive or lambda/stream-based programming in Java. To detect lambda evasion (see Listing 1.6), we can add the following inspec- tion to a test case. It scans for lambda functions that make use of block state- ments in their definition. To use blocks in lambda functions might indicate a kind of “problem evasion” in a submission – a student may try to evade from lambda programming into more simple statement based programming (which is likely not the intent of the assignment). Listing 1.15. Example for a lambda evasion check in JEdUnit. @Inspection(description=Lambda evasion inspection) public void assignmentInspections () { penalize (25, No blocks in lambdas., () - inspect(Main.java, ast - ast.select(LAMBDA) .select(BLOCK) .annotate(No blocks in lambdas.) .exists () ) ); } So, this inspection would effectively detect submissions like the one already presented in Listing 1.6. To detect recursion evasion (see Listing 1.4) we simply have to search for methods outside the main() method that make use of loop statements.
- 39. How Programming Students Trick and What JEdUnit Can Do Against It 19 Table 6. Predefined code inspections provided by JEdUnit. Check option Default Description CHECK IMPORTS True Checks if only cleared libraries are imported. java.util.* is allowed per default CHECK COLLECTION INTERFACES True Checks if collections are accessed via their interfaces only (e.g. List instead of LinkedList is used in method parameters, method return types, variable and datafield declarators). To use concrete collection classes like ArrayList instead of their interface List is penalized by default, the same applies for Map. This check can be deactivated ALLOW LOOPS True for, while, do while, forEach() loops are allowed per default. This can be deactivated and penalized. E.g., in case if methods must be implemented recursively ALLOW METHODS True Methods are allowed per default. Can be deactivated and penalized in special cases, like to enforce the usage of lambda functions ALLOW LAMBDAS True Lambdas are allowed per default. Can be deactivated and penalized in special cases, like to enforce the usage of methods ALLOW INNER CLASSES False Inner classes are penalized by default. This check can be deactivated if inner classes shall be allowed ALLOW DATAFIELDS False Checks if datafields (global variables) are used. This is penalized by default. However, this check must be deactivated for object-oriented contexts ALLOW CONSOLE OUTPUT False By default System.out.print() statements are not allowed and penalized outside the scope of the main() method. This check can be deactivated
- 40. 20 N. Kratzke Listing 1.16. Example to detect recursion evasion. inspect(cu , ast - ast.select(METHOD , [name != main]) .select(FOR , FOREACH , WHILE , DOWHILE) .annotate(no loops allowed) .exists () ); The same technique can be used to detect submissions that make use of System.out.println() or System.exit() calls that might indicate injection attacks. Listing 1.17. Example to detect suspect method calls (used JEdUnit internally). inspect(cu , ast - ast.select(METHODCALL) .filter(c - c.toString (). startsWith(System.exit))) .annotate(c - [CHEAT] Forbidden call: + c) .exists () ); The reader may notice that these selector-based code inspections are quite powerful and flexible to formalize and detect arbitrary violation patterns in student code submissions. JEdUnit makes intensively use of this code inspection technique and provides several predefined code inspections that can be activated by config options. Table 6 lists some of these predefined inspections. 5.3 How Does JEdUnit Supports Separation of Evaluation and Submission Logic? The main problem of injection attacks is that the submission logic and the eval- uation logic make use of the same console streams (stdout/stderr). The problem is that the grading component interprets this console output and this output could be compromised by student submission’s outputs (intentionally or unin- tentionally). JEdUnit solves this problem simply be redirecting the stdout/stderr console streams to other streams. Simple methods like this do this redirection. Listing 1.18. Redirection of stdout console stream. /** * Creates a file called console.log that stores all * console output generated by the submitted logic. * Used to isolate the evaluation output from the * submitted logic output to prevent injection attacks. * @since 0.2.1 */ public void redirectStdOut () throws Exception { this.redirected = new PrintStream( Config. STD_OUT_REDIRECTION ); System.setOut(this.redirected ); }
- 41. How Programming Students Trick and What JEdUnit Can Do Against It 21 The stdout/stderr streams are switched whenever a submission logic is called and switched again when the submission logic returns. Because the submission logic has no access to the evaluation library logic of JEdUnit, it can not figure out the current state of the redirection and is therefore not capable of reversing this redirection. In consequence, JEdUnit generates two stdout/stderr streams. One for the evaluation logic that is passed to the VPL grading component, and one for the submission logic that could be presented as feedback to the user for debugging purposes. However, currently, JEdUnit is simply ignoring the submission output streams. Figure 4 shows how JEdUnit changes the control flow (right-hand side) in comparison to the standard VPL approach (left-hand side). This stream redirection effectively separates the submission logic streams from the evaluation logic streams, and no stream injections can occur. Fig. 4. Isolation of console streams in JEdUnit to prevent injection. Additionally, JEdUnit prevents that student submissions can stop their exe- cution via System.exit() or similar calls to bypass the control flow of the evaluation logic. This is done by prohibiting the reflection API2 and calls to System.exit() by code inspection means already explained in the previous Sect. 5.2. 6 Discussion of Results and Threats of Validity If we compare the number of detected cheats in the systematization phase with the validation phase (see Table 3 in Sect. 3), we see the impact of applying solutions like JEdUnit. In the systematization phase cheat detection has been 2 The reflection API would enable to formulate arbitrary calling indirections that could be not identified by code inspections.
- 42. 22 N. Kratzke done manually and without notice to the students. In the validation phase cheat detection has been done automatically by JEdUnit and been reported immedi- ately as JEdUnit evaluation feedback to the students. The consequence was that cheating occurred only in the first weeks. What is more, the cheating detected in the validation phase has been only observed in intermediate outcomes and not in the final submissions (while the cheating reported in the systematization phase is cheating that made it to the final submissions). So, we can conclude that JEdUnit effectively detects the four identified pat- terns of cheating (overfitting, evasion, redirection, and injection). Cheating was not even applied in intermediate results and came entirely to an end in the second half of the semester. It was no effective mean anymore from a students perspective. However, we should consider and discuss the following threats of internal and external validity [4] that apply to our study, and that might limit its conclusions. Selection Bias and Maturation. We should consider that the target audience of the systematization and validation phase differed a bit. In the systematiza- tion phase, we worked with first-semester novice programmers of a computer science and information technology and design study programme. In the vali- dation phase, we worked with second-semester computer science students that made some programming progress during their first semester. This increase in expertise might be one reason why a general decrease in cheating was observable in the second semester. However, that explains not the fact that cheating comes entirely to an end in the second half of the semester. This decrease might be only explained by the fact that students learned that the grading automat “was not so dumb” anymore. Contextual Factors. This threat occurs due to specific conditions under which research is conducted that might limit its generalisability. In this case, we were bound to a Moodle-based APAAS solution. The study would not have been possible outside this technical scope. We decided to work with VPL because it is the only mature-enough open source solution for Moodle. Therefore, the study should not be taken to conclude on all existing APAAS systems. Especially JEdUnit is a Moodle/VPL specific outcome. However, it seems to be worth to check existing APAAS solutions whether they are aware of the four identified cheat-patterns (or attack vectors from a system security perspective) and what can be transferred from JEdUnit into further APAAS solutions. Hawthorne Effects. This threat occurs due to participants’ reactions to being studied or tested. It alters their behaviour and therefore, the study results. We can observe the Hawthorne effect quite obvious in Table 3. The reader should compare the systematization phase (unaware of observation) and the validation phase (noticing a more clever grading automat capable of detecting cheats). In the validation phase cheating was drastically reduced, only occurred in inter- mediate outcomes, and even comes entirely to an end in the second half of the
- 43. How Programming Students Trick and What JEdUnit Can Do Against It 23 semester. JEdUnit made the grading component more“clever” and changed the behaviour of students not to cheat. Because of this internal feedback loop, the study should not be taken to draw any conclusions on the quantitative aspects of cheating. Furthermore, the reader should additionally take all the threat to validity discussion of the systematiza- tion phase [11] into account in order to avoid drawing wrong conclusions. 7 Conclusion Students trick – at least 15% of them. Maybe because grading components of automated programming assessment systems can be tricked very easily. Even first-year students are clever enough to do this. We identified recurring pat- terns like overfitting, evasion, redirection, and even injection tricks. Most APAAS solutions provide sandbox mechanisms and code plagiarism detection to protect the host against hostile code or to detect “copy-paste” cheating. How- ever, these measures do not prevent submissions like System.out.println(Grade := 100); System.exit(0); which would give full points (in a Moodle/VPL setting) regardless of the assign- ment complexity. These two lines (or the injection idea behind) are like an “atomic bomb” for a grading component that is unaware to many program- ming instructors. Current APAAS solutions can little do against it. This study aimed to systematize such kind of recurring programming student cheat patterns and to search for mitigation options. To handle these kinds of cheats we do not only need sandboxing nor code plagiarism detection (that almost all APAAS solutions provide) but additionally means 1. to randomise test cases (contributes to 63% of all cheats, see Table 4), 2. to provide pragmatic code inspection techniques (contributes to 36% of all cheats, see Table 4), 3. and to isolate the submission and the evaluation logic consequently in separate processes (contributes to 1% of all cheats, see Table 4). Therefore, this paper presented and evaluated the solution proposal JEdUnit that handles these problems. According to our validation in three courses with over 260 students and more than 3.600 submissions of programming assignments, JEdUnit can make grading components much more “intelligent”. We showed how to make overfitting inefficient, how to detect evasion and redirection, and how to deny injection cheat patterns. When students learn that the grading component is no dumb automat anymore, cheating decreases immediately. In our case, cheating comes even entirely to an end. However, as far as the author oversees the APAAS landscape, exactly these mentioned features are only incompletely provided by current APAAS solutions. Most APAAS solutions focus on sandboxing and code plagiarism detection but
- 44. 24 N. Kratzke oversee the cheating cleverness of students. JEdUnit is not perfect as well, but it focuses on the cheating cleverness of students and is meant as fortifying add-on to existing APAAS solutions like Moodle/VPL. However, its biggest shortcoming might be its current technological dependence on Moodle/VPL and Java. Further research should focus on aspects of how the JEdUnit lessons learned can be transferred into further APAAS solutions. To do this, we have to ask how and which of the JEdUnit principles can be made programming language and APAAS solution agnostic? Therefore, JEdUnits working state can be inspected on GitHub3 to foster the adaption of the ideas and concepts behind JEdUnit. References 1. Ala-Mutka, K.M.: A survey of automated assessment approaches for programming assignments. Comput. Sci. Educ. 15(2), 83–102 (2005). https://doi.org/10.1080/ 08993400500150747 2. Alraimi, K.M., Zo, H., Ciganek, A.P.: Understanding the MOOCs continuance: the role of openness and reputation. Comput. Educ. 80, 28–38 (2015). https:// doi.org/10.1016/j.compedu.2014.08.006, http://www.sciencedirect.com/science/ article/pii/S0360131514001791 3. Caiza, J.C., Alamo Ramiro, J.M.d.: Automatic grading: review of tools and imple- mentations. In: Proceedings of 7th International Technology, Education and Devel- opment Conference (INTED2013) (2013) 4. Campbell, D.T., Stanley, J.C.: Experimental and Quasi-experimental Designs for Research. Houghton Mifflin Company, Boston (2003). Reprint 5. Douce, C., Livingstone, D., Orwell, J.: Automatic test-based assessment of pro- gramming: a review. J. Educ. Resour. Comput. 5(3) (2005). https://doi.org/10. 1145/1163405.1163409, http://doi.acm.org/10.1145/1163405.1163409 6. Gupta, S., Gupta, B.B.: Cross-site scripting (xss) attacks and defense mechanisms: classification and state-of-the-art. Int. J. Syst. Assurance Eng. Manage. 8(1), 512– 530 (2017). https://doi.org/10.1007/s13198-015-0376-0 7. Halfond, W.G.J., Orso, A.: Amnesia: analysis and monitoring for neutralizing SQL-injection attacks. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, ASE 2005, pp. 174–183. ACM, New York (2005). https://doi.org/10.1145/1101908.1101935, http://doi.acm.org/ 10.1145/1101908.1101935 8. Hunter, J.D.: Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9(3), 90–95 (2007). https://doi.org/10.1109/MCSE.2007.55 9. Ihantola, P., Ahoniemi, T., Karavirta, V., Seppälä, O.: Review of recent systems for automatic assessment of programming assignments. In: Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling 2010, pp. 86–93. ACM, New York (2010). https://doi.org/10.1145/1930464. 1930480, http://doi.acm.org/10.1145/1930464.1930480 10. Kluyver, T., et al.: Jupyter notebooks - a publishing format for reproducible com- putational workflows. In: Loizides, F., Schmidt, B. (eds.) Positioning and Power in Academic Publishing: Players, Agents and Agendas, pp. 87–90. IOS Press (2016) 11. McLaren, B.M., Reilly, R., Zvacek, S., Uhomoibhi, J. (eds.): CSEDU 2018. CCIS, vol. 1022. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21151-6 3 https://github.com/nkratzke/JEdUnit.
- 45. How Programming Students Trick and What JEdUnit Can Do Against It 25 12. Oliphant, T.: A Guide to NumPy. Trelgol Publishing (2006) 13. del Pino, J.C.R., Rubio-Royo, E., Hernández-Figueroa, Z.J.: A virtual program- ming lab for moodle with automatic assessment and anti-plagiarism features. In: Proceedings of the 2012 International Conference on e-Learning, e-Business, Enter- prise Information Systems, and e-Government (2012) 14. Pomerol, J.C., Epelboin, Y., Thoury, C.: What is a MOOC?, Chapter 1, pp. 1– 17. Wiley-Blackwell (2015). https://doi.org/10.1002/9781119081364.ch1, https:// onlinelibrary.wiley.com/doi/abs/10.1002/9781119081364.ch1 15. Ray, D., Ligatti, J.: Defining code-injection attacks. SIGPLAN Not. 47(1), 179–190 (2012). https://doi.org/10.1145/2103621.2103678, http://doi.acm.org/10. 1145/2103621.2103678 16. Rodrı́guez, J., Rubio-Royo, E., Hernández, Z.: Fighting plagiarism: metrics and methods to measure and find similarities among source code of computer pro- grams in VPL. In: EDULEARN11 Proceedings, 3rd International Conference on Education and New Learning Technologies, IATED, pp. 4339–4346, 4–6 July 2011 17. Romli, R., Mahzan, N., Mahmod, M., Omar, M.: Test data generation approaches for structural testing and automatic programming assessment: a systematic liter- ature review. Adv. Sci. Let. 23(5), 3984–3989 (2017). https://doi.org/10.1166/asl. 2017.8294 18. Smith, N., van Bruggen, D., Tomassetti, F.: JavaParser: Visited. Leanpub (2018) 19. Staubitz, T., Klement, H., Renz, J., Teusner, R., Meinel, C.: Towards practical programming exercises and automated assessment in massive open online courses. In: 2015 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), pp. 23–30, December 2015. https://doi.org/10.1109/TALE. 2015.7386010 20. Su, Z., Wassermann, G.: The essence of command injection attacks in web appli- cations. SIGPLAN Not. 41(1), 372–382 (2006). https://doi.org/10.1145/1111320. 1111070, http://doi.acm.org/10.1145/1111320.1111070 21. Thiébaut, D.: Automatic evaluation of computer programs using moodle’s virtual programming lab (vpl) plug-in. J. Comput. Sci. Coll. 30(6), 145–151 (Jun 2015), http://dl.acm.org/citation.cfm?id=2753024.2753053 22. Thompson, K.: Programming techniques: regular expression search algorithm. Commun. ACM 11(6), 419–422 (1968)
- 46. Exploring the Affordances of SimReal for Learning Mathematics in Teacher Education: A Socio-Cultural Perspective Said Hadjerrouit(B) University of Agder, Kristiansand, Norway [email protected] Abstract. SimReal is an innovative technology that emerged the last few years in mathematics education. It provides new potentialities for mathematical learning by means of dynamic and interactive visualizations of mathematical concepts. This paper uses SimReal in the context of teacher education to explore the affordances and constraints of the tool for the learning of mathematics. It presents a framework that combines Vygotsky’s socio-cultural theory, the role of mediation, and the notion of affordance at a technological, pedagogical, and socio-cultural level. The aim of the article is to explore the extent to which SimReal as a mediating artifact affords students’ mathematical learning in teacher education. Keywords: Affordance · Agency · Mathematical learning · SimReal · Teacher education · Visualization 1 Introduction SimReal is a visualization tool that is used to teach a wide range of topics ranging from school to undergraduate mathematics. While the suitability of the tool has been evaluated in university mathematics courses, it is not the case in teacher education, without an appropriate theoretical framework. Several theoretical approaches can be used to address the use of digital tools in mathematics education such as instrumental or documentational approach, theory of didactical situations, or anthropological theory of didactics [1]. However, these approaches are not ready-made to assess the affordances of SimReal to support the learning of mathematics and explore the extent to which the tool is useful in teacher education [2]. The notion of affordance combined with a socio-cultural perspective provides a powerful framework to address the impacts of SimReal by considering the tool as a mediating artefact with affordances and constraints at the technological, pedagogical, and socio-cultural level. This suits well with the way mathematics is taught in the socio-cultural context of teacher education. This paper is an extended version of the article that was published in the proceedings of CSEDU 2019 [3]. This paper has a different title and structure. The sections that are extended are Introduction, Literature Review, Theoretical Framework, Discussion, and Conclusion and Future Work. The abstract has changed, and research questions © Springer Nature Switzerland AG 2020 H. C. Lane et al. (Eds.): CSEDU 2019, CCIS 1220, pp. 26–50, 2020. https://doi.org/10.1007/978-3-030-58459-7_2
- 47. Exploring the Affordances of SimReal for Learning Mathematics 27 are included. The result section contains more findings and information on affordances and programming issues. Blocks of text are also partly rewritten. This new version is structured as follows. Firstly, the visualization tool SimReal is presented. Secondly, the theoretical framework is described. This is followed by the context of the study, the research questions and methods. The results are then presented and discussed. Finally, a summary of the results, and recommendations for future work conclude the article. 2 Literature Review 2.1 The Notion of Visualization Mathematics educators have used visualizations for many years [4, 5]. Textbooks and Web sites are filled with pictures, illustrations, animations, diagrams, and graphs. Like- wise, graphing calculators and videos have become integral part of mathematics class- room. According to Arcavi [6], visualization is defined as the ability to use and reflect upon pictures, graphs, animations, images, and diagrams on paper or with digital tools with the goal of communicating information, thinking about and advancing understand- ings. Visualizations include both the process and product of creation, and reflection upon pictures, images in the mind internally, and on paper or with digital tools exter- nally [6]. Another use of the notion is visual representation, but sometimes no differ- ence is made between mathematical visualizations (pictures, images, and diagrams) and mathematical representations (verbal, graphical, and symbolical). Finally, Presmeg [5] uses similar terms to define the notion of visualization. This includes processes of con- struction and transformation of both visual mental images and inscriptions (or external representations) of a spatial nature that may be implicated in doing mathematics. 2.2 The Visualization Tool SimReal SimReal is a visualization tool for a range number of mathematical subjects ranging from school to higher education. It uses a combination of graphic calculator, video lessons, video live streaming, and interactive simulations [7]. An example of SimReal utilization in mathematics education is given in Fig. 1. SimReal has more than 5000 applications, exercises, and tasks [8]. The tool can be divided into small subsets, while keeping the same basic structure and user interface. Hogstad, Ghislain, and Vos [9] investigated a subset of SimReal called Sim2Bil, which provides 4 windows for visualizations: simulation, graph, formula, and menu window. Some studies on SimReal focus on mathematics at the undergraduate university level [8, 10, 11], and aimed to report on students’ views of the tool as a supplement to ordinary teaching and its usefulness in difficult and abstract mathematical areas. Students’ views were mostly positive, except some technical difficulties, e.g., navigating through the tool. In engineering education, Hogstad, Ghislain, and Vos [9] studied a subset of SimReal called Sim2bil that aims at exploring how students use visualizations in their mathematical activities. Furthermore, Hadjerrouit and Gautestad [13] used the theory of instrumental orchestration and usability criteria to analyze teachers’ use of SimReal in an engineering class. Another study addressed the use of SimReal in an upper
- 48. 28 S. Hadjerrouit Fig. 1. An example of SimReal utilization in mathematics education [3]. secondary school [12]. It reports on positive students’ attitudes towards the use of the tool inclassroom. Somestudents didnot foundvisualizations useful, andtheintegrationof the tool into the curriculum was not easy. Other studies were carried out in teacher education in a technology-based course. Firstly, Hadjerrouit [14] evaluated the suitability of the tool using usability criteria. Secondly, Hadjerrouit [15] addressed students’ perceptions of SimReal impacts on learning using a survey and open-ended questions based on three categories of affordances. Finally, Hadjerrouit [3] is a continuation of these two studies and aimed at using SimReal to investigate the affordances of the tool in a more in- depth way by means of specific mathematical tasks - Pythagoras’ and Square theorem - and programming languages to create digital visualizations. 3 Theoretical Framework 3.1 Socio-Cultural Theory and Mediating Artifacts An important element of Vygotsky’s socio-cultural theory [16] is the role of tools in edu- cational settings, and the way they mediate teaching and learning activities. A crucial issue is thus the role of mediating artifacts and how teachers and students take advan- tage of artifacts, such as mathematical symbols, textbooks, blackboard, and in particular digital artifacts, such as GeoGebra, Excel, and SimReal. These artifacts have both affor- dances and constraints. Based on the writings of Vygotsky, the role of mediating artifacts in mathematics education can be represented as follows (Fig. 2). In this paper, the subject is the student, the object is the mathematical concept to be learned, e.g., Pythagoras’ theorem, and the mediating artifact is SimReal. Among several theoretical approaches that can be applied to explore the impacts of SimReal, such as the instrumental approach [17], the notion of affordance combined with Vygotsky’s socio- cultural stance provides the most appropriate framework to address the affordances and constraints of SimReal in a teacher education context.
- 49. Exploring the Affordances of SimReal for Learning Mathematics 29 SimReal as mediating artifact with affordances and constraints Subject (Student) Object (mathemaƟcal learning) Fig. 2. SimReal as mediating artifact with its affordances and constraints. 3.2 The Notion of Affordance The notion of affordance was originally proposed by Gibson [18]. It refers to the relation- ship between an object’s physical properties and the characteristics of a user that enables particular interactions between user and object. More specifically, Gibson used the term “affordance” to describe the action possibilities offered to an animal by the environment with reference to the animal’s action capabilities [18, 19]. A typical example is a tall tree that has the affordance of food for a giraffe because it has a long neck. Gibsons’ notion of affordance implies a complementarity between the animal and the environment. The notion of affordance was introduced to the computer world by Norman [20]. It refers to the perceived and actual properties of the thing, primarily those fundamental propertiesthatdeterminejusthowthethingcouldpossiblybeused.AccordingtoNorman [20], an affordance is the design aspect of an object, which suggests how the object should be used and a visual clue to its function and use. Examples of affordances are user interface elements, which directly suggest suitable actions: Clickable geometrical figures, draggable sliders, pressable buttons, selectable menus for figures or algebraic calculations, etc. Several research studies used Norman’s ideas to implement the notion of affordance in various educational settings. For example, Turner and Turner [21] specified a three- layer articulation of affordances: Perceived affordances, ergonomic affordances, and cultural affordances. Likewise, Kirchner, Strijbos, Kreijns, and Beers [22] described a three-layer definition of affordance: Technological affordances that cover usability issues, educational affordances to facilitate teaching and learning, and, social affordances to foster social interactions. In mathematics education, Chiappini [23] applied the notions ofperceived,ergonomic,andculturalaffordancestoAlnuset,adigitaltoolforhighschool algebra. Finally, Hadjerrouit [3, 15] presented a model with three levels of affordances of digital tools in teacher education. The same model is used in this paper. De Landa [24] emphasized that affordances are not intrinsic properties of the object. Rather affordances become actualized in a specific context, e.g., the socio-cultural con- text of the classroom. In other words, affordances emerge from the relationship between the object, e.g., a digital tool, and the particular environment with which it is interacting. From this perspective, the specific context of the mathematics classroom in teacher edu- cation may include several artifacts with their affordances and constraints, which interact with the teacher and students. The artifacts used in classroom may include paper-pencil techniques, the blackboard, Interactive White Board (IWB), PowerPoint slides, and
- 50. 30 S. Hadjerrouit diverse digital tools, such as Smartphones, IPad, GeoGebra, SimReal, and mathematical symbols, notations, and representations, etc. Hence, artifacts with their affordances and constraints mediate between the subject (student) and the object (mathematics). This view of digital tools is in line with Vygotsky’s socio-cultural stance. 3.3 SimReal Affordances Based on the research literature described above, the specificities of mathematics edu- cation, and the socio-cultural stance as an overreaching framework, this study proposes three categories of affordances at six different levels (Fig. 3): Fig. 3. Three categories of SimReal affordances at six different levels [3]. a) Technological affordances that describe the tool’s functionalities b) Pedagogical affordances at four levels: • Pedagogical affordances at the student level or mathematical task level • Pedagogical affordances at the classroom or student-teacher interaction level • Pedagogical affordances at the mathematics subject level • Pedagogical affordances at the assessment level c) Socio-cultural affordances that cover curricular, cultural, and ethical issues The affordances that are highlighted in this section are only a subset of potential affordances that could emerge in specific classroom contexts. These affordances are
- 51. Exploring the Affordances of SimReal for Learning Mathematics 31 some of the most important ones from the researcher’s perspective, but as De Landa [24] pointed out affordances are not intrinsic properties of the object, of the visualization tool SimReal in this study. Rather affordances of SimReal become actualized in specific contexts, e.g., the socio-cultural context of the classroom in teacher education. Regarding the affordance model presented in Fig. 3, two types of technological affor- dances can be distinguished: Ergonomic and functional affordances. From the ergonomic point of view, these are ease-of-use, ease-of-navigation, and accessibility of SimReal at any time and place, accuracy and quick completion of mathematical activities. From the functional point of view, SimReal provides support for performing calculations, drawing graphs and functions, solving equations, constructing diagrams, measuring figures and shapes, and presenting mathematical concepts visually and dynamically. Technological affordances are a pre-requisite for any digital tool and provide the ground for pedagogical affordances in educational settings. There are several pedagogical affordances at the student level, e.g., using SimReal to freely build and transform mathematical expressions that support conceptual under- standing, such as collecting real data, creating a mathematical model, using a slider to vary a parameter or drag a corner of a triangle, moving between symbolic, numerical, and graphical representations, simulating mathematical concepts, or exploring regularity and change [25]. At this level, the motivational factor is important to engage students in mathematical problem solving. Furthermore, feedback of the tool in various forms to students’ actions may foster mathematical thinking, e.g., whether an answer given by the student is correct or not. Programming mathematical tasks also provide affordances that may foster mathematical learning and understanding. Likewise, several pedagogical affordances can be provided at the classroom level [25]. Firstly, affordances that result in changes of interpersonal dimensions, such as change of teachers’ and students’ role, less teacher-directed and more student-oriented instruction. Secondly, affordances that foster more learner autonomy, resulting in stu- dents taking greater control over their own learning, and using SimReal as a “new” authority in assessing their own learning. Other affordances at this level are change of social dynamics and more focus on collaborative learning and group work. These affordances have an impact on the didactical contract and the role of the teacher as an authority in classroom [26]. Variation in teaching and differentiation of tasks are other affordances of digital tools at this level [27]. Other affordances at this level may result in flipping the classroom, which is an alternative way of using SimReal out-of-class. Furthermore, three types of pedagogical affordances can be provided at the math- ematics subject level [25]. The first one is fostering mathematical fidelity according to “ideal” mathematics, the congruence between machine mathematics and “ideal” and paper-pencil mathematics and promoting faithfulness of machine mathematics [28]. The second affordance is amplifying and reorganizing the mathematical subject. The former is accepting teachers’ and curricular goals to achieve those goals better. Reorganizing the mathematical subject means changing the goals by replacing some issues, adding and reordering others. For example, in advanced topics such as calculus there might be less focus on skills and more on mathematical concepts [25]. In geometry, there might be and
- 52. 32 S. Hadjerrouit emphasis on more abstract geometry, and away from facts, more argumentation and con- jecturing [25]. Likewise, it may be useful to support tasks that encourage metacognition, e.g., starting with real-world tasks, and using SimReal to generate results. Affordances at the assessment level consist of summative and formative assessment. Summative assessment is for testing, scoring and grading, and it can be provided in the form of statistics that the tool can generate. Formative assessment is crucial for the learning process. Feedback is an essential component of formative assessment, and it can take many forms, e.g., immediate feedback to students’ actions, a combination of conceptual, procedural, and corrective information to the students, or asking question types, etc. Finally, several socio-cultural affordances can emerge at this level. Firstly, SimReal should provide opportunities to concretize the mathematics subject curriculum in teacher education. Secondly, SimReal should be tied to teaching mathematics in schools, and support the learning of mathematics at the primary, secondary, and upper secondary school level. In other words, teachers using SimReal should consider the requirement of adapted education. Finally, other socio-cultural affordances are associated with ethical, gender, and multi-cultural issues, which are too complex to address in this paper. 4 The Study 4.1 Context of the Study and Participants The study was conducted at the University of Agder in the context of a technology-based course for teacher educators. The course covered various digital tools and their didactical use in classroom, such as Excel, GeoGebra, Aplusix, Numbas, Smart Board, Campus Inkrement, diverse pedagogical software, and SimReal. Fifteen teacher students (N = 15) were enrolled in the course in 2016. The stu- dents had very different knowledge background in mathematics ranging from primary to upper secondary and university level. The students were categorized in four groups according to their knowledge level in mathematics associated with their study pro- grammes: Primary teacher education level 1–7, primary teacher education level 5–10, advanced teacher education level 8–13, and mathematics master’s programme. The par- ticipants had varied experience in using digital tools and resources inside or outside classroom settings. In terms of mathematics, the basic requirement was the comple- tion of a bachelor-degree in teacher or mathematics education, which presupposes at least the acquisition of mathematics at the primary and middle school level. In terms of digital tools, the recommended prerequisites were basic knowledge in information and communication technologies and some experiences with standard digital tools like text processing, spreadsheets, calculators and Internet. None of the students had any prior experience with SimReal. 4.2 Research Questions Based on the results achieved in the paper presented in Hadjerrouit [3], the purpose of this work is to extend the investigation of affordances and constraints of SimReal and their impact on students’ mathematical learning in teacher education. It addresses two main research questions:
- 53. Exploring the Affordances of SimReal for Learning Mathematics 33 a) What are the students’ perceptions of SimReal’s affordances and constraints? b) What are the impacts of SimReal on mathematical learning in terms of affordances and constraints in teacher education? The first research question is addressed in the result section, and the second in the discussion. 4.3 Teaching Activities A digital learning environment centered around SimReal was created over three weeks, starting from 25 August to 8 September 2016. The SimReal environment included video lectures, visualizations, and simulations of basic, elementary, and advanced mathemat- ics, and diverse online teaching material. Basic mathematics focused on games, dices, towerofHanoi,andprison.Elementarymathematicsconsistedofmultiplication,algebra, Pythagoras’ and Square theorem, and reflection. The topics of advanced mathematics were measurement, trigonometry, conic section, parameter, differentiation, Fourier, and programming issues. To assess students’ experiences on specific mathematical topics that are of consid- erable interest for the participants, two specific tasks were chosen. The first one was Pythagoras’ theorem, and the many ways of representing it. The theorem has also been given numerous proofs, including both geometric and algebraic proofs, e.g., proofs by dissection and rearrangement, Euclid’s proof, and algebraic proofs. Thus, Pythagoras’ theorem is more than just a way of calculating the lengths of a triangle. An example of representing the theorem is given in the following figure (Fig. 4). Fig. 4. An example of representation of Pythagoras’ theorem [3]. The second task was the Square theorem. Like Pythagoras’ theorem, there are many ways of using and representing the theorem (Fig. 5). 4.4 Methods This work is a single case study in teacher education. It aims at exploring the affor- dances of SimReal for mathematical learning in teacher education. Both quantitative and qualitative methods were used to collect and analyze the affordances and constraints of SimReal and the students’ experiences with the tool. The following methods were used:
- 54. 34 S. Hadjerrouit Fig. 5. An example of representation of the Square theorem [3]. a) A survey questionnaire with a five-point Likert scale from 1 to 5, and quantitative analysis of the results b) Students’ comments in their own words to each of the statements of the survey c) Students’ written answers to open-ended questions d) Qualitative analysis of students’ comments on point b and answers to open-ended questions to point c e) Task-based questions on Pythagoras’ and Square theorem, and programming issues The design of the survey was guided by the theoretical framework and the research questions. To measure students’ perceptions of SimReal, a survey questionnaire with a five-point Likert scale from 1 to 5 was used, where 1 was coded as the highest and 5 as the lowest (1 = “Strongly Agree”; 2 = “Agree”; 3 = “Neither Agree nor Disagree”; 4 = “Disagree”; 5 = “Strongly Disagree”). The average score (MEAN) was calculated, and the responses to open-ended questions were analyzed qualitatively. The survey included 72 statements that were distributed as follows: Technological affordances (12), pedagogical affordances at the student level (11), classroom level (19), mathematical subject level (9), assessment level (10), and finally socio-cultural level (11). The students were asked to respond to the survey using the five-point Likert scale and to comment each of the statements in their own words. The students were also asked to provide written answers in their own words to open-ended questions. The responses to students’ comments to the survey and open-ended questions were analyzed qualitatively. Of particular importance are task-based questions on Pythagoras’ and Square theo- rem to collect data on affordances of these two mathematical tasks. Additional questions on programming issues were given to the students to assess the affordances of program- ming languages for the learning of mathematics. Asking task-based questions provides information on the affordances of SimReal. This method also provides more nuanced and in-depth information about the students’ experiences with SimReal. The analysis of the data was guided by the specified affordances of the theoretical framework presented in Fig. 3, and open coding to bring to the fore information about affordances that was not covered by the theoretical framework.
- 55. Discovering Diverse Content Through Random Scribd Documents
- 57. F M E C H A N I C K S . 2 An Explication of the Second P l a t e. IGURE 1. Is the deceitful Balance; which yet is in Æquilibrio because the Weights 23 and 24 are reciprocally proportional to their Distances from the Center of Motion. Now this Cheat is easily discover'd by changing the Position of the Weights, and putting each of them into the other Scale, which will then be very unequal, or nearly as 11 to 12. Fig. 2. Is that sort of Balance which is called a Stiliard, and of frequent Use among us. It is only a Common Balance, with Weights at Distances from the Center of Motion reciprocally Proportionable to themselves: Only here the Length of Part of the Beam is compensated by a large Ball or Weight B, fixed to the shorter Beam; and one Weight as w removed along equal Divisions is made use of to weigh several others, as 6 w. c. Fig. 3. Is design'd to shew how any Force is diminish'd by its Obliquity; and that a Weight hung obliquely at 3, 2, 1, in the Circumference of a Circle or Wheel, is of no more Efficacy, as to the turning of the Wheel round, than if it were hung perpendicularly at the corresponding Points 3, 2, 1, in the Semidiameter of the same Circle. Fig. 4. Is the Demonstration of the former Case, by shewing that in those Circumstances the Force P B is resolved into two B F and B G, of which B F pulls directly from the Center, and is of no Use to the turning the Wheel round: And so all the remaining Force is represented by the perpendicular Force B G, which is wholly spent in turning it round. So that as B P is to B G, so is the whole oblique
- 58. Force, to the real or direct Force: Or so, in the similar Triangle B E C, is B C the whole oblique Radius, to C E the Perpendicular: Or so in the foregoing Figure is O 1, O 2, O 3, the common Hypotenuse or entire Radius, to O 1, O 2, O 3, the Bases or shorter Radij, where the String cuts the entire Radius perpendicularly. Fig. 5. Is the first Sort of Lever, where C the Prop is between the Resistance to be overcome, or Weight to be moved 5 w, and w 1 the Power or Weight to move the other by: And is so like the Case of the Balance or Stiliard, that it needs no particular Explication. A Crow of Iron is of this Sort. Fig. 6. Is the second Sort of Lever, where the Resistance to be overcome, or Weight to be moved w 3, is between the Prop C and the Point A, to which by the means of the Pulley P, the Power or Weight to move the other by, is applied. Bakers Knives for cutting Bread are commonly of this Sort. Fig. 7. Is the third Sort of Lever, where the Resistance to be overcome, or Weight to be moved, w 2 is at one End, the Prop at the other, and the Power or Weight w 3 between them. A Ladder lifted up by the Middle, in order to be rear'd, where one End is fixed, is of this Sort. Only the Force being in this Case nearer the Prop than the Resistance to be overcome, or Weight to be moved, this Sort of Lever diminishes Force instead of increasing it, and is therefore of little Use. Fig. 8. Is a common Lever of the first Sort, with its Prop and equal Divisions, fit to be used as the Stiliard. Fig. 9. Is a compound Lever of the first Sort, as long as the single one just above it, where a Weight at G, by being doubled three several Times, will raise eight Times its own Weight at A, as well as the other does it at once. This last is therefore of the same Force as the former, and no more; and by being compounded, is less considerable than the other.
- 59. N. B. Had the Proportion in the Compound Lever, Fig. 9. been otherwise, as suppose the Part B C on one Side of the Prop B three Times the Length of A B on the other Side, and the same in the other two Levers C E and E G; then the Weight G being but the 27th Part of the Weight at A, will be in Æquilibrio with it. Fig. 10. Is a bended Lever of the first Sort, where C the Prop is at an Angle, and the Force is increas'd with C H, the Distance of the Weight w 1, which by the means of the Pulley P, is applied to the longer Part of the Lever; and in this Lever, the Power is to the Resistance reciprocally as their Distances. An Hammer drawing out a Nail is such a bended Lever. Fig. 11, 12. Shew that Levers or Balances that are even when horizontal, may be uneven in other Positions; that is, too light when the Center of Gravity of one Weight is fix'd to the Lever or Balance above, and it is elevated; or below, and depress'd: Because the Perpendicular cuts the horizontal Line too near the Center in these Cases.
- 61. F M E C H A N I C K S . 3 An Explication of the Third P l a t e. IGURE 1. Is a Sort of Compound Lever of the second Kind, where the Weight H 6 is unequally born by the Weights F 4 and G 2, which are reciprocally proportional to the Distances C B and C A; and are accordingly in Æquilibrio. Whence we see how two Men may bear unequal Parts of the same Weight, in Proportion to their Nearness thereto. Fig. 2. Is another Engine of the same Nature with the former; where the Lines D C, A E, B F, and the Lever A B, are parallel to the Horizon; but the Lines on which the Weights hang D w 7, E w 5, F w 2, are perpendicular thereto; and here a Force or Weight pulling at the Point C sustains the unequal Weights w 5 and w 2 in Æquilibrio: Provided the Distances C B and C A be reciprocally proportional to those Weights. Whence we learn, how Horses of unequal Strength may be duly fitted to preserve equally in their Labour; viz. by taking care that the Beam by which they both draw a Weight or Waggon, may be divided at the Point of Traction as C, in reciprocal Proportion to such their Strength. Fig. 3. A B is an upper Pulley, of no direct Advantage, but for Readiness of the Motion, as increasing not the Power at all; equal Weights being ever required to raise others. Fig. 4. Is an upper and an under Pulley connected together; where the upper being of no Efficacy, the lower does however double the Force, as is ever the Case in such Pulleys.
- 62. Fig. 5. Is a Compound Pulley of three upper and three under Pulleys, all communicating together; where therefore the whole Weight is divided among 6 Strings; and so 1 Pound balances 6 Pound. The last String B M 1, as passing beyond the last upper Pulley, not being here to be reckon'd of any Consequence. Fig. 6. and 7. These are Boxes of the same Number of upper and under Pulleys with the former; only in other Positions, and depend on the same Principle entirely.
- 64. F M E C H A N I C K S . 4 An Explication of the Fourth P l a t e. IGURE 1. Is a System of Pulleys connected together, whereby the Force is increased by Addition in Proportion to the Number of Cords; so that one Pound, w 1, sustains five Pounds, w 5, as must happen from the Equality of the stretching of the whole Cord, and the consequent Division of the whole Weight into five equal Parts, as equally supported by them all. Fig. 2. Is a System of Pulleys not connected together, whereby the Force is increas'd, for every lower Pulley; according to the Numbers, 2, 4, 8, in a double Proportion; because every lower Pulley doubles the Force of the former; as is evident at the first Sight; since the Velocity of Ascent or Descent of the greater Weight is every Time but half so great as before. Fig. 3. Is the Axis in Peritrochio; or Wheel, with its Axel; where any Weight or Force applied round E F, or C D, or A B, has just so much greater Power to move the Wheel, or entire Machine about the Axis, as the Velocity or Distance from the Geometrical Axis it self is greater. Nor is there any farther Difficulty in this plain Engine. Fig. 4. This is only a Train of Wheel-work; which by Composition of Wheels vastly increases the Force. Thus suppose the Diameter of the Barrel E F, be ten times the Diameter of the Pinion G: And the Diameter, or Number of equal Teeth in G, be one tenth of the Diameter, or Number of equal Teeth in H I: And the Diameter and Velocity of the Teeth in H I, be ten times the Diameter and Velocity of the Pinion K; and the Diameter or Number of equal Teeth in K, be one tenth of the Diameter, or Number of equal Teeth in L M; And
- 65. that the Barrel N O, be of the same Diameter with the Wheel L M. Then a Weight on the Barrel E F will balance a Weight one hundred times as heavy upon the Barrel N O; which is done by its moving an hundred Times as swift as the other. For the Velocity in the first Barrel E F, to that of its Pinion G, is as ten to one; and that in the Wheel H I, to that in its Pinion K, is also as ten to one. While the Velocities at each Wheel, and its corresponding Pinion in the other Wheel, as well as at the Wheel L M, and its Barrel N O, are equal. Fig. 5. Is a compound Engine, to prove that in a Wedge, as E M G, depress'd by a Weight w, or by its own Weight, or by a Stroke, the Force is diminished in Proportion to the Sine of its Aperture, compar'd with the Line of its Depth: So that when the former Sine is double or triple, c. the Force is diminished one half, or one third, c. This is here prov'd by the Wedges separating two Cylinders, which are drawn together by other Weights, in the Scales R and S beneath, when its Sides are screw'd nearer or farther off, to adjust their Distance to those Weights perpetually. Fig. 6. Is a Wedge by it self, where the Force is increas'd in the Proportion of the Sines of the Angles of Aperture, D F and D E, to the Radius D B; or is resolv'd into two Forces, the one perpendicular, and the other parallel to the Plain of the Tree or Timber it is to reeve: And this because the Velocity downward is ever to the Velocity side-ways in the Proportion of D B to D F and D E, or to 2 D F. i. e. by the Similitude of Triangles, as A B or C B to A C. Fig. 7. Is a Paper Wedge, H F G coil'd round a Cylinder, and so representing a Screw; and shews that its Force must be increas'd in Proportion to the Progress along its Cylinder, when it is compar'd with the Circumferences on the same Cylindrical Surface, or as H F to H G. Fig. 8. Is a compound Engine to explain and measure the Power of the Screw: from whence it appears, that the Force of Screws is
- 66. reciprocally proportional to the Distance of the Helix's or Threads which compose them.
- 68. F M E C H A N I C K S . 5 An Explication of the Fifth P l a t e. IGURE 1. Is a Compound Engine in which all the several Mechanical Powers are combin'd: as the Wheel and Axle G H: The Balance or Lever I K: the Screw F; which includes the Wedge: and the Pulley L M. The entire Force of this Engine is to be computed by compounding the separate Forces together. Fig. 2. Is a Windmill; whose Force is here represented, by its raising a Weight on a Barrel. The Wind is supposed to blow parallel to the Axis, from E towards D; its several Sails have their Plains nearly 45 Degrees oblique to the Plain through the middle of those Sails: Two of them inclining, and two reclining. By this Means the Wind falling at about 45 Degrees obliquity on the Plain of each Sail; the Breadth of each Sail is a Diagonal of a square, one of whose Sides is parallel to the Direction of the circular Motion, and has its full Force; and the other is perpendicular thereto, and so has no Effect as to that circular Motion at all. And as much as the Side of a Square is lesser than the Diagonal, so much of the whole Quantity of the Wind is lost on every single Sail. But then each Pair along the same Line, by the different Situation of those Sails, agreeing in the same Motion, the whole united Quantity is more than the single Quantity upon one equal Sail directly expos'd to the same Wind, as much as two Sides of a Square are greater than the Diagonal. But this without the Consideration of the weakning of the Force of the Wind by the Obliquity of Incidence; which alters the former Proportion: for this also diminishing the Force in the same Proportion with the former Diminution of the Quantity of the Wind, the whole Diminution will ever be as the Squares of that Quantity; or as the
- 69. Squares of the Sines of the Angles of Incidence: wherefore in this Case of Four oblique Sails of 45 Degrees will be equivalent to Two direct ones. Fig. 3. Is the elastical spiral Spring of a Watch, out of its Box, and unwinding it self more weakly, as it is less restrained. Fig. 4. Is the same Spring in its Barrel A B join'd by a Chain to its Fusee C D, or spiral Line about a Cone, which Cone has the Semidiameter or Distance from its Axis in the very same Proportion, greater as the Spring is weaker, and lesser as the Spring is stronger: that so the absolute Force on the Wheels of the Watch may be ever the same, for the exact Equality of their Motion in all Cases. Fig. 5. Is an Imitation of a Waggon or Coach, with its fore Wheels E F, either equal (as here,) or else lesser, or greater, than the hinder G H; to be drawn by a Weight w in the Scale, either upon an Horizontal, or upon an Inclined Plain A B, and to get over any Obstacle as C D: The Quadrant M, and Bullet N, are to shew the Quantity of the Elevation of that Plain, for the Tryal of Experiments relating to all such Sort of Vehicles. Fig. 6. Is a strong Machine, with a Wheel O P, and its Winch R, and String O P L K, its lesser Barrel K L, circular Table A B, Scale with a Weight w, suspended by a String that comes through the hollow Axis C D, and oblique Tube G C, in which Mercury or a Bullet is included; its Screw H; its Balls I and B, and their Strings; To shew that Motion once begun always continues, till some other Cause stops it: That absolute and respective Motion are entirely different: And to shew withal the Endeavour of Bodies that move circularly to recede from the Center of their Motion, on inclined, as well as horizontal Plains, and that in the same Circle in a duplicate Proportion to their Velocity.
- 71. F M E C H A N I C K S . 6 An Explication of the Sixth P l a t e. IGURE 1. Is an Instrument to shew the various Parabola's that are made by Projectils, and particularly the Truth of the several Rules in the Art of Gunnery. Wherein A B is a Tunnel full of Quicksilver, D K is a Glass Tube, let into a Groove or Frame of Wood for its Support, and at K is a fine Stem, accommodated to the Arch of a Quadrant L M, and turning upon its Center, to direct the projected Quicksilver to any Angle; while the Tube's perpendicular Altitude, or the Force that produces the Projection, is either the same, or altered by a different Inclination at Pleasure, according to the Nature of the several Experiments. Fig. 2. Is a Cycloid with its equal Sides A B, A C, and pendulous Body E, oscillating therein. And, Note, That by the Make of the Figure, the Line B C is equal to the Circumference of the Circle D G F, by which it was describ'd; that the Length of the Cycloid it self is four times that Circle's Diameter; that every Part of it from F the Vertex is still double to the Chord of the Correspondent circular Arch G F; that its included Area B D C F, is Three times the Area of the former Circle; that the Force upon the Pendulum at any Point E, is exactly proportional to the Distance along the Cycloid of the Point from the Vertex, as E F; and that therefore the Time of every Oscillation, in all Angles whatsoever, is always equal. Fig. 3. A C B is a Syphon with Quicksilver from A to C, and a Pendulum of half that Length; to shew here also that the Force is as the Line to be describ'd, and that by Consequence the Vibrations in the Syphon are all equal: as also to shew that they are equal to those of a Pendulum, of half the same Length: As is plain from the
- 72. former Case of the Cycloid, where the Length of the Pendulum is half that of the Cycloid in which the Body moves. Fig. 4. A B are two Spheres, to denote the several Laws of Motion in the Collision of Bodies, whether Elastical or not Elastical, to be tried in the Cycloid, or in a Circle, with proper Corrections: Which Experiments yet are most of them too difficult for such a Course as this is. Fig. 5. Is an Instrument to explain muscular Motion; supposing the Muscles to be some way like a String of Bladders; by shewing that a smaller Quantity of an elastical Fluid may equally raise equal Weights with a larger; and to shew exactly what Quantity is necessary for any particular Effect. For thus will the lesser Quantity of Air, (measured in both Cases by the Gage C A K, as condens'd by the Syringe H A) equally raise an equal Weight to the same Height by the lesser three Bladders, that the greater Quantity raises the same by the one larger Bladder. Fig. 6. Are several Pendulums of several Sorts of Matter, heavy and light; where the Centers of Suspension and Oscillation are equally distant, and the Times of those Oscillations are all equal. This also hints the other remarkable Phænomena of Pendulums; viz. that the Semicircular and Cycloidal Times of Oscillation are to each other as 34 to 29: That in both the Length of the Strings of Pendulums are in a duplicate Proportion to their Times of Oscillation; and that the Heights of Roofs, c. may be found from the Times of the Oscillations of Pendulous Bodies fixed to them, on the known Hypothesis that a Pendulum of 39.2 Inches vibrates in one Second of Time. Fig. 7. Is a Fountain running on Wheels, and made by Air condens'd on the Surface of Quicksilver, and so forcing the Quicksilver to ascend through the Pipe G: And is to shew that the Lines of Projectils, or other Bodies, are not alter'd by the common Motion of the whole Instrument or Floor on which they are plac'd;
- 73. and that all Motions on the Earth, if it move, will be the same as if it stand still. Fig. 8. Is a Parabola with the several Lines belonging to it, in order to demonstrate the Doctrine of Projectils; and particularly the Art of Gunnery. Fig. 9. Is an Engine moving on Wheels, that lets a Ball fall down from a Groove through a Hole, as it is in Motion; to shew that it will then fall on the same Point of the Frame that it falls upon when it is at rest; as does a Stone let fall from the Top of the Mast of a Ship under Sail: and that all respective Motions on the Earth must be the very same, while it self moves as if it were at rest. Fig. 10. Is a Cylindrical Iron A B, swinging on a Pin E F, in the very same time that a pendulous Body D of two thirds of its Length C D does; to shew that two thirds is the Center of Oscillation or Percussion in all such prismatick or cylindrical Bodies.
- 75. F 7 O P T I C K S . An Explication of the First P l a t e. IGURE 1. Represents the Foundation of Vision, and of all Opticks whatsoever, by exhibiting to the Eye a Specimen how the Rays of Light do as well originally, as after Reflection or Refraction, spread themselves in right Lines from each Point in every visible Object, as P, to each other Point, as R, R, R, R, R, every way, to be receiv'd by the Eye in any direct Position whatsoever. Fig. 2. Represents the known Law of Reflection; that the Angle of Incidence C P D, is equal to that of Reflection C P E, or that the Angle of Inclination D P A is equal to the other E P B. Fig. 3. Shews the Reason why a plain Looking-Glass, as A E F B, exhibits the Object C D by the Image c d, which is equal to C D, and equidistant from the Glass A c = A C: And in an erect Posture; all depending only on the Equality of the Triangles, whose Vertices are C c : D d, and have their common Bases below E and above F, which Glass by forming the same Image c d, so to the Eye, as if the real Object C D was at c d, must needs shew that Picture in the Place assign'd, without any Inequality of Distance or Magnitude, or any Inversion.
- 76. Fig. 4. Shews the Reason why the same or equal Object, as A B, C D, E F, appears larger when it is nearer, and smaller when farther off: viz. on account of the Inequality of the Angles A G B, or M G N, and C G D, or K G L, and E G F or H G I, and the consequent Inequality of the Pictures made by the Rays at the Bottom of the Eye. Fig. 5. Shews the Reason why a Convex Looking-Glass, as A E F B, exhibits Object C D by the Image c d, both nearer to the Glass, and lesser than it self; but still in an erect Posture. All depending only on the different Bend of the Circle between E and its lower Point, between F and its upper Point; which cannot make the Angles of Reflection or Inclination equal, as they must needs be in all such Reflections, without making the Vertices of the Angles, as c and d, nearer the Glass than C and D: And so the apparent Picture or Diameter c d lesser than that of the Object C D, though without any Inversion. Fig. 6. Shews the Reason why a Concave Glass, as A E F B, exhibits an Object plac'd nearer the Glass than the Center, as C D by the Image c d, remoter from the Glass, and larger than it self, viz. for Reasons just contrary to those under the fifth Figure foregoing. Fig. 7. Shews the Reason why a Concave Glass, as C D E F, exhibits an Object, if it be plac'd remoter than the Center, as A B, inverted, and at different Distances between the Eye and the Glass; according to the Length or Shortness of its own Distance, as B C or A D, viz. Because the Rays from the same Point still cross one another, as at G and H, before they fall upon the Eye; and so by forming an inverted Image make it impossible for the Eye to see the Object in any other Position than that the Image has; which Image indeed it self is the only proper Object of the Eye, in all such Cases whatsoever. Fig. 8. Is a Picture in Confusion; but rectified by a Convex Cylinder, and thereby brought into exact Order again.
- 77. Fig. 9. Represents an Image in a Cylindrical Concave Surface, when the Eye is in a Plain perpendicular to its Axis; so that lengthways it is as a Plain, and breadthways as a Concave Speculum: Which therefore makes the Picture longer, but not wider. The contrary will happen in a Convex Speculum, which will make it shorter but not narrower, for the like Reason.
- 79. F O P T I C K S . 8 An Explication of the Second P l a t e. IGURE 1. Shews that an Object, as K, seen through a plain Glass, whose Sides A B, C D, are parallel, by the Eye at G, appears out of its true Place; and this so much the more as the Glass is thicker: While at the same time the two Surfaces do exactly balance each other's Refraction, and make the two Rays H K, G F exactly parallel. Fig. 2. Exhibits a plain Method of measuring the Refraction of Fluids at all Angles, and of proving thereby that it is always in one fixed Proportion of the Sines, as the next Figure will explain it. For if the moveable Rule K C L, with its measuring Circle A B D E fix'd by the Prop E, to a heavy Pedestal F G, in a large Glass A H I D, be so far immers'd in the Fluid, that the Center C may be in the Surface of the Fluid, and one of its Legs C L be so far bent from a rectilinear Position, that the Refraction of the Fluid can just make it appear as if it were in a strait Line, the Angle B C K, or its equal M C E, is the Angle of Incidence: And L C E the Angle of Refraction: And L C M the Difference, or the refracted Angle. Fig. 3. Is for the Illustration of the former Proposition, and shews the Sines afore-mentioned; as A D or G N (for they are suppos'd equal, and the Line A C N one strait Line,) is the Sine of the Angle of Incidence, and F E the Sine of the Angle of Refraction, which Sines do in the same Fluid at all Angles bear one and the same Proportion to each other; till at last, if the Refraction be out of a thick Medium into a thin one, and makes the second Sine equal to the Radius, that Ray cannot emerge at all, but will be reflected back by the Surface into the same Medium whence it came, along the Line C R.
- 80. Fig. 4. Is a Bason of Water, or other Fluid; to shew the common Experiment of Refraction; where a Shilling, or other Object at A, (which is so plac'd that it cannot be seen by the Eye at O, the Side of the Bason C interposing) is readily seen there, as soon as the Water or other Fluid is put in to the same Bason, and appears to be remov'd to the Point B. Fig. 5. Is the Alteration of a round white Object D, as seen through a Triangular Glass Prism A B C, by the Eye at G, where the double Refraction of the Glass at E and F makes the Object appear at d; and that as an oblong colour'd Image; wherein the upper Part is made by the violet Rays, which are most refrangible; and the lower by the red Rays, which are least so; and the intermediate Parts by those that are refrangible in a mean Degree; after the Order of the Colours of the Rainbow. Fig. 6. Shews the Nature of a multiplying Glass A D, and its Plains A B, B C, C D, c. and the Reason why the different Refraction of every oblique Plain, as A B, C D, c. exhibits the same Object K as a different Object k, k, c. according to the Number of the oblique Plains: While the direct Plain B C shews it still in its own Place: And while the Convolution of the Glass on the Axis K L removes all the oblique Images, but does not remove the direct one, on Account of the Change of the Position of those oblique Plains, and of the unchanged Position of the direct Plain. Fig. 7. Shews the Effect of the Lens, or double Convex Glass, in gathering parallel Rays, as G L, H M, A B, I N, K O, c. towards a Point, as D; because, as in the Case of the Prism above, the Refraction to the perpendicular in the Entrance, and from it in the Exit of those Rays, do still, by the different Position of that Perpendicular, conspire to unite the same Rays. Fig. 8. Shews the contrary Effect of the double Concave Glass, in scattering the parallel Rays; and that exactly on the like Account; and so this needs no new Explication.
- 81. Fig. 9. Shews the Reason why a Lens, or double Convex, shews a near Object at Q, as more remote at q, because it refracts it so that the Rays from the same Point meet more backward than before: And why it shews the same Object larger also: Which must needs be, because every Point in the Object appearing so much more backward, and yet in the same apparent Angle, its Length and Breadth must every where be proportionably enlarg'd. Fig. 10. Shews how such a Lens inverts Objects, as A, B, b a, which it does on Account of the Intersection of the Rays from each Point, in or near the Lens it self: Which necessarily infers such an Alteration: just as the Images of all Objects are in the Eye in an inverted Position, on the like Account. Fig. 11. Shews how a Lens does so refract the Rays from every Point of an Object, that is in its Focus C, and B, and A, that the Rays from each of those Points do become parallel afterward; and also how parallel Rays of different Positions are gather'd in that Focus. Fig. 12. Is the Nature of direct Vision by the Eye, in some Conformity to the 10th Figure: only in this Case the Crystalline Humour is the Lens. Fig. 13. Is the Case of a Concavo-convex Glass, with its parallel Surfaces, as in Fig. 1.
- 83. F O P T I C K S . 9 An Explication of the Third P l a t e. IGURE 1. Is a Telescope, with two Convex Glasses, the one towards the Object and the Segments of a great Sphere, the other near the Eye, the Segments of a small Sphere g h i, and they are to be so placed that the distinct Base or Image may, by the Collection of the Rays, be in the common Focus of both the Glasses f e d. By these two Glasses the parallel Rays, or those nearly so, as proceeding from the same Point of the Object A B C, (which is to be suppos'd considerably remote) are made to meet in the intermediate Image f e d, at f, and e, and d; and again at the Bottom of the Eye, at r, and s, and t; but in an erect Position; and therefore so as to shew the Object inverted. Fig. 2. Is a Telescope with four Convex Glasses, the one towards the Object, and three nearer the Eye: Whose Images are made in the common Focus of two Glasses, as before. This is like the former; but only that two of the Eye Glasses serve merely to reinvert, or to erect the Image, that so it may be inverted at the Bottom of the Eye; and therefore may shew the Object in its true or erect Position. Fig. 3. Is a Telescope, with a Convex Object Glass, and a Concave Eye Glass; which last, by scattering the Rays, as if they came from a nearer Point, makes the Image inverted in the Bottom of the Eye, and therefore shews the Object in its true or erect Position. Only this takes in but a small Part of an Object, an so is less used than the two former Telescopes. Fig. 4. Is a Telescope with a Triangular Prism D B in its Axis; and that Prism's Gage F G for the Demonstration of the Refraction out of
- 84. Vacuum into Air, and out of thinner Air into thicker; and both by the Means of an Object seen through the Prism, as well when the Air is condensed, as when it is exhausted. Where in the first Case the Object is seen higher, and in the other lower than in its natural Situation; as the two following Figures demonstrate. Fig. 5. Shews how the Object or Circle which was low at first, is to Appearance rais'd as it passes through condens'd Air; by being refracted towards the perpendicular, in its Ingress into a Glass Prism, and from it in its Egress into the common Air again. Fig. 6. Shews how the same Object or Circle, which was high at first, is to Appearance depress'd, as it passes through the Vacuum; by being refracted from the Perpendicular, in its Ingress into the Prism, and towards it, in its Egress into the common Air again. Fig. 7. Is a Triangular Glass Prism, fitted to receive all sorts of Fluids, and when rightly apply'd to the Semi-circle of the next Figure, does exactly measure the refractive Power of all those Fluids. Where the vertical Angle G D H is 45 Degrees; and by consequence the half Angles C D H, C D G, C H G, are 22° 30′, and where all is to be so contriv'd, that the Rays within the Glass may be parallel to G H, and perpendicular to C D, and may fall on each side Plain of the Glass Prism in an Angle of 22° 30′ from their Perpendiculars; that so the Refractions at the Ingress and Egress may be equal, and the Computations easy. Fig. 8. Is the Semicircle, with the Glass Prism full of its Liquor rightly apply'd thereto; and both Arms of the Index E D, F D, equally elevated above the horizontal Line A C. This shews the Proportion of the Sine of the Angle of Incidence to that of Refraction, in this Incidence of 22° 30′; which Proportion of Sines being the same in all other Angles, we hence learn that Proportion accurately and universally.
- 86. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com