Test Driven Development of A Static Code Analyzer

Test-Driven
Development of a Static
Code Analyzer in Python
terry@odd-e.com

2
3 Steps
Make it work
Make it right
Make it fast
This formulation of this statement has been attributed to KentBeck;
it has existed as part of the UnixWay for a long time.

3
Story of Lizard
HFCCA
https://github.com/terryyin/lizard

Cyclomatic Complexity
The cyclomatic complexity of a section of source code
(functions in C) is the count of the number of linearly
independent paths through the source code. For
instance,
• if the source code contained no decision points such
as IF statements or FOR loops, the complexity would
be 1
• If the code had a single IF statement then it’s 2
• For every IF/FOR or OR/AND in the code +1
Most studies ﬁnd a strong positive correlation between cyclomatic complexity and
defects: modules that have the highest complexity tend to also contain the most
defects. We use 15 as the threshold for ‘complex function’, which is already a VERY
BIG number.

It also has extensions …
5
lizard -EwordCount

Pycodestyle in 8 years
lizard -Ewordcount ../pycodestyle/

7
A list of supported languages:

• C/C++ (works with C++14)

• Java

• C# (C Sharp)

• JavaScript

• Objective C

• Swift

• Python

• Ruby

• TTCN-3

• PHP

• Scala

• GDScript

It was driven by tests
8
Spec Code
Refactoring

How would you start to build a code
clone detector?
11

I just start with some simple
speculation:
Perhaps I will need to do some
hashing …
12

14
Refactoring
I did some small refactoring to get my test helper ready.

15
lizard_ext/lizardduplicate.py
test/test_extensions/testDuplicate.py

16
Refactoring
Yeah! We have our first failing test!!!
Spec

17

18
Refactoring
I made my test pass by just returning an empty list…
And I’m not kidding.
Spec Code

20
Add a new test

21
Refactoring
Nothing to refactor, so I add a new test…
Spec Code
Spec

22

23
Refactoring
As long as there’s code (token), it will return 1
Spec Code
Spec Code

25

26
Refactoring
Nothing to refactor in production code,
But there’s “duplication” in test code
Spec Code
Spec Code
Refactoring

27

28
Refactoring
New test for the repeating times.
Now you should know how to make it pass …
Spec Code
Spec Code
Refactoring
Spec

30
Refactoring
Fake it, when you can…
Implement it when you have to.
Spec Code
Spec Code
Refactoring
Spec Code

32
Refactoring
It now looks like …
Spec Code
Spec Code
Refactoring
Spec Code

34
Refactoring
We refactor to make the code “right” again
Spec Code
Spec Code
Refactoring
Spec Code
Refactoring

38
Refactoring
At some point, our code becomes like this
Spec Code
Spec Code
Refactoring
Spec Code
Refactoring
Refactoring
Spec Code
Spec Code
Refactoring
Spec Code

39
Code, listen you shall.
Let the design be the
way it wants to be!
Data Do Not Speak, Code Sometimes Does

40
Refactoring
Let the design emerge from working software
Spec Code
Spec Code
Refactoring
Spec Code
Refactoring
Refactoring
Spec Code
Spec Code
Refactoring
Spec Code
Refactoring

41
Basic Duplicate Detection
def test_empty_file(self):
def test_two_functions_that_are_exactly_the_same(self):
def test_two_functions_that_are_exactly_the_same_with_start_end_line_number(self):
def test_two_5_line_functions_that_are_exactly_the_same_detail(self):
def test_two_functions_that_are_not_the_same(self):
def test_2_duplicates(self):
def test_1_duplicate_3_times(self):
Detection Ignoring const, variable names similar operators
def test_duplicate_with_different_integer_value(self):
def test_duplicate_with_different_string_value(self):
def test_duplicate_with_different_variable_name(self):
def test_duplicate_with_different_operator(self):
def test_duplicate_with_value_dense_block__value_should_be_kept(self):
def test_duplicate_with_value_dense_block_in_brackets(self):
Merging big duplicate blocks
def test_duplicate_with_2_big_blocks(self):
def test_no_duplicate_with_1_big_blocks_of_the_same_number(self):
def test_many_identifiers_together(self):
def test_repeating_patterns(self):
def test_repeating_patterns_multiple_matches(self):
Duplicate threshold
def test_threshold(self):
def test_threshold_with_too_many_repeatings(self):
def test_threshold_exceeded(self):
Multiple Files
def test_2_files_basic_duplicate(self):
def test_multiple_files_more_duplicate(self):

Hash?
42
Yah, that was my initial speculation. Eventually,
my implementation didn’t use hash.
Which is ﬁne.

What about performance?
43
Not an excuse to
write smelly code.

44
Rules Of Optimization
•FirstRuleOfOptimization - Don't.
•SecondRuleOfOptimization - Don't... yet.
•ProfileBeforeOptimizing
Source: Michael Jackson

45
Without much optimisation
•The lizard duplicate detector can process the entire
ansible codebase (700K+ NLOC) in less than 2
minutes.

46
Duplication may be
the root of all evil in
software.
Clean Code, Robert C. Martin

47
Duplicate block:
--------------------------
../django/django/conf/locale/en_AU/formats.py:17 ~ 38
../django/django/conf/locale/en_GB/formats.py:17 ~ 38
../django/django/conf/locale/cy/formats.py:17 ~ 34
^^^^^^^^^^^^^^^^^^^^^^^^^
Duplicate block:
--------------------------
../django/django/conf/locale/en_GB/formats.py:8 ~ 39
../django/django/conf/locale/cy/formats.py:8 ~ 35
^^^^^^^^^^^^^^^^^^^^^^^^^^
Duplicate block:
--------------------------
../django/django/shortcuts.py:74 ~ 92
../django/django/shortcuts.py:98 ~ 113
^^^^^^^^^^^^^^^^^^^^^^^^^^
Total duplicate rate: 23.71%
Using 50 tokens as the minimum duplicated block size.
The duplicate rate is high than actual.

48
Minimum Token 50
Files Avg CCN Duplicate rate
django 1933 1.8 23.7%
Flask 69 1.5 6.41%
Tornado 118 1.8 5.25%
Youtube-dl 845 4.3 17.43%
Ansible 3237 4.0 117.30%
Puppet (Ruby) 2419 1.5 23.93%

49
— Kent Beck, inventor of XP

Test Driven Development of A Static Code Analyzer

Recommended

More Related Content

What's hot (20)

Similar to Test Driven Development of A Static Code Analyzer (20)

More from Terry Yin (6)

Recently uploaded (20)

Test Driven Development of A Static Code Analyzer