Netzob Documentation
Netzob Documentation
Release 0.4.1
Contents
1 2
Contact information Netzob Overview 2.1 Overview of Netzob . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tutorials 3.1 Getting started with Netzob . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Auto generation of Peach pit les/fuzzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Export Wireshark dissectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Installation Guides 4.1 Installation Guide . . . . . . . . . . . 4.2 Installation documentation . . . . . . . 4.3 Installation documentation on Debian . 4.4 Installation documentation on Gentoo . 4.5 Installation documentation on Windows User Guide 5.1 Importing Data . . . . . . . . . . . . . 5.2 Protocol inference . . . . . . . . . . . 5.3 Generating trafc and simulating actors 5.4 Exporting protocol model . . . . . . .
3 5 5 9 9 12 17 21 21 21 23 24 25 27 27 29 33 34 35 35 77 79
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
ii
Netzob is an open source tool for reverse engineering, trafc generation and fuzzing of communication protocols. It allows to infer the message format and the state machine of a protocol through passive and active processes. The model can afterward be used to simulate realistic and controllable trac. The main features of Netzob are: Vocabulary Inference Netzob includes a complete model to represents the message format of a protocol (aka its vocabulary). Using specic algorithms, it allows to learn it from provided traces. Grammar Inference The state machine of a protocol (aka its grammar) denes the valid sequences of exchanged messages. Netzob allows to learn it semi-automaticaly using specic algorithms. Protocol simulation To support the inferring process, a dynamic analysis is perfomed based on simulated actors. These can initiate and take part in a complex communication following the infered protocol.
Contents
Contents
CHAPTER 1
Contact information
Website http://www.netzob.org Email [email protected] Mailing list Two lists are available, use the SYMPA web interface to register. IRC You can hang-out with us on Freenodes IRC channel #netzob @ freenode.org. Wiki Discuss strategy on Netzobs wiki Twitter Follow Netzobs ofcial accounts (@Netzob)
CHAPTER 2
Netzob Overview
The following sections will describe in more details the available mechanisms.
etc.) and the format (PCAP, hex, raw binary ows, etc.). The gure below describes the multiple communication channels and therefore possible snifng points Netzob aims at addressing. Figure 2.2: Multiple communication ows arround an application The current version (version 0.4) of Netzob deals with the following data sources : Live network communications Captured network communications (PCAPs) Inter-Process Communications (IPCs) Text and binary les API ows through oSpy le format support Otherwise, if you plan to reverse a protocol implemented over an supported communication channel, Netzobs can manipulates any communications ow through an XML representation. Therefore, this situation only requires a specic development to capture the targeted ow and to save it using a compatible XML. Figure 2.3: Importing data from an unknown communication channel using the XML denition
Step 1: clustering Messages and Partitioning in Fields To discover the format of a symbol, Netzob supports different partitioning approaches. In this article we describe the most accurate one, that leverages sequence alignment processes. This technique permits to align invariants in a set of messages. The Needleman-Wunsh algorithm performs this task optimally. Needleman-Wunsh is particularly effective on protocols where dynamic elds have variable lengths (as shown on the following picture). When partitioning and clustering processes are done, we obtain a relevant rst approximation of the overall message formats. The next step consists in determining the characteristics of the elds. 6 Chapter 2. Netzob Overview
Figure 2.5: Sequence alignment with Needleman-Wunsh algorithm If the size of those elds is xed, as in TCP and IP headers, it is preferable to apply a basic partitioning, also provided by Netzob. Such partitioning works by aligning each message by the left, and then separating successive xed columns from successive dynamic columns. To regroup aligned messages by similarity, the Needleman-Wunsh algorithm is used in conjunction with a clustering algorithm. The applied algorithm is UPGMA. Step 2 : characterization of Fields The eld type identication partially derives from the partitioning inference step. For elds containing only invariants, the type merely corresponds to the invariant value. For other elds, the type is automatically materialized, in rst approximation, with a regular expression, as shown on next gure. This form allows to easily validate the data conformity with a specic type. Moreover, Netzob offers the possibility to visualize the denition domain of a eld. This helps to manually rene the type associated with a eld. Figure 2.6: Characterization of eld type Some intra-symbol dependencies are automatically identied. The size eld, present in many protocol formats, is an example of intra-symbol dependency. A search algorithm has been designed to look for potential size elds and their associated payloads. By extension, this technique permits to discover encapsulated protocol payloads. Environmental dependencies are also identied by looking for specic values retrieved during message capture. Such specic values consist of characteristics of the underlying hardware, operating system and network conguration. During the dependency analysis, these characteristics are searched in various encoding. Step 3: inferring the Transition Graph of the Protocol The third step of the learning process discovers and extracts the transition graph from a targeted protocol (also called the grammar). More formally, the grammar of a communication protocol denes the set of valid sentences which can be produced by a communication. A sentence is a sorted set of words which may be received or emmited by a protocol handler. An exemple of a simple sentence is :
["attack www.google.fr", "attack has failed", "attack www.kernel.org", "root access granted."]
which can be described using the following simple automata with S0 the initial state : Figure 2.7: Schema of a simple grammar The learning process step is achieved by a set of active experiments that stimulate a real client or server implementation using successive sequences of input symbols and analyze its responses. In Netzob, the automata used to represent or model a communication protocol is an extended version of a Mealy automata which includes semi-stochastic transitions, contextualized and parametrized inputs and outputs. The rst academic presention of this model is included in a dedicated scientic paper provided in the documentation section. The model is inferred through a dedicated active process which consists in stimulating an implementation and to analyze its responses. In this process, we use the previously infered vocabulary to discover and to learn the grammar of the communication protocol. Each stimulation is computed following an extension of the Angluin L algorithm*.
CHAPTER 3
Tutorials
As shown on the picture, you have to launch the capture at the Layer 4 on the localhost lo interface. As the targeted protocol works over UDP, youll be able to capture only the UDP payloads. Then launch the server of the targeted protocol and then the client. This one will send different commands to the server and wait for the response. Once you have captured one session, you have to select the messages you want to import (you should import everything) and click the Import messages button. A popup will ask you if you want to allow duplicate messages. Its better to not do so, to avoid unnecessary messages. We recommend to repeat this import process 4 times, in order to have enough variation between messages.
The main window shows each message in raw hexadecimal format. You can play with visualization attributes : right click on the symbol, then select Visualization and the attribute you want to change (hex, decimal or even string format, the unit size and potentially the sign and endianess). The following picture shows the rendering of the messages in hex format (on the left) and string format (on the right). You can then see that messages contain some interesting strings (api_identify, api_encrypt, api_decrypt, etc.).
You can use the lter functionality to display messages that contain a specic pattern. Here, we lter with the api_identify pattern. This lter permits to easily retrieve the messages associated with a potential identication command. You can see that a # character is present in each messages. You can try to split the messages by forcing their partitioning with a specic delimiter. To do so, use the Force partitioning functionality available in the symbol list (either with a right click on a symbol, or by selecting a symbol with its checkbox and then clicking on the Force partitioning button right above). Using the # string delimiter, youll have the following result: You may also want to play with Sequence alignment. This partitionment allows to align messages according to their common patterns. After playing with the different partitionment available, you are able to retrieve the different commands associated with the targeted protocol, as shown on the following picture. According to the name of the commands, you can see that a api_encrypt command is available. Lets have a look at its message format, which looks like:
10
Chapter 3. Tutorials
[command]#[dataToEncrypt][padding]
Netzob allows you to indicate that a specic eld has a mutable content, which means its data is not xed (such as the # delimiter) nor part of a set of xed elements (such as the command string).To specify the structure of a eld and its attributes, right click on a eld and select Edit Variable. A popup dialog displays a rooted tree that corresponds to the inferred structure of the eld. For example, you should have all the observed values of the eld (materialized through DataVariable leafs) under an AlternateVariable node variable. Regarding the targeted protocol, as we want to allow any data for the current eld, we rst have to delete the AlternateVariableNode and modify the root node to a DataVariable that has a mutable behavior, as shown on the following picture. You can visualize the associated message format on bottom-left corner. Its should display something like this: Now that we have rened the api_encrypt command message, we have to do the same for other commands that also take as parameter a user data: api_identify, api_authentify and api_decrypt, but also for some response messages such as resp_decrypt and resp_encrypt. At this time, you have a satisfactory approximation the vocabulary. You can now start to construct the state machine of the protocol.
11
3.2.2 Prerequisite
You need Netzob in version 0.4.1 or above.
12
Chapter 3. Tutorials
This tutorial assumes that the user have previously followed the Getting started with Netzob tutorial and have a complete Netzob project (or at least some format messages). The protocol implementation contains several vulnerabilities that should be detected during fuzzing. Moreover it assumes that the user has Peach 2.3.8 installed.
3.2.3 Export
To export the project go in File > Export the project > Peach pit file. The window below should appears : The window is composed of three panels. The left one lists all fuzzer available. They differ on the state representation. There are three kinds of fuzzer available: Randomized state order fuzzer: one state is created for each symbols of Netzob and at each step, the fuzzer changes of state for a randomly chosen one. Randomized transitions stateful fuzzer: one state is created for each symbols of Netzob and the transitions bewteen these states are based on those Netzob allows, weight by their probability. One-state fuzzer: one state is created corresponding to the chosen symbol. When the fuzzer is on a particular state, it sends fuzzed data that corresponds to the associated symbol to the target. Choose one of them. The right panel shows the fuzzer. It gives the user a small idea of what he is doing and what changes between two congurations. The bottom panel has two options: The rst options Fuzzing based on tells on which Netzob data model the fuzzing is based: Variable: use the Netzob variables to make Peach data models. It makes more fuzzy but less smart fuzzer. Regex: use the Netzob Regex (which are displayed on the top of the symbol visualization), it is the simplest solution. The second options Mutate static fields tells if the static elds in the Netzob data model are fuzzed or not. The Export button exports the fuzzer into a user dened le. Use this fuzzer into Peach Export this fuzzer directly through the Export button to a le named test.xml into the directory of Peach. It should create a PeachzobAddons.py le, which is essential for Peach to leverage Netzob capabilities as xup. The test.xml le should look like this. Look closely to the few XML comments.
<?xml version="1.0" encoding="utf-8"?> <Peach xmlns="http://phed.org/2008/Peach" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:s <Include ns="default" src="file:defaults.xml"/> <Import import="PeachzobAddons"/> <DataModel name="dataModel 1"> <Blob name="Field 0_0" valueType="hex" value="6170695f"/>
13
<Blob name="Field 1_0" valueType="hex"> <Fixup class="PeachzobAddons.Or"> <Param name="values" value="Blob,696e666f; Blob,7374617473"/> </Fixup>
</Blob> <Blob name="Field 2_0" valueType="hex" value="2300000000000000000000000000000000000000000000"/> <Blob name="Field 3_0" valueType="hex" value="00"/> </DataModel> <DataModel name="dataModel 2"> <Blob name="Field 0_0" valueType="hex" value="6170695f6279652300000000000000000000000000000000000
</DataModel> <DataModel name="dataModel 3"> <Blob name="Field 0_0" valueType="hex" value="6170695f6964656e74696679236672656400000000000000000 </DataModel> <DataModel name="dataModel 4"> <Blob name="Field 0_0" valueType="hex" value="6170695f61757468656e74696679236d7950617373776421000
</DataModel> <DataModel name="dataModel 5"> <Blob name="Field 0_0" valueType="hex" value="6170695f656e637279707423313233343536746573740000000 </DataModel> <DataModel name="dataModel 6"> <Blob name="Field 0_0" valueType="hex" value="6170695f6465637279707423737071767774362731360000000 </DataModel> <DataModel name="dataModel 7"> <Blob name="Default-1_0" valueType="hex" value="00000000"/> <Blob name="Default-2-1_0" valueType="hex" value="23"/> <Blob name="Default-2-2-1-1_0" valueType="hex"> <Fixup class="PeachzobAddons.Or">
<Param name="values" value="Blob,00000000000000; Blob,00000004000000; Blob,00000005000000; Bl </Fixup> </Blob> <Blob name="Default-2-2-1-2_0" valueType="hex"> <Fixup class="PeachzobAddons.Or"> <Param name="values" value="Blob,00000000000000000000; Blob,31323334353674657374; Blob,696e66 </Fixup> </Blob> <Blob name="Default-2-2-2_0" valueType="hex" value="00000000000000000000"/> </DataModel> <DataModel name="dataModel 9"> <Blob name="Field 0"> <Fixup class="PeachzobAddons.RandomField"> <Param name="minlen" value="0"/> <Param name="maxlen" value="1024"/> <Param name="type" value="Blob"/> </Fixup> </Blob> </DataModel>
14
Chapter 3. Tutorials
<StateModel initialState="state 0" name="stateModel"> <State name="state 0"> <Action ref="state 1" type="changeState" when="random.randint(1,8)==1"/> <Action ref="state 2" type="changeState" when="random.randint(1,7)==1"/> <Action ref="state 3" type="changeState" when="random.randint(1,6)==1"/> <Action <Action <Action <Action <Action </State> ref="state ref="state ref="state ref="state ref="state 4" 5" 6" 7" 9" type="changeState" when="random.randint(1,5)==1"/> type="changeState" when="random.randint(1,4)==1"/> type="changeState" when="random.randint(1,3)==1"/> type="changeState" when="random.randint(1,2)==1"/> type="changeState"/>
<State name="state 1"> <Action type="output"> <DataModel ref="dataModel 1"/> <Data name="data"/> </Action> </State> <State name="state 2"> <Action type="output"> <DataModel ref="dataModel 2"/> <Data name="data"/> </Action> </State> <State name="state 3"> <Action type="output"> <DataModel ref="dataModel 3"/> <Data name="data"/> </Action> </State> <State name="state 4"> <Action type="output"> <DataModel ref="dataModel 4"/> <Data name="data"/> </Action> </State> <State name="state 5"> <Action type="output"> <DataModel ref="dataModel 5"/> <Data name="data"/> </Action> </State> <State name="state 6"> <Action type="output"> <DataModel ref="dataModel 6"/> <Data name="data"/> </Action> </State> <State name="state 7"> <Action type="output"> <DataModel ref="dataModel 7"/>
15
<Data name="data"/> </Action> </State> <State name="state 9"> <Action type="output"> <DataModel ref="dataModel 9"/> <Data name="data"/> </Action> </State> </StateModel> <Agent name="DefaultAgent"> <!--Todo: Configure the Agents.--> </Agent> <Test name="DefaultTest"> <!--Todo: Enable Agent <Agent ref="TheAgent"/> --> <StateModel ref="stateModel"/> <Publisher class="udp.Udp"> <Param name="host" value="127.0.0.1"/> <Param name="port" value="4242"/> </Publisher> <Publisher class="udp.Udp">
<Param name="host" value="127.0.0.1"/> <Param name="port" value="10000"/> </Publisher> <!--The Netzob project has several simulator actors, so this file have several publishers. Choose </Test> <Run name="DefaultRun"> <!--Todo: Configure the run.--> <Logger class="logger.Filesystem"> <Param name="path" value="logs"/> </Logger> <Test ref="DefaultTest"/> </Run> </Peach>
This tutorial will not talk about Peach agents but conguring one of them could be useful. In the Test block, there is as many publishers as the Netzob simulator has actors. One publisher is needed, remove the others. If there is no publishers, create one according to the model above. On this example, the tester remove the second publisher.
Assuming that the user exports the test.xml le into the Peach directory, you can now start the fuzzer:
python peach.py test.xml
After few seconds, you should trigger a segfault or a stack smashing detection.
16
Chapter 3. Tutorials
-> Read: api_identify#fred Command: api_identify Arg: fred <- Send: Return value: 0 Size of data buffer: 13 Data buffer: DATA: 72 65 73 70 5f 69 64 65 6e 74 69 66 79 -> Read: api_identify#f Command: api_identify Arg: f
"resp_identify"
*** stack smashing detected ***: ./server terminated ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(__fortify_fail+0x45)[0xcec045] /lib/i386-linux-gnu/libc.so.6(+0x103ffa)[0xcebffa] ./server[0x8048a3c] ./server[0x8048eb4] ./server[0x8048985] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xc014d3] ./server[0x8048831] ======= Memory map: ======== 00289000-0028a000 r-xp 00000000 00:00 0 [vdso] 002fb000-00317000 r-xp 00000000 08:03 2605207 /lib/i386-linux-gnu/libgcc_s.so.1 00317000-00318000 r--p 0001b000 08:03 2605207 /lib/i386-linux-gnu/libgcc_s.so.1 00318000-00319000 rw-p 0001c000 08:03 2605207 /lib/i386-linux-gnu/libgcc_s.so.1 00bb4000-00bd4000 r-xp 00000000 08:03 673152 /lib/i386-linux-gnu/ld-2.15.so 00bd4000-00bd5000 r--p 0001f000 08:03 673152 /lib/i386-linux-gnu/ld-2.15.so 00bd5000-00bd6000 rw-p 00020000 08:03 673152 /lib/i386-linux-gnu/ld-2.15.so 00be8000-00d8b000 r-xp 00000000 08:03 672879 /lib/i386-linux-gnu/libc-2.15.so 00d8b000-00d8c000 ---p 001a3000 08:03 672879 /lib/i386-linux-gnu/libc-2.15.so 00d8c000-00d8e000 r--p 001a3000 08:03 672879 /lib/i386-linux-gnu/libc-2.15.so 00d8e000-00d8f000 rw-p 001a5000 08:03 672879 /lib/i386-linux-gnu/libc-2.15.so 00d8f000-00d92000 rw-p 00000000 00:00 0 08048000-0804a000 r-xp 00000000 08:03 6488874 /home/sygus/travaux/netzob/target_protocol/server 0804a000-0804b000 r--p 00001000 08:03 6488874 /home/sygus/travaux/netzob/target_protocol/server 0804b000-0804c000 rw-p 00002000 08:03 6488874 /home/sygus/travaux/netzob/target_protocol/server 09e0d000-09e2e000 rw-p 00000000 00:00 0 [heap] b778b000-b778c000 rw-p 00000000 00:00 0 b77a8000-b77ac000 rw-p 00000000 00:00 0 bf90f000-bf930000 rw-p 00000000 00:00 0 [stack] Abandon (core dumped)
17
Netzob provides a powerful datamodel in which elds are described with the following information: Regular expression (xed or dynamic size) Name (textual representation) Format Size Endianess Signing All this information is gathered to generate a script including a dissector used by Wireshark.
3.3.2 Language
Wireshark can be statically extended with C modules similar to core dissectors. Optionally, Wireshark can be congured to embed a Lua interpretor. For modularity purposes, the Lua engine has been choosed to extend Wireshark with Netzob generated dissectors.
3.3.3 Prerequisite
You need Netzob in version 0.4.1 or above. The wireshark exporter functionality is provided as a netzob core plugin (which is included in the 0.4.1 version). This tutorial assumes that the user have previously inferred the specication of the targeted protocol. An example of protocol inference is avaibale in the Getting started with Netzob tutorial.
3.3.4 Usage
1. Check that Wireshark supports Lua
2. Select a project Given a partitioned symbol in a project you can generate a wireshark dissector using the Export project menu item, then by selecting Wireshark.
You should get a popup with the LUA script automatically generated: 3. Import into wireshark Two methods are available: Evaluate the Lua script in a Wireshark instance. In wireshark, select Tools > Lua > Evaluate and paste the generated code. Start wireshark with a specic Lua script. Start wireshark with the following parameters: wireshark -X lua_script:PATH_OF_LUA_SCRIPT This will automatically import the Lua script on start.
18
Chapter 3. Tutorials
4. Dissect data packets Within the lower panel of Wireshark, you should get the dissected packets:
3.3.5 Limitations
Variable size elds cannot be easily exported to the datamodel used by Wireshark when we dont know the expected size. In this case, an error message will popup preventing Netzob from generating the dissector. If this happen, you have to complete the protocol model in order to nd the expected size of the dynamic eld.
3.3.6 Improvements
These ideas could be use to enhance dissection: Use relations (eld / size, repeat ...) Look at future biteld core implementation
19
20
Chapter 3. Tutorials
CHAPTER 4
Installation Guides
21
where XXX corresponds to the version you want to install (see http://www.netzob.org/download for available versions). Then, you can either install a package locally (developer mode) or on the system. Install Netzob locally (developer mode) Once the required dependencies are installed, you can install Netzob on its current directory:
$ python setup.py build $ python setup.py develop --user
Install Netzob on the system You can also install Netzob as a Python system package:
$ python setup.py build $ python setup.py develop $ python setup.py install
22
Activate debug information (DEBUG, INFO, WARNING, ERROR, CRITICAL) Manage Netzobs plugins: --plugin-list List the available plugins
4.2.6 Miscellaneous
Conguration requirements for Network and PCAP input:
$ sudo setcap cap_net_raw=ep /usr/bin/python2.XX
Stable (Squeeze)
deb http://apt.netzob.org/debian/ squeeze-backports main deb-src http://apt.netzob.org/debian/ squeeze-backports main
23
Import GPG key The repository is signed, so APT may complain until you register the archive key 0xE57AEA26 to your APT keyring. The ngerprint of the key is D865 DCF0 9B9A 195C 49F0 E3F3 F750 1A13 E57A EA26 and has been signed by the followings: 0xA255A6A3 : Georges Bossert <[email protected]> 0x561F7A47 : Frederic Guihery <[email protected]> 0x04B1A89C : Olivier Tetard <[email protected]> To import the key of the APT repository you can execute the following commands :
# wget https://dev.netzob.org/misc/debian_archive.asc -O -| gpg --import # gpg --export -a 0xF7501A13E57AEA26 | sudo apt-key add -
Once the requirements are fulllled you can download the debian le (i386 or amd64) and install it using the following command for an i386 architecture (32 bits) :
# dpkg -i netzob_0.3.0-1_i386.deb
24
# emerge app-portage/layman
Then, declare this repository in the portage conguration le /etc/make.conf by adding this line:
PORTDIR_OVERLAY="/home/USER/netzob-gentoo/"
Synchronize portage
# emerge --sync
stable systems:
# ACCEPT_KEYWORDS="~x86" emerge -av netzob
25
26
CHAPTER 5
User Guide
Netzob can handle multiple kinds of input data. Hence, you can analyze network trafc, IPC communications, les structures, etc. Import can either be done by using a dedicated captor or by providing already captured messages in a specic format. Current accepted formats are: PCAP les Structured les Netzob XML les (used by Netzob for its internal representation of messages) Current supported captors are:
27
Network captor, based on the XXX library Intra Process communication captor (API calls), based on API hooking Inter Process Communication captor (pipes, shared memory and local sockets), based on system call hooking Imported messages are manipulated by Netzob through specic Python objects which contains metadata that describes contextual parameters (timestamp or even IP source/destination for example). All the Python object that describe messages derived from an abstract object : AbstractMessage. The next part of this section details the composition of each message object.
5.1.1 AbstractMessage
All the messages inherits from this denition and therefore has the following parameters : a unique ID a data eld represented with an array of hex
5.1.2 NetworkMessage
A network message is dened with the following parameters : a timestamp the ip source the ip target the protocol (TCP/UDP/ICMP...) the layer 4 source port the layer 4 target port Denition of a NetworkMessage :
5.1.3 FileMessage
A le message is dened with the following parameters : a lename the line number in the le the creation date of the le the last modication date of the le the owner of the le the size of the le Denition of a NetworkMessage : Denition of the factory for XML processing of a FileMessage :
28
29
Grammar inference Identication of the automata of the protocol Fields dependencies with messages of previous states All the functionalities of the framework are detailled in this chapter. Vocabular inference
Structure inference Regoupment of similar structures
Options during alignment process read-only process (do not require a participation in the communication). Identify the xed and dynamic elds of all the messages. Regroups equivalent messages depending of their elds structures. Clustering (Regroups equivalent messages using) : an UPGMA Algorithm to regroup similar messages an openMP and MPI implementation Sequencing, Alignment (Identication of elds in messages) : Needleman & Wunsch Implementation Needleman and Wunsch algorithm Originaly a bio-informatic algorithm (sequencing DNA) Align two messages and identify common patterns and eld structure Computes an alignment score representing the efciency of the alignment The following picture shows the sequence alignment of two messages.
UPGMA algorithm Identify equivalent messages based on their alignment score. Build a hierarchical organization of the messages with the UPGMA algorithm (Unweighted Pair Group Method with Arithmetic Mean) The following picture shows a regroupment of similar messages based on the result of the clustering process.
30
Abstraction of a set of message The abstraction is the process of substituting the dynamic elds with their representation as a regex. An example of abstraction is shown on the follinw picture.
Analyses after alignment process aaa Message contextual menu aaa Group contextual menu aaa Rene regexes aaa Slick regexes aaa Concatenate aaa Split column aaa
31
Visualization options aaa Type structure contextual menu aaa Messages distribution This function shows a graphical representation of the distribution of bytes per offset for each message of the current group. This function helps to identify entropy variation of each elds. Entropy variation combined with byte distribution help the user to infer the eld type. [INCLUDE GRAPH] Data typing Primary types [binary, ascii, num, base64...] Denition domain, unique elements and intervals Data carving (tar gz, png, jpg, ...) Semantic data identication (emails, IP ...) Domain of denition aaa Change type representation aaa
Field dependencies from the same message and from the environment
Fields dependancies identication Length elds and associated payloads Encapsulated messages identications And from the environment... Payload extraction The function Find Size Fields, as its name suggests, is dedicated to nd elds that contain any length value as well as the associated payload. It does this on each group. Netzob supports different encoding of the size eld : big and little endian binary values are supported through size of 1, 2 and 4 bytes. The algorithm used to nd the size elds and their associated payloads is desribed in the table XXX. [INCLUDE ALGORITHM] The following picture represents the application of the function on a trace example. It shows the automated extraction of the IP and UDP payloads from an Ethernet frame.
32
Data carving Data carving is the process of extracting semantic information from elds or messages. Netzob allows the extraction of the following semantic information : URL email IP address [INCLUDE FIGURE] Search aaa Properties aaa Grammar inference
Identication of the automata of the protocol Fields dependencies with messages of previous states
33
34
CHAPTER 6
Developer Guide
Read the Developer Guide. See how you can contribute to Netzob API explanation:
35
6.1.7 Subpackages
Common Package
ApplicativeData Module Automata Module
class netzob.Common.Automata.Automata(type) Bases: object MMSTD_TYPE = mmstd getType() static loadFromXML(xmlRoot, vocabulary, namespace, version) save(root, namespace)
BugReporter Module CommandLine Module
class netzob.Common.CommandLine.CommandLine Bases: object Reads, validates and parses the command line arguments provided by users configure() Congure the parser based on Netzobs usage and the denition of its options and arguments getConfiguredParser() Return (if available) the parser congured to manage provided arguments and options by user. @return: the parser getOptions() isManagePluginsRequested() Compute and return is the user has requested to manage the plugins isStartGUIRequested() Compute and return if the user requested (through the command line arguments and options) to start the GTK GUI parse() Read and parse the provided arguments and options
DepCheck Module
class netzob.Common.DepCheck.DepCheck Bases: object Dependency checker. Provides multiple static method to check is required and optionnal dependency are available. static checkCExtensions() static checkRequiredDependency()
36
class netzob.Common.EnvironmentalDependency.EnvironmentalDependency(name, aType, value) Bases: object getName() getType() getValue() save(root, namespace) setName(name) setType(type) setValue(value)
ExecutionContext Module
class netzob.Common.Grammar.Grammar Bases: object addsequence(sequence) getAutomata() getSequences() static loadGrammar(xmlRoot, vocabulary, namespace, version) save(root, namespace) setAutomata(automata)
37
setSequences(sequences) update(vocabulary) update: veries the grammar and its automata is still valid and that none of its symbols has been deprecated
ImportedTrace Module LoggingConfiguration Module
netzob.Common.LoggingConfiguration.LoggingConfiguration(*args, **kwargs) netzob.Common.LoggingConfiguration.singleton(cls, *args, **kwargs) This decorator allows to implement some kind of Singleton design pattern. In our case, we only allow one instanciation.
NetzobException Module
exception netzob.Common.NetzobException.NetzobException(value) Bases: exceptions.Exception Class of handling Netzob specic exceptions exception netzob.Common.NetzobException.NetzobImportException(source, message, statusCode=None, subCode=None) Bases: netzob.Common.NetzobException.NetzobException Raised if an error was encountered while importing data
Order Module
class netzob.Common.Order.Order(value) Bases: object addMessage(message) getMessages() getValue() static loadFromXML(xmlRoot, vocabulary, namespace, version) removeMessage(message) save(root, namespace) setMesages(messages) setValue(value)
Process Module
class netzob.Common.Process.Process(name, pid, user) Bases: object getName() getPid() 38 Chapter 6. Developer Guide
class netzob.Common.Property.Property(name, format, currentValue) Bases: object getCurrentValue() getFormat() getName() getPossibleValues() hasEntry() isEditable() setCurrentValue(currentValue) setFormat(format) setHasEntry(hasEntry) setIsEditable(isEditable) setName(name) setPossibleValues(possibleValues)
PropertyList Module
class netzob.Common.PropertyList.PropertyList Bases: object This class allow to store Property items that are internally stored in a dict(). Items are accessible like any other list or dict. append(prop) clear() keys() list of keys in PropertyList values() list of values in PropertyList
39
class netzob.Common.Sequence.Sequence(id, name, description) Bases: object addMessage(message, orderNumber) addOrder(order) getDescription() getID() getName() getOrderByValue(value) getSortedOrders() static loadFromXML(xmlRoot, vocabulary, namespace, version) save(root, namespace) setDescription(description) setID(id) setName(name)
Session Module SharedLib Module
class netzob.Common.SharedLib.SharedLib(name, version, path) Bases: object static findNameAndVersion(path) getFunctions() getName() getPath() getVersion() static loadFromXML(rootElement) setFunctions(functions) setName(name) setPath(path) setVersion(version)
40
SignalsManager Module
class netzob.Common.SignalsManager.SignalsManager Bases: object Manage the signals for feature availability SIG_FIELDS_MULTIPLE_SELECTION = eld.multiple_selection SIG_FIELDS_NO_SELECTION = elds.no_selection SIG_FIELDS_SINGLE_SELECTION = eld.single_selection SIG_MESSAGES_MULTIPLE_SELECTION = messages.multiple_selection SIG_MESSAGES_NO_SELECTION = messages.no_selection SIG_MESSAGES_SINGLE_SELECTION = messages.single_selection SIG_PROJECT_CLOSE = project.close SIG_PROJECT_OPEN = project.open SIG_SYMBOLS_MULTIPLE_CHECKED = symbols.multiple_checked SIG_SYMBOLS_MULTIPLE_SELECTION = symbols.multiple_selection SIG_SYMBOLS_NONE_CHECKED = symbols.none_checked SIG_SYMBOLS_NO_SELECTION = symbols.no_selection SIG_SYMBOLS_SINGLE_CHECKED = symbols.single_checked SIG_SYMBOLS_SINGLE_SELECTION = symbols.single_selection attach(methodToExecute, signals) emitSignal(signal, *cb_args, **cb_kwargs) emitSignals(signals) getListenersMethodsForSignal(signal)
41
Simulator Module SplashScreen Module Symbol Module Token Module TrashSymbol Module Vocabulary Module Workspace Module XSDResolver Module
C_Extensions Package WrapperArgsFactory Module WrapperMessage Module Functions Package EncodingFunction Module class netzob.Common.Functions.EncodingFunction.EncodingFunction(type, name) Bases: netzob.Common.Functions.RenderingFunction.RenderingFunction TYPE = EncodingFunction apply(message) getConversionAddressingTable(message) getName() getType() setName(name) setType(type)
42
FunctionApplicationTable Module class netzob.Common.Functions.FunctionApplicationTable.FunctionApplicationTable(splittedData) Bases: object applyFunction(function, i_start, i_end) getInitialConversionAddressingTable() getResult() getSegments(i_start, i_end) getTags(col, i_local) insertTagInEncoded(col, i_local, i_global, tag, currentValue) registerTag(i_col, idTag, i, tag) updateConversionAddressingTable(old_start, old_end, new_start, new_end) updateConversionAddressingTableWithTable(table) RenderingFunction Module class netzob.Common.Functions.RenderingFunction.RenderingFunction(superType) Bases: object getID() getPriority() getSuperType() static loadFromXML(rootElement, namespace, version) loadFromXML: Function which parses an XML and extract from it the denition of a rendering function @param rootElement: XML root of the function @return an instance of a function @throw NameError if XML invalid static save(function, root, namespace_workspace) setPriority(priority) TransformationFunction Module class netzob.Common.Functions.TransformationFunction.TransformationFunction(type, name) Bases: netzob.Common.Functions.RenderingFunction.RenderingFunction TYPE = TransformationFunction apply(message) apply: Apply the function on the provided message getConversionAddressingTable(message) getName() getType() reverse(message) reverse: Apply the reverse function on the provided message save(root, namespace_common) Save under the provided XML root the current transformation function setName(name)
43
setType(type) VisualizationFunction Module class netzob.Common.Functions.VisualizationFunction.VisualizationFunction(type, name) Bases: netzob.Common.Functions.RenderingFunction.RenderingFunction TYPE = VisualizationFunction getName() getTags() getType() setName(name) setType(type) Subpackages Encoding Package FormatFunction Module Transformation Package BZ2Function Module Base64Function Module CustomFunction Module GZipFunction Module Visualization Package
44
TextColorFunction Module class netzob.Common.Functions.Visualization.TextColorFunction.TextColorFunction(name, color) Bases: netzob.Common.Functions.VisualizationFunction.VisualizationFunction TYPE = TextColorFunction getTags() MMSTD Package MMSTD Module Subpackages Actors Package AbstractChannel Module MMSTDVisitor Module SimpleCommunicationChannel Module Subpackages NetworkChannels Package InstanciatedNetworkServer Module NetworkClient Module NetworkServer Module Dictionary Package AbstractionLayer Module
45
DictionaryEntry Module class netzob.Common.MMSTD.Dictionary.DictionaryEntry.DictionaryEntry(id, name, value) compare(val, indice, negative, dictionary) forget(temporary) generate() getID() getName() getValue() isActive() learn(val, indice) restore() send(negative, dictionary) setID(id) setName(name) undoForget() MMSTDDictionary Module class netzob.Common.MMSTD.Dictionary.MMSTDDictionary.MMSTDDictionary(variables, entries) getEntries() getEntry(id) getVariableByID(id) getVariables() Memory Module _Variable Module class netzob.Common.MMSTD.Dictionary._Variable.Variable(typeVariable, idVar, name) Variable: Denition of a variable dened in a dictionary compare(value, indice, negative, vocabulary, memory) compare: Compare the current variable to the end (starting at the indice-th character) of value. @type value: bitarray.bitarray @param value: a bit array a subarray of which we compare to the current variable binray value. @type indice: integer @param indice: the starting point of comparison in value. @type negative: boolean @param negative: tells if we use the variable or a logical not of it. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. @rtype: integer @return: the number of letters that matches, -1 if it does not match.
46
getDescription(negative, vocabulary, memory) getDescription: Get the full description of the variable. @type negative: boolean @param negative: tells if we use the variable or a logical not of it. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. @rtype: string @return: the full description of the variable. getID() getName() getTypeVariable() getUncontextualizedDescription() getUncontextualizedDescription: Get the uncontextualized description of the variable (no use of memory or vocabulary). @rtype: string @return: the uncontextualized description of the variable. getValue(negative, vocabulary, memory) getValue: Get the current value of the variable it can be the original value if its set and not forget or the value in memory if it has one else its NONE. @type negative: boolean @param negative: tells if we use the variable or a logical not of it. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. @rtype: (bitarray.bitarray, string) @return: the current or memorized value of the variable in both bitarray and string format. getValueToSend(negative, vocabulary, memory) getValueToSend: Get the current value of the variable it can be the original value if its set and not forget or the value in memory if it has one or it generates one and save its value in memory. @type negative: boolean @param negative: tells if we use the variable or a logical not of it. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. @rtype: (bitarray.bitarray, string) @return: the current, memorized value or a generated value of the variable in both bitarray and string format. isMutable() learn(value, indice, negative, vocabulary, memory) learn: Compare the current variable to the end (starting at the indice-th character) of value. Moreover it stores learns from the provided message. @type value: bitarray.bitarray @param value: a bit array a subarray of which we compare to the current variable binray value. @type indice: integer @param indice: the starting point of comparison in value. @type negative: boolean @param negative: tells if we use the variable or a logical not of it. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. @rtype: integer @return: the number of letters that matches, -1 if it does not match. static loadFromXML(xmlRoot, namespace, version) loadFromXML: Load a variable from an XML denition. Calls its proper heir function for it to create the variable. @type xmlRoot: lxml.etree.Element @param xmlRoot: the root of the XML denition. @type namespace: string @param namespace: the namespace (precision) associated to the variable in the XML
47
denition. @type version: string @param version: the version of the load needed. @rtype: netzob.Common.MMSTD.Dictionary.Variable.Variable @return: a variable constructed from this XML denition. restore(vocabulary, memory) restore: Restore learned value from the last execution of the variable. @type vocabulary: netzob.Common.Vocabulary.Vocabulary @param vocabulary: the vocabulary of the current project. @type memory: netzob.Common.MMSTD.Memory.Memory @param memory: a memory which can contain a former value of the variable. setID(idVar) setName(name) setTypeVariable(typeVariable) toXML(root, namespace) toXML: Create the xml tree associated to this variable. @type root: lxml.etree.Element @param root: the root of the tree being built. @type namespace: string @param namespace: a namespace used as a precision in the variable tree. Subpackages DataTypes Package AbstractType Module AbstractWordType Module BinaryType Module DecimalWordType Module HexWordType Module IPv4WordType Module IntegerType Module MACWordType Module WordType Module RelationTypes Package
48
AbstractRelationType: It denes the type of a relation variable. computeValue(writingToken) computeValue: Compute the value of the variable according to its type.
@type writingToken: netzob.Common.MMSTD.Dictionary.VariableProcessingToken.VariableWritingToken.VariableWriting @param writingToken: a token which contains all critical information on this access. @rtype: bitarray @return: the computed value. getAssociatedDataType() getFactor() getNbBits() getNbChars() getOffset() getType() getType: Return a string describing the current Type. @rtype: string @return: the current type in string format. makeAssociatedDataType(sized, minChars, maxChars, delimiter) Make the associated data type. @type minChars: integer @type sized: boolean @param sized: tell if the variable can be delimited by a size or by a delimiter. @param minChars: the minimum number of elementary character the value of this variable can have. @type maxChars: integer @param maxChars: the maximum number of elementary character the value of this variable can have. @type delimiter: bitarray @param delimiter: a set of bits that tells where the associated variable ends. static makeType(typeString, sized, minChars, maxChars, delimiter, factor, offset) setAssociatedDataType(dataType) toString() BinarySizeRelationType Module WordSizeRelationType Module Values Package AbstractValue Module 6.1. netzob Package 49
Aggregate Module EndValue Module TextValue Module VarValue Module VariableProcessingToken Package
AbstractVariableProcessingToken: A communication token used by variable when they are processed. appendLinkedValue(value) getIndex() getLinkedValue() getMemory() getNegative() getValue() getVocabulary() isOk() setOk(ok) setValue(value) VariableReadingToken Module VariableWritingToken Module Variables Package AbstractLeafVariable Module AbstractNodeVariable Module AbstractVariable Module 50 Chapter 6. Developer Guide
AggregateVariable Module AlternateVariable Module ComputedRelationVariable Module DataVariable Module RepeatVariable Module _AbstractRelationVariable Module _AggregateVariable Module _AlternateVariable Module class netzob.Common.MMSTD.Dictionary.Variables._AlternateVariable.AlternateVariable(idVar, name, children) Bases: netzob.Common.MMSTD.Dictionary._Variable.Variable AlternateVariable: A node variable that makes an alternative of all its children. addChild(child) compare(value, indice, negative, vocabulary, memory) compare: Compare the current variable to the end (starting at the indice-th character) of value. Return the number of letters that matches, -1 if it does not match. getChildren() getDescription(negative, vocabulary, memory) getDescription: Get the full description of the variable. getUncontextualizedDescription() getUncontextualizedDescription: Get the uncontextualized description of the variable (no use of memory or vocabulary). getValue(negative, vocabulary, memory) getValue: Get the current value of the variable it can be the original value if its set and not forget or the value in memory if it has one else its NONE. getValueToSend(negative, vocabulary, memory) getValueToSend: Get the current value of the variable it can be the original value if its set and not forget or the value in memory if it has one or it generates one and save its value in memory. learn(value, indice, negative, vocabulary, memory) learn: Compare the current variable to the end (starting at the indice-th character) of value. Moreover it stores learns from the provided message. Return the number of letters that matches, -1 if it does not match. static loadFromXML(xmlRoot, namespace, version) loadFromXML: Load an alternate variable from an XML denition. restore(vocabulary, memory) restore: Restore learned value from the last execution of the variable.
51
toXML(root, namespace) toXML: Create the xml tree associated to this variable. _BinaryVariable Module _ComputedRelationVariable Module _DecimalWordVariable Module _DirectRelationVariable Module _DynLenStringVariable Module _HexVariable Module _IPVariable Module class netzob.Common.MMSTD.Dictionary.Variables._IPVariable.IPVariable(id, name, defaultVar) Bases: netzob.Common.MMSTD.Dictionary._Variable.Variable generateValue(negative, dictionary) getValue(negative, dictionary) learn(val, indice, isForced, dictionary) string2bin(aStr) _IPv4Variable Module _IntVariable Module _MD5Variable Module _ReferencedVariable Module _WordVariable Module MemOpexs Package
52
MemOpex Module class netzob.Common.MMSTD.MemOpexs.MemOpex.MemOpex(type, id, transitionId) getID() getTransitionID() getType() static loadFromXML(xmlRoot, namespace, version) save(root, namespace) setID(id) setTransitionID(transitionID) Subpackages impl Package ForgetMemOpex Module class netzob.Common.MMSTD.MemOpexs.impl.ForgetMemOpex.ForgetMemOpex(id, transitionId, variableId) Bases: netzob.Common.MMSTD.MemOpexs.MemOpex.MemOpex getVariableID() static loadFromXML(xmlRoot, namespace, version) save(root, namespace) setVariableID(variableID) States Package AbstractState Module class netzob.Common.MMSTD.States.AbstractState.AbstractState(type, id, name) activate() addMemOpex(memOpex) deactivate() executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer) getID() getMemOpexs() getName() getTransitions() getType()
53
isActive() static loadFromXML(xmlRoot, namespace, version) registerTransition(transition) removeMemOpex(memOpex) save(root, namespace) setID(id) setName(name) Subpackages impl Package NormalState Module class netzob.Common.MMSTD.States.impl.NormalState.NormalState(id, name) Bases: netzob.Common.MMSTD.States.AbstractState.AbstractState executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer) getID() getName() getTransitions() static loadFromXML(xmlRoot, namespace, version) registerTransition(transition) save(root, namespace) setID(id) setName(name) unregisterTransition(transition) StartState Module class netzob.Common.MMSTD.States.impl.StartState.StartState(id, name) Bases: netzob.Common.MMSTD.States.AbstractState.AbstractState executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer) getID() getName() getTransitions() static loadFromXML(xmlRoot, namespace, version) registerTransition(transition) save(root, namespace)
54
setID(id) setName(name) Symbols Package AbstractSymbol Module class netzob.Common.MMSTD.Symbols.AbstractSymbol.AbstractSymbol(type) getType() Subpackages impl Package DictionarySymbol Module class netzob.Common.MMSTD.Symbols.impl.DictionarySymbol.DictionarySymbol(dictionaryEntry) Bases: netzob.Common.MMSTD.Symbols.AbstractSymbol.AbstractSymbol TYPE = DictionarySymbol getEntry() getID() getName() isEquivalent(symbol) setEntry(entry) setID(id) write(writingToken) write: Grant a writing access to the entry. Retrieve and return the value issued from this access.
@type writingToken: netzob.Common.MMSTD.Dictionary.VariableProcessingToken.VariableWritingToken.VariableWriting @param writingToken: a token which contains all critical information on this writing access. @rtype: bitarray @return: the value this acces writes. EmptySymbol Module UnknownSymbol Module Transitions Package
55
AbstractTransition Module class netzob.Common.MMSTD.Transitions.AbstractTransition.AbstractTransition(type, id, name, inputState, outputState) activate() deactivate() executeAsClient(abstractionLayer) executeAsMaster(abstractLayer) getDescription() getID() getInputState() getName() getOutputState() getType() isActive() isValid(inputSymbol) static loadFromXML(states, vocabulary, xmlRoot, namespace, version) setID(id) setInputState(inputState) setName(name) setOutputState(outputState) toXMLString(idStartState) Subpackages impl Package CloseChannelTransition Module
56
class netzob.Common.MMSTD.Transitions.impl.CloseChannelTransition.CloseChannelTransition(id, name inputState, outputState, disconnectionTime) Bases: netzob.Common.MMSTD.Transitions.AbstractTransition.AbstractTransition TYPE = CloseChannel closeConnection(abstractionLayer) executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer) getDescription() getDisconnectionTime() isValid(receivedSymbol) static loadFromXML(states, xmlTransition, namespace, version) save(root, namespace)
OpenChannelTransition Module class netzob.Common.MMSTD.Transitions.impl.OpenChannelTransition.OpenChannelTransition(id, name, inputState, outputState, connectionTime, maxNum berOfAttempt) Bases: netzob.Common.MMSTD.Transitions.AbstractTransition.AbstractTransition TYPE = OpenChannel executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer)
57
getConnectionTime() getDescription() getMaxNumberOfAttempt() isValid(receivedSymbol) static loadFromXML(states, xmlTransition, namespace, version) openConnection(abstractionLayer) save(root, namespace) SemiStochasticTransition Module SimpleTransition Module class netzob.Common.MMSTD.Transitions.impl.SimpleTransition.SimpleTransition(id, name, inputState, outputState, timeBeforeActing, outputSymbol) Bases: netzob.Common.MMSTD.Transitions.AbstractTransition.AbstractTransition executeAsClient(abstractionLayer) executeAsMaster(abstractionLayer) getDescription() getOutputSymbol() getTimeBeforeActing() isValid(receivedSymbol) static parse(xmlTransition, dictionary, states) toXMLString(idStartState) Models Package AbstractMessage Module FileMessage Module 58 Chapter 6. Developer Guide
IPCMessage Module IRPDeviceIoControlMessage Module IRPMessage Module L2NetworkMessage Module L3NetworkMessage Module L4NetworkMessage Module RawMessage Module Subpackages Factories Package AbstractMessageFactory Module FileMessageFactory Module IPCMessageFactory Module class netzob.Common.Models.Factories.IPCMessageFactory.IPCMessageFactory Bases: object XML_SCHEMA_TYPE = netzob-common:IPCMessage static addPropertiesToElement(xmlMessage, message, namespace) static loadFromXML(rootElement, namespace, version) static save(message, xmlMessage, namespace_project, namespace) IRPDeviceIoControlMessageFactory Module IRPMessageFactory Module L2NetworkMessageFactory Module L3NetworkMessageFactory Module L4NetworkMessageFactory Module OldFormatNetworkMessageFactory Module 6.1. netzob Package 59
RawMessageFactory Module Plugins Package AbstractPluginController Module AbstractPluginView Module class netzob.Common.Plugins.AbstractPluginView.AbstractPluginView(plugin, troller) Bases: object Regroup methods any plugins view must be able to access getController() Returns the controller attached to the current plugin @return: the controller getPlugin() Returns the current plugin denition class @return: the plugin denition CapturerPlugin Module ExporterPlugin Module FileImporterPlugin Module NetzobPlugin Module PluginChecker Module Subpackages Capturers Package AbstractCapturer Module AbstractCapturerController Module AbstractCapturerView Module Exporters Package AbstractExporterController Module AbstractExporterView Module
con-
60
Extensions Package
CapturerMenuExtension Module class netzob.Common.Plugins.Extensions.CapturerMenuExtension.CapturerMenuExtension(netzob, controller, actionName, menuText, menuStock=None, menuAccel=None, menuTooltip=None) Bases: netzob.Common.Plugins.Extensions.GlobalMenuExtension.GlobalMenuExtension executeAction(widget, vocabularyView, data=None) getActions() getUIDefinition() ExportMenuExtension Module class netzob.Common.Plugins.Extensions.ExportMenuExtension.ExportMenuExtension(netzob, controller, actionName, menuText, menuStock=None, menuAccel=None, menuTooltip=None) Bases: netzob.Common.Plugins.Extensions.GlobalMenuExtension.GlobalMenuExtension ExportMenuExtension: A generic export menu extension. executeAction(widget, data=None) executeAction: Start the controller. Called when the menu entry is selected. getActions() getActions:
61
@rtype: List @return: the list of all information on the action associated to the menu extension. getUIDefinition() getUIDenition: Return the User Interface denition of this menu extension. @rtype: string @return: an xml string representing the menu extension. GlobalMenuExtension Module class netzob.Common.Plugins.Extensions.GlobalMenuExtension.GlobalMenuExtension Bases: netzob.Common.Plugins.Extensions.NetzobExtension.NetzobExtension NetzobExtension Module class netzob.Common.Plugins.Extensions.NetzobExtension.NetzobExtension Bases: object Importers Package AbstractFileImporterController Module AbstractFileImporterView Module AbstractImporterController Module AbstractImporterView Module Threads Package Job Module class netzob.Common.Threads.Job.Job(generator) Bases: object Wrap a co-routines that yields asynchronous tasks (see Task class). Task Module class netzob.Common.Threads.Task.Task Bases: object Base class for asynchronous tasks. cancel() config(return_cb, exception_cb) Set return and exception callbacks. run() Subpackages Tasks Package
62
SleepTask Module ThreadedTask Module Type Package Endianess Module class netzob.Common.Type.Endianess.Endianess BIG = big-endian LITTLE = little-endian Format Module class netzob.Common.Type.Format.Format ALPHA = alpha ALPHA_NUM = alphanum ASCII = ascii BASE64_DEC = base64dec BASE64_ENC = base64enc BINARY = binary DECIMAL = decimal FLOAT = oat HEX = hex IP = ip IPv4 = ipv4 NUM = num OCTAL = octal STRING = string formatVisualizationUnitSizes = {binary: 1, hex: 4, string: 8} static getExtendedSupportedFormats() static getPossibleFormats(values) getPossibleFormats: Given the provided values, it computes the possible formats the values can be. returns a list of format static getSupportedFormats() static getUnitSize(format)
63
Sign Module class netzob.Common.Type.Sign.Sign SIGNED = signed UNSIGNED = unsigned TypeConvertor Module TypeIdentifier Module class netzob.Common.Type.TypeIdentifier.TypeIdentifier getTypes(val) isAscii(val) isBase64(val) isHexString(val) UnitSize Module class netzob.Common.Type.UnitSize.UnitSize BIT = bit BITS16 = 16-bits BITS32 = 32-bits BITS4 = 4-bits BITS64 = 64-bits BITS8 = 8-bits NONE = none static getMaxDigitForTypeAndUnitSize(aType, unitSize) static getPackDefiniton(unitSize) static getSizeInBits(unitSize) unitSizeInBits = {32-bits: 32, 16-bits: 16, 8-bits: 8, bit: 1, 4-bits: 4, 64-bits: 64} Export Package
RawExport Module
64
ScapyExport Module
Import Package
AbstractImporter Module ApiImport Module ThirdPartyImport Module XMLImport Module Subpackages
GOTPoisoning Package GOTPoisoner Module class netzob.Import.GOTPoisoning.GOTPoisoner.GOTPoisoner(parasite, injector) injectProcess(pid) HijackedFunction Module class netzob.Import.GOTPoisoning.HijackedFunction.HijackedFunction(name, returnType, parameters, source) getEndOfFunction() getName() getParameters() getParasiteFunctionDeclaration() getParasitePrototype()
65
getPrototype() getReturnType() getSource() static loadFromXML(rootElement) setName(name) setParameters(parameters) setReturnType(returnType) setSource(source) InjectorGenerator Module class netzob.Import.GOTPoisoning.InjectorGenerator.InjectorGenerator(tmp_folder, parasite) compileInjector() getFolder() getLibName() getLibPath() getParasiteSignature() getShellCode() getSourceCode() produceShellCode() writeInjectorToFile() ParasiteGenerator Module class netzob.Import.GOTPoisoning.ParasiteGenerator.ParasiteGenerator(tmp_folder) addAnHijackedFunctions(function) compileParasite() getFifoFile() getFunctions() getParasitesSignature() getSourceCode() getSourceCodeHeader() getSourceCodeOfWriteFunction() getSourceCodeParasiteCoreFunctions() linkParasite() writeParasiteToFile()
66
PrototypesRepositoryParser Module class netzob.Import.GOTPoisoning.PrototypesRepositoryParser.PrototypesRepositoryParser static loadFromXML(le) ThirdParties Package AbstractThirdPartyImporter Module class netzob.Import.ThirdParties.AbstractThirdPartyImporter.AbstractThirdPartyImporter(id, name) Bases: object getId() getName() parse(paths) OSpy Module TreeViews Package TreeProcessesGenerator Module Inference Package
Subpackages
Grammar Package Angluin Module AutomaticGrammarAbstractionView Module AutomaticGrammarInferenceView Module GrammarInferer Module LearningAlgorithm Module MQCache Module Subpackages EquivalenceOracles Package
67
AbstractEquivalenceOracle Module class netzob.Inference.Grammar.EquivalenceOracles.AbstractEquivalenceOracle.AbstractEquivalen Bases: object findCounterExample(mmstd) WMethodNetworkEquivalenceOracle Module Oracles Package AbstractOracle Module class netzob.Inference.Grammar.Oracles.AbstractOracle.AbstractOracle(type) Bases: object start(mmstd) stop() NetworkOracle Module Queries Package MembershipQuery Module Vocabulary Package DataCarver Module SearchResult Module class netzob.Inference.Vocabulary.SearchResult.SearchResult(message, description) Bases: object addSegment(i_start, i_end) getDescription() getMessage() getSegments() getVariationDescription() setDescription(description) setVariationDescription(variationDescription)
68
SearchTask Module class netzob.Inference.Vocabulary.SearchTask.SearchTask(description, data, type) Bases: object getDescription() getResults() getVariations() registerResults(r, v) registerVariation(data, description) setDescription(description) SearchView Module Searcher Module SizeFieldIdentifier Module _VariableView Module Subpackages Alignment Package AsciiAlign Module NeedlemanAndWunsch Module UPGMA Module Simulator Package
XDotWidget Module
UI Package
ModelReturnCodes Module
69
NetzobAbstractController Module NetzobAbstractPerspectiveController Module NetzobAbstractPerspectiveView Module NetzobAbstractView Module NetzobWidgets Module Subpackages
Common Package AboutDialog Module Subpackages Controllers Package AvailablePluginsController Module BugReporterController Module CustomTransformationFunctionController Module MoveMessageController Module ProjectExportController Module ProjectImportController Module ProjectPropertiesController Module WorkspaceConfigurationController Module WorkspaceSelectorController Module Views Package AvailablePluginsView Module BugReporterCertificateErrorView Module 70 Chapter 6. Developer Guide
BugReporterView Module CustomTransformationFunctionView Module MoveMessageView Module ProjectExportView Module ProjectImportView Module ProjectPropertiesView Module WorkspaceConfigurationView Module WorkspaceSelectorView Module Export Package Subpackages Controllers Package RawExportController Module ScapyExportController Module TextExportController Module Views Package RawExportView Module ScapyExportView Module TextExportView Module Fuzzing Package Subpackages Grammar Package 6.1. netzob Package 71
Subpackages Controllers Package CreateCloseChannelTransitionController Module CreateOpenChannelTransitionController Module CreateSemiStochasticTransitionController Module CreateStateController Module DeleteStateController Module DeleteTransitionController Module EditStateController Module GrammarController Module Subpackages Menus Package ContextualMenuOnStateController Module Views Package CreateCloseChannelTransitionView Module CreateOpenChannelTransitionView Module CreateSemiStochasticTransitionView Module CreateStateView Module DeleteStateView Module DeleteTransitionView Module EditStateView Module 72 Chapter 6. Developer Guide
GrammarView Module GrammarXDotWidget Module Subpackages Menus Package ContextualMenuOnStateView Module Import Package ImportFileChooserDialog Module Subpackages Controllers Package ConfirmImportMessagesController Module Views Package ConfirmImportMessagesView Module Simulator Package Subpackages Controllers Package CreateNetworkActorController Module SimulatorController Module Views Package CreateNetworkActorView Module SimulatorView Module TraceManager Package 6.1. netzob Package 73
Subpackages Controllers Package ApplicativeDataImportController Module ApplicativeDataManagerController Module TraceManagerController Module Views Package ApplicativeDataImportView Module ApplicativeDataManagerView Module TraceManagerView Module Vocabulary Package Subpackages Controllers Package EnvironmentDependenciesSearcherController Module FieldAnalysisController Module FilterMessagesController Module FindSizeFieldsController Module MessageTableController Module MessagesDistributionController Module PopupEditFieldController Module RelationsController Module ResearchController Module 74 Chapter 6. Developer Guide
SplitFieldController Module VariableController Module VariableDisplayerController Module VocabularyController Module Subpackages Menus Package ContextualMenuOnFieldController Module ContextualMenuOnLayerController Module Partitioning Package ForcePartitioningController Module ResetPartitioningController Module SequenceAlignmentController Module SimplePartitioningController Module SmoothPartitioningController Module Views Package EnvironmentDependenciesSearcherView Module FieldAnalysisView Module FilterMessagesView Module MessageTableView Module MessagesDistributionView Module PopupEditFieldView Module 6.1. netzob Package 75
RelationsView Module ResearchView Module SearchResultsView Module SplitFieldView Module VariableDisplayerView Module VariableView Module VocabularyView Module Subpackages Menus Package ContextualMenuOnFieldView Module ContextualMenuOnLayerView Module Partitioning Package ForcePartitioningView Module ResetPartitioningView Module SequenceAlignmentView Module SimplePartitioningView Module SmoothPartitioningView Module
76
CHAPTER 7
77
78
netzob.Common.MMSTD.Dictionary.Variables._IPVariabl 52 netzob.Common.Automata, 36 netzob.Common.MMSTD.MemOpexs.impl.ForgetMemOpex, netzob.Common.CommandLine, 36 53 netzob.Common.DepCheck, 36 netzob.Common.EnvironmentalDependencies, netzob.Common.MMSTD.MemOpexs.MemOpex, 53 37 netzob.Common.MMSTD.States.AbstractState, netzob.Common.EnvironmentalDependency, 53 37 netzob.Common.MMSTD.States.impl.NormalState, netzob.Common.ExecutionContext, 37 54 netzob.Common.Functions.EncodingFunction, netzob.Common.MMSTD.States.impl.StartState, 42 54, netzob.Common.Functions.FunctionApplicationTable netzob.Common.MMSTD.Symbols.AbstractSymbol, 43 55 netzob.Common.Functions.RenderingFunction, netzob.Common.MMSTD.Symbols.impl.DictionarySymbol, 43 netzob.Common.Functions.TransformationFunction, 55 netzob.Common.MMSTD.Transitions.AbstractTransition, 43 56 netzob.Common.Functions.Visualization.BackgroundColorFunction , netzob.Common.MMSTD.Transitions.impl.CloseChannelTr 44 56 netzob.Common.Functions.Visualization.TextColorFunction , netzob.Common.MMSTD.Transitions.impl.OpenChannelTra 45 netzob.Common.Functions.VisualizationFunction, 57 netzob.Common.MMSTD.Transitions.impl.SimpleTransiti 44 58 netzob.Common.Grammar, 37 netzob.Common.Models.Factories.IPCMessageFactory , netzob.Common.LoggingConfiguration, 38 59 netzob.Common.MMSTD.Dictionary._Variable, netzob.Common.NetzobException, 38 46 netzob.Common.Order , 38 netzob.Common.MMSTD.Dictionary.DictionaryEntry , netzob.Common.Plugins.AbstractPluginView , 46 60 netzob.Common.MMSTD.Dictionary.MMSTDDictionary, netzob.Common.Plugins.Extensions.CapturerMenuExtens 46 61 netzob.Common.MMSTD.Dictionary.RelationTypes.AbstractRelationType , netzob.Common.Plugins.Extensions.ExportMenuExtensio 49 61 netzob.Common.MMSTD.Dictionary.VariableProcessingToken.AbstractVariableProcessingToken , netzob.Common.Plugins.Extensions.GlobalMenuExtensio 50 62 netzob.Common.MMSTD.Dictionary.Variables._AlternateVariable , netzob.Common.Plugins.Extensions.NetzobExtension , 51 62 79
netzob.Common.Process, 38 netzob.Common.Property, 39 netzob.Common.PropertyList, 39 netzob.Common.Sequence, 40 netzob.Common.SharedLib, 40 netzob.Common.SignalsManager, 41 netzob.Common.Threads.Job, 62 netzob.Common.Threads.Task, 62 netzob.Common.Type.Endianess, 63 netzob.Common.Type.Format, 63 netzob.Common.Type.Sign, 64 netzob.Common.Type.TypeIdentifier, 64 netzob.Common.Type.UnitSize, 64 netzob.Common.XSDResolver, 42 netzob.Export.RawExport, 64 netzob.Export.ScapyExport, 65 netzob.Export.TextExport, 65 netzob.Import.GOTPoisoning.GOTPoisoner, 65 netzob.Import.GOTPoisoning.HijackedFunction, 65 netzob.Import.GOTPoisoning.InjectorGenerator, 66 netzob.Import.GOTPoisoning.ParasiteGenerator, 66 netzob.Import.GOTPoisoning.PrototypesRepositoryParser, 67 netzob.Import.ThirdParties.AbstractThirdPartyImporter, 67 netzob.Inference.Grammar.EquivalenceOracles.AbstractEquivalenceOracle, 68 netzob.Inference.Grammar.Oracles.AbstractOracle, 68 netzob.Inference.Vocabulary.SearchResult, 68 netzob.Inference.Vocabulary.SearchTask, 69 netzob.NetzobPluginManagement, 35 netzob.NetzobResources, 35 netzob.release, 35 netzob.UI.ModelReturnCodes, 69
80