From NS-2 to NS-3 – Implementation and Evaluation

Nattavit Kamoltham Kulit Na Nakorn Kultida Rojviboonchai*

Department of Computer Engineering Department of Computer Engineering Department of Computer Engineering
Faculty of Engineering, Faculty of Engineering, Faculty of Engineering,
Chulalongkorn University Chulalongkorn University Chulalongkorn University
Bangkok, Thailand Bangkok, Thailand Bangkok, Thailand
[email protected] [email protected] [email protected]

Abstract—NS-2 is the most famous simulator among other ones for their works. However, the shortcoming of using NS-2
network simulators. Most of researchers use NS-2 to evaluate is that the implementation cannot be reused or tested on a real
their new protocols/architectures. Although NS-2 has much system. Researchers need to do more works or make new
resource and many available tools to generate different implementation to evaluate their work on a real system.
characteristics and network scenarios, source codes created on
NS-2 is not able to be reused for real implementation. The NS-3[5] is the new simulator for researchers. It has been
releasing of NS-3 changes the way researchers work. NS-3 allows developed with whole new idea. Its structure is completely
the researchers to work on both simulations and emulations with different from NS-2’s. NS-3 supports on both simulation and
the same implementation and source codes. These can help emulation. This can reduce time of implementation[10][11].
researchers to evaluate their works on a single computer for Researchers can implement their works only one time. Then,
simulation or on a real system without wasting their time to they can use the same source codes to evaluate their works on
implement their works twice; one in NS-2 simulation and the both simulation and emulation.
other in real system. In this paper, we elaborate different points
of protocol implementation between NS-2 and NS-3. We Our contribution is divided into threefold. Firstly, we
recommend a setting for mobility trace in order to obtain correct elaborate an outline implementation structure when a protocol
results between NS-2 and NS-3. Then, as a case study, we use that is already implemented in ns-2 needs to be transferred to
DECA, which is a reliable broadcasting protocol for VANETs ns-3 platform. Our outline implementation structure is general
and previously implemented on NS-2. How to transfer DECA so it can be applied to any protocols. We also have a
from NS-2 to NS-3 and how to validate its performance are suggestion on setting of mobility trace. Secondly, we show
shown. Moreover, an emulation of DECA on real system using how to transfer a protocol from NS-2 to NS-3 and then
NS-3 is described. The emulation results show a problem caused evaluate its performance. Thirdly, according to our experiment
by asymmetric links which is not concerned in most of using ns-3 emulation over real testbeds, we found a problem
literatures. In this paper, therefore, we also propose a simple caused by asymmetric links. We propose a simple solution
solution to help protocols’ operation in the asymmetric link using RSSI information that is mentioned in [14] to estimate
scenarios. The simulation results show that our solution can the interference and carrier sensing of protocol to help select a
improve protocol performance.
better node for broadcasting a message.
Keywords-NS-2; NS-3; Simulator; Emulator; Asymmetric Link; In this paper, as a case study, we use DECA [6] (Density-
Vehicular Networks Aware Reliable Broadcasting Protocol), one of protocols on
VANET that has previously been implemented on NS-2.
I. INTRODUCTION DECA is a reliable broadcasting protocol for data
dissemination in vehicular ad-hoc network. This protocol uses
Vehicular Ad-Hoc Networks (VANETs) is one of the most
store and forward technique. It also reduces the redundant data
interesting topics on ad-hoc networks. This is because VANET
retransmission by using only density information around nodes.
can support many applications on Intelligent Transportation
We select this protocol because of two reasons; the former is
System (ITS). In VANET, most of researchers evaluate their
that DECA is already implemented in NS-2, and the latter is
work on simulation because the simulation does not require
that DECA uses less information than other protocols. It can
any equipment, high-cost device and cars. We have found that
perform without any position knowledge or GPS. The detail of
most of literatures using NS-2[4] for their evaluation rather
this protocol can be found in [6].
than using other simulators such as OmNeT++[1], QualNet[2]
and OPNET[3]. We implement DECA in NS-3 and validate the protocol by
using the same traffic scenarios to simulate on both NS-2 and
NS-2[4] is the most famous simulator in network fields
NS-3. In this process, we have found a problem of NS-3
because it is an open-source simulator that comes with many
mobility file reader that has error about 6-7% in every scenario.
tools and resources. It has user community to help and support
Then we use implementation on NS-3 to test on emulation. The
other users. Especially in VANETs works, more tools such as
emulation results have shown the overhead problem of the
traffic simulation convertor and specific propagation models
protocol that is different from simulation result. After
are needed. Researchers can choose the proper tool from many
investigation, we have found that the protocol has inefficient

*Corresponding author

978-1-4577-1719-2/12/$26.00 ©2012 IEEE 35

operation from an asymmetric link problem. On the real
system, each computer has the different wireless device and
also different antenna. These affect to computer’s transmission
power and to performance of the protocol. Most of papers does
not concern about this problem because the evaluation is done
only on simulation.
To solve the asymmetric link problem, we use RSSI level
information helping protocol decision. This sample information
can help protocol to know the probability that node will
succeed retransmit the data. This solution can be applied on
any of protocol including protocol that using GPS information
such as POCA[7], CAR[8] and DV-Cast[9]. The results have
shown that our solution can help protocol to improve their
performance on the asymmetric scenarios.
The rest of paper is organized as follows. In section II, the
different of NS-2 and NS-3 will be described. In section III
shows the solution to improve protocol in an asymmetric link a) Protocol Addition Outline on NS-2 and NS-3
scenario. In section IV, performance evaluations are shown.
Finally, this paper is concluded in section V.

II. DIFFERENCES BETWEEN NS-2 AND NS-3

A. Differences Between NS-2 and NS-3

NS-2 is based on C++ on its core system with Tcl script
for controlling simulation. On the other hand, NS-3 is based
on C++ for the whole system and users can use Python for any
other scripts. Therefore, the implementation on NS-2 cannot
be reused in NS-3 and need to be carefully and manually
transferred.
In this paper, we analyze two main differences between
NS-2 and NS-3 including an addition procedure of a new
protocol and an outline implementation structure of NS-2 and
NS-3.
b) Protocol Implementation Outline on NS-2 and NS-3
Firstly, the addition of a new protocol is a step to add a Figure 1. Differences Between NS-2 and NS-3
new protocol to the simulation program. In NS-2, we begin
with adding packet header of the new protocol to packet.h, and
then add the new protocol to Tcl Library. Next, we add the B. Importing Mobility Trace on NS-2 and NS-3
Protocol Source to Makefile.in. Finally, we execute program NS-3 has ability to use the same mobility trace file as in
with make command. On the other hand, NS-3 with purely NS-2 by helping of NS2MobilityHelper. This helper will
C++ implementation makes the procedure easier. We start change the mobility trace in NS-2 format to NS-3 format. In
with adding packet header, the new protocol to wscript in their NS-2, a node will move following its assigned velocity vector.
folder to make those objects linked together to become parts However, in NS-3, a node will use different speed in X, Y and
of the ns3 library. And then, we execute ./waf command. Z axis instead of vector direction. At the first step, we have
Secondly, the outline implementation structure of NS-2 tried to validate nodes’ movement trace. We found that using
and NS-3 is the other different point that we would like to the default setting of NS-3 makes nodes in NS-3 have
show in this paper. In NS-2, after running the Tcl Script, Tcl different distances from nodes in NS-2 up to 200 meters at the
Binding to the protocol object implemented with C++ is same time of simulation. Such different distances can provide
created in the simulator. Note that in this paper we focus on significant performance difference.
Network Layer Protocol. Then, protocol calls the Packet This problem comes from class NS2MobilityHelper. The
Header struct including Timer. However, in NS-3, it is source code from this class has a code that makes nodes stop
different from NS-2. In NS-3, after we execute ./waf moving for a small period of time before changing their
command, the simulator which is completely implemented by movement direction. This causes the different distance that a
C++ calls the protocol directly without Tcl Binding. Then, the node can move when comparing to NS-2. To solve this
protocol can call Packet Header and Timer as ordinary C++ problem, our suggestion is to disable the following code in
objects. Fig.1a shows the protocol addition step for each NS2MobilityHelper.cc at line 614 as shown in Fig2.
networks simulator version and Fig.1b shows the different in
term of implementation outline.

36
 if (time >= 0) nodes within transmission range receive the packet, they store
{ the message and set a waiting timeout. The preferred node
Simulator::Schedule (Seconds (at +
time),&ConstantVelocityMobilityModel::SetVelocity, selects the next rebroadcast node and rebroadcasts the
model, Vector (0, 0, 0)); message immediately. If the preferred node does not
}
rebroadcast that message, other nodes with the shortest
Figure 2. Disable Code in NS2MobilityHelper.cc waiting timeout will do it instead. In case of nodes detect
In order to show that our suggestion can help, we some neighbor nodes that miss the packet, they will set
generated scenarios using the traffic simulator Simulation of waiting timeout and a node with a shortest waiting time will
Urban and Mobility, called SUMO [12], and then converted to rebroadcast the packet.
NS-2 format by Traffic and Network Simulation Environment, To validate our DECA implementation on NS-3, we use the
called TraNS[13]. We used 4 kilometers with 2 lanes as same traffic scenarios as above which are 4 kilometers straight
highway scenarios. Then, we plot distances of all nodes. Fig.3 road with 2 lanes. The traffic scenarios have been generated
shows movement of the same node in NS-2 and in NS-3 at from SUMO then they are converted to NS-2 trace format by
density 30 veh/km. TABLE I shows average movement error TraNS. Then we use the modified NS-2 trace reader (NS-
of all nodes when comparing to the node movement in NS-2 at 2MobilityHelper) which provides nodes’ movement in NS-3 to
be the same as shown in NS-2. Each node has 250 meters
various vehicle density. It can be seen from both Fig. 3 and
transmission range with MAC 802.11. Other parameters setting
TABLE I that our suggestion to modify NS2MobilityHelper
for simulations are configured as indicated in TABLE II.
can make the movement trace used in NS-3 to be
approximately the same as the movement trace used in NS-2. A source node will send data message every 10 seconds 5
This can help researchers to evaluate their work accurately and times. Evaluation metrics include reliability and overhead.
repeatedly. Reliability is a percentage of number of node that received the
message to number of total nodes. Overhead is the cost of
C. Simulation Results Between NS-2 and NS-3 message retransmission from every node in scenario. Every
To show that our outline implementation structure result will be averaged from total scenarios.
described in Section II.A is workable, as a case study, we use The simulation results show in Fig.4 and Fig.5. Fig.4 shows
DECA, Density-aware reliable Broadcasting protocol [6], the reliability results. The reliability of DECA implemented in
which is previously implemented and evaluated on NS-2, to NS-3 provides the same amount as DECA implemented in NS-
implement on NS-3. DECA is a store and forward protocol. It 2 except for 2 veh/km scenario. However, it is slightly different
uses periodic beacons to discover neighbors and exchange and not significant. The overhead results are shown in Fig.5.
information in 1-hop neighbors. The main mechanism is Similarly, the overhead results of both NS-2 and NS-3 are
occurred when a source node would like to broadcast a approximately the same. Hence, we can conclude from the
message; it will select a preferred node with the highest simulation results that DECA implemented in NS-3 has the
number of neighbors and broadcast the message. When all same performance as DECA implemented in NS-2.
TABLE I. AVERAGE MOVEMENT ERROR (%) TABLE II. PARAMETERS SETTING
a
Scenarios Density (veh/km) Message life time 10 seconds
Number of Messages 5
2 10 30 60 120 Size of a broadcast message 512 bytes
NS-3 : Original Simulation Time 50 seconds
6.6853 6.1845 6.1958 7.8375 7.4393 Maximum vehicle speed (km/h) 50, 80
NS-2MobilityHelper
Vehicles density (veh/km) 2, 10, 30, 60, 120
NS-3 : Modified Networks Simulation NS-2.34, NS-3.10
0.0013 0.0009 0.0019 0.0036 0.3843
NS-2MobilityHelper a. An interval from packet was released until packet is expired

Figure 3. A Node Movement in NS-2 and NS-3 Figure 4. Reliability Results

37
 As a result, the sending node will keep sending this message
until the node that misses the message get closer to it.
This problem is not found in the simulation results because
nodes in simulation always have exactly the same transmission
range. That’s why most of researchers do not concern about the
asymmetric link problem on their works. In fact, it affects to all
protocols’ performance.

B. A Proposed Solution forAsymmetric Links Problem

In this section, we propose a simple solution to tackle with
the asymmetric link problem mentioned in the previous section.
Due to local information such as density and position
knowledge cannot provide stability of links, we propose a
Figure 5. Overhead Results
simple solution to use RSSI level as optional information. After
each protocol makes a decision by its own mechanism, our
III. EMULATION AND AN ASYMMETRIC LINK PROBLEM solution will consider more on RSSI information.

A. Emulation Setup and Results Let us show an example of our solution in DECA that can
be also applied to other protocols as well. Nodes in DECA will
As we mentioned before, the key ability of NS-3 is select the next rebroadcast node with the highest density. This
emulation. We can evaluate our works on a real testbeds with can maximize the number of received nodes in each time of
the same implementation as in simulation. In this section, we rebroadcasting. But in an asymmetric link scenario, a node
tested DECA on a real system. We use 4 notebooks to with the highest density may have short transmission range and
represent nodes in the system. Each of them has their own hence could not do it. Because of variable transmission range,
wireless communication. The emulations have been tested in 2 if that node has a short transmission range, although it has
scenarios. The first one, every node is fixed to the same many neighbors, its rebroadcasting to neighbors has low
position all the time in experiment. Nodes will be aligned as probability of success. Therefore, the preferred node selection
illustrated in Fig.6a. The distance between nodes is about 150 factors should take not only density but also RSSI level into
meters. In the second scenario, three nodes will move considerations. This would be the better solution in this
backward and forward as shown in Fig.6b. scenario. In the case that a node has the highest of RSSI level
According to the testing result, the reliability is 100% on and density, this node will be the next rebroadcast node. If
both of the scenarios in every test. Nevertheless, we have found there is a node that has a lower density not less than 5
the significant difference from overhead results. In some tests, compared to a node with the highest density and this node has
there are a large number of redundant retransmissions. The the higher RSSI compared to a node with the highest density,
overhead results are increased at least 5 times comparing to a this node will be selected. This can help the protocol to select
normal case. We investigate that the reason is from asymmetric the most efficient node to rebroadcast the message. For more
links. Specifically, a node that does not have the message will clarification on our mechanism, we provide pseudo code of
send beacon to others in its transmission range. So when others selection algorithm as shown in Fig. 7.
receive this beacon, they will try to send the missing message
back to that node. Unfortunately, the transmission range of the maxNumberOfNeighbor = 0
sending node is shorter than others. rssi = rssi level of first neighbor in neighbor table
nextRebroadcastNode
for( i = first neighbor -> last neighbor ){
if( number of Neighbor of neighbor(i) >=
maxNumberOfNeighbor ){
maxNumberOfNeighbor = number of Neighbor of
Neighbor(i)
if( RSSI level of neighbor(i) >= rssi ){
rssi = RSSI level of neighbor(i)
nextRebroadcastNode = neighbor(i)
}
}
(a) Static Node Scenario.
if( number of Neighbor of neighbor(i) >=
maxNumberOfNeighbor - 5){
if( RSSI level of neighbor(i) > rssi ){
rssi = RSSI level of neighbor(i)
nextRebroadcastNode = neighbor(i)
}
}
return nextRebroadcastNode

(b) Moving Node Scenario. Figure 7. Pseudo-code of Selection Algorithm

Figure 6. Real Experiments Scenarios

38
 IV. PERFORMANCE EVALUATION TABLE III. PARAMETERS SETTING
a
Message life time 10 seconds
A. Simulation Setup Number of Messages 5
To evaluate performance of our solution proposed in the Size of a broadcast message 512 bytes
Simulation Time 50 seconds
previous section, we need to change the propagation loss model
Maximum vehicle speed (km/h) 50, 80
and nodes’ transmission range. Because Two-Ray ground Vehicles density (veh/km) 2, 10, 30, 60, 120
propagation loss model is a function loss model that returns the Network Simulation NS-2.34, NS-3.10
exact value of receiving signal depending on distance between Wireless Communacation • NS-2 : 250m / Two-Ray Ground
nodes, it cannot provide the realistic asymmetric link scenarios. (Max Transmission Range (m)/
• NS-3 : 250m / Two-Ray Ground
Therefore, we use Nakagami propagation loss model that is the Propagation Loss Model)
• NS-3 : 250m / Nakagami
famous propagation loss model for VANETs simulation. The • NS-3 : 150m, 250m / Two-Ray
simulations consist of 2 groups of nodes; nodes with Ground
transmission range 250 meters and nodes with transmission • NS-3 : 150m, 250m/ Nakagami
range 150 meters. The highway scenarios used in Section II are
a. An interval from packet was released until packet is expired
used in this simulation. The simulations have been setup as
shown in TABLE III.

B. Metrics
The following two metrics are considered for performance
evaluation.
• Reliability is measured as a percentage of number of
nodes that received the message at the end of simulation.
• Overhead is measured from bandwidth consumption
which is from message retransmission and it is
normalized with reliability.
The results will be averaged from total messages in
simulations.
C. Simulation Results Figure 8. Reliability Results

The reliability results and overhead results are shown in

Fig.8 and Fig.9, respectively.
Reliability : In Fig.8, we can notice that nodes with
different transmission range and propagation loss model can
affect to protocol performance. This shows the problem caused
by asymmetric links. However, in higher density scenarios,
nodes become close so the problem of asymmetric links is not
severe. That’s why the reliability in high density scenarios
increases. Our RSSI solutions can provide more reliability than
original protocol’s mechanism. The results show that in same
scenarios RSSI information can increase the reliability up to
5%.
Figure 9. Overhead Results
Overhead : In Fig.9, the original protocol have much more
overhead in the asymmetric link scenarios than normal scenario structure from NS-2 to NS-3 in general. Then, we select DECA
especially in low density scenarios. Our solution can decrease which is a reliable broadcast protocol on VANETs. DECA is
retransmission overhead. It can increase reliability with lower already previously implemented in NS-2. So we implement it
retransmission comparing to the original protocol. This is on NS-3 according to our outline structure and validate the
because our solution tries to maximize number of received performance. Moreover, we evaluate the protocol on emulation
nodes by selecting the highest RSSI level node. mode to discover the problem that cannot be found on
simulation. The emulation results show the asymmetric link
V. CONCLUSION problem which affect to the protocol efficiency. This problem
can be occurred due to signal interference between devices and
Most of researches have been evaluated on NS-2 which is obstacles or the different wireless devices. But it can hardly be
available for simulation. But NS-3 has more ability than its noticed in such a simulation scenario. We propose a simple
predecessor, NS-2. NS-3 provides emulation ability so solution to use RSSI information to be an option for any
researchers can implement their work only one time, and then protocol to make better decision. Then we simulate it with
they can evaluate their works on both simulation and asymmetric link scenarios. The simulation results show that
emulation. In this paper, we elaborate outline implementation our mechanism can help the protocol to increase its reliability

39
up to 5% and decrease its retransmission overhead that [6] N. Na Nakorn, and K. Rojviboonchai, “DECA: Density-Aware Reliable
normalized with reliability. In future works, we will improve Broadcasting in Vehicular Ad-Hoc Networks,” IEEE the 7th ECTI-
CON2010, Chiang Mai, Thailand, May 19-21, 2010.
the real testbeds to be more realistic on vehicular scenarios and
[7] K. Na Nakorn and K. Rojviboonchai, “POCA : Position-Aware Reliable
improve performance of protocols with mechanisms that Broadcasting in VANET,” to be appear in Proc. the 2nd Asia-Pacific
concern more about the noise signal and the obstacles between Conference of Information Processing APCIP2010, Nanchang, China,
nodes. September 17-18, 2010.
[8] V. Naumov and T. R. Gross,"Connectivity-Aware Routing (CAR) in
Vehicular Ad Hoc Networks," IEEE the 26th International Conference
ACKNOWLEDGMENTS on Computer Communications (INFOCOM’07), Anchorage, Alaska,
USA, May 6-12, 2007.
This research was supported in part by the TRF (Thailand [9] O. K. Tonguz, N. Wisitpongphan and F. Bai, “DV-CAST: A distributed
Research Fund) [MRG5380164], Thailand, and the vehicular broadcast protocol for vehicular ad hoc networks,” Proc. IEEE
Ratchadaphiseksomphot Endowment Fund, Chulalongkorn Wireless Communications, Vol.17 April, 2010.
University, Bangkok, Thailand. The authors would like to [10] E. Weingartner, H.vom Lehn and K. Wehrle, "A Performance
Comparison of Recent NetworkSimulators," IEEE International
thank Prof. Dr. Prabhas Chongstitvatana for his useful Conference on Communications(ICC '09), 2009.
discussion on this work. [11] T. R. Henderson, S. Roy, S. Floyd and G. F. Riley, “ns-3 Project Goals,”
Available at : www.nsnam.org/docs/meetings/wNS-2/wNS-2-NS-3.pdf ,
REFERENCES October 10,2006.
[12] “Simulation of Urban MObility (SUMO).” Internet:
[1] A. Varga, "The OMNeT++ discrete event simulation system." Internet: http://sumo.sourceforge.net/.
http:// www.omnetpp.org,2001.
[13] “Traffic and Network Simulation Environment (TraNS).” Internet:
[2] QualNet, "Scalable Network Technologies, Inc." Internet: http://lca.epfl.ch/projects/trans /.
http://www.scalable-networks.com, 2008.
[14] Jeongkeun Lee; and et al , "RSS-based Carrier Sensing and Interference
[3] OPNET, “Technologies Inc. OPNET modeler website.” Internet: Estimation in 802.11 Wireless Networks," Sensor, Mesh and Ad Hoc
http://www.opnet.com/solution/network\_rd/modeler.html. Communications and Networks, 2007. (SECON '07). 4th Annual IEEE
[4] “The network simulator NS-2.” Internet: http://www.isi.edu/nsnam/ns/. Communications Society Conference on , pp.491-500, 18-21 June 2007
[5] “The network simulator NS-3.” Internet: http:/www.nsnam.org/.

40