Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Contents lists available at ScienceDirect

Renewable and Sustainable Energy Reviews

journal homepage: www.elsevier.com/locate/rser

Evaluating the sustainability of urban passenger transportation by Monte T

Carlo simulation
⁎
Vanessa de Almeida Guimarãesa, , Ilton Curty Leal Juniorb, Marcelino Aurélio Vieira da Silvac
a
Centro Federal de Educação Tecnológica Celso Suckow da Fonseca - CEFET/RJ, Rua do Areal, 522, Sala 4D, Parque Mambucaba, Angra dos Reis 23.953-030, RJ, Brazil
b
Programa de Pós-Graduação em Administração, Universidade Federal Fluminense, Rua Desembargador Ellis Hermydio Figueira, 783, Bloco A, Sala 304, Aterrado, Volta
Redonda 27.213-145, RJ, Brazil
c
Programa de Engenharia de Transportes, Universidade Federal do Rio de Janeiro, COPPE/UFRJ, Centro de Tecnologia, Bloco H, sala 111, Cidade Universitária, Rio de
Janeiro 21.949-900, RJ, Brazil

A R T I C LE I N FO A B S T R A C T

Keywords: Evaluating the performance of transportation usually is based on average indicator values, incurring a risk of
Performance evaluation failure to diagnose the system's real performance. Besides this, despite the effort to improve sustainability in this
Sustainability sector, the evaluations generally focus on economic and environmental aspects, avoiding the social ones due to
Probabilistic behavior the difficult in measuring it. Therefore, the general objective of this article is to assess the performance of
Monte Carlo simulation
passenger transportation alternatives by applying Monte Carlo simulation to economic, social and environmental
indicators. For this purpose, we examined the alternatives for the route between the cities of Rio de Janeiro and
Niterói (located on opposite sides of Guanabara Bay), because of its importance to circulation in the Rio de
Janeiro metropolitan region. Besides this, we compared the results of the simulation with the performance
ranking obtained by applying average values of the selected indicators. We found that the use of mean values can
underestimate or overestimate the performance of the alternatives, besides not considering the uncertainty as-
sociated with the indicators. The collective transportation alternatives performed best in the assessment with
average indicators, with a tendency to show better performance levels in the probabilistic evaluation. Finally,
the indicator with the strongest influence on the performance of the individual transportation alternatives was
cost, while trip time was most relevant in the collective alternatives.

1. Introduction dispersion of daily activities in relation to residential areas increases

demand for mobility [7–9]. In Brazil, 92% of passenger flows are by
The sustainability of the transportation sector has attracted a great roadway and most cities face intense traffic problems [10], so the
deal of interest by the scientific community [1] and public policy- search for more sustainable transportation alternatives is particularly
makers [2], because at the same time that transport activities have urgent since the impacts of transportation are generally more acute in
economic importance, they also have environmental and social impacts urban regions [8].
[3,4]. Therefore, sustainable management of transport systems should One of the challenges of sustainability is performance assessment, in
be integrated with the strategies for sustainable development in general which respect studies [7,11,12] in general use deterministic values of
[5]. performance indicators to represent the system's behavior, given the
For a transportation system to be considered sustainable, it should difficulty of obtaining data. However, statistically speaking these values
meet the needs of commercial relations, be easily accessible to the may not be sufficient to reflect the true dimensions of the performance
population and at the same time be structured so as to minimize energy of a given transportation alternative [13,14]. The use of means, for
consumption and carbon emissions, besides being safe and assuring instance, if related to high standard deviation, can lead to partial
high quality of life [6]. As framed by Santos et al. [2], sustainability in conclusions that may not represent the case under assessment in its real
the transport sector should be examined considering the social, en- perspective.
vironmental and economic aspects (a position corroborated by Litman Therefore, simulation techniques can be applied to consider the
[1] and Guimarães and Leal Junior [7]). uncertainties of performance assessments, since they adopt the con-
In the case of passenger transportation in urban centers, the fidence intervals – a numerical interval that represents an estimated

⁎
Corresponding author.
E-mail addresses: [email protected]ff.br (V.A. Guimarães), [email protected]ff.br (I.C. Leal Junior), [email protected] (M.A.V. da Silva).

https://doi.org/10.1016/j.rser.2018.05.015
Received 7 May 2017; Received in revised form 30 April 2018; Accepted 13 May 2018
Available online 30 June 2018
1364-0321/ © 2018 Elsevier Ltd. All rights reserved.
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Nomenclature Solution
C Cargo
MCS Monte Carlo simulation P Passenger
RJMR Rio de Janeiro metropolitan region B5 Name of the blend of petroleum diesel oil and a percentage
AHP Analytic Hierarchy Process 5% of biodiesel
GRA Grey Relational Analysis UN United Nations
TOPSIS Technique for Order Preference by Similarity to Ideal

parameter of the population [15] – instead of means values of the the results; and (iv) final considerations.
variables. Then, it comes to Monte Carlo simulation (MCS), a technique
used in many areas of science to simulate problems that can be re-
2. Sustainability involving transportation
presented by stochastic processes [16].
MCS relies on a sequence of random numbers to carry out a simu-
The discussion of sustainable development, originally focused on
lation [16]. This enables obtaining a probability distribution of the
consumption of resources, is now defined more broadly as involving a
performance of transportation alternatives regarding sustainability in-
combination of economic and social well-being, fairness, human health
stead of a single value to represent this performance. Thus, the decision
and environmental integrity [3]. Therefore, the planning of sustainable
making process might become more accurate since the manager would
transportation entails recognition that the decisions in this sector cause
decide based on a set of possible performances related to the several
varied impacts that need to be considered in the decision-making pro-
values that an indicator can have in a interval of data. Then, it also
cess [1].
could picture more scenarios than the classic sensitivity analysis, de-
Besides covering economic aspects, many works discuss the en-
pending on the number of simulations performed (as presented in the
vironmental impacts of transportation systems or modes [12,19–24],
literature discussion in Section 2).
while social aspects are examined by others [4,8,18,25–27]. We sear-
Considering the possibility of modal choice (and of the transporta-
ched the international literature for works on sustainability of transport
tion alternatives that compose each mode), the following question
systems, finding that the theme has attracted a good deal of attention
arises: How accurate would estimation of the sustainability of urban
from the scientific community. The discussion is concentrated on the
passenger transportation alternatives be considering that the system has
proposal of indicators to assess the sustainability in this sector.
probabilistic behavior?
From the 96 papers raised, we evaluated those that consider, at
Therefore, the main objective of this article is to assess the perfor-
least, three dimensions of sustainability (economic, social and en-
mance of passenger transportation alternatives based on application of
vironmental), as presented in Table 1. We verified that the theme is
Monte Carlo simulation to the economic, social and environmental in-
investigated either to passenger or freight transportation. Further Boj-
dicators. For this purpose, we focus on the alternatives for the route
kovi et al. [28], Litman [1] and Litman and Burwell [3] evaluate these
between the cities or Rio de Janeiro and Niterói, chosen because of its
two modalities of transportation in their research.
importance to the circulation of people in the Rio de Janeiro me-
Besides the traditional dimensions of sustainability, some authors as
tropolitan region [17]. This route can be covered basically by two
Litman [1], Rajak et al. [33] and Miller et al. [34] propose other ca-
means: (i) private vehicle or bus over the Rio-Niterói Bridge; or (ii)
tegories that should be included in the assessment, such as: Transpor-
passenger ferry between the downtown regions of the two cities.
tation System Effectiveness and Good Governance and Planning. In the
The specific objectives are: (i) to formulate indicators that can be
other hand, although [7] deal with the traditional dimensions, the case
used to evaluate the sustainable performance of the passenger trans-
of study only considers the environmental and economic ones.
portation alternatives between the two cities; (ii) to apply the method
All papers in Table 1 propose and/or apply indicators related to
proposed by Guimarães and Leal Junior [7] to evaluate the sustainable
different dimensions of sustainability. Figueroa and Ribeiro [18],
performance of those alternatives; and (iii) to compare the results found
Litman [1], Litman and Burwell [3], Santos and Ribeiro [4] and Rajak
with MCS to those obtained by application of mean values.
et al. [33] perform a literature review and propose indicators to assess
This research is relevant since the performance evaluation in
the sustainability of the transportation system or specific modes. The
transport sector usually applies scenarios or sensitivity analysis to deal
others go beyond and evaluate the sustainable performance in transport
with the stochastic behavior of the data; while MCS can combine a huge
with some of the proposed indicators.
set of possibilities (100,000 cases, in this paper), ranging the values of
There is certain coherence in relation to the indicators. Regarding
different indicators at the same time. Then, future academic researches,
the economic dimension, they are mainly related to productivity, im-
feasibility projects or new public polices in transport could be more
pacts of transport activity in GDP, costs and investments in transpor-
accurate if they adopted MCS in the assessments.
tation. With respect to the environmental aspect, the papers deal mostly
Besides, including the social dimension in the performance analysis
with energy consumption, atmospheric emissions, noise pollution and
is a great contribution of this paper, considering that the literature
land use. Concerning to social dimension, there is greater occurrence of
surveyed (and presented in Section 2) points out the difficult in in-
indicators related to: health and safety of population, mobility, acces-
corporate this aspect in the assessment due to the complexity in mea-
sibility, trip time and equity.
suring it.
Table 1 also shows that the papers consider the different transpor-
Moreover, sustainability is a topic of interest of different stake-
tation alternatives available, either to freight or passenger transporta-
holders due to the several kinds of negative impacts that come from the
tion. About the methodological concerns, we found that they apply
transport activities, crossing over other sectors (e.g. influencing the
literature review and content analysis methods, fuzzy logic, multi-
economic feasibility of a whole supply chain, or environmentally, by
criteria decision aid techniques (as GRA, AHP, TOPSIS) or a combina-
pressing for extra carbon credits needs). Therefore, as propelling of
tion of two or more techniques. Nevertheless, only Awasthi et al. [29]
economic development, transportation sector must be aligned to the
and Guimarães and Leal Junior [7] perform sensitivity analysis. Despite
sustainable principles [4,5,18].
of applying MCS, Miller et al. [34] did not generate the probability
The article is divided into four more sections after this introduction:
curves of performance with the sustainable indicators. The authors
(i) discussion of sustainability involving transportation; (ii) description
analyzed the changes in the performance rankings by modifying the
of the methodological procedures; (iii) presentation and discussion of
weights of the selected variables.

733
 Table 1
Studies of transportation sustainability.
Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

[8] Analysis of the sustainability of the urban • Economic; Social; Environmental Coverage ratio for public Railway and roadway Normalization and Cluster Analysis D 1
passenger transportation systems of 23 transport;
European cities based on available indicators. Ratio between cost of transport
for user and GDP per capita;
Time spent travelling per
capita;
Traffic fatalities per capita;
Public transport network
density;
Reduction of public transport
fares for students and elderly;
Land use for transport
infrastructure;
Public transport energy
consumption per user;
Public transport emissions per
user.

[29] Analysis of alternatives to attain a sustainable • Economic; Social; Environmental Operating costs; Roadway(3) Fuzzy TOPSIS (Multicriteria) and D 1
passenger transportation system. Costs of the operator for sensitivity analysis
running the transportation
service;
Safety offered by the

734
transportation system;
Security from theft and
vandalism offered by the
transportation system;
Ability to perform the promised
service dependably and
accurately;
Air pollutants from the
transportation system;
Noise from the transportation
system;
GHG emissions from the
transportation system;
Usage of fossil fuels, e.g. petrol,
diesel;
Costs for travelling between
any given stations;
Waste from road transport, e.g.
number of end-of-life vehicles,
number of used tires;
Energy consumption by the
transportation system;
Land space used for running the
transportation service;
Access to residential areas,
activity areas and other
transportation modes;
(continued on next page)
Renewable and Sustainable Energy Reviews 93 (2018) 732–752
 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

Benefits to economy from the

transportation mode e.g. labor
employment;
State of the art of technology,
equipment and infrastructure
employed by the transportation
mode;
Equity across genders, age
groups, handicapped people;
Ability to expand the service, if
required;
Ability to service over the
transportation area;
Ability to achieve performance
targets;
Capacity utilization of
transportation mode;
Convenience in using the
transportation service;
Quality of service provided by
the transportation employees.

[28] Evaluation of the sustainability of the • • Economic; Social; Environmental Energy intensity; Railway, roadway and Analytic Hierarchy Process D 1
transportation systems of 12 countries in Car share of inland passenger waterway (considering the

735
Central Europe, by applying the analytic transport; system as a whole and not
hierarchy process (AHP) to sustainability Road share of inland freight modes separately)
indicators. transport;
Volume of freight transport
relative to GDP;
Volume of passenger transport
relative to GDP;
Investments in transport
infrastructure;
GHG emissions;
People killed in road accidents;
Emissions of particulate matter
from transport;
Emissions of ozone precursors.

[18] Examination of the relation between energy • Economic; Social; Environmental Reduced travel time, travel cost Roadway Systematic literature review NA(4) NA
policies (seeking to reduce oil consumption and trip uncertainty;
and greenhouse gas emissions) and Percent of lane-km by
sustainable development goals. pavement and infrastructure
maintenance expenses;
Indirect jobs supported
(created);
Lost time due to congestion;
Reduced access time from
facility to destination;
Good Conditions for biking/
walking;
Passenger trips per household;
Renewable and Sustainable Energy Reviews 93 (2018) 732–752

(continued on next page)

 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

Public acceptability and

participation in decision
making process;
Reduction of road traffic deaths
and injuries;
Access provided for the elderly
and people with disabilities;
Public health benefits from
increased physical activity;
Air pollution level reduction;
Noise level reduction;
Land use area;
Biodiversity protection;
GHG emission reduction;
Alternative fuel consumption;
Sector is secure, ready and
resilient to threats and hazards;
Percentage of reduction in use
of fossil fuels.

[7] Proposal of a method to evaluate the • Despite the approach of the three Number of passengers moved; Roadway and waterway Grey Relational Analysis and D 1
performance of passenger transportation dimensions, the illustrative Total energy use; sensitivity analysis
systems. application of the method considered Total use of non-renewable
only environmental and economic energy;

736
CO2 emission;
Total atmospheric pollutant
emission.

[25] Evaluation of the impact of public and private • Economic; Social; Environmental Cost per person-km; Railway and roadway Comparative analysis of indicators D 1
transportation on the sustainability of the Estimated impacts of air
metropolitan region of Toronto. pollution;
GHG emissions;
Energy intensity;
Level of service;
Employment;
Accidents;
Insurance Costs.

[1] Discussion of how sustainability indicators can • • Economic; Social; Environmental and The author presents a set of 41 Not specify but applies to Literature research and Content NA NA
be used to assess and plan transportation Good Governance and Planning indicators related to the any mode of transport analysis
systems. following sustainability
objectives:
Economic productivity;
Economic development;
Energy efficiency;
Affordability;
Operational efficiency;
Equity / Fairness;
Safety, security and health;
Community development;
Heritage protection;
(continued on next page)
Renewable and Sustainable Energy Reviews 93 (2018) 732–752
 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

Climate stability;
Air pollution prevention;
Noise prevention;
Water pollution;
Open space preservation;
Good planning;
Efficient Pricing.

[3] Identification of questions related to the • • Economic; Social; Environmental The author presents a set of 23 Not specify but applies to Literature research and content NA NA
definition, evaluation and implementation of indicators related to the any mode of transport analysis
sustainable transportation. following sustainability
objectives:
Accessibility;
Transport diversity;
Facility Freight Planning;
Safety;
Health and fitness;
Community liveability;
Equity;
Citizen involvement;
Climate change emissions;
Other air pollution;
Noise pollution;
Water pollution;

737
Habitat protection;
Resource efficiency.

[30] Evaluation of the sustainability of navigation • Economic; Social; Environmental Accessibility; Waterway Literature research for selection and D 1
in Europe via the Rhine River, based on Transport operation; descriptive analysis of indicators
selected indicators. Productivity and efficiency;
Costs to economy;
Benefits to economy;
Resource use;
Direct ecological intrusion;
Emissions to air;
Emissions to soil and water;
Noise;
Waste;
Safety and security;
Working condition in transport
sector.

[4] Selection of 20 sustainability indicators and • Economic; Social; Environmental CO2 emissions; Railway and roadway Literature research NA 1
evaluation of whether they are applicable to Land consumption for transport
monitor the transport actions that compose infrastructure;
the Climate Plan of the State of Rio de Janeiro.
Per capita energy consumption;
Air and noise pollution
exposure and health impacts;
Vehicle travel by mode;
Land use density;
Per capita congestion costs;
(continued on next page)
Renewable and Sustainable Energy Reviews 93 (2018) 732–752
 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

Total transport expenditures;

Household expenditure
allocated to transport;
Expenditures on transportation
for local government;
Transparency of costs and
investments;
Harmful subsidies and green
fiscal policies;
Transport system diversity/
transportation variety;
Quality of transport for
disadvantaged people;
Access to public transport;
Fatality and injured of traffic
accidents;
Satisfaction of citizens and
variety and quality of transport
options;
Safety;
Health;
Gender equality/equity
between societies and groups.

738
[31] Evaluation of 15 transportation strategies in • Economic; Social; Environmental Alternative and renewable Roadway Fuzzy Cognitive Maps and the D 5
the city of Tapei (associated with the use of energy consumption; Analytic Hierarchy Process
clean energy, land use and modal transport), Modal split of transit;
applying sustainability indicators. Emission intensity of GHG;
Emission intensity of air
pollutants;
Proximity of transport
infrastructure;
Traffic accidents;
Mobility and transport for
elderly and disabled persons;
Transport infrastructure in
remote areas;
Transit subsidy in remote areas.

[27] Proposal of a method to evaluate car fleets of a • Economic; Social; Environmental Indicators related to: Roadway Process Analysis Method (proposed D 1
determined city, based on the sustainability Atmospheric emissions; by the authors)
concept. Use of material;
Water contamination;
Workforce;
Accident costs;
Health costs caused by
environmental impacts;
Jobs generated;
Passengers transported;

People killed or hospitalized for

traffic accidents.
Renewable and Sustainable Energy Reviews 93 (2018) 732–752

(continued on next page)

 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

[32] Development of an emission inventory of ships • Economic; Social; Environmental Social cost; Waterway Activity-based methodology and D 1
also considering the associated social cost, and Emission social cost per data based on automatic
analysis of the performance of the Port of throughput; identification system-were used in
Yangshan in light of the eco-efficiency the study for the emissions
Emission social cost per ship
concept. inventory
call;
Emission social cost per port
revenue;
Total GHG emissions;
Total atmospheric pollutants
emissions.

[33] Present a model for transport sustainability • Economic; Social; Environmental; The author presents a set of 60 Not specify but applies to Fuzzy logic D 1
performance evaluation Transportation System Effectiveness. indicators related to: any mode of transport
Affordability;
Climate change prevention and
mitigation;
Community cohesion;
Community livability;
Cultural preservation;
Economic development;
Economic productivity;
Equity/Fairness;
Government efficiency;

739
Human safety, security and
health;
Hydrologic impacts;
Improve mobility;
Improve system performance;
Non-Renewablere source
conservation;
Open space preservation and
biodiversity protection;
Pollution prevention;
Resource efficiency;
Stakeholder involvement;
Traffic congestion;
Transportation reliability.

[34] Introduce the Public Transit Sustainable • Economic; Social; Environmental; Low Income Population; Railway, roadway and Public Transit Sustainable Mobility D 6
Mobility Analysis Tool framework, which uses Transportation System Effectiveness. Population; waterway (considering the Analysis Tool, Normalization and
composite sustainability index techniques Jobs; system as a whole and not Monte Carlo Analysis
along with research into transport Collision cost savings; modes separately)
sustainability to propose a new transit analysis Capital;
tool that can be used in both planning/ Operating;
decision making and research contexts. Contributions GDP;
Travel Time;

(continued on next page)

Renewable and Sustainable Energy Reviews 93 (2018) 732–752
 Table 1 (continued)

Authors Objective Focus(1) Dimensions of sustainability Indicators Transport Alternatives Method Stoc./ Scenarios
Det.(2)
C P
V.A. Guimarães et al.

Change in transit GHG;

Transit GHG from construction;
Total pollutants emissions;
Trip generation;
Modeshare.

[35] Examine an integrated approach for assessing • Economic; Social; Environmental. Energy efficiency; Not specify but apply to any Surveys and Fuzzy Analytic D 1
the sustainability of the current transportation Purchasing Power; mode of transport Hierarchy Process
system design, based on a policy making Business opportunities;
problem, aiming at providing decision makers Public finance;
with a framework allowing them to choose the Pollution;
most eco-responsible policy amongst many Recyclability;
alternatives. Ecological footprint;
Energy used;
Affordability;
Accessibility;
Information system;
Technology acceptability;
Mobility.

740
Notes: (1) C - cargo; P – passenger; (2) S – stochastic; D – Deterministic; (3) Do not specify, but mention the transport system and some indicator related to the roadway. (4) NA - not applicable Notes: (1) C - cargo; P –
passenger; (2) S – stochastic; D – Deterministic; (3) Do not specify, but mention the transport system and some indicator related to the roadway. (4) NA - not applicable.
Renewable and Sustainable Energy Reviews 93 (2018) 732–752
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Table 2
Indicators to evaluate the sustainability of passenger transportation.

Aspects Attributes Indicators Measurement Unit

Economic Monetary Value Monetary value associated with volume of passengers carried R$
Total cost to carry passengers (operator’s perspective) R$
Total cost to carry passengers (users’ perspective) R$
Costs of congestion R$
Variable cost associated with individual transportation R$
Fixed cost associated with individual transportation R$
Cost of collective transportation R$
Government revenue generated by taxes (direct and indirect) associated with transportation/ R$
mobility
Government revenue generated by direct taxes associated with transportation/mobility R$
Government revenue generated by indirect taxes associated with transportation/mobility R$

Service Produced Number of passengers carried Quant.

Quantities of physical resources (equipment and/or installations) used to provide transportation Quant.
services

Environmental Greenhouse Gas Emissions Emission of CO2 t

Emission of CO2-eq (equivalent) t
Emission of water vapor t
Emission of CH4 t
Emission of N2O t
Emission of CFC t
Total emission of greenhouse gases t

Atmospheric Pollutant Emissions Emission of hydrocarbons t

Emission of aldehydes t
Emission of CO gas t
Emission of acidifying gases (SOx, NOx) t
Emission of particulate matter t
Average vehicle age Years

Energy Use Total energy consumption MJ

Consumption of renewable energy MJ

Land Use Proximity of transportation infrastructure to environmental preservation areas Km

Total area occupied by transportation infrastructure Km2

Sound Pollution Intensity of noise emitted by vehicles Db

Water Pollution Quantity of wastes generated by transportation operations t

Quantity of oil leaked from vehicles l
Number of end-of-life vehicles Quant.

Social Motorization Number of cars in a determined region Quant.

Number of non-motorized trips Quant.
Number of motorized trips Quant.

Mobility1 Total number of trips Quant.

Number of trips taken by elderly people2 Quant.
Number of trips taken by handicapped people2 Quant.
Number of trips taken by low-income people2 Quant.
Number of trips taken by individual transportation Quant.
Number of trips taken by collective transportation Quant.

Safety Total number of accidents Quant.

Number of fatal accidents Quant.
Number of accidents with severe injuries Quant.
Total cost of accidents R$
Number of muggings and other crimes suffered in route R$

Accessibility Distance traveled Km

Distance traveled by individual transportation Km
Distance traveled by collective transportation Km
Distance traveled by non-motorized means Km
Number of residences with public transportation services within 400 m Quant.

Rapidity Total trip time h

Time to reach the nearest terminal h
Waiting time at terminals h
Time stopped in traffic h
Time for boarding and/or deboarding h
Total commuting time h

741
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Besides, a deterministic approach is adopted in the case studies, analyses [44,45]. It is a stochastic technique based on the use of
presenting sustainability indexes without consider the ranging in the random numbers and probabilities to investigate a determined problem
values of the indicators. Yet, Shiau [31] and Miller et al. [34] consider [14,46]. As stated by Vanorio and Mera [13], the use of probability
the stochastic behavior of the indicators by scenario analysis, while distributions allows representing the uncertainty of the variables more
other papers deal with only one scenario, considering single values to realistically.
the indicators. During MCS, the values of the input variables are selected randomly
However, according to Holden et al. [36], there is no consensus based on the probability distributions associated with them [13,46]. At
about the definition of sustainability in passenger transportation, so each iteration, the result is recorded, creating a probability distribution
further studies are necessary on this theme, as well as to identify and of the possible results and the probability of the occurrence of a certain
select appropriate indicators of sustainability for application to trans- event, in the case here, of sustainable performance [13,42].
portation questions [3,4,8,27,37–40]. Table 2 presents the indicators With respect to application of MCS in transport sectors, the majority
proposed in the literature to assess the sustainability of passenger of the works found apply it to assess the risks of accidents and other
transportation. undesired events [13,47,48], or the feasibility of projects
In relation to evaluation of transportation performance, the tradi- [14,41,44,49,50,51,52], or the planning of transportation systems,
tional approach focuses on economic and environmental aspects, re- especially regarding the capacity/reliability of the system and/or traffic
legating social aspect to the second echelon, if that (as presented by forecasts [42,46,53–58].
Markovich and Lucas [26]). Nevertheless, according to Alonso et al. [8] We also found applications of MCS to estimate greenhouse gas
and Guimarães and Leal Junior [7], to achieve sustainability in pas- emissions by transportation modes [59,60]. However, none of the
senger transportation, the environmental, economic and social dimen- works found address the assessment of the performance of transporta-
sions should be considered. tion systems (as a whole or parts), or consider the concept of sustain-
Besides this, average values are normally used to represent the in- ability. Thus, the application of the proposal put forward in this article
dicators. This can lead to mistaken decisions, since those values do not can lead to more informed decisions by policymakers.
incorporate the risk of uncertainties, and can either overestimate or
underestimate the real behavior of the system. According to Lemp and
3. Methodological procedures
Kockelman [41] and Salling [14], the normal approach is to conduct
analyses of sensitivity and/or scenarios to consider these uncertainties.
We consulted the academic literature and official documents to se-
Nevertheless, these techniques are also based on deterministic data, so
lect the performance evaluation method as well as to formulate ap-
they rarely capture the variability present in the systems [14]. In this
propriate economic, environmental and social indicators (Table 2) for
context, Monte Carlo simulation (MCS) can be a good alternative.
evaluation of the sustainable performance of the transportation alter-
Widely used in the sciences [16,42,43], MCS is a technique to
natives examined here as a case study.
support decision making that permits evaluating risk in quantitative
We clarify that to select the papers presented in Table 1 and the

STEP 1 STEP 2 STEP 3 STEP 4

INITIAL DEFINITIONS ATTRIBUTES AND WEIGHTS INDICATORS AND MEASURES PERFORMANCE STANDARDS

-Selection of the route and

Level 1 characterization of the
transport network. - Survey of the
Transport Level 1 attributes to be Level 1 - Survey of the indicators
-Definition of transport
Research considered in the study. Research associated with each
modes to be evaluated.
attribute
Level 2 - Types of displacements (or
Passenger travel) and its characteristics - Study of the benchmark performance
- Weighting of the levels for each measure
- Categories and aspects for aspects
Level 2
Level 3 evaluation; - Determination of
Selection and Level 2
Performance - Point of view of the - Selection, priorization indicators and measures
evaluation; weighting Choice
and weighting of the to be used
- Level of analysis; attributes
-Scope of the evaluation.
OUTPUT OUTPUT OUTPUT
OUTPUT
List of aspects, attributes and their Table with aspects, attributes, indicators Table or list with benchmark performance
Description of the basic parameters to
respective weights and measures for performance assessment levels for each measure
orient the performance evaluation

STEP 8 STEP 7 STEP 6 STEP 5

IMPROVEMENTS PROCESSING AND ANALYSIS DETERMINATION DATA COLLECTION
- Application of the
Level 1 - Sensitivity analysis Level 1
selected aggregation
Sensitivity Application technique Level 1
- Proposal of Measures - Calculation of
improvement actions Level 2 - Determination of the measures
Level 2 - Creating scenarios Determination performence level
Upgrading based on improvement
actions Level 3 - Performance analysis - Determination of the values of the
Analysis Level 2 - Selection of indicators
- Performance Aggregation aggregation technique
Level 3 simulation considering Level 4 technique to be used for
- Comparison with the
Simulation the improvement Comparison evaluation
benchmark(s) values
actions
OUTPUT
OUTPUT OUTPUT OUTPUT
New result of the evaluation considering
Result of performance assessment in Table with values of each measure and Table with results of the indicators
the implementation of improvement
passenger transport and its analysis. description of the selected technique
actions.

Fig. 1. Performance assessment and evaluation method for passenger transportation.

Source: [7].

742
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

n
indicators proposed in Table 2, we conducted a systematic review in- ∑i = 1 x i fi
volving Brazilian and international literature, not being restricted to a x = n for i:0…n
∑i = 1 fi (3)
specific database. Most of the articles published in international jour-
nals were obtained from Periódicos Capes – that is a Brazilian repository where: x i denotes the values of each alternative of each indicator; and fi
where researchers can find papers from different databases (e.g. Web of refers to the weights associated with each indicator.
Science and Scopus) and journals. The papers were searched con- We did not carry out the traditional analysis of scenarios (opti-
sidering the combination of the following keywords: “transport”; mistic, business as usual, pessimistic) in Step 7 of the method proposed
“passenger”; “passenger transport”; “sustainability”; “sustainable”; by Guimarães and Leal Junior [7], considering that MCS would offer
“performance”; “ecoefficiency”; "eco-efficiency"; “efficiency”; "perfor- more realistic results. The MCS was composed of 100 thousand simu-
mance assessment"; "performance evaluation”. Besides, we searched for lations, calculated with the @Risk software [62].
papers in specific journals (as Natural Resources Forum supported by Even this simulation can have margins for error in relation to the
United Nations – UN) related to the sustainable subject. Then, we se- results, since the number of iterations directly influences the accuracy.
lected the papers concerned only to passenger assessment and sus- To calculate the error, we applied Eq. (4), proposed by Fernandes [63].
tainability to performed the analysis. 3σ
The analysis is based on the method of Guimarães and Leal Junior ε=
N (4)
[7], presented in Fig. 1. In step 6 of the method, we used the normal-
ization proposed by Deng [61] to harmonize indicators expressed in where: ε = error; σ = standard deviation of the simulation; and N =
different measurement units, allowing comparative analysis. Besides number of iterations.
this, we performed normalization by amplitudes, which is suitable We believe that the combination of the normalization proposed by
when uncertainty exists in the data being analyzed. Deng [61] with MCS brought flexibility to the analysis, allowing eva-
For situations where the higher the indicator value the better (or luation of many situations, since the two techniques take into con-
more desirable) the result, we applied Eq. (1). sideration the shortage and uncertainty of information. To facilitate
discussion of the data, the results are presented in aggregate form,
x i (o) (k ) − min(x i (o) (k )) without indicating to what step and level of the method they belong.
∀i
x ′i (k ) = for i: 1 … m , k: 1 … n
max(x i (o) (k )) − min(x i (o) (k ))
∀ i ∀i (1) 4. Discussion of the results

where: x ′i (k ) is the normalized value of an indicator k for an original

To evaluate the proposal, we selected the route between the
observation x i (o) .
downtown areas of the cities of Rio de Janeiro and Niterói because of its
For situations where the lower the indicator value the better (or
importance for mobility in the Rio de Janeiro metropolitan region
more desirable) the result, we applied Eq. (2).
(RJMR), as shown by the data of the Rio de Janeiro Urban Transport
max(x i (o) (k )) − x i (o) (k ) Master Plan [17] and because it has different transportation alter-
x ′i (k ) = ∀ i
for i: 0 … m , k: 1 … n natives (Table 4). Data of the [17] indicate that the city of Rio de Ja-
max(x i (o) (k )) − min(x i (o) (k )) neiro is the main origin of trips in the RJMR (61.3%), while Niterói is in
∀ i ∀i (2)
fourth position (with 5.6%). However, Niterói is the city that accounts
To determine the sustainability index of each transportation alter- for the largest number of trips per inhabitant (2.45 trips/person), fol-
native (overall performance value), we used the weighted average of lowed by Rio de Janeiro with 1.99 trips/person.
the normalized values of the indicators (Eq. (3)). This index varies from In average, about 232,000 people circulate over this route daily
0 to 1 (the nearer to 1, the better the performance). [64], by road and passenger ferry [17], of which 34% ride the ferry. All

Fig. 2. Paths covered by the alternatives presented in Table 4. Note: (1) Waterway; (2) Road.
Source: Elaborated by the authors with GoogleMaps.

743
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Table 3 number of fatal accidents and total trip time. The values of the in-
Description of the transport alternatives under analysis. dicators were obtained from Guimarães [58] and are reported in
Source: [17] and [64]. Table 5.
Mode Type ID Alternative1 Fuel Maximum We applied Eqs. (1)–(3) to the mean values of Table 5 for definition
capacity [pass/ of the overall sustainability indexes, assigning the same weights to each
vehicle] aspect (0.333), since they have the same importance according to the
sustainability concept [5]. The results are presented in Fig. 3.
Road Individual A1 Flex car Gasoline 5
A2 Flex car Ethanol 5 To support the analysis of the data, we applied Pareto logic to
A3 Motorcycle Gasoline 2 classify the sustainability patterns of the alternatives, to determine: (i)
Collective A4 Bus Diesel (B5) 80 superior sustainability – sustainability indexes greater than or equal to
Waterway Collective A5 Ferry Maritime fuel 2000
0.8; (ii) intermediate sustainability – sustainability indexes greater than
or equal to 0.5 but smaller than 0.8; or (iii) inferior sustainability –
Notes: (1) Non-motorized transportation is not feasible due to the prohibition of
movement of pedestrian and bicycle in Rio – Niterói bridge. sustainability indexes less than or equal to 0.5.
As can be seen in Fig. 3, the collective transportation alternatives
Table 4
attained the best performances (above 0.6), followed by car burning
Parameters of the assessment. ethanol (0.503). Unexpectedly, ferry, the alternative with the largest
capacity, did not attain the best performance. Car burning gasoline and
Parameter Description
motorcycle presented similar performance (below 0.4), placing them
Passengers and their Those who travel to study, work and/or leasure the inferior sustainability category. None of the alternatives reached the
features superior sustainability level.
Network and transport Network and alternatives presented in Table 4 On this point it should be mentioned that the use of average values
modes to calculate the performance of the alternatives can underestimate (or
Category Sustainability
Aspects Economic, social and environmental
overestimate) the performance, since each indicator can have a range of
Standpoint Governamental values (as shown in Table 5), depending on the context, day of the
Analysis level Strategic week, time of day, type of vehicle and source of energy, among other
Scope Part of the transport system, considering all the factors.
existing alternatives in the route.
Therefore, with the aim of mapping the possible results of each al-
Temporal scope Not applicable
Geographical scope Praça XV - Araribóia route to ferry (5 km) and Terminal ternative and the corresponding probabilities of occurrence, we applied
Alfredo Agache - Terminal João Goulart (20,7 km) to MCS to the data of Table 5. Table 6 summarizes the main results ob-
the other alternatives tained for each alternative.
Note that according to the most frequent performance in the ex-
periment (the mode), the ranking of the alternatives would be similar to
told, we evaluated five transportation alternatives between the two that found using the average value of the indicators, with alteration of
cities (Fig. 2), whose characteristics are described in Table 3. The only the first two positions. However, a variation exists between the
parameters adopted are detailed in Table 4. possible minimum and maximum performance values of each alter-
Based on the availability of data, we selected the following in- native.
dicators from those in Table 2 for the sustainability aspects: (i) eco- This needs to be investigated, especially if the decision-maker is
nomic – total transport cost and government revenue generated by di- assessing new transportation policies that require prioritizing some al-
rect taxes on transportation/mobility; (ii) environmental – total ternative. It can be necessary to evaluate, for example, the probability
consumption of renewable energy and CO2-eq emission; and (iii) social –

Table 5
Indicators to evaluate the sustainable performance of the selected route.
Source: [58].
ID Economic Social

Total Cost Government Revenue Total trip time

[R$/pass/trip] [R$/pass/trip] [h/pass/trip]

Minimum Average Maximum Minimum Average Maximum Minimum Average Maximum

A1 1.069 145.052 620.287 0.020 0.172 0.348 0.296 0.585 1.285

A2 1.228 145.982 621.977 0.020 0.172 0.348 0.296 0.585 1.285
A3 0.651 98.690 388.142 0.019 0.045 0.053 0.296 0.470 0.664
A4 4.900 5.900 7.350 0.571 0.687 0.856 0.634 1.013 2.626
A5 3.500 3.800 5.000 0.323 0.351 0.462 0.214 0.744 1.116

ID Social Environmental

Fatal Accidents Consumption of renewable energy Emission of CO2-eq

[qtd/pass] [MJ/pass/trip] [kg/pass/trip]

Minimum Average Maximum Minimum Average Maximum Minimum Average Maximum

A1 0.054 0.056 0.058 1.703 7.397 10.720 632.597 2748.465 3983.021

A2 0.054 0.056 0.058 9.405 41.378 60.555 568.412 2500.655 3659.641
A3 0.729 0.751 0.774 1.044 2.592 3.096 387.822 963.072 1150.501
A4 0.019 0.020 0.021 0.095 0.262 0.624 156.971 431.656 1027.446
A5 1.2 × 10−5 1.3 × 10−5 1.3 × 10−5 0.000 0.000 0.000 154.651 499.196 1797.967

Note: To use the triangular distribution, the average value was considered as the peak value.

744
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 3. Ranking of sustainable performance of transport alternatives.

Table 6 being in the 0.6 – |0.7 range, while the most probable performance of
Summary results of Monte Carlo Simulation. ethanol car is in the 0.5 – |0.6 range, motorcycle is in the 0.4 – |0.5
Statistics Gasoline car Ethanol car Motorcycle Bus Ferry range and gasoline car is in the 0.3 – |0.4 interval. Only ethanol car and
bus have results in the 0.8 – |0.9 range.
Minimum 0.1660 0.3207 0.1718 0.4959 0.4089 No alternative is in the best performance interval (0.9 – |1.0), and
Maximum 0.7212 0.8547 0.5255 0.8165 0.7932 only gasoline car and motorcycle have any cases with performance
Mean 0.3983 0.5447 0.3916 0.6625 0.6765
Standard 0.0790 0.0797 0.0530 0.0182 0.0519
below 0.2 (0.26% and 0.01%, respectively).
deviation The coefficient of variation is also an important metric to be ana-
(sample) lyzed, since the greater this coefficient is, the higher the probability will
Variance 0.0062 0.0064 0.0028 0.0003 0.0027 be that the results found by using deterministic indicators will lead to
Coefficient of 19.83% 14.64% 13.54% 2.75% 7.67%
erroneous decisions. Besides this, analysis of the coefficient should be
variation
Median 0.3942 0.5411 0.3984 0.6624 0.6839 combined with evaluation of performance curves. A small coefficient of
Mode 0.3896 0.5206 0.4284 0.6627 0.6993 variation associated with performance concentrated in the 0.2 to |0.3
range, for example, indicates that the decision-maker will have diffi-
Note: The Tukey test was performed using the Portal Action supplement and it culty in promoting improvements in an alternative's performance, since
was found that the mean performances of the alternatives are statistically dif- the result will be relatively insensitive to alterations in the values of the
ferent.
indicators.
In turn, when a small variation is associated with performance le-
of the occurrence of extreme values (outliers), as well as in what per-
vels in the superior sustainability range (above 0.8), this would be a
formance range the highest probability of occurrence is contained.
positive situation, because even with variations, the sustainability index
Therefore, to facilitate the comparative analysis we divided the possible
would tend to be high. Therefore, since even analysis by classes can
performance values into classes, as presented in Fig. 4.
obfuscate certain results, we decided to analyze the performance of the
The collective transportation alternatives have the highest prob-
alternatives on a case-by-case basis, in disaggregated form.
abilities of attaining superior performance: bus and ferry have values
For the gasoline car alternative, there is a 90% probability of its
higher than 0.5 in 100% and 99.77% of the occurrences, respectively.
performance being concentrated between 0.269 and 0.531 (Fig. 5).
Besides this, the 0.5 - |0.6 range, for example, contains the best per-
However, in 89.5% of the cases, its performance is in the inferior sus-
formance by motorcycle and the worst performance by bus.
tainability category (lower than 0.5). Thus despite being possible, the
There is a higher probability of the performance of ferry and bus
“intermediate sustainability” category only contains 10.5% of the

Fig. 4. Comparison of the performance of transport alternatives (accumulated).

745
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 5. Distribution of gasoline car performance.

events. highest probability of occurrence (90%) of the gasoline car alternative.

The gasoline car alternative never reaches the superior sustain- Besides this, in 15.55% of the simulated results, gasoline car performs
ability level (index greater than 0.8), with its best index being 0.7212. worse than the lowest index attained by ethanol car.
Furthermore, for one combination of indicators, this alternative has the In approximately 70.95% of the possible combinations of the in-
worst performance among all of them (equal to 0.1660). dicator values, the ethanol car alternative is in the intermediate sus-
Comparison of the results of Fig. 5 with the performance presented tainability category. The probability of its being in the superior cate-
in Fig. 3 (by applying average values of the indicators) reveals its gory is only 0.06%, and its best performance is 0.8547.
performance was underestimated: in 73.63% of the combinations of Considering the results found in Fig. 3, the performance of the
indicators selected, gasoline car recorded performance greater than ethanol car alternative is also underestimated, since in 69.31% of the
0.349. However, the chances of attaining a result lower than the occurrences it presents better performance than the average result
average (26.37%) needs to be observed carefully by the decision-maker. (0.503). However, there is a relevant probability (30.69%) of the oc-
This substantial dispersion of the possible results is ratified by the currence of values below 0.503, something that needs to be investigated
coefficient of variation (19.83%), indicating that its performance has by the decision-maker.
the largest range of variation around the mean of all the alternatives In complementary fashion, this alternative has a high coefficient of
evaluated (Table 6). variation compared to the collective alternatives (14.64%). However,
This makes management decisions difficult, since the performance when compared to gasoline car, it has a higher probability of attaining
of this alternative is very sensitive to modifications of the variables. better performance levels, since 70.95% of the occurrences are in the
However, this also means that an improvement in the indicator can lead intermediate sustainability category, while for gasoline car this is true
to more sensitive improvements in its performance. It is important to for only 10.5% of the occurrences.
reflect, nevertheless, that this alternative should be prioritized for With respect to motorcycle, its highest probability of occurrence is
policy decisions, since its most probable performance is in the inter- concentrated in the interval from 0.301 to 0.469 (Fig. 7). In its worst
mediate sustainability category. performance, its sustainability indexes equal to 0.1718, while for ga-
In the ethanol car case, the highest probability of occurrence is soline car this figure is 0.1660 and for ethanol car it reaches 0.3207.
concentrated in the interval between 0.415 and 0.679 (Fig. 6). This Nevertheless, for practically all the possible results, the motorcycle al-
demonstrates a higher probability of this alternative having better ternative is in the inferior sustainability category.
sustainability performance than gasoline car. The worst performance The best performance of the motorcycle alternative (0.5255) be-
found for ethanol car (0.3207), for example, is in the interval with longs to the most probable range of performance values (90%) of the

Fig. 6. Distribution of ethanol car performance.

746
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 7. Distribution of motorcycle performance. Note: Motorcycle is the only alternative that has just two possible occupancy rates (one or two people). It could
explain why this was the only curve that did not follow the normal distribution format.

previous alternatives. For example, 5.68% of the occurrences for ga- intermediate sustainability category (Fig. 8). Additionally, this alter-
soline car are higher than 0.5255 (the highest being 0.7212). Besides native reached the superior sustainability level in only 0.01% of the
this, approximately 58% of the results of ethanol car are higher than the simulated cases. Although its best performance is lower than the result
best index reached by motorcycle (0.8547). for the ethanol car alternative (0.8165 versus 0.8547), in general the
In the calculation of performance using the mean values, in 52.62% bus alternative performs better since the probability of having perfor-
of the combinations the motorcycle alternative attains performance mance below 0.638 is 5%, while for ethanol this occurs in 87.12% of
higher than 0.395. However, in approximately 47% of the cases, mo- the cases.
torcycle has lower performance than that presented in Fig. 3. This de- The worst performance of the bus alternative is 0.4959, a figure
monstrates the possibility of making erroneous decisions based on di- higher than 99.58% of all the occurrences for motorcycle, 88.32% of
agnoses carried out with deterministic indicators (here average values). the possible performance levels of gasoline car and 26.67% of the
Finally, motorcycle has a coefficient of variation near that of possible results for ethanol car. Besides this, performance indexes below
ethanol car (14.64 versus 13.54). Therefore, consideration should go to 0.5 account for only 0.002% of the cases investigated. Finally, in
which alternative has the potential to present better results from the 69.42% of the cases, bus is higher than 0.661, demonstrating that the
sustainability standpoint, if it is necessary to prioritize the selection and result for the bus alternative also was underestimated in Fig. 3.
implementation of improvements. The bus alternative also presented the lowest coefficient of variation
Since the law in Brazil, based on the principles of sustainable mo- among the alternatives analyzed (2.75%). This small dispersion means
bility, establishes incentives for use of collective transportation alter- the possible results of this alternative are near those found in Fig. 3.
natives (Law 12,587/12 [65]), if a public manager had to choose in- However, it also indicates that the bus alternative needs significant
centive policies for only one individual transportation mode to compose improvements in its indicators to obtain a small increase in its sus-
the set of sustainable alternatives for this origin-destination pair of ci- tainability index.
ties, the best choice would be ethanol car in light of the possible per- In light of this finding, a manager needs to evaluate whether the
formance levels presented. effort (know-how, financial, institutional, among other aspects) neces-
With respect to the bus alternative, 90% of the possible performance sary to improve the performance of this transportation alternative
levels are concentrated between 0.638 and 0.697, putting it in the would be viable, or if modifications in another individual or collective

Fig. 8. Distribution of bus performance.

747
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

alternative might produce better improvement of the sustainability larger alterations in performance in relation to the mean). In contrast,
index. the performance of the collective transportation alternatives is not
For the ferry alternative, the range of most probable occurrences highly sensitive to this indicator. Instead, it is most strongly influenced
(90%) varies from 0.579 to 0.748 (Fig. 9). Like for the other alter- by the “trip time” factor. This factor also has the second strongest in-
natives, there is a combination of alternatives that classify it in the fluence on the performance of the two car alternatives. The “fatal ac-
inferior sustainability category (although in only 0.23% of the cases). cident” indicator is not relevant for any of the alternatives analyzed,
Nevertheless, its lowest index is 0.4089, which is in the range of most while “trip time” and “CO2-eq emission” have an influence, to a greater
probable values (90%) of the gasoline car and motorcycle modes. or lesser extent, on all the alternatives.
The probability of the motorcycle alternative performing worse than It should be noted that the performance of a determined alternative
the ferry, for example, is 56.46%, followed by gasoline car with 53.03% can be influenced by the performance of the others, as well as by the
and ethanol car with 4.27%. The bus option, in turn, never performs values of its indicators. This happens because due to the normalization
worse than 0.4089. used, the performance is comparative. Therefore, the improved per-
This alternative never reaches the superior sustainability category, formance of a given alternative can cause worse performance of an-
because its best performance is 0.7932. However, values greater than other, even without a change in the values of the latter's indicators.
0.697 (the upper limit of the range concentrating 90% of the results, In the case of gasoline car, environmental factors are the most nu-
considering the other alternatives) account for only 5.07% of the pos- merous among those with influence on its result, although the most
sible results for bus travel, followed by 3.33% for ethanol car and relevant are economic (total cost) and social (trip time). The economic
0.008% for gasoline car, while approximately 39.33% of the results for factor has the strongest influence on the performance of this alternative
ferry are above this value. This indicates that, although not reaching the (the best average output result could be reached with an alteration in
level of superior sustainability, the ferry alternative has a much higher the total cost of gasoline car use, reaching 0.479), while the social
probability of performing better than the other alternatives evaluated. factor has the largest negative influence (able to reduce the average
Considering the performance levels found with application of performance to its worst result – 0.332). Besides this, the results for bus
average values of the indicators, the result for ferry was under- and motorcycle impact the performance of this alternative. “Total cost”,
estimated, since in 68.76% of the cases its performance is superior to “CO2-eq emission” and “renewable energy consumption” are the most
0.657. Finally, just as for bus, the coefficient of variation of ferry is frequent indicators, as shown in Fig. 11.
small (7.67%) compared to the individual transportation alternatives. Therefore, if the intention is to improve the performance of the
Fig. 10 facilitates comparison of the results. The overlaps demon- gasoline car mode, priority should be given to actions that allow re-
strate that the alternatives can have the same performance, depending ducing the total cost and CO2-eq emission of this mode, with secondary
on the combination of indicators, but the probability of occurrence is attention to trip time and the total cost of the ethanol car alternative (a
different. lower ethanol car cost has a negative influence on gasoline car per-
For example, all the alternatives can have performance in the range formance).
0.5 - |0.6, although this is most likely to occur for ethanol car (46.35%). From the standpoint of management of the system, priority should
Likewise, with the exception of motorcycle, the other alternatives can be given to actions that allow improving the system as a whole rather
have performance levels in the 0.7 - |0.8 range, with ferry having the than penalizing one alternative to benefit another, although the lit-
highest probability of this occurring (37.33%), followed by bus (4.6%), erature generally encourages restrictions on individual transportation
ethanol car (2.97%) and gasoline car (0.01%). to favor collective transport (e.g., [27]). However, as already eluci-
Besides this, through the @Risk software [62], it is possible to dated, individual alternatives (such as ethanol car between the city pair
identify which indicators influence the result of each alternative, on in analysis) also can attain sustainable performance levels. Hence,
average, as presented in Figs. 11–15. The position of the bars on the managers should try to find a balance between collective and individual
right side indicates a positive influence on the result, because the alternatives to achieve a sustainable transportation system.
average is shifted to a higher value than that presented in Table 6. In In relation to ethanol car, it is most often affected by economic
turn, shifts to the left indicate a negative influence, reducing the factors, although only one of them is among the five indicators with
average. strongest influence. “Total cost” and “CO2-eq emission” have the
Note that the “total cost” factor has the strongest influence on the strongest positive impact on the results (able to shift the average per-
result of the individual transportation alternatives (ability to cause formance levels of ethanol car of 0.626 and 0.617, respectively). The

Fig. 9. Distribution of ferry performance.

748
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 10. Comparison of the performance of transport alternatives (relative).

Note: The results found can be considered accurate since the margin of error, calculated by the application of Eq. (4), for all the alternatives, is less than 1%.

“trip time” factor, in turn, has the strongest negative influence (able to Regarding bus, environmental and social factors are preponderant,
reduce the average from 0.5447 to 0.504). Besides this, the results for while the economic aspect does not directly influence the performance
bus and ferry affect the performance of this alternative. “Total cost” is of this alternative. This influence is indirect, since the economic in-
the most frequent indicator, followed by “CO2-eq emission” and “trip dicators compose the set of indicators that determine the performance
time”, as shown in Fig. 12. Therefore, analogously to the gasoline car of the alternatives under analysis. The “trip time” indicator has the
case, if the intention is to improve the performance of ethanol car, greatest positive influence, while “CO2-eq emission” is the main negative
actions to reduce its total cost and CO2-eq emission should be prior- influence. Besides these indicators, only the results of motorcycle and
itized. renewable energy consumption directly influence the performance of
With respect to motorcycle (Fig. 13), environmental factors are most the bus alternative, as shown in Fig. 14.
frequent, although the three most relevant indicators are economic With respect to ferry, social and environmental factors are pre-
ones. “Total cost” and “CO2-eq emission” have the greatest positive in- ponderant, with the highlights being “trip time” and “CO2-eq emission”.
fluence on the results, while “trip time” has the strongest negative in- This is the only alternative whose relevant economic indicator is
fluence. Nevertheless, at the same time the total cost of this alternative “governmental revenue” (instead of “total cost”). Indeed, economic
has the strongest positive influence on the result (reaching 0.455), it factors do not appear on the graph of the bus alternative. Moreover,
can also be responsible for the worst result (0.333). Furthermore, the only the results of motorcycle and gasoline car, besides renewable en-
results for bus and ferry affect the performance of the motorcycle mode, ergy consumption, have a direct influence on the ferry performance, as
in which case “total cost” and “CO2-eq emission” are the most frequent shown in Fig. 15. The three most relevant indicators for its performance
indicators in Fig. 13. have greater chances of causing negative results on its performance.

Fig. 11. Indicators that influence the gasoline car performance.

749
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 12. Indicators that influence the ethanol car performance.

Fig. 13. Indicators that influence the motorcycle performance.

Fig. 14. Indicators that influence the bus performance.

This can be explained by the fact that ferry is the alternative with number of possible occurrences can be analyzed (100,000 here).
highest probability of having sustainable performance levels higher
than 0.697.
This analysis adds further support to the need to evaluate the per- 5. Final considerations
formance of the transportation sector by applying techniques that
consider the probabilistic behavior of the indicators. Based on the dis- This work has shown how sustainability has been addressed in in-
cussion in this section, it is possible to understand that assessment using vestigations about performance assessment of the transportation sector.
point data (and/or means) can lead to mistaken decisions, since in some Traditionally, these investigations have considered deterministic values
cases the performance can be better (or worse) than that indicated in (means and/or point data) of the indicators, be it for analysis of the
Fig. 3. Besides this, MCS allows considering situations that not even risks, feasibility of projects or performance of the system. However, the
traditional analysis of scenarios (or sensitivity) can reveal, since a huge results demonstrate that the use of techniques that allow considering
the uncertainty of the assessment, such as Monte Carlo simulation, can

750
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

Fig. 15. Indicators that influence the ferry performance.

provide more realistic diagnosis of the behavior and/or performance of adopted a triangular distribution for application of the MCS.
the system. Regarding suggestions for future research, we recommend using
Besides, the literature review has shown that, most times, the social more recent data and also historical numbers to generate probability
dimension is not considered in the assessments due to the difficult in curves that represent the input indicators; selecting indicators based on
measuring this kind of impact. Thus, the indicators proposed in Table 2 the opinion of specialists; increasing the number of simulations; in-
helps to fill a gap indicated in the previous studies by showing metrics corporating alternatives envisioned in future projects for the city pair
that could be applied to evaluate the social aspect. Therefore, the ob- analyzed; analyzing other origin-destination pairs where modal choice
jectives of this study were attained. is possible; comparing results with other studies of transport sectors;
In the context investigated, the collective transportation alternatives proposing actions to improve the performance of the alternatives; and
had the best performance considering the indicators based on de- simulating their impact on the performance curves. Finally, we re-
terministic values, and also tended to show the best performance in the commend using MCS in future studies, both academic and govern-
probabilistic evaluation. The ethanol car alternative was the one at- mental, to obtain more accurate results on which to base decisions.
taining the best performance among all the alternatives (0.8547), al-
though the highest probability of its performance was concentrated in Acknowledgements
the range from 0.5 to |0.6, while the highest probabilities for bus and
ferry were in the 0.6 to |0.7 range (95.26% and 53.61%, respectively). We thank CNPq (Process 310825/2017-2 Research Productivity
Besides this, gasoline car, the option most used to travel between the Fellowship Holder-CNPq Brazil) for financial support.
two cities studied, showed the possibility of having the worst sustain-
ability index (0.1660) among all the alternatives (although the most References
probable performance levels were concentrated between 0.3 and |0.4).
By comparing the results depicted in Fig. 3 with those found by [1] Litman T. Well measured: developing indicators for sustainable and livable trans-
applying MCS, it was possible to verify that the assessment considering port planning. Vic Transp Policy Inst 2015:10–5.
[2] Santos G, Maoh H, Potoglou D, von Brunn T. Factors influencing modal split of
deterministic values can underestimate or overestimate the results of commuting journeys in medium-size European cities. J Transp Geogr
the alternatives. For example, 47% of the results of the motorcycle 2013;30:127–37. http://dx.doi.org/10.1016/j.jtrangeo.2013.04.005.
mode were worse than the average performance shown in Fig. 3, while [3] Litman T, Burwell D. Issues in sustainable transportation. Int J Glob Environ Issues
2006;6:331–47. http://dx.doi.org/10.1504/IJGENVI.2006.010889.
the results for ferry were higher in 68.78% of the cases. [4] Santos AS, Ribeiro SK. The use of sustainability indicators in urban passenger
Therefore, MCS can be used as an additional tool for analysis of transport during the decision-making process: the case of Rio de Janeiro, Brazil.
transportation systems (to improve performance of existing ones or for Curr Opin Environ Sustain 2013;5:251–60. http://dx.doi.org/10.1016/j.cosust.
2013.04.010.
new projects), to reduce the chance of misinterpretation of the results
[5] United Nations – Department of Economic and Social Affairs. Sustanaible transport
and thus erroneous decisions about public policies. In this respect, the 〈https://sustainabledevelopment.un.org/topics/susta〉; 2015.
greater the coefficient of variation of the results is, the higher the [6] Buehler R, Pucher J, Kunert U. Making transportation sustainable: insights from
Germany; 2009.
probability will be that the evaluation considering only deterministic
[7] Guimarães V de A, Leal Jr IC. Performance assessment and evaluation method for
values will lead to a result divorced from reality. passenger transportation: a step toward sustainability. J Clean Prod
Finally, it was possible to observe that the indicator with the 2017;142:297–307.
greatest influence on the individual transportation alternatives is cost, [8] Alonso A, Monzón A, Cascajo R. Comparative analysis of passenger transport sus-
tainability in European cities. Ecol Indic 2014;48:578–92. http://dx.doi.org/10.
while the most relevant one for the collective alternatives is trip time. 1016/j.ecolind.2014.09.022.
Moreover, we found which indicator influenced (positively or nega- [9] Barandier Jr JR. Future pathways of latin american cities: land use policy for
tively) the results of each alternative under assessment. Yet, trip time achieving sustainable mobility. A case study of the Rio de Janeiro metropolitan
area. Eur Transp Conf 2014:1–17.
and emission of CO2-eq, e.g., impacted the performance of all alter- [10] MME – Ministério de Minas e Energia. Plano decenal de expansão de energia 2024;
natives. These findings can help managers make decisions about the 2015.
actions that should be taken to improve the performance of transpor- [11] EPA – United StatesEnvironmental Protection Agency. Guide to sustainable trans-
portation performance measures; 2011.
tation system, to make it more sustainable. [12] Leal Jr IC, Garcia PAA, D’Agosto M de A. A data envelopment analysis approach to
We stress that the results might have been influenced by the in- choose transport modes based on eco-efficiency. Environ Dev Sustain
dicators chosen and the number of iterations carried out in the simu- 2012;14:767–81. http://dx.doi.org/10.1007/s10668-012-9352-x.
[13] Vanorio G, Mera JM. Methodology for risk analysis in railway tunnels using Monte
lation. Besides this, we used secondary data, which in some cases may Carlo simulation. Comput Railw XIII 2012;127:673–83. http://dx.doi.org/10.2495/
have been outdated. Finally, due to the absence of historical data, we CR120571.

751
V.A. Guimarães et al. Renewable and Sustainable Energy Reviews 93 (2018) 732–752

[14] Salling KB. Appraisal a new approach to feasibility risk assessment within transport 2010;15:179–88. http://dx.doi.org/10.1016/j.trd.2009.09.002.
infrastructure appraisal. Procedia - Soc Behav Sci 2013;74:468–77. http://dx.doi. [38] Shen LY, Jorge Ochoa J, Shah MN, Zhang X. The application of urban sustainability
org/10.1016/j.sbspro.2013.03.047. indicators – a comparison between various practices. Habitat Int 2011;35:17–29.
[15] Montgomery DC, Runger GC. Applied statistics and probability for engineers. 6th http://dx.doi.org/10.1016/j.habitatint.2010.03.006.
ed. Wiley; 2013. [39] Toth-Szabo Z, Várhelyi A. Indicator framework for measuring sustainability of
[16] Yoriyaz H. Método de Monte Carlo: princípios e aplicações em Física Médica Monte transport in the city. Procedia - Soc Behav Sci 2012;48:2035–47. http://dx.doi.org/
Carlo method: principles and applications in medical physics. Rev Bras Fìsica Méd 10.1016/j.sbspro.2012.06.1177.
2009;3:141–9. [40] Shiau TA, Liu JS. Developing an indicator system for local governments to evaluate
[17] PDTU. Plano Diretor de Transporte Urbano da Região Metropolitana do Rio de transport sustainability strategies. Ecol Indic 2013;34:361–71. http://dx.doi.org/
Janeiro. Rio de Janeiro; 2013. 10.1016/j.ecolind.2013.06.001.
[18] Figueroa MJ, Ribeiro SK. Energy for road passenger transport and sustainable de- [41] Lemp JD, Kockelman KM. Understanding and accommodating risk and uncertainty
velopment: assessing policies and goals interactions. Curr Opin Environ Sustain in toll road projects. Transp Res Rec J Transp Res Board 2009;2132:106–12. http://
2013;5:152–62. http://dx.doi.org/10.1016/j.cosust.2013.04.004. dx.doi.org/10.3141/2132-12.
[19] Mattila T, Antikainen R. Backcasting sustainable freight transport systems for [42] Du Y, Geng Y, Sun L. Simulation model based on Monte Carlo method for traffic
Europe in 2050. Energy Policy 2011;39:1241–8. http://dx.doi.org/10.1016/j. assignment in local area road network. Front Archit Civ Eng China 2009;3:195–203.
enpol.2010.11.051. http://dx.doi.org/10.1007/s11709-009-0032-3.
[20] Schmitt WF, Szklo A, Schaeffer R. Policies for improving the efficiency of the [43] Shi FY, Gao XD, Xing ZY, Chen YJ. Reliability analysis of the subway door system
Brazilian light-duty vehicle fleet and their implications for fuel use, greenhouse gas based on Monte Carlo. In: Proceedings of International Conference Comput Inf Syst
emissions and land use. Energy Policy 2011;39:3163–76. http://dx.doi.org/10. Ind Appl (CISIA 2015); 2015. p. 330–33.
1016/j.enpol.2011.02.067. [44] Tan F, Makwasha T. Best practice cost estimation in land transport infrastructure
[21] Baptista C, Silva CM, Farias TL, Heywood JB. Energy and environmental impacts of projects. In: Proceedings of Australas Transp Res Forum; 2010. p. 1–15.
alternative pathways for the Portuguese road transportation sector. Energy Police [45] Rangarajan K. Sustainable engineering practices in transportation projects; 2012.
2012;51:802–15. http://dx.doi.org/10.1016/j.enpol.2012.09.025. [46] Ali M, Najife FT. A cost effective methodology for pedestrian road crossing for
[22] Bastani P, Heywood JB, Hope C. The effect of uncertainty on US transport-related developing countries. In: Proceedings of thr 120th ASEE Annual Conference Expo;
GHG emissions and fuel consumption out to 2050. Transp Res Part A Policy Pract 2000.
2012;46:517–48. http://dx.doi.org/10.1016/j.tra.2011.11.011. [47] Gotoh H, Takezawa M, Maeno Y. Risk analysis of airplane accidents due to bird
[23] Hao H, Wang H, Ouyang M. Fuel consumption and life cycle GHG emissions by strikes using Monte Carlo simulations. Risk Anal VIII 2012;44:393–403. http://dx.
China's on-road trucks: future trends through 2050 and evaluation of mitigation doi.org/10.2495/RISK120331.
measures. Energy Policy 2012;43:244–51. http://dx.doi.org/10.1016/j.enpol.2011. [48] Faghih-Roohi S, Xie M, Ng KM. Accident risk assessment in marine transportation
12.061. via Markov modelling and Markov Chain Monte Carlo simulation. Ocean Eng
[24] Skippon S, Veeraraghavan S, Ma H, Gadd P, Tait N. Combining technology devel- 2014;91:363–70. http://dx.doi.org/10.1016/j.oceaneng.2014.09.029.
opment and behaviour change to meet CO2 cumulative emission budgets for road [49] Su CW, Cheng MY, Lin FB. Simulation – enhanced approach for ranking major
transport: case studies for the USA and Europe. Transp Res Part A transport projects. J Civ Eng Manag 2016;3730:285–91. http://dx.doi.org/10.
2012;46:1405–23. http://dx.doi.org/10.1016/j.tra.2012.05.021. 1080/13923730.2006.9636405.
[25] Kennedy CA. A comparison of the sustainability of public and private transportation [50] Salling KB. Assessment of transport projects risk analysis and decision support;
systems: study of the Greater Toronto Area. Transportation (Amst) 2002;29:459–93. 2008.
http://dx.doi.org/10.1023/A:1016302913909. [51] Mudova N, Salling KB. Transport assessment and risk analysis: the case of the 2nd
[26] Markovich J, Lucas K. The social and distributional impacts of transport: a literature fixed link across the Danube River. Trafik Aalb Univ 2009;2009:1–18.
review; 2011. [52] Salling KB, Leleur S. Modelling of transport project uncertainties: feasibility risk
[27] Smith TW, Axon CJ, Darton RC. A methodology for measuring the sustainability of assessment and scenario analysis. EJTIR 2012;12:21–38.
car transport systems. Transp Policy 2013;30:308–17. http://dx.doi.org/10.1016/j. [53] Nojima N. Prioritization in upgrading seismic performance of road network based
tranpol.2013.09.019. on system reliability. Third China-Japan-US Trilateral Symp. Lifeline Earthq. Eng;
[28] Bojkovi N, Macura D, Pej S. A comparative assessment of transport- sustainability in 1998.
central and eastern European countries with a brief reference to the Republic of [54] O’Connor O, O’Brien O, Eugene J. Mathematical traffic load modelling and factors
Serbia. Int J Sustain Transp 2015;5:319–44. http://dx.doi.org/10.1080/15568318. influencing the accuracy of predicted. Can J Od Civ Eng 2005;32:270–8.
2010.539664. [55] Enright B, O’Brien EJ. Monte Carlo simulation of extreme traffic loading on short
[29] Awasthi A, Chauhan SS, Omrani H. Application of fuzzy TOPSIS in evaluating and medium span bridges. Struct Infrastruct Eng 2012;2479:37–41. http://dx.doi.
sustainable transportation systems. Expert Syst Appl 2011;38:12270–80. http://dx. org/10.1080/15732479.2012.688753.
doi.org/10.1016/j.eswa.2011.04.005. [56] Kuang A, Tang Z, Shan L. Road network capacity reliability considering travel time
[30] Pauli G. Sustainable transport: a case study of Rhine navigation. Nat Resour Forum reliability. Procedia Soc Behav Sci 2013;96:1818–27. http://dx.doi.org/10.1016/j.
2010;34:236–54. sbspro.2013.08.207.
[31] Shiau T. Evaluating sustainable transport strategies with incomplete information for [57] Macfarlane GS, Brakewood C, Fischer JM, Poznanski A. Monte Carlo simulation for
Taipei City. Transp Res Part D 2012;17:427–32. http://dx.doi.org/10.1016/j.trd. transit transfer volumes. Data Anal Compet ABJ70 Artif Intell Comm Transp Res
2012.05.002. Board 2013:1–6.
[32] Song S. Ship emissions inventory, social cost and eco-efficiency in Shanghai [58] Guimarães VdeA. Avaliação do desempenho sustentável das alternativas de trans-
Yangshan port. Atmos Environ 2014;82:288–97. http://dx.doi.org/10.1016/j. porte urbano de passageiros; 2016.
atmosenv.2013.10.006. [59] Kouridis C, Gkatzoflias D, Kioutsioukis I, Ntziachristos L. Uncertainty estimates and
[33] Rajak S, Parthiban P, Dhanalakshmi R. Sustainable transportation systems perfor- guidance for road transport emission calculations 〈http://dx.doi.org/10.2788/
mance evaluation using fuzzy logic. Ecol Indic 2016;71:503–13. http://dx.doi.org/ 78236〉; 2009.
10.1016/j.ecolind.2016.07.031. [60] Torok A. Monte-Carlo simulation of road transport emission. Int J Traffic Transp
[34] Miller P, de Barros AG, Kattan L, Wirasinghe SC. Analyzing the sustainability per- Eng 2015;5:278–85.
formance of public transit. Transp Res Part D Transp Environ 2016;44:177–98. [61] Deng J. Introduction to grey system theory. J Grey Syst 1989;1:1–24.
http://dx.doi.org/10.1016/j.trd.2016.02.012. [62] Palisade. @Risk; 2016.
[35] Ngossaha JM, Ngouna RH, Archimède B, Nlong JM. Sustainability assessment of a [63] Fernandes CABdeA. Gerenciamento de riscos em projetos: como usar o Microsoft
transportation system under uncertainty: an integrated multicriteria approach. Excel para realizar a simulação de Monte Carlo; 2005.
IFAC-Pap 2017;50:7481–6. http://dx.doi.org/10.1016/j.ifacol.2017.08.1064. [64] Silva SS, Leal Jr IC, Guimarães V de A, Garcia PAA. Modal choice method to pas-
[36] Holden E, Linnerud K, Banister D. Sustainable passenger transport: back to senger transportation based on eco-efficiency measures. Congr Ing Del Transp - CIT
Brundtland. Transp Res Part A Policy Pract 2013;54:67–77. http://dx.doi.org/10. 2014.
1016/j.tra.2013.07.012. [65] Brasil. Lei n° 12.587/12. Institui as diretrizes da Política Nacional de Mobilidade
[37] Castillo H, Pitfield DE. ELASTIC – a methodological framework for identifying and Urbana e dá outras providências; 2012.
selecting sustainable transport indicators. Transp Res Part D Transp Environ

752