Development of Provisions for Evaluating the Effectiveness of Public Passenger Transport Systems // Transport Problems: X International Scientific Conference, Katowice, 27-29 июня 2018 г. - Silesian University of Technology, 2018. - P. 529-544

16 Pages·0.79 MB·Russian

by  Mochalin S. , Kasper M. , Nikiforov O. , Levkin G. , Kurshakova N.

Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
TRANSPORT PROBLEMS 2018
X INTERNATIONAL CONFERENCE

Key words: passenger public transport; the evaluation of the system's effectiveness;
planned and actual indicators; the indexation-rating model; the Pareto’s chart

Sergey MOCHALIN, Mariya KASPER
Siberian State Automobile and Highway University
Mira av., 5, 644080, Omsk, Russia
Oleg NIKIFOROV*
Murmansk Branch of Emperor Alexander I St. Petersburg State Transport University
Rybny proezd, 3, 183038, Murmansk, Russia
Grigory LEVKIN, Natali KURSHAKOVA
Omsk State Transport University
Marx av., 35, 644046 Omsk, Russia
*Corresponding author. e-mail: [email protected]

DEVELOPMENT OF PROVISIONS FOR EVALUATING THE
EFFECTIVENESS OF PUBLIC PASSENGER TRANSPORT SYSTEMS

Summary. The article defines the urgency of developing scientifically sound solutions
and effective tools in the evaluation of the effectiveness of the public passenger transport
system. The work presents a methodology that allows evaluate the effectiveness of the
system in accordance with accepted standards in terms of the three components of the
"passenger-carrier-department" system, taking into account the interests of each side. To
develop a methodology for the evaluation of the effectiveness of passenger enterprises, a
model of indexation is used, where the whole set of carefully selected indicators is
converted from absolute values to relative ones. The article presents a classification
model for the formation of a basic system of indicators for passenger enterprise and a
functional scheme for grouping and ranking them. The developed tool can be used for
evaluating the public passenger transport systems, its application is illustrated by
examples.

1. INTRODUCTION

In the ISO 9000:2000 standards, effectiveness is defined as the degree of implementation of the
planned activity and achievement of the planned results. That is, if the goal is achieved more precisely,
the effectiveness is the better. According to ISO 9000:2000, efficiency is the ratio of achieved results
and used resources. Thus, the effectiveness may increase, while the efficiency falls [15; 16].
Uniform methodology for the integrated evaluation of the effectiveness of public passenger
transport system has not been developed yet. This is stated in recent works by the direction of
efficiency, quality, accessibility, and effectiveness of public passenger transport. A comprehensive
evaluation of the effectiveness of public passenger transport system is usually carried out through
surveys involving only a small number of people using passenger transport. At the same time, in the
market of passenger transport services, there are many different enterprises and organizations, from
small to large ones [13; 21]. So, it is necessary to evaluate the functioning of the system in accordance
with accepted standards in terms of the three components of the "passenger - carrier - department of
transport" system, taking into account the interests of each side. Evaluation of the effectiveness should
be able to identify which indicators need to be adjusted and what activities can be carried out to
improve the effectiveness of the system and its subsystems [6; 8; 9; 20].
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
530 S. Mochalin, M. Kasper, O. Nikiforov, G. Levkin

2. THE SYSTEM AND PROBLEM

At present, the task of evaluating the effectiveness of functioning of the public passenger transport
system is solved in a fragmented manner. For example, the most common means of evaluation the
effectiveness of enterprises is the analysis of isolated, mostly financial, indicators of the enterprise for
the planning period (year). The evaluation of the effectiveness of the public passenger transport
system is often confused with the evaluation of efficiency. At the same time, the evaluation of
efficiency is carried out periodically by the city administration, but usually without taking into account
the interests of passengers who are consumers of services. The lack of a comprehensive evaluation of
the effectiveness causes economic losses, increased risks and untimely management decisions to
eliminate negative factors, which affect the company's efficiency and financial result, the quality of
services and competitiveness [11, рр. 25-34; 24].
The process of transport service should be viewed as an organized set of interrelated actions that
give a valuable result to the consumer. Thus, we can distinguish different levels of effectiveness:
individual (passenger), organizational (carrier), process (department of transport). All these levels are
interrelated and permeated with processes implemented in the environment. Each of the levels
involves the formation of appropriate goals. The passenger wants to get the highest possible quality of
services (the recommended interval is 5 ... 7 minutes at all hours of the day and all days of the week)
with minimal travel costs (cost, transfers, physiological condition). The carrier is interested in
obtaining the maximum profit (at the same time it is advantageous for him to raise the tariff to a
certain value) and minimum investments (balance "tariff-quality") [21]. But at the same time he takes
into account the balances "tariff-volume of transportation" and "quality-volume transportation ". A
carrier often reduces the number of "unprofitable" flights for him during peak periods and on
weekends, which negatively affects the quality. But the carrier's work on the line at such hours and
days requires subsidies. At the same time, the department of transport is interested in reducing
subsidies, which make up the difference between the fare and the cost of transportation, and in
ensuring the maximum quality of passenger transportation, which directly affects the social and
economic development of the city. Because the interests of the administration are to create an
environment for successful management at all levels and to achieve the region's strategic goals
towards sustainable development. But despite the difference in the interests of the passenger, the
carrier and the administration of city, there are areas in which interests coincide (regularity, use of
capacity, etc.). In conditions of competitive struggle of carriers for ensuring the quality of the transport
service and obtaining competitive advantages, the significance of determining the effectiveness of
functioning of the public passenger transport system and its subsystems is growing [2, рр. 21-45; 10;
11, pp. 30-44]. It will help to identify of "problem areas" and determinate of solutions contributing to
the elimination of causes negatively affecting transport process. Thus, new scientifically grounded
solutions which take into account the methodological basis for evaluating the effectiveness of
complex systems and consider the evaluation from three sides (carriers, passengers, department of
transport) are required.
The concept of quality of transport service is inextricably linked with the demands of its consumer
The process of transport service should be built based on customer requests while the interests of the
carrier which is ready to work in this market should be taken into account.
The methodology for the effectiveness evaluation of the public passenger transport system must
meet all the necessary general requirements for evaluation methodology, namely:
 be clear, consistent, independent, reasonable;
 contain elements of a dynamic system, taking into account uncertainties;
 allow to identify and describe the functioning processes;
 determine the relationship between the indicators of effectiveness;
 include tasks of all types of planning and management;
 provide a comprehensive (integrated) evaluation of the results in the shortest possible time;
 identify the cause-effect relationships of reduction of effectiveness;
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
Development of provisions for evaluating the effectiveness of public passenger transport…531

 increase the effectiveness of the functioning of the passenger public transport system and the
reliability of managerial decisions for optimizing the work of enterprises [4; 21].

3. METHODOLOGICAL APPROACH

3.1. Literature review

Theoretical and methodological basis was the works of domestic and foreign scientists devoted to
problems of the effectiveness evaluation of complex systems.
Analysis of various approaches to effectiveness evaluation [5; 6; 8; 12; 13; 17, pp. 21-37; 22; 24]
showed that the effectiveness evaluation of public passenger transport systems is one of the unsolved
tasks in the field of improving the efficiency and quality of passenger road transport.
The analysis of works in the field of evaluation the effectiveness of complex systems makes it
possible to identify known methods, published in [20]:
Existing evaluations of the effectiveness of complex system have certain significant disadvantages:
a simplified approach that does not take into account the degree of influence of each indicator on the
effectiveness as a whole; the impossibility to express the dynamics of indicators in their mutual
relation; the impossibility of applying the approach when working with absolute indicators. Variants
of the effectiveness evaluation of processes in complex systems are based on poorly formalized
methods with low accuracy and reliability.

3.2. Assumptions

For the development a methodology for evaluation the effectiveness of the passenger public
transport system the authors propose to apply the model of indexation (MINOR) based of measuring
proximity of two rank orderings of growth rates (plan and fact) using Spearman and Kendall's rank
correlation coefficients, generalized and positive coefficients of evaluation [14, pp. 201-206; 22; 23,
pp. 151-158].
In accordance with this model, the entire set of key indicators is converted from absolute values to
relative values (growth rates of these indicators). This allows operations for indicators with different
units of measure and to estimate the value of the increment of indicators. The ranking allows to
express the dynamics of the indicators in their mutual relation, to evaluate the property of the system,
which can not be estimated by any of the indicators separately [24].
Further, according to the rule of the Pareto-analysis "80/20", the factors that have the most negative
impact on effectiveness are identified. For an objective evaluation of the effectiveness of the transport
process, the indicators should be complex and multifactorial, including both qualitative, economic and
natural criteria, necessarily using international criteria [1; 3].

3.3. Basic structure of methodology

The algorithm for evaluating the effectiveness is shown in Fig. 1, 2.
1. Identification of the system
At this stage, the characteristics of the system are given: conditions and features of functioning, the
duration of the investigated cycle, relationships within the system, incoming and outgoing flows are
determined.
2. Formation a set of effectiveness indicators
A specific set of indicators of the effectiveness depends on the specifics of the system. A set of
indicators can consist of two subsets - a basic and an additional.
For the carrier, for example, it is necessary to determine such indicators that can be calculated or
obtained from the reporting documentation (the plan can be planned or taken from the previous year's
fact). It is important to include those indicators that directly affect the evaluation of the effectiveness
by the passenger (1/3 part of all indicators) and by the department of transport, leaving 1/3 for a part
of the indicators that are important only for the enterprise (usually the financial result).
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
532 S. Mochalin, M. Kasper, O. Nikiforov, G. Levkin

3. Formation the model of the standard ranking of the indicators i = 1 ... n
A measure of effectiveness is the forming prioritized benchmark series of estimates of the growth
rates of planned indicators R , i.е. set of ranks of the "plan" of indicators from 1 to n, where N is the
пi
set of indicators.

Fig. 1. The algorithm for evaluating the effectiveness of the passenger public transport system (part 1)
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
Development of provisions for evaluating the effectiveness of public passenger transport…533

Fig. 2. The algorithm for evaluating the effectiveness of the passenger public transport system (part 2)
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
534 S. Mochalin, M. Kasper, O. Nikiforov, G. Levkin

If the set of indicators is rather large and exceeds 12 units, then it is necessary to organize subsets
(groups of indicators with common characteristics that combine a certain number of indicators) in
order of priority and establish the priority of indicators within each group (item 3 of the algorithm).
For the formation of the basic subsets of indicators for the carrier, it is proposed to build on the
following levels:
- The "Passenger" is a perspective that evaluates the results of satisfying the interests of consumers
(regularity factor, speeds, etc.), characterizes the quality of services.
- The "Department of transport" is a perspective that evaluates results, which affects the satisfaction of
the department of transport (subsidies, traffic volume, etc.), characterizes the volume of services.
- The "Black box" of enterprise processes is a perspective that evaluates the effectiveness of the
processes performed by the organization that have a direct impact on the degree of coherence of
processes at the enterprise, these indicators are not of interest to the passenger and the department.
All three groups have intersection areas (Fig. 3).

Fig. 3. Functional diagram of the grouping of the carrier's indicators

For the ranking of indicators, the following procedure is suggested:
1) to choose the indicators Ni with the planned value and the actual value included in group No.
1, which directly affect the evaluation of the effectiveness on the part of both the passenger and the
department of transport, and the enterprise (quality-efficiency-volume: K-E-V);
2) choose the indicators Ni for group No.2, which directly affect the evaluation of effectiveness
from both the passenger and the enterprise, and do not interest the department of transport (K-E);
3) choose the indicators Ni for group No.3, which directly affect the evaluation of effectiveness
from both the department of transport and the enterprise, and do not interest the passenger (V-E);
4) choose the indicators Ni included in group No.4 - the "black box" of enterprise, which are
important only for the enterprise (E).
5) rank the indicators within each group - establish the priority (significance) of indicators.
Systematized indicators of the work of enterprises should be ranked in order of importance,
primarily for the passenger, then for the department of transport, and in the last instance for the
enterprise, since in market conditions effectiveness is not only a good financial result, but also, first of
all, the competitiveness of the enterprise in terms of the quality of services and the amount of
subsidies. As a result, an analytical model is formed in the form of a standard rank of indicators.
4. Formation of an array of data for i = 1 ... n with assignment of the rank "fact"
This stage is the filling of the data array:
1) input of planned values of indicatorsП , П 0, which correspond to the rank "plan"R ;
i i пi
2) input of actual values of indicators Ф , Ф 0;
i i
3) determination of the rate of growth of the "fact" of the indicatorsr (Fig. 4) shows the growth
фi
dynamics of each indicator:
Ф
r  i , (1)
фi П
i
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
Development of provisions for evaluating the effectiveness of public passenger transport…535

where Ф - the actual value (fact) of the indicator for the period; П - the planned value (plan) of the
i i
indicator for the period.
4) sorting the table by increasing the ratio r ;
фi
5) arrangement of ranks "fact" R in accordance with the relations r ;
фi фi
6) sorting the table by increasing the rank "plan"R (Fig. 4).
пi

Fig. 4. Calculation of correlation coefficients with ordering according to the rank of the "plan"

5. Evaluation of the relationships between the indicators i = 1 ... n (Fig. 2)
At this stage, the joint and separate influence of many interrelated and simultaneously acting
indicators is determined. This stage is based on nonparametric analysis. To measure proximity of two
rank orderings of growth rates (for plan and fact), the rank correlation coefficients of Spearman (by
deviation) and Kendall (by inversions) are used. The coefficients give an estimate of the closeness of
ranks in the interval from +1 to -1. The +1 score is obtained if the one set of ranks has a coincidence
with the another, -1 if the one set of ranks is inverted in relation to the other [14, pp. 201-206; 22; 23,
pp. 151-158].
This stage contains:
1) calculation of the Spearman coefficient according to the formula:
d2
К 16 i , (2)
откл n(n2 1)
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
536 S. Mochalin, M. Kasper, O. Nikiforov, G. Levkin

where n – the number of values of the variables, in the particular case n = 27, d2 – the sum of
i
squares of differences ranks (Fig. 5).

d  R R . (3)
i пi фi
2) calculation of the Kendall coefficient according to the formula:
4R
К  1, (4)
инв n(n1)
where n – the number of values of the variables, in the particular case n = 27, R– the sum of ranks
- the column "search" (Fig. 5),
n
 
R  1
(5)
i1 ji,
R R
фj фi
The obtained values can be represented in the form of histogram, where the values of the OX axis
correspond to the values of the reference order of the rank of the "plan", and the magnitude of the
columns OY corresponds to the values of the rank "fact" of the i-indicators.
The dashed line is a trend line approximating the ranks "fact". The smaller the deviation of the
actual effectiveness from the y = x, the closer the relationship between them, and, consequently, the
higher the effectiveness of the system.
6. Evaluation of the economic regime of the system
Evaluation of the economic regime of the system contains a calculation of generalized and positive
effectiveness evaluation [14, pp. 244-247; 22; 23, pp. 151-158].
1) calculation of the generalized effectiveness evaluation:
A generalized evaluation of the effectiveness is varied from 0 to 1 and determined by the formula:
(1К )(1К )
ОР  откл инв 100%, (6)
4
where К – the coefficient of rank correlation by deviation (Spearman); К – the coefficient of
откл инв
rank correlation by inversions (Kendall).
2) calculation of the positive effectiveness evaluation [22]:
It is necessary to normalize the positive component of effectiveness OP on the interval [0.25; 1]:
ОР P
ПР min 100% (7)
Р P
max min
where Р и P – the maximum and minimum values of the OP at the normalized interval, 1.00
max min
and 0.25, respectively.
On the interval [0; 0.25] ПР = 0. The obtained values of efficiency can be represented in the form
of the curve of effectiveness with values of the generalized and positive evaluation by years.
7. Factor analysis of variations for i = 1 ... n
For the analysis, the method of isolated influence of factors on the result change is used. The
influence of factors is estimated by the extent to which they have lowered the evaluation of
effectiveness compared to the standard.
1) the deviation P of each indicator is calculated by the formula:
i
P 1r (8)
i фi
2) the influence of the i-th index on the decrease of the integral performance score is determined.
Indicators with the change causes a decrease in effectiveness are remained. It does not take into
account the deviation of those indicators whose change in one direction or the other does not affect the
reduction in effectiveness (Fig. 5).
3) the modules of nonzero values:
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
Development of provisions for evaluating the effectiveness of public passenger transport…537

P : P (9)
Аi i
4) the data P is sorted in descending order;
Аi

Fig. 5. Results of factor analysis calculations

5) the integral coefficient of effectiveness is calculated on the basis of the values of the deviations
of the indicators selected in 2) and 3):
n
K P (10)
рез Ai
i1
The integral coefficient of effectiveness shows what the total deviation of those indicators, the
change in which in one direction or another becomes the reason for the decrease in effectiveness.
6) the cumulative deviation P forP is calculated:
Кi Аi
i
P P ; (11)
Кi Aj
j1
7) the percentage of influence of each indicator on the general decrease is calculated:
P
  Ai 100%; (12)
i K
рез
8) the cumulative percentage (with accumulation) is calculated;
i
   (13)
Ki j
j1
9) a threshold level of significance (80%) is determined for representation in the diagram.
8. Analysis of the degree of impact of indicators on results
The Pareto chart is constructed on the basis of the deviation values of each indicator [1; 3].
9. Identification of cause-effect relationships
This stage includes the study, search, collection and formation of the most significant factors that
are the causes of the problems [4; 24].
10. Development of preventive and corrective measures
11. Economic evaluation of improvement measures
This stage includes the choice of the method of calculating the costs for each planned activity,
calculating the costs in the amount necessary to carry out all planned activities and assessing the each
element of costs in order to identify opportunities to reduce them.
Copyright ОАО «ЦКБ «БИБКОМ» & ООО «Aгентство Kнига-Cервис»
538 S. Mochalin, M. Kasper, O. Nikiforov, G. Levkin

4. APPLICATION OF THE PROPOSED METHODOLOGY

4.1. Inputs

To assess the effectiveness of the public passenger transport system, the data of the documentation
of the Department of Transport and large passenger enterprises of Omsk [19], the surveys of experts -
the heads of the Department of Transport and enterprises were used. The set of indicators for
evaluating the effectiveness by passengers and by the department of transport is fully with the Social
standard for transport services for population in the Russian Federation [18] (the first 13 base units)
and supplemented with indicators that are also significant from the point of view of the experts.

4.2. Results

In Fig. 6 there is an example of the histogram of effectiveness for Passenger Enterprise No. 4.

Fig. 6. The histogram of effectiveness for Passenger Enterprise No. 4 in 2016

The proposed tool for evaluation of the effectiveness allows to evaluate the effectiveness of the
carrier by years (Table 1) and compare the effectiveness of carriers for a certain period (Figure 7).
Table 1
The results of the evaluation of the system "Passenger Enterprise No. 4"
Year
Indicator
2014 2015 2016
Spearmen coefficient 0,161 0,062 -0,231
Kendall coefficient 0,077 0,077 -0,191
ОР, % 31,3 28,6 15,6
ПР, % 8,3 4,8 0

Fig. 7. Effectiveness of the enterprises regime in 2016