Anwendungen und Konzepte der Wirtschaftsinformatik

104 Pages·2017·13.43 MB·German

Anwendungen und Konzepte der
Wirtschaftsinformatik
Nummer 5 (2017)
DOI: 10.5281/zenodo.579639
Editorial
Liebe Leserinnen und Leser,
vor Ihnen liegt die fünfte Ausgabe des E-Journals Anwendungen und Konzepte in der
Wirtschaftsinformatik (AKWI).
Dieses Heft enthält einen Beitrag in der Rubrik Grundlagen und zwei Beiträge in der
Rubrik Praxis. Acht Beiträge sind aus hervorragenden Abschlussarbeiten entstanden und Christian Müller
in Zusammenarbeit mit dem zuständigen Professor bzw. der zuständigen Professorin für
unsere Zeitschrift aufbereitet worden. Die Themen der Beiträge stammen aus den Berei-
chen Digitalisierung, Logistik, Geschäftsprozessoptimierung und Produktionsplanung.
Die Autoren stammen aus den Hochschulen der angewandten Wissenschaft und aus
Unternehmen. Die anwendungsorientierte Arbeit der Autoren ist auch aus den Titeln der
Arbeiten ersichtlich.
Alle Beiträge wurden von zwei unabhängigen Gutachtern begutachtet und von den
Autoren anschließend überarbeitet. Dieser Prozess nimmt naturgemäß viel Zeit in
Anspruch, da sämtliche Redakteure, Gutachterinnen und Gutachter ihre Arbeit in der
immer spärlicher werdenden Freizeit leisten. Dafür gebührt ihnen unser Dank. Aus Konrad Marfurt
diesem Grund haben wir uns entschlossen, das Team der Herausgeber um zwei weitere
Kollegen – Frank Herrmann und Norbert Ketterer – zu erweitern.
Die Zeitschrift wird weiterhin als E-Journal an der Hochschule Luzern unter Feder-
führung von Konrad Marfurt gehosted. Auch dafür danken wir bestens.
Unsere Zeitschrift steht kostenfrei unter http://akwi.hswlu.ch im Netz zur Verfügung. Für
unsere Autoren entstehen durch die Publikation keine Kosten, allerdings erhalten sie auch
keine Honorare. Damit wir unter diesen Bedingungen erfolgreich arbeiten können,
reichen die Autoren druckfertige Manuskripte, die unserer Formatvorlage genügen, in
deutscher oder englischer Sprache ein.
Norbert Ketterer
Außerdem bitten wir unsere Autoren um eine Einverständniserklärung zur Publikation
und eine Selbsteinschätzung, welcher der Rubriken Grundlagen, Trends, Praxis, Kurz
erklärt, Buchbesprechung oder Abschlussarbeit ihr Beitrag zugeordnet werden soll. Bei
Abschlussarbeiten gehen wir davon aus, dass es sich um Zusammenfassungen hervor-
ragender Thesisarbeiten handelt, die zusammen mit dem betreuenden Hochschullehrer
eingereicht werden. Nach der Einreichung beginnt sofort unser Begutachtungsprozess.
Frank Herrmann
Nach dieser ausführlichen Beschreibung des Procederes der Beitragseinreichung hoffen wir, Sie zu einer solchen moti-
viert zu haben. In diesem Sinne verbleiben wir, stellvertretend für alle Herausgeber
Wildau und Luzern, im April 2017 Christian Müller, Konrad Marfurt,
Norbert Ketterer, Frank Herrmann
Applied Concepts of Probabilistic Programming
Olga Ivanova
Business informatics
E-Mail: [email protected]
ABSTRACT general is. Although none of them is universally
applicable or standardized, most of them tend to share
Probabilistic programming is one of the auspicious, fast-
substantial similarity.
growing fields of IT research, which is applicable in the
Probabilistic programs are defined by A. D. Gordon as
context of machine learning. Probabilistic programming
"usual functional or imperative programs with two
looks into the possibilities of mapping theoretical
added constructs: (1) the ability to draw values at
concepts of probability theory onto suitable practical
random from distributions, and (2) the ability to
programming techniques to handle uncertainty in data.
condition values of variables in a program via
This paper provides an overview of the applied concepts
observations".[GordHenzNo] F. Wood asserts that
of probabilistic programming and main groups of
"probabilistic programs are written with parts not fixed
probabilistic programming tools, as well as outlines the
in advance that instead take values generated at runtime
theoretical context and open issues of probabilistic
by random sampling procedures".[WoodMeMan] In a
programming.
similar way N. D. Goodman remarks that probabilistic
programming languages "in their simplest form ...
KEYWORDS extend a well-specified deterministic programming
language with primitive constructs for random
Probabilistic system, reasoning pattern, evidence,
choice".[Good]
inference, probability query, MAP.
Given uncertain or incomplete knowledge the agent
(whether human or not) is required in most cases to
INTRODUCTION
make an analysis of reliant data and make an
Probabilistic approach in programming has gained assumption of it thus, restoring to some extent the
considerable attention of academic community over the missing information. As D. Koller states, "Most tasks
last decade. The area of the research keeps growing, as require a person or an automated system to reason: to
the approach in question explores new ways and take the available information and reach conclusions,
techniques of massive data processing and decision both about what might be true in the world and about
making. As B. Cronin remarks, "probabilistic how to act".[KollFried] According to A. Pfeffer,
programming languages are in the spotlight".[Cron] M. probabilistic programming is a way to create systems,
Hicks describes probabilistic programming as "an supporting decision-making in the face of uncertainty.
exciting, and growing, area of research", with "people in He also points out that probabilistic approach combines
both AI/ML and PL working together and making the knowledge of a situation with the laws of probability
strides".[ Hicks] to determine those unobserved factors that are critical to
There exist a number of reasons accounting for this rise the decision.[Pfeff3] Performing such tasks as
of interest. However, the main of them comes down to preliminary analysis of bulk data as well as subsequent
the fact that constantly growing vast amount of data decision-making in case of incomplete knowledge
demands new techniques of automation, prediction, entails an extensive use of applied probabilistic tools
analysis and modelling. This ongoing search of new and techniques. Transferring the latter from the field of
ways to make IT systems more intelligent and mathematical reasoning into the field of applied
sophisticated has boosted the development of the whole informatics implies describing probability distributions
domain of machine learning. and providing ways of drawing probabilistic inference.
Handling uncertainty is one of numerous challenges This can be performed in the "traditional" imperative
machine learning is facing, as IT systems initially have programming with the help of complex cumbersome
been very restricted by means of processing these control flows but as A. D. Gordon noticed, the purpose
uncertainties. Applied concepts of probabilistic of probabilistic programming is to make probabilistic
programming could provide machine learning with modelling accessible to a programmer without expert
suitable tools for dealing with uncertainty of data, thus, knowledge of probability theory, i.e. without revealing
enabling ML to combine the available knowledge of the the details of inference implementation.[GordHenzNo]
subject or situation with mathematical probability rules.
There exist a number of definitions of what a
probabilistic program or probabilistic programming in
Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 1
BASIC COMPONENTS OF A PROBABILISTIC
REASONING SYSTEM. KEY TERMS AND
DEFINITIONS.
A probabilistic reasoning system presupposes a coherent
interaction of its components. Despite the fact that
approaches to separate out the components of such a
system differ in the level of abstraction, the underlying
principles and ideas bear a certain resemblance to each
other.
According to A. Pfeffer, components of a probabilistic
reasoning system generally include a probabilistic
model, general and evidential knowledge, query and
Figure 1: Probabilistic Reasoning System [Pfeff2]
inference engine.
Probabilistic model is an integral part of any
A similar approach has been given by D. Koller and N.
probabilistic reasoning system encompassing the most
Friedman, who singled out three major components or
relevant information about specifics of the particular
rather layers in any complex reasoning system, namely
domain in a suitable form for formal processing. A.
representation, inference and learning. In their
Pffefer describes a probabilistic model as "an encoding
perception, declarative representation is a reasonable
of general knowledge about a domain in quantitative,
encoding of the world model, used to answer the
probabilistic terms". He also stresses that "each model
questions of interest. Inference is viewed as "answering
has an element of inherent randomness".[Pfeff3] D.
queries using the distribution as our model of the world"
Koller also points out that a probabilistic model
in particular that process is carried out by "computing
"encodes our knowledge of how the system works in a
the posterior probability of some variables given
computer-readable form".[KollFried] In terms of
evidence on others".[KollFried]
programming a probabilistic model consists of
As far as the stage of learning is concerned, D. Koller
variables, dependencies between these variables,
and N. Friedman assume that models can be constructed
numerical parameters as values of these variables and
either with the help of a human expert or automatically,
the so-called functional forms of dependencies (e.g. the
"by learning from data a model that provides a good
possibility of modelling as the result of a coin toss with
approximation to our past experience". Furthermore,
a certain weight).[Pfeff2]
data-driven approach to model construction with human
B. Cronin emphasizes the importance of "clean
experts setting only guidelines for an automatic supply
separation between modelling and inference", as it "can
of details, was acknowledged to be more effective
vastly reduce the time and effort associated with
compared to purely human-constructed.[KollFried]
implementing new models and understanding data".
A. Pfeffer remarks that as far as learning is concerned,
According to his comparison, probabilistic languages
there are two main things to be done. The most obvious
can "free the developer from the complexities of high-
one is to "learn from the past to improve your general
performance probabilistic inference" the way "high-
knowledge" and as a result "to better predict a specific
level programming languages transformed developer
future situation". However, it is also possible to go
productivity by abstracting away the details of the
further learning from the past, that is "to improve the
processor and memory architecture".[Cron] In other
model itself", especially if "a lot of past experiences to
words, loose coupling between the model and the
draw on" is available. In this case the goal of a learning
inference engine enables the system to process different
algorithm is to produce a new model, not to answer
models and thus, serve as a generic tool.
queries. The learning algorithm begins with the original
A. Pfeffer differentiates between general knowledge
model and updates it based on the experience to produce
embracing "what you know to hold true of your domain
the new model. The new model can be used then to
in general terms, without considering the details of a
answer queries in the future in a "better-informed" way.
particular situation" and evidence as "specific
[Pfeff2]
information about a particular situation".[Pfeff3]
Query is elucidated as a property of some particular
situation which is looked for. In this interpretation
probabilistic inference is defined as "the process of
using the model to answer queries based on the
evidence".[Pfeff3]
It is also of importance that the relations of all
components of any probabilistic reasoning system
strictly comply with mathematical laws of probability.
General, and evidential information is treated in
"quantitative, probabilistic terms".[Pfeff2]
Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 2
pprrooggrraammmmiinngg llaanngguuaaggee wwiitthh pprriimmiittiivvee ccoonnssttrruuccttss ffoorr
random choice".[Good]
BB.. CCrroonniinn ddeeffiinneess aa pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg llaanngguuaaggee
as "a high-lleevveell llaanngguuaaggee tthhaatt mmaakkeess iitt eeaassyy ffoorr aa
ddeevveellooppeerr ttoo ddeeffiinnee pprroobbaabbiilliittyy mmooddeellss aanndd tthheenn "solve"
tthheessee mmooddeellss aauuttoommaattiiccaallllyy"" aanndd rreemmaarrkkss tthhaatt tthheessee kkiinndd
ooff llaanngguuaaggeess ""iinnccoorrppoorraattee rraanndom events as primitives
aanndd tthheeiirr rruunnttiimmee eennvviirroonnmmeenntt hhaannddlleess
inference".[Cron]
DD.. PPoooollee mmeennttiioonnss tthhaatt "most of the work in
pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg llaanngguuaaggeess hhaass bbeeeenn iinn tthhee
context of specific languages"". He tries to abstract from
thorough considerationn ooff ssppeecciiffiicc pprroobbaabbiilliissttiicc
pprrooggrraammmmiinngg llaanngguuaaggeess aanndd ttoo ffooccuuss oonn tthhee ddeessiiggnn ooff
tthheemm,, ssiinngglliinngg oouutt tthhrreeee aaddddiittiioonnaall ffeeaattuurreess,, wwhhiicchh aarree
FFiigguurree 22:: PPrroobbaabbiilliissttiicc RReeaassoonniinngg SSyysstteemm wwiitthh aa LLeeaarrnniinngg iinnhheerreenntt iinn aallll pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg llaanngguuaaggeess::
Component [Pfeff2] • ccoonnddiittiioonniinngg aass tthhee aabbiilliittyy ttoo mmaakkee

observations about some variaabblleess iinn tthhee ssiimmuullaattiioonn aanndd
NN.. DD.. GGooooddmmaann aanndd JJ.. BB.. TTeenneennbbaauumm developed an
ttoo ccoommppuuttee tthhee ppoosstteerriioorr pprroobbaabbiilliittyy ooff aarrbbiittrraarryy
aapppprrooaacchh ttoo tthhee ssttuuddyy ooff pprroobbaabbiilliissttiicc ssyysstteemmss oonn tthhee
pprrooppoossiittiioonnss wwiitthh rreessppeecctt ttoo tthheessee oobbsseerrvvaattiioonnss..
basis of the concept "ggeenneerraattiivvee mmooddeellss", which
• inference
rreepprreesseenntt kknnoowwlleeddggee aabboouutt tthhee ccaauussaall ssttrruuccttuurree ooff tthhee
• lleeaarrnniinngg aass tthhee aabbiilliittyy ttoo lleeaarrnn pprroobbaabbiilliittiieess
wwoorrlldd iinn aa ssiimmpplliiffiieedd ffoorrmm.. AA ggeenneerraattiivvee mmooddeell iinn tthhiiss
from data.[Poole]
theory is used to describe some pprroocceessss ooff tthhee rreeaalliittyy,,
AAss ffaarr aass tthhee iimmpplleemmeennttaattiioonn ppaarraaddiiggmm iiss ccoonncceerrnneed,
wwhhiicchh pprroodduucceess oobbsseerrvvaabbllee ddaattaa.. PPrroobbaabbiilliissttiicc
mmoosstt aauutthhoorrss ddiivviiddee tthhee eexxiissttiinngg pprroobbaabbiilliissttiicc
ggeenneerraattiivvee mmooddeellss aarree ddeeffiinneedd aass mmooddeellss ooff pprroocceesssseess,,
pprrooggrraammmmiinngg llaanngguuaaggeess aanndd ssyysstteemmss iinnttoo sseevveerraall mmaajjoorr
"wwhhiicchh uunnffoolldd wwiitthh ssoommee aammoouunntt ooff rraannddoommnneessss" and
groups.
ccaann bbee uusseedd ttoo iinnqquuiirree aabboouutt tthheessee pprroocceesssseess wwiitthh tthhee
AAccccoorrddiinngg ttoo AA.. PPffeeffffeerr,, ssoommee llaanngguuaaggeess bbeelloonngg ttoo tthhee
help of probabilistic infereennccee.. TThhee mmaaiinn iiddeeaa hheerree iiss ttoo
logic-bbaasseedd ggrroouupp ((PPRRIISSMM,, BBLLOOGG,, MMaarrkkoovv LLooggiicc)),,
ddeeaall wwiitthh aa pprroocceessss wwiitthh uunncceerrttaaiinnttyy aass wwiitthh
ootthheerrss ccoonnssttiittuuttee tthhee ggrroouuppss bbaasseedd oonn pprriinncciples of
ccoommppuuttaattiioonn,, wwhhiicchh iinnvvoollvveess rraannddoomm cchhooiicceess.. TThhee
ffuunnccttiioonnaall ((IIBBAALL,, CChhuurrcchh)) aanndd iimmppeerraattiivvee
ssiimmuullaattiioonn ooff ssuucchh pprroocceesssseess iiss iinneevviittaabbllyy ccoonnnneecctteedd
((FFAACCTTOORRIIEE,, PPiiccttuurree)) pprrooggrraammmmiinngg ppaarraaddiiggmmss.. OObbjjeecctt-
wwiitthh tthhee ddeeggrreeee ooff bbeelliieeff,, aass tthhee eexxppeecctteedd oouuttccoommeess aarree
oorriieenntteedd aapppprrooaacchh iiss aallssoo ssttaatteedd ttoo hhaavvee sseevveerraall
formalized as probabilittyy ddiissttrriibbuuttiioonn.. [[GGooooddTTeenneenn]]
aaddvvaannttaaggeess iinn tthhee ddoommaaiinn ooff pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg

aanndd FFiiggaarroo iiss mmeennttiioonneedd aass aann eexxaammppllee ooff aann oobbjjeecctt-
OOVVEERRVVIIEEWW OOFF EEXXIISSTTIINNGG PPRROOBBAABBIILLIISSTTIICC
oriieenntteedd pprroobbaabbiilliissttiicc llaanngguuaaggee..[[PPffeeffff11]]
SYSTEMS AND TOOLS AA.. DD.. GGoorrddoonn,, TT.. AA.. HHeennzziinnggeerr,, AA.. VV.. NNoorrii aallssoo ssiinnggllee
oouutt tthhrreeee ppaarraaddiiggmmss iinn tthhee ddiivveerrssiittyy ooff pprroobbaabbiilliissttiicc
TThhee ggrroowwiinngg iinntteerreesstt iinn tthhee pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg
llaanngguuaaggeess:: iimmppeerraattiivvee,, ffuunnccttiioonnaall,, aanndd llooggiiccaall.. PPRROOBB,,
aapppprrooaacchh wwiitthhiinn tthhee aaccaaddeemmiicc ccoommmmuunniittyy hhaass rreessuulltteedd
IInnffeerr..NNEETT aarree mmeennttiioonneedd aass eexxaammpplleess ooff iimmppeerraattiivvee
iinn tthhee eemmeerrggeennccee ooff nneeww llaanngguuaaggeess aanndd ffrraammeewwoorrkkss
languaaggeess,, ffuunnccttiioonnaall ppaarraaddiiggmm iiss tthhee bbaassee ffoorr BBUUGGSS,,
designed and iimmpplleemmeenntteedd ttoo ppeerrffoorrmm ttaasskkss ssppeecciiffiicc ttoo
IIBBAALL aanndd CChhuurrcchh,, wwhheerreeaass pprroobbaabbiilliissttiicc llooggiicc
tthhee ddoommaaiinn ooff pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg.. TThhee wwiikkii-list
llaanngguuaaggeess iinncclluuddee BBLLOOGG,, AAllcchheemmyy,, aanndd
ooff eexxiissttiinngg pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg ssyysstteemmss ccoonnttaaiinnss
Tuffy.[GordHenzNo]
more than 20 entries.[WikiPP]
IItt sshhoouulldd aallssoo bbee ppooiinntteedd oouutt,, tthhaatt mmoosstt aauutthhoorrss uussee tthhee
DDeessppiittee nnuummeerroouuss ddiiffffeerreenncceess ccoonncceerrnniinngg tthhee ppaarraaddiiggmm
notions "pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg llaanngguuaaggee" and
and implementation, proobbaabbiilliissttiicc llaanngguuaaggeess aanndd
"pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg ssyysstteemm" interchangeably.
lliibbrraarriieess hhaavvee mmuucchh iinn ccoommmmoonn.. TThhee mmoosstt iimmppoorrttaanntt
TToo bbee pprreecciissee,, oonnllyy aa ssmmaallll nnuummbbeerr ooff tthhee eexxiissttiinngg
ssiimmiillaarriittyy bbeettwweeeenn tthheessee llaanngguuaaggeess rreellaatteess ttoo tthheeiirr
probabilistic programming ssyysstteemmss aarree TTuurriinngg ccoommpplleettee
common purpose, namely to "aallllooww pprrooggrraammmmeerrss ttoo
proggrraammmmiinngg llaanngguuaaggeess ssuucchh aass VVeennttuurree.. MMoosstt ooff tthheemm
ffrreeeellyy mmiixx ddeetteerrmmiinniissttiicc aanndd ssttoocchhaassttiicc eelleemmeennttss" and
pprreesseenntt aann eexxtteennssiioonn ((ee..gg.. CChhuurrcchh eexxtteennddiinngg SScchheeme
"to specify a stochastic prroocceessss uussiinngg ssyynnttaaxx tthhaatt
wwiitthh pprroobbaabbiilliissttiicc sseemmaannttiiccss,, PPrroobbLLoogg eexxtteennddiinngg PPrroolloogg))
rreesseemmbblleess mmooddeerrnn pprrooggrraammmmiinngg
oorr aa ffrraammeewwoorrkk ((ee..gg.. IInnffeerr..NNEETT ffoorr CC##,, PPFFPP ffoorr HHaasskkeellll))
languages".[WinStuhGood]
ooff aann eexxiisstteenntt ggeenneerraall ppuurrppoossee llaanngguuaaggee..
NN.. DD.. GGooooddmmaann oobbsseerrvveess tthhaatt pprroobbaabbiilliissttiicc llaanngguuaaggeess
DDeessppiittee tthhee aappppaarreenntt vvaarriieettyy ooff tthhee eexxiissttiinngg pprroobbaabbiilliissttiic
"pprroovviiddee ccoommppoossiittiioonnaall mmeeaannss ffoorr ddeessccrriibbiinngg ccoommpplleexx
pprrooggrraammmmiinngg ssyysstteemmss ((bbootthh llaannguages and
probability distributions", "pprroovviiddee ggeenneerriicc iinnffeerreennccee
frraammeewwoorrkkss)),, tthhee eexxppeerriimmeennttaall cchhaarraacctteerr ooff tthhee mmaajjoorriittyy
engines: tools for ppeerrffoorrmmiinngg eeffffiicciieenntt pprroobbaabbiilliissttiicc
ooff tthheemm mmiigghhtt pprreesseenntt aa cceerrttaaiinn ddiiffffiiccuullttyy wwhheenn uusseedd iinn aa
inference over an arbitrary program" aanndd ppooiinnttss oouutt tthhaatt
real-lliiffee pprroojjeecctt ffoorr aapppplliieedd rraatthheerr tthhaann aaccaaddeemmiicc
"iinn tthheeiirr ssiimmpplleesstt ffoorrmm,, pprroobbaabbiilliissttiicc pprrooggrraammmmiinngg
ppuurrppoosseess.. IInn tthhiiss ccaassee,, ppuurree pprroobbaabbiilliissttiicc llaanngguuaaggeess aarree
languages extend a well-ssppeecciiffiieedd ddeetteerrmmiinniissttiicc

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 3
placed in an unfavourable position compared to the K. Murphy also notes that "the basic rules of probability
frameworks and libraries extending general purpose theory are the same, no matter which interpretation is
languages, because of the restricted number of their adopted" and chooses the Bayesian interpretation for his
users as well as lack of community knowledge and research. [Murphy]
support. S. J. Russell and P. Norvig single out three main
interpretations of probabilities, namely frequentist,
INTERPRETATION OF PROBABILITY objectivist and subjectivist and describe them as
follows. According to the frequentist position "the
The notion of probability belongs to the fundamental
numbers can come only from experiments. The
concepts of probabilistic programming. However, the
objectivist view is that probabilities are real aspects of
interpretation of probability is not always unambiguous.
the universe - propensities of objects to behave in
Generally speaking, there exist two common
certain ways - rather than being just descriptions of an
interpretations of probability, namely frequentist and
observer's degree of belief. In this view, frequentist
Bayesian (or subjective).
measurements are attempts to observe the real
D. Koller writes, "The frequentist interpretation views
probability value. The subjectivist view describes
probabilities as frequencies of events. More precisely,
probabilities as a way of characterizing an agent's
the probability of an event is the fraction of times the
beliefs, rather than having any external physical
event occurs if we repeat the experiment indefinitely".
significance. [RussNor]
[KollFried] K. Murphy observes that in frequentist
S. J. Russell and P. Norvig note, "in the end, even a
interpretation "probabilities represent long run
strict frequentist position involves subjective analysis,
frequencies of events". [Murphy] This interpretation
so the difference probably has little practical
suits for describing events that can be repeated a
importance". It is also pointed out that the total refusal
number of times, like flip of a coin, random choice of a
of subjective methods will inevitably result in the
card, etc. However, it can become problematic to
reference class problem. The reference class problem
describe the probability of an occurrence of a one-time
arises when everything is known about an object. That
time event in future (e.g. the probability of precipitation
makes the object unique and, as a result, devoid of any
the next day, stock exchange course tomorrow, etc.). D.
reference class, needed to collect experimental data.
Koller remarks, "several attempts have been made to
That was characterised as "a vexing problem in the
define the probability for such an event by finding a
philosophy of science". [RussNor]
reference class of similar events for which frequencies
Summing up, it is obvious that the pure frequentist
are well defined; however, none of them has proved
approach of defining probabilities is not sufficient in a
entirely satisfactory". [KollFried] That is when the
number of cases, where an event cannot take place
subjective (Bayesian) interpretation comes into play. D.
multiple times. Moreover, a certain degree of
Koller describes probabilities as "subjective degrees of
subjectivity is unavoidable at the stage of singling out
belief" within this interpretation, observing that "the
relevant properties of an object to be able to assign it to
statement P(α) = 0.3 represents a subjective statement
a reference class. No matter which interpretation of
about one’s own degree of belief that the event α will
probabilities is chosen for a particular case, the most
come about". [KollFried] Similarly, K. Murphy notes,
important thing is compliance with the rules of
"in this [Bayesian] view, probability is used to quantify
probability theory.
our uncertainty about something; hence it is

fundamentally related to information rather than
REASONING PATTERNS
repeated trials", adding that a major advantage of
Bayesian approach is that it provides means to "model Probabilistic reasoning systems are characterized by a
our uncertainty about events that do not have long term high degree of flexibility. The latter is essential, as the
frequencies". [Murphy] system should enable to query about different
Although the subjective interpretation enables the aspects/properties of a particular probabilistically
quantification of uncertainty of events that happen zero modelled situation given evidence about other aspects
or one time but can hardly happen repeatedly, the or properties. Approaches to differentiation between
approach still possesses certain flaws. D. Koller types of inference vary in literature.
criticizes it for being unable to determine "what exactly According to A. Pfeffer, there exist three kinds of
it means to hold a particular degree of belief". The reasoning that probabilistic systems can do:
source of the problem in her view is that "we need to 1. Predict future events. The evidence will
explain how subjective degrees of beliefs (something typically consist of information about the current
that is internal to each one of us) are reflected in our situation.
actions". She suggests employing indirect ways of 2. Infer the cause of events. The evidence here is
attributing degrees of beliefs (e.g. by a betting game) the same as before, together with an additional fact that
where it is possible. Nevertheless it is also pointed out the event of interest has happened.
that "both interpretations lead to the same mathematical 3. Learn from past events to better predict future
rules" and as a result of that "the technical definitions events. The evidence includes all evidence from last
hold for both interpretations". [KollFried] time (making a note that it was from last time), as well
as the new information about the current situation. In

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 4
answering the query, the inference algorithm first infers observations; this is known as explaining
properties of the situation that led to the present. It then away.[GoodTenen]
uses these updated properties to make a prediction. Screening off, as stated by N. D. Goodman and J. B.
The third type of reasoning is characterized by A. Tenenbaum, implies that if the statistical dependence
Pfeffer as "a kind of machine learning".[Pfeff3] between two events A and B is only indirect, mediated
D. Koller introduces more formal terminology in this strictly by one or more other events C, then observing C
context. According to her view, queries with predicted should render A and B statistically independent. This
"downstream" effects of various factors are instances of can occur if events A and B are connected by one or
causal reasoning or prediction, whereas queries, where more causal chains, and all such chains run through the
one reasons from effects to causes, are instances of set of events C, or if C comprises one or more common
evidential reasoning or explanation. It must be pointed causes of A and B.
out that D. Koller’s interpretation of the first two types In case of explaining away if two events A and B are
of reasoning is equivalent to that of A. Pfeffer. The third statistically independent, but they are both causes of one
pattern of reasoning examined by D. Koller is or more other events C, then conditioning on C can
intercausal reasoning, "where different causes of the render A and B statistically dependent.[GoodTenen]
same effect can interact". The subtype explaining away The main types of probabilistic inference can be
is treated as an instance of intercausal reasoning. illustrated with the example of a student's work as
However, D. Koller remarks that "explaining away is follows:
not the only form of intercausal reasoning" and that "the  Causal reasoning implies that a hard working
influence can go in any direction".[KollFried] student is more likely to understand the
S. J. Russell and P. Norvig differentiate between four material, which in turn makes them more likely
distinct types of inference, namely: to be successful with their homework grade.
 diagnostic inferences: from effects to causes  According to the evidential reasoning, flowing
 causal inferences: from causes to effects in the opposite direction, observing a high
 intercausal inferences: between causes of a mark of the student's homework provides
common effect (also known as explaining evidence that the student understood the
away) material, which in turn increases the
 mixed inferences: combining two or more of probability that the student works hard.
the above.[RussNor]  The case of mixed reasoning (composed of the
N. D. Goodman and J. B. Tenenbaum, however, adopt a causal and evidential types) presupposes that if
considerably different approach to differentiation types a student earned a good exam grade, that
of reasoning. Causal relations are considered to be the provides evidence, that they understood the
basic type, encoding the knowledge of the dependencies material, which in turn makes it more likely
in the real world within causal models. They are that they also received a high homework grade.
described as "local, modular, and directed". It is further However, it must be pointed out that the nodes
elaborated that a causal structure is local in the sense "Exam Grade" and "Homework Grade" are
that many related events are not related directly, but conditionally independent given the node
rather are connected only through causal chains of "Understands Material". In other words, if it is
several steps, a series of intermediate and more local already known that the student understands the
dependencies. Causal relations are also described as material, then the fact of the student's receiving
modular in the sense that any two arbitrary events in the a good exam grade does not deliver any new
world are most likely to be causally unrelated, or information about the homework grade.
independent. Causal relations are always directed, as  In case of intercausal reasoning, also called
causal influence flows only one way along a causal explaining away, if the value of the node
relation.[GoodTenen] "Understands Material" (as a common effect)
Causal dependence is opposed to statistical dependence is unknown, then values of the nodes "Smart"
or correlation. According to N. D. Goodman and J. B. and "Hard working" are independent.
Tenenbaum, two events may be statistically dependent However, if it is known that "Understands
even if there is no causal chain running between them, Material" is true, then the fact of "Smart" being
as long as they have a common cause (direct or true reduces the probability that "Hard
indirect). working" is true.[CS181-Lec]
e.g. Cough and fever are not causally dependent but
they are statistically dependent, because they both
depend on cold.[GoodTenen]
However, events that are considered to be statistically
dependent a priori may become independent when
conditioned on some other observation; this is called
screening off, or context-specific independence. Also,
events that are statistically independent initially may
become dependent when conditioned on other

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 5
tells you the most likely state of variables in the
model".[Pfeff2]
According to L. E. Sucar, "the MPE or abduction
problem consists in determining the most probable
values for a subset of variables (explanation subset) in a
BN given some evidence". It is also underlined that "the
MPE is not the same as the union of the most probable
value for each individual variable in the explanation
subset".[Suc]
D. Koller considers MAP queries to fulfil "a second
important type of task" consisting in "finding a high-
probability joint assignment to some subset of

Figure 3: Student's work [CS181-Lec] variables". So, according to D. Koller, MPE query’s aim
is "to find the MAP assignment - the most likely
To summarize, causal, evidential, mixed and intercausal assignment to all of the (non-evidence) variables" or if
patterns of reasoning seem to be essential means of defined more formally:
constructing relationships within a probabilistic if we let W=X−E, our task is to find the most likely
reasoning system. Although the details of different assignment to the variables in W given the evidence
classifications of reasoning patterns (such as naming � =�:
and nesting of elements) tend to vary, the pragmatic MAP(W | e)= argmax P(w,e), where, in general,
�
logic behind them shows a certain degree of similarity. argmax f(x) represents the value of x for which f(x) is
x
maximal.[KollFried]
TYPES OF QUERIES (PROBABILITY QUERIES Addressing the difference between probability queries
and MAP queries, D. Koller states, "in a MAP query,
AND MAPS)
we are finding the most likely joint assignment to W. To
According to D. Koller and N. Friedman, two main find the most likely assignment to a single variable A,
types of queries can be singled out in the probabilistic we could simply compute P(A | e) and then pick the
context, namely probability queries and MAP most likely value. However, the assignment where each
(maximum a posterior) or MPE (Most Probable variable individually picks its most likely value can be
Explanation) queries.[KollFried] K. Karkera’s quite different from the most likely joint assignment to
classification adheres to the same types.[Kark] all variables simultaneously".[KollFried]
D. Koller characterizes probability queries as "perhaps Likewise, K. Karkera defines MAP as "the highest
the most common query type", which is comprised of probability joint assignment to some subsets of
two types: variables", emphasizes that "the MAP assignment
 the evidence: a subset E of random variables in cannot be obtained by simply taking the maximum
the model, and an instantiation e to these probability value in the marginal distribution for each
variables; random variable" [Kark] and illustrates it with the
 the query variables: a subset Y of random following example.
variables in the network.[KollFried] e.g. There are two non-independent random variables X
According to D. Koller and N. Friedman, the task and Y, where Y is dependent on
consists in the computation of P(Y | E=e), "that is, the X. The MAP assignment for the random variable X is
posterior probability distribution over the values y of Y, X1 since it has a higher value.
conditioned on the fact that � =�. This expression can
also be viewed as the marginal over Y, in the Table 1: Probability Distribution over X [Kark]
distribution we obtain by conditioning on e".[KollFried]
However, there exist situations, when the most probable X X
0 1
result is of interest. That is where MAP queries, also 0.4 0.6
known as MPE queries come in play.
A. Pfeffer describes MPE query as the one to be Table 2: Probability Distribution P(Y | X) [Kark]
employed, when it is needed "to know the world that is
the most probable explanation of the data", noting that P(Y | X) Y Y
0 1
"sometimes, rather than knowing a probability X 0.1 0.9
0
distribution over outcomes, you want to know which X 0.5 0.5
1
outcomes are the most likely". He underscores that "the
goal of probabilistic inference in this case can be to find However, the MAP assignment to random variables (X,
out the most likely state of the system", because Y) in the joint distribution is (X , Y ), and the MAP
0 1
"identifying the most likely state tells you the most assignment to X (X ) is not a part of the MAP of the
1
likely cause of the problems you’re seeing". So, joint assignment.
according to A. Pfeffer, MPE query is "the query that
Table 3: The Joint Distribution over X and Y [Kark]

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 6
Assignment Value A CPD specifies a probability distribution over the child
X , Y 0.04 variable given the values of its parents. A CPD
0 0
X , Y 0.36 considers every possible assignment of values to the
0 1
X , Y 0.3 parents, when the value of a parent can be any value in
1 0
X , Y 0.3 its domain. For each such assignment, it defines a
1 1
probability distribution over the child. When a variable
The marginal MAP query can be regarded as a more has no parents, the CPD just specifies a single
probability distribution over the variable".[Pfeff2]
general query type, which consists of "elements of both
a conditional probability query and a MAP query". e.g. The following simple model contains five random
[KollFried] variables with corresponding CPDs: the student’s
Thus, with a subset of variables Y of the query and with intelligence, the course difficulty, the grade, the
the task to find the most likely assignment to the student’s SAT score, and the quality of the
recommendation letter.
variables in Y given the evidence � =� and �=�−

�−�:
��� (� | �)= ������ ∑ �(�,� | �) , marginal
��� � �
MAP contains "both summations and
maximizations".[KollFried]

PROBABILISTIC GRAPHICAL MODELS
Probabilistic programming needs a formal
representation of real life situations to perform
reasoning under uncertainty. Introduction of variables
denoting the quantified knowledge of the situation, its
agents and objects is an essential step to enable this kind
of reasoning. As D. Koller remarks "domains can be
characterized in terms of a set of random variables,
where the value of each variable defines an important

property of the world", emphasizing that "the set of
Figure 4: Student Bayesian network [KollFried]
possible variables and their values is an important

design decision, and it depends strongly on the
A Markov network is defined by A. Pfeffer as a
questions we may wish to answer about the
representation of a probabilistic model consisting of
domain".[KollFried] However, the introduction of
three things:
variables formally representing elements of a particular
1. A set of variables. Each variable has a domain,
situation is not sufficient for building a viable model of
which is the set of possible values of the variable.
this situation. It is also the interaction of the elements,
2. An undirected graph in which the nodes are
their mutual influence that needs to be reflected in the
variables. The edges between nodes are undirected. This
model. In other words, there should be means of
graph is allowed to have cycles.
encoding dependencies. As A. Pfeffer states,
3. A set of potentials, providing the numerical
"dependencies capture relationships between variables"
parameters of the model.[Pfeff2]
and he singles out two general kinds of them, namely
As opposed to Bayesian networks, where each variable
"directed dependencies, which express asymmetric
is characterised by a CPD, variables in Markov
relationships, and undirected dependencies, which turn
networks do not have their own numerical parameters.
into symmetric relationships", pointing out that
The interaction between variables can be represented
"probabilistic models essentially boil down to a
and quantified with the help of a function called a
collection of directed and undirected
potential. As stated by A. Pfeffer, "When there’s a
dependencies".[Pfeff2] The two main frameworks that
symmetric dependency, some joint states of the
are used for this kind of dependency-encoding are
variables that are dependent on each other are more
Bayesian networks and Markov networks, expressing
likely than others, all else being equal. The potential
directed and undirected dependencies respectively.
specifies a weight for each such joint state. Joint states
A. Pfeffer treats the Bayesian network as "a
with high weights are more likely than joint states with
representation of a probabilistic model consisting of
low weight, all else being equal. The relative probability
three components:
of the two joint states is equal to the ratio between their
1. A set of variables with their corresponding
weights, again all else being equal".[Pfeff2] He defines
domains. The domain of a variable specifies which
a potential as "simply a function from the values of
values are possible for that variable.
variables to real numbers", stressing the fact that only
2. A directed acyclic graph in which each variable
positive real numbers or zero are allowed as the values
is a node.
of a potential". Describing the interaction of potential
3. For each variable, a conditional probability
functions with the graph structure, A. Pfeffer singles out
distribution (CPD) over the variable given its parents.
two main rules, namely:

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 7
1. A potential function can only mention perfectly computes the probabilities", one might "think
variables that are connected in the graph. that it’s not suitable for real-world applications with
2. If two variables are connected in the graph, complex models", which is not the case. Variable
they must be mentioned together by some potential Elimination is frequently employed, as long as the
function.[Pfeff2] model has the right structure. In particular, it is of
e.g. The following model describes students teamwork importance whether variables can be eliminated
in pairs. The following pairs can work well together: "without adding too many edges to the VE graph,
Alice and Bob; Bob and Charles; Charles and Debbie; leaving the size of the largest clique [set of nodes in a
and Debbie and Alice. Each interaction is described graph, which are all connected with each other] in the
with a factor. For instance, � (A,B) means that Alice VE graph small and the complexity low". Hidden
�
and Bob tend to agree with each other. Markov models with a possible application of speech
recognition and also parse trees in natural language
processing are among structures, which allow running
inference with VE.[Pfeff2]
According to A. Pfeffer, an approximate algorithm
Belief Propagation has fewer limitations of use
compared to VE and for a "model with discrete

Figure 5: Students Teamwork Model [KollFried] variables, BP is a good candidate technique to use".
Possible applications of BP include Markov networks
INFERENCE: MAIN GROUPS OF INFERENCE for image analysis and loopy Bayesian networks for
medical diagnostics. The higher applicability of BP is a
ALGORITHMS
result of the fact that BP "operates using the moral
The variety of algorithms dealing with the task of graph (the initial VE graph), without adding edges".
drawing inference can be examined in different However, since adding these edges is necessary for
perspectives. Thus, for example, A. Pfeffer puts stress correct inference, not adding them will result in errors.
on differentiating between factored and sampling Nevertheless, inference can be approximately correct
algorithms, which he defines as follows: with a certain error margin even when these edges
 Factored algorithms - group of algorithms that aren’t added. [Pfeff2]
operate on data structures called factors that D. Koller also emphasizes the importance of choosing
capture the probabilistic model being reasoned the right algorithm, as she addresses the problem of
about (e.g. Variable Elimination (VE) inference complexity, noting that "exponential blowup
algorithm and Belief Propagation (BP) of the inference task is (almost certainly) unavoidable in
algorithm). the worst case: the problem of inference in graphical
 Sampling algorithms are algorithms creating models is NP-hard, and therefore it probably requires
examples of possible worlds from the exponential time in the worst case. Even worse,
probability distribution and using those approximate inference is also NP-hard".[KollFried]
examples to answer queries (MCMC algo- Nevertheless, she also stresses, "the story does not end
rithms).[Pfeff2] with this negative result. In general, we care not about
K. Karkera makes use of juxtaposition of exact the worst case, but about the cases that we encounter in
inference (e.g. Variable Elimination, Tree Algorithms) practice" and that "many real-world applications can be
and approximate inference methods (MCMC group), tackled very effectively using exact or approximate
parallelly addressing the problem of complexity of inference algorithms for graphical models".[KollFried]
inference tasks with the words, "even approximate
inference is NP-hard". He notes, that inference might PROBABILISTIC OBJECT-ORIENTED
seem to be "a hopeless task, but that is only in the worst
KNOWLEDGE REPRESENTATION
case" and that generally exact inference can successfully
serve "to solve certain classes of real-world problems Object-oriented programming paradigm has become an
(such as Bayesian networks that have a small number of inalienable part of the modern landscape of software
discrete random variables)", whereas approximate development. Hence, it is appropriate to consider most
inference is required "for larger problems".[Kark] general concepts of exercising probabilistic
Other scholars, such as D. Koller, S. J. Russel and P. programming in the context of OOP.
Norvig also hold to the classification of inference A. Pfeffer accentuates the following two advantages of
algorithms in two major groups, namely exact inference OOP, namely
algorithms (with VE and clustering algorithms as  Providing structure to complex programs.
classical examples) and approximate inference Objects are coherent units that capture a set of
algorithms, including a family of sampling methods. data and behaviors. An object provides a
It must be pointed out that the choice of a suitable uniform interface to these data and behaviors,
inference algorithm depends on the structure of the and the internals of the object are encapsulated
model. For example, A. Pfeffer remarks that since from the rest of the program. This allows the
Variable Elimination "is an exact algorithm that programmer to modify the internals of the

Anwendungen und Konzepte der Wirtschaftsinformatik (ISSN: 2296-4592) http://akwi.hswlu.ch Nr. 5 (2017) 8