CENTURION: Incentivizing Multi-Requester Mobile Crowd Sensing

0.38 MB·

by  Haiming Jin

CENTURION: Incentivizing Multi-Requester
Mobile Crowd Sensing
Haiming Jin∗, Lu Su†, Klara Nahrstedt∗
∗Department of Computer Science, University of Illinois at Urbana-Champaign, IL, USA
†Department of Computer Science and Engineering, State University of New York at Buffalo, NY, USA
Email: [email protected], [email protected], [email protected]
Abstract—The recent proliferation of increasingly capable Therefore, it is necessary to design an effective incentive
7
mobile devices has given rise to mobile crowd sensing (MCS) mechanism that can achieve the maximum user participation.
1
systems that outsource the collection of sensory data to a crowd
Dueto the paramountimportanceof stimulatingparticipation,
0
of participating workers that carry various mobile devices.
2 manyincentivemechanisms[7–31]havebeenproposedbythe
Aware of the paramount importance of effectively incentiviz-
n ing participation in such systems, the research community has research community. However, most of these aforementioned
a proposed a wide variety of incentive mechanisms. However, past literature assume that there is only one data requester
J different from most of these existing mechanisms which assume who also serves as the platform in the MCS system. In
the existence of only one data requester, we consider MCS
6 practice, however, there are usually multiple data requesters
systems with multiple data requesters, which are actually more
competingforhumanresources,whousuallyoutsourceworker
common in practice. Specifically, our incentive mechanism is
]
T basedondoubleauction,andisabletostimulatetheparticipation recruiting to third-party platforms (e.g., Amazon Mechanical
G of both data requesters and workers. In real practice, the Turk [32]) that have already gathered a large number of
incentive mechanism is typically not an isolated module, but workers. Therefore, in this paper, we focus on such MCS
.
s interacts with the data aggregation mechanism that aggregates systems where three parties, including the data requesters,
c workers’ data. For this reason, we propose CENTURION, a
a platform (i.e., a cloud-based central server), as well as a
[ novel integrated framework for multi-requester MCS systems,
consisting of the aforementioned incentive and data aggregation crowd of participatingworkersco-exist, and aim to developa
1
mechanism. CENTURION’s incentive mechanism satisfies truth- newincentivemechanismthatcandecidewhichworkerserves
v
fulness, individual rationality, computational efficiency, as well as which data requester at what price.
3
guaranteeingnon-negativesocialwelfare,anditsdataaggregation
3 In real practice, the sensory data provided by individual
mechanism generates highly accurate aggregated results. The
5 workers are usually quite unreliable due to various factors
desirablepropertiesofCENTURIONarevalidatedthroughboth
1 theoretical analysis and extensive simulations. (e.g., poor sensor quality, lack of sensor calibration, envi-
0
ronment noise). Hence, in order to cancel out the possible
.
1 I. INTRODUCTION errors from individual workers, it is highly necessary that the
0
Recentyearshavewitnessedthe riseof mobilecrowdsens- platform utilizes a data aggregation mechanism to properly
7
1 ing(MCS),anewly-emergedsensingparadigmthatoutsources aggregate their noisy and even conflicting data. In an MCS
: thecollectionofsensorydatatoacrowdofparticipatingusers, system, the incentive and the data aggregation mechanism
v
namely (crowd) workers, who usually carry increasingly ca- are usually not isolated from each other. In fact, the data
i
X pablemobiledevices(e.g.,smartphones,smartwatches,smart- aggregation mechanism typically interacts with the incentive
r glasses) with a plethora of on-board sensors (e.g., gyroscope, mechanism, and thus, affects its design and performance.
a
camera, GPS, compass, accelerometer). Currently, a large Intuitively, if the platform aggregates workers’ data in naive
variety of MCS systems [1–6] have been deployed that cover ways (e.g., voting and average) that treat all workers’ data
almost every aspect of our lives, including healthcare, smart equally,the incentivemechanismdoesnotneed to distinguish
transportation, environmentalmonitoring, and many others. them with respect to their reliability. However, a weighted
To perform the sensing tasks, the participating workers aggregation method that puts higher weights on more reli-
typicallyconsumetheirown resourcessuch as computingand able workers is much more desirable, because it shifts the
communicating energy, and expose themselves to potential aggregated results towards the data provided by the workers
privacythreatsbysharingtheirpersonaldata.Forthisreason,a with higher reliability. Accordingly, the incentive mechanism
participantwouldnotbeinterestedinparticipatinginthesens- should also incorporateworkers’ reliability, and selects work-
ing tasks, unless she receives a satisfying reward to compen- ers that are more likely to provide reliable data.
sate her resource consumption and potential privacy breach. Therefore, different from most of the aforementioned ex-
isting work [7–31], we propose CENTURION1, a novel in-
Wesincerely thankProfessorJuliaChuzhoyforhervaluable contribution. tegrated framework for multi-requester MCS systems, which
WegratefullyacknowledgethesupportofNationalScienceFoundationgrants
CNS-1330491, and 1566374. The views and conclusions contained in this
documentarethoseoftheauthorsandshouldnotbeinterpretedasnecessarily 1The name CENTURION comes from inCENTivizing mUlti-Requester
representingtheofficialpolicies,eitherexpressedorimplied,ofthesponsors. mobIlecrOwdseNsing.
consists of a weighted data aggregation mechanism that con- III. PRELIMINARIES
siders workers’ diverse reliability in the calculation of the
Inthissection,weintroducethesystemoverview,reliability
aggregatedresults, together with an incentive mechanism that
level model, auction model, as well as the design objectives.
selects workers who potentially will provide more reliable
data. Specifically, CENTURION’s incentive mechanism is A. System Overview
based ondoubleauction[33], which involvesauctionsamong CENTURION is an MCS system framework consisting
not only the workers, but also the data requesters, and is able of a cloud-based platform, a set of participating workers,
to incentivize the participation of both data requesters and denoted as W = {w ,··· ,w }, and a set of requesters,
1 N
workers. This paper makes the following contributions. denoted as R = {r ,··· ,r }. Each requester r ∈ R has
1 M j
• Differentfromexistingwork,weproposeanovelintegrated a sensing task τ to be executed by the workers. The set of
j
framework for multi-requester MCS systems, called CEN- all requesters’ tasks is denoted as T = {τ ,··· ,τ }. We
1 M
TURION, consisting of a data aggregationand an incentive are specifically interested in the scenario where T is a set of
mechanism. Such an integrated design, which captures the M differentbinary classificationtasks that requireworkersto
interactive effects between the two mechanisms, is much locally decide the classes of the events or objects, and report
more complicated and challenging than designing them to the platform their local decisions (i.e., the labels of the
separately. observed events or objects). Such MCS systems, collecting
• CENTURION’s double auction-based incentive mechanism binarylabels from the crowd,constitute a large portionof the
isabletoincentivizetheparticipationofbothdatarequesters currently deployed MCS systems (e.g., congestion detection
andworkers,andbearsmanydesirableproperties,including systems that decide whether or not particular road segments
truthfulness,individualrationality,computationalefficiency, are congested [2], geotagging campaigns that tag whether
as well as non-negative social welfare. bumps or potholes exist on specific segments of road surface
• The data aggregation mechanism of CENTURION takes [1, 3]).
intoconsiderationworkers’reliability,and calculateshighly Each task τ has a true label l ∈ {−1,+1}, unknown to
j j
accurate aggregated results. the requesters, the platform, and the workers. If a worker w
i
In the rest of this paper, we first discuss the past literature is chosen to execute task τ , she will provide to the platform
j
that are related to this work in Section II, and introduce a label l . We define l = [l ] ∈ {−1,+1,⊥}N×M as the
i,j i,j
the preliminaries in Section III. Then, the design details of matrix containing all workers’ labels, where l = ⊥ means
i,j
CENTURION’sdataaggregationandincentivemechanismare thattaskτ isnotexecutedbyworkerw .Foreverytaskτ ,the
j i j
described in Section IV. In Section V, we conduct extensive platform aggregatesworkers’ labels into an aggregatedresult,
simulations to validate the desirable properties of CENTU- denoted as l , so as to cancel out the errors from individual
j
RION. Finally in Section VI, we conclude this paper. workers. The framework of CENTURION is given in Figure
1, and we dbescribe its workflow as follows.
II. RELATEDWORK
…
Aware of the paramount importance of attracting worker Requesters
poAapmretdoicnivgpaarttihiooeunms,,inthgceaemntreiev-setehamerocerhcehtiaccnoimisnmmcesunn[t7iivt–ye31mh]aefscohrraeMncieCsnmStlsysy[7sdt–ee2vm8esl]-,. gnisneS1stseuqeR sdiB2 stnemyaP8 detagerggA7stluseR
which utilize either auction [10–13, 18–28] or other game- Incentive Platform Data Aggregation
theoretic models [8, 9, 14–17], have gained increasing popu- Mechanism 5Winner Sets Mechanism
larityduetotheirabilitytotackleworkers’selfishandstrategic
bpelahtafovriomr’ss. pTrhoefiste[m14e–c2h3a]niosmr ssotcyiaplicwalelylfaariem[9to–1m3]a,xainmdizmeitnhie- teS ksaT3 sdiB4 stnemyaP9 slebaL6
…
mize the platform’spayment[7, 8, 24–27] or socialcost [28].
Workers
Different from most of the aforementioned past literature
which assume that there exists only one data requester, we
Fig. 1. Framework of CENTURION (where circled numbers represent the
propose a novel incentive mechanism for MCS systems with orderoftheevents).
multipledatarequestersthatcompeteforhumanresources.In • Incentive Mechanism. Firstly, in the double auction-based
fact,theredoexistseveralpriorwork[19,22,28]designingin- incentive mechanism, each requester r submits to the
j
centivemechanismsforthemulti-requesterscenario.However, platform a sensing request containing the sensing task τ
j
they do not provide any joint design of the data aggregation to be executed (step 1), and a bid a , the amount she is
j
and the incentive mechanism as in this paper, which is willing to pay if the task is executed (step 2). Then, the
much more challenging than designing the two mechanisms platformannouncesthesetofsensingtasksT totheworkers
as isolated modules. Moreover, although similar integrated (step 3).Afterreceivingthetaskset,everyworkerw sends
i
designs that consider the two mechanisms are proposed in totheplatformthesetoftasksshewantstoexecute,denoted
some existing work [25, 26], as previously mentioned, they as Γ ⊆ T, as well as a bid b , which is her bidding price
i i
assumethatonlyonedatarequesterexistsintheMCSsystem. for executing them (step 4). Based on received bids, the
platform determines the set of winning requesters S , the
R is willing to pay for the execution of her task. Each worker
set of winning workers S , as well as the payment pr
W j w is interested in executing one subset of the tasks, denoted
i
charged from every winning requester r and the payment
j as Γ ⊆T, and bids to the platform b , her bidding price for
pw paid to every winning worker w (step 5). Note that i i
i i executing these tasks. Her actual sensing cost for executing
losing requesters’ tasks are not executed, and thus, they do
all tasks in Γ is denoted as c . Both the requesters’ values
i i
not submit any payment. Similarly, losing workers do not
and workers’ costs are unknown to the platform.
receive any payment, as they do not execute any task.
Then, we define a requester’s and worker’s utility, as well
• Data AggregationMechanism. Next, the platformcollects as the platform’s profit in Definition 3, 4, and 5.
the labels submitted by the winning workers (step 6),
Definition 3 (Requester’s Utility). A requester r ’s utility is
calculates the aggregated results, and sends them to the j
defined as
winning requesters (step 7).
• Finally, the platform charges prj from winning requester rj ur = vj −prj, if rj ∈SR. (2)
(step 8), and pays pwi to winning worker wi (step 9). j (0, otherwise
We denote the requesters’ and workers’ bid profile as a =
(a1,··· ,aM) and b = (b1,··· ,bN), respectively. Moreover, Definition4(Worker’sUtility). Aworkerwi’sutilityisdefined
the requesters’ and workers’ payment profile is denoted as as
pw−c , if w ∈S
pr =(pr1,··· ,prM) and pw =(pw1,··· ,pwN), respectively. uwi =(0,i i otheirwiseW. (3)
B. Reliability Level Model
Definition 5 (Platform’s Profit). The profit of the platform is
Beforeworkerw executestask τ ,herlabelaboutthistask
i j defined as
can be regarded as a random variable L . Then, we define
i,j u = pr− pw.
the reliability level of a worker in Definition 1. 0 j i (4)
Definition1(ReliabilityLevel). Aworkerw ’sreliabilitylevel j:rXj∈SR i:wXi∈SW
i
θ abouttaskτ isdefinedastheprobabilitythatsheprovides BasedonDefinition3,4,and5,wedefinethesocialwelfare
i,j j
a correct label about this task, i.e., of the MCS system in Definition 6.
Definition 6 (Social Welfare). The social welfare of the MCS
θ =Pr[L =l ]∈[0,1]. (1)
i,j i,j j system is defined as
Moreover, we denote the workers’ reliability level matrix as u =u + uw+ ur
θ=[θ ]∈[0,1]N×M. social 0 i j
We ia,jssume that the platform knows the reliability level i:wXi∈W j:Xrj∈R
= v − c . (5)
matrix θ a priori, and maintains a historical record of it. j i
In practice, the platform could obtain θ through various j:rXj∈SR i:wXi∈SW
approaches.For example,as, in manyscenarios, workerstend Clearly,thesocialwelfareisthesumoftheplatform’sprofit
to havesimilar reliabilitylevelsforsimilar tasks, the platform and all requesters’ and workers’ utilities.
couldassigntoworkerssometaskswithknownlabels,anduse
D. Design Objectives
workers’ labels about these tasks to estimate their reliability
levelsforsimilar tasksas in[34]. Incaseswheregroundtruth Inthispaper,weaimtoensurethatCENTURIONbearsthe
labelsarenotavailable,θcanstillbeeffectivelyinferredfrom following advantageous properties.
workers’characteristics(e.g.,thepricesofa worker’ssensors, Sincetherequestersarestrategicandselfishinourmodel,it
a worker’s experience and reputation for similar tasks) using ispossiblethatanyrequesterrj submitsa bidaj thatdeviates
the algorithmsproposed in [35], or estimated using the labels from vj (i.e., her value for task τj). Similarly, any worker wi
previously submitted by workers about similar tasks by the might also submit a bid bi that differs from ci (i.e., her cost
methods in [36, 37]. for executing all tasks in Γi). Thus, one of our objectives is
todesigna truthfulincentivemechanismdefinedinDefinition
C. Auction Model 7.
Inthispaper,weconsiderthescenariowherebothrequesters Definition 7 (Truthfulness). A MELON double auction is
andworkersarestrategicandselfishthataimtomaximizetheir truthfulifandonlyifbiddingvj andciisthedominantstrategy
own utilities. Since CENTURION involves auctions among for each requester rj and worker wi, i.e., bidding vj and ci
not only the workers, but also the requesters, we utilize the maximizes, respectively, the utility of each requester rj and
following double auction for Multi-rEquester mobiLe crOwd worker wi, regardless of other requesters’ and workers’ bids.
seNsing (MELON double auction), formally defined in Defi- By definition 7, we aim to ensure that both requesters and
nition 2, as the incentive mechanism. workersbidtruthfullytotheplatform.Apartfromtruthfulness,
Definition 2 (MELON Double Auction). In a double auction anotherdesirablepropertythatweaimtoachieveisindividual
for multi-requester mobile crowd sensing (MELON double rationality defined in Definition 8.
auction), each requester r obtains a value v , if her task Definition 8 (IndividualRationality). A MELON double auc-
j j
τ is executed, and bids to the platform a , the amount she tion is individual rational if and only if no requesters or
j j
workers receive negative utilities, i.e., we have ur ≥ 0, and Algorithm 1: Data Aggregation Mechanism
j
uwi ≥0, for every requester rj and worker wi, respectively. Input:θ, l, Γ, SR, SW;
Individual rationality is a crucial property to stimulate the Output: lj|rj ∈SR ;
participationofbothrequestersandworkers,becauseitensures 1 foreach j(cid:8)s.t. rj ∈SR(cid:9)do
b
that the charge to a requester is no larger than her value, and 2 lj ← i:wi∈SW,τj∈Γi 2θi,j−1 li,j;
a worker’s sensing cost is also totally compensated. As men- P (cid:0) (cid:1)
tioned in Section III-A, CENTURION aggregates workers’ 3 retubrn lj|rj ∈SR ;
(cid:8) (cid:9)
labels to ensure that the aggregated results have satisfactory a winner bby the incentive mechanism. The aggregated result
accuracy, which is mathematically defined in Definition 9. l for each winning requester r ’s task τ is calculated (line
j j j
Definition 9 (βj-Accuracy). A task τj is executed with βj- 1-2) using Equation (6) with the weight
accuracy if and only if Pr[Lj 6= lj] ≤ βj, where βj ∈ (0,1), b
λ =2θ −1, ∀r ∈S , w ∈S , τ ∈Γ . (7)
and L denotes the random variable representing the aggre- i,j i,j j R i W j i
j
gated result for task τ . b
j By Equation(7), wehavethatλ ,i.e.,workerw ’sweight
i,j i
BybDefinition 9, βj-accuracy ensures that the aggregated fortaskτj,increaseswithθi,j,whichconformstoourintuition
resultequalsto the truelabel with highprobability.Note that, thatthehighertheprobabilitythatworkerw providesacorrect
i
foreverytaskτj,βj isaparameterchosenbytheplatform,and label about task τj, the more her label li,j should be counted
a smaller βj implies a stronger requirement for the accuracy. in the calculation of the aggregated result about this task.
In short, our objectives are to ensure that the proposed We provide the formal analysis about the data aggregation
CENTURIONframeworkprovidessatisfactoryaccuracyguar- mechanism in Section IV-A2.
antee for the aggregated results of all executed tasks, and
2) Analysis
incentivizes the participation of both requesters and workers
In Theorem 1, we prove that the aggregated results calcu-
in a truthful and individual rational manner.
lated by Algorithm 1 has desirable accuracy guarantee.
IV. DESIGN DETAILS Theorem 1. For each executed task τ , the data aggregation
j
Inthissection,wepresentthedesigndetailsoftheincentive mechanismgiveninAlgorithm1minimizestheupperboundof
and data aggregation mechanism of CENTURION. the error probabilityof the aggregatedresult, i.e., Pr[Lj 6=lj]
(whereL istherandomvariablerepresentingtheaggregated
j
A. Data Aggregation Mechanism
resultfor task τ mentionedin Definition9), andsatisbfies that
j
1) Proposed Mechanism b
(2θ −1)2
Although the data aggregation mechanism follows the in- Pr[L 6=l ]≤exp − i:wi∈SW,τj∈Γi i,j . (8)
j j
centivemechanisminCENTURION’sworkflow,weintroduce P 2 !
it first, as it affects the design of the incentive mechanism.
Proofb. WedenoteX astherandomvariableforworkerw ’s
In order to capture the effect of workers’ diverse reliability i,j i
weighted label about task τ , i.e., X = λ l with proba-
on the calculation of the aggregated results, CENTURION j i,j i,j j
bility θ , and X =−λ l with probability1−θ . Then,
adopts the following weighted aggregation method. That is, i,j i,j i,j j i,j
we define X = X , and thus, E[X ] =
theaggregatedresultlj foreveryexecutedtaskτj iscalculated j E[Xi:w]i=∈SW,τj∈Γi i,j l λ (2θ −j1).
as i:wi∈SW,τj∈Γi Pi,j i:wi∈SW,τj∈Γi j i,j i,j
b The error probabilityof the aggregatedresult can be calcu-
lj =sign λi,jli,j , (6) lPatedas Pr[L 6=l ]=Pr[XP<0|l =1]Pr[l =1]+Pr[X ≥
! j j j j j j
i:wi∈SXW,τj∈Γi 0|lj =−1]Pr[lj =−1], and based on the Chernoff-Hoeffding
b
whereλ >0isworkerw ’sweightontaskτ .Furthermore, bound, we hbave
i,j i j
thefunctionsign(x)equalsto+1,ifx≥0,and−1otherwise. Pr[Xj <0|lj =1]=Pr[E[Xj]−Xj >E[Xj]|lj =1]
Intuitively, higher weights should be assigned to workers
whoaremorelikelytosubmitcorrectlabels,whichmakesthe ≤exp − 2(E[Xj|lj =1])2
aggregated results closer to the labels provided by more reli- i:wi∈SW,τj∈Γi(2λi,j)2!
2
able workers. In fact, many state-of-the-art literature [36, 37] =exp − P i:wi∈SW,τj∈Γiλi,j(2θi,j−1) .
utilize such weighted aggregation method to aggregate work- (cid:0)P 2 i:wi∈SW,τj∈Γiλ2i,j (cid:1) !
ers’data.Astheweightλ ’shighlyaffecttheaccuracyofthe
aggregatedresults, we proi,pjose, in the following Algorithm 1, Then, we define the vector λj P= [λi,j] for every executed
the data aggregation mechanism of CENTURION. task τj, which contains every λi,j such that wi ∈ SW, and
Algorithm 1 takes as inputs the reliability level matrix θ, τj ∈ Γi. Therefore, minimizing the upper bound of Pr[Xj <
the workers’ label matrix l, the profile of workers’ interested 0|lj =1]isequivalenttofindingthevectorλj thatmaximizes
tasksets,denotedasΓ=(Γ1,··· ,ΓN),thewinningrequester the function f(λj) defined as
setS ,andthewinningworkerset S .Note thata largeθ
R W i,j 2
indicates that a worker wi has a high reliability level for task f(λ )= i:wi∈SW,τj∈Γiλi,j(2θi,j −1) .
τj, and any worker wi with θi,j ≤0.5 will not be selected as j (cid:0)P i:wi∈SW,τj∈Γiλ2i,j (cid:1)
P
Based on the Cauchy-Schwarz inequality, we have Corollary 1 gives us a sufficient condition, represented by
Inequality (11), that the set of winning workers S selected
W
f(λj)≤ (cid:0)Pi:wi∈SW,τj∈Γiλ2i,j(cid:1)(cid:0)Pi:wi∈SWλ2,τj∈Γi(2θi,j−1)2(cid:1) bsaytitshfeyisnoceanstitvoeamcheicehvaenβisjm-ac(pcruorpacoysefdorineaScehcteioxnecIuVt-eBd)tasshkouτljd.
i:wi∈SW,τj∈Γi i,j
P 2 B. Incentive Mechanism
= (2θi,j−1) ,
i:wi∈SXW,τj∈Γi Now, we introduce the design details of CENTURION’s
incentive mechanism, including its mathematical formulation,
and equality is achieved if and only if λ ∝2θ −1. Thus,
i,j i,j the hardness proof of the formulated integer program, the
Pr[Xj <0|lj =1]≤exp − i:wi∈SW,τj∈Γi(2θi,j−1)2 . (9) p1r)oMpoastehdemmaetcichaalniFsomr,mauslawtieolnl as the corresponding analysis.
P 2 !
As mentioned in Section III-C, CENTURION’s incentive
Similarly, from the Chernoff-Hoeffdingbound, we have
mechanism is based on the MELON double auction defined
Pr[Xj ≥0|lj =−1]≤exp − (cid:0)Pi:w2i∈SWi:w,τij∈∈SΓWi,λτij,∈j(Γ2iθλi,2ij,j−1)(cid:1)2!. gidnuoauDrbaelnefitneaietuiinocgntio2sna.tIitnshfaattchtiomsraypxaipdmeairzt,aewsaetghageirmegsaototciioadnlesiwagcenclfuaarraeMc,yE.wLThOihlNee
The upper bound of Pr[Xj >0|lPj =−1] is also minimized formal mathematical formulation of its winner selection
if and only if λi,j ∝ 2θi,j −1 based on the Cauchy-Schwarz problem is providedin the following MELON double auction
inequality, and we have social welfare maximization (MELON-SWM) problem.
Pr[Xj ≥0|lj =−1]≤exp − i:wi∈SW,τj∈Γi(2θi,j−1)2 . (10) MELON-SWM Problem:
P 2 !
max ajyj− bixi (13)
From Inequality (9) and (10), we have that when λ =
i,j j:Xτj∈T i:wXi∈W
2θ −1, the upper bound of Pr[L 6=l ] is minimized, and
i,j j j 2 1
Pr[L 6=l ]≤exp − i:wi∈bSW,τj∈Γi(2θi,j −1)2 , s.t.i:wi∈XW,τj∈Γi(2θi,j−1) xi≥2ln(cid:18)βj(cid:19)yj, ∀τj ∈T (14)
j j
P 2 ! xi,yj ∈{0,1}, ∀wi∈W, τj ∈T (15)
whichbexactly proves Theorem 1. Constants.TheMELON-SWMproblemtakesasinputsthe
tasksetT,theworkersetW,therequesters’andworkers’bid
By Theorem 1, we have that the data aggregation mecha- profile a and b, the profile of workers’ interested task sets Γ,
nism proposed in Algorithm 1 upper bounds the error proba- the workers’ reliability level matrix θ, and the β vector.
bility Pr[Lj 6=lj] by exp − 21 i:wi∈SW,τj∈Γi(2θi,j −1)2 , Variables. On onehand,the MELON-SWM problemhasa
which in factis the minimumupperboundof thisprobability. vector of M binary variables, denoted as y = (y ,··· ,y ).
(cid:0) P (cid:1) 1 M
Next, webderive Corollary 1, which is directly utilized in our Any y =1 indicates that task τ will be executed, and thus,
j j
design of the incentive mechanism in Section IV-B. requester r is a winning requester (i.e., r ∈ S ), whereas
j j R
Corollary1. Foreveryexecutedtaskτ ,thedataaggregation y = 0 means r 6∈ S . On the other hand, the problem
j j j R
mechanism proposed in Algorithm 1 satisfies that if has another vector of N binary variables, denoted as x =
(x ,··· ,x ), where x = 1 indicates that worker w is a
1 1 N i i
(2θi,j −1)2 ≥2ln β , (11) winningworker(i.e., wi ∈SW), andxi =0 meanswi 6∈SW.
i:wi∈SXW,τj∈Γi (cid:18) j(cid:19) Objective function. The objective function satisfies that
a y − b x = a − b ,
then Pr[Lj 6= lj] ≤ βj, i.e., βj-accuracy is satisfied for this whji:cτhj∈iTs ejxajctly tih:wei∈sWociail iwelfarej:rdje∈fiSnRedjin Die:wfiin∈itSiWon i6
task τj, where βj ∈ (0,1) is a platform chosen parameter. bPased on the reqPuesters’ and worPkers’ bids. P
Moreovebr, we define β as the vector (β1,··· ,βM). Constraints. For each task τ , Constraint (14) naturally
j
holds, if y = 0. When y = 1, it is equivalent to Inequality
Proof. By setting the upper bound of Pr[L 6= l ] given in j j
j j
(11) given in Corollary 1, which specifies the condition that
Theorem 1 to be no greater than β ∈(0,1), we have
j
the set of selected winning workers S should satisfy in
b W
exp − i:wi∈SW,τj∈Γi(2θi,j −1)2 ≤β , order to guarantee βj-accuracy for task τj. To simplify the
2 j presentation, we introduce the following notations, namely
P !
q = (2θ −1)2, q = [q ] ∈ [0,1]N×M, Q = 2ln 1 ,
which is equivalent to ain,dj Q = [iQ,j ] ∈ [0,+∞)Mi,×j1. Thus, Constrainjt (14) canβjbe
j (cid:0) (cid:1)
1 simplified as
(2θi,j −1)2 ≥2ln . (12)
β
i:wi∈SXW,τj∈Γi (cid:18) j(cid:19) qi,jxi ≥Qjyj, ∀τj ∈T. (16)
Hence,togetherwith Theorem1, we havethatInequality(12) i:wi∈XW,τj∈Γi
indicates that Pr[L 6=l ]≤β . Besides, we say a task τ is covered by a solution, if y =1.
j j j j j
b
2) Hardness Proof • The 2ndsubsetE2 containsm tasksτ1,··· ,τm. Eachτk ∈
E belongstoallsetscorrespondingtoC andC ,i.e.,τ
We prove the NP-hardness of the MELON-SWM problem 2 k k+1 k
belongsto all sets {Γ(C ,α)|α∈A }∪{Γ(C ,α′)|α′ ∈
by performinga polynomial-timereductionfrom the 3SAT(5) k k k+1
A } with the subscripts being modulo m. The Q value
problem which is formally defined in Definition 10. k+1 k
of each such τ is set as 2, and its value v is set as Y.
Definition 10 (3SAT(5) Problem). In a 3SAT(5) problem, we k k
aregivenasetO={z ,··· ,z }ofnBooleanvariables,and • The 3rd subset E3 contains a task τ(Ck) for each clause
1 n C , and τ(C ) belongs to set Γ(C ,α) for each α ∈ A .
a collection C ,··· ,C of m clauses. Each clause is an OR k k k k
1 m The Q value of the task τ corresponding to τ(C ) is set
of exactly three literals, and every literal is either a variable j j k
as 1, and its value v is set as Z.
of O or its negation. Moreover, every variable participates in j
exactly 5 clauses. Therefore, m = 5n. Given some constant • The4thsubsetE4 containsasingletaskτ∗,whoseQj value
3 issetas1andvaluev issetasX.Thetaskτ∗ onlybelongs
0 < ǫ < 1, a 3SAT(5) instance ϕ is a Yes-Instance if j
to set Γ .
there is an assignment to the variables of O satisfying all 0
Thisfinishesthedescriptionofthereduction.Clearly,given
clauses, whereas it is a No-Instance (with respect to ǫ), if
a 3SAT(5) instance ϕ, we can construct an instance of the
every assignment to the variables satisfies at most (1−ǫ)m
MELON-SWM problem in time polynomial in n.
clauses. An algorithm A distinguishes between the Yes- and
No-instancesoftheproblem,if,givenaYes-Instance,itreturns We now analyze the optimal social welfare for an instance
a “YES” answer, and given a No-Instance it returns a “NO” of the MELON-SWM problemthatcorrespondsto a 3SAT(5)
answer. instance ϕ, when ϕ is a Yes- or No-Instance. Note that the
Regarding the hardness of the 3SAT(5) problem, we intro- following analysis uses the same reduction as in Theorem 2.
ducewithoutproofthefollowingwell-knownLemma1,which Theorem 3. If the 3SAT(5)instance ϕ is a Yes-Instance, then
is a consequence of the PCP theorem [38]. there is a solution to the resulting instance of the MELON-
Lemma 1. There is some constant 0 < ǫ < 1, such SWM problem whose social welfare is X. If ϕ is a No-
that distinguishing between the Yes- and No-instances of the Instance, then any solution has social welfare at most 0.
3SAT(5) problem, defined with respect to ǫ, is NP-complete.
Proof. LetϕbeaYes-Instance,andAbeanassignmenttothe
Next, we introduce Theorem 2 and 3 that will be utilized
variables satisfying all clauses. We construct a solution S′ to
to prove the NP-hardness of the MELON-SWM problem.
the MELON-SWM problem. Firstly, we add Γ to S′. Next,
0
Theorem 2. Any 3SAT(5) instance is polynomial-time re- for each clause C , we add to S′ the unique set Γ(C ,α),
k k
ducible to an instance of the MELON-SWM problem. where α is the assignment consistent with A. Then |S′|=m,
Proof. The reduction goes as follows. Assume there is a andthetotalcostofallsetsis(Y +Z+3)m.We nowanalyze
3SAT(5) instance ϕ on n variables and m clauses. We define the number of tasks covered by S′, and their values. Clearly,
3 parameters: X = ǫm (0 < ǫ < 1), Y = mnX, and τ∗ iscoveredbyS′,anditcontributesX tothesolutionvalue.
100
Z =mnY.TheexactvaluesofY andZ arenotimportant.We • For each clause Ck ∈ ϕ, the unique task τ(Ck) ∈ E3 is
justneedtoensureZ ≫Y ≫X.We constructaninstanceof covered.Thus, all tasks in E3 are covered,and overallthey
the MELON-SWM problem corresponding to ϕ, by defining contribute value mZ to the solution.
the task set T, and the profile of workers’ interested task sets • Consider some τk ∈E2. S′ contains one set corresponding
Γ. toCk andCk+1,respectively.Sinceτk belongstoboththese
Out of the 8 possible assignments to the variables of some sets, and its Qk is 2, it is covered.Thus, all tasks in E2 are
clause C ∈ ϕ, exactly one does not satisfy C . Let A be covered, and they contribute value mY to the solution.
k k k
the set of the remaining 7 assignments. We define a set of • Considersomevariablezk ∈O, andletγk ∈{T,F}be the
tasks Γ(Ck,α) for each clause Ck and assignment α ∈ Ak, assignmenttozk underA.IfCk isanyclausecontainingzk,
let Γ = [Γ(Ck,α)] for each clause Ck ∈ ϕ and assignment and Γ(Ck,α) is the set that belongsto S′, then α gives the
α∈Ak, set the qi,j valueof eachworkerwi andtask τj ∈Γi assignment γk to zk. Thus, for all five clauses containing
as qi,j =1, andset her bid as bi =3+Y +Z. We also create zk, the corresponding sets chosen to S′ contain τ(zk,γk),
a dummy worker w0, with q0 = 1, b0 = 0, and Γ0 being and this task is covered.So the total numberof tasks of E1
her interested task set. We start with all set Γ(C ,α)’s being coveredby S′ is n. Each such task contributesvalue 5, and
k
empty,graduallydefine the tasks, and specify which sets they the total value contributed by the tasks in E is 5n=3m.
1
belong to. The task set T consists of 4 subsets. Therefore,the overallsocialwelfareofthis solutionis X+
• The 1st subset E1 contains a task τ(zl,γ) for each vari- mZ+mY +3m−(Z+Y +3)m=X.
able z ∈ O and assignment γ ∈ {T,F} to this vari- Assume now that ϕ is a No-Instance, and let S′ be any
l
able. τ(z ,γ) belongs to each set Γ(C ,α), such that z solution with positive social welfare. We can assume that
l k l
participates in C , and the assignment α to the vari- Γ ∈ S′, and task τ∗ is covered by S′. We then introduce
k 0
ables of C gives assignment γ to z . The Q value the following observations, whose proofs are provided in the
k l j
of the task τ corresponding to τ(z ,γ) is set as 5 − appendices.
j l
the number of the clauses containing z ,andthevaluev of Observation 1. For every clause C of ϕ, at most one of the
l j k
this task is set as 5. sets {Γ(C ,α)|α∈A } belongs to S′, and |S′|=m.
k k
Observation 2. For every variable z ∈ O, at most one of ratio, and propose the following MELON double auction that
k
the two tasks τ(z ,T) and τ(z ,F) is covered by S′. aimstoensurenon-negativesocialwelfare,instead.Itswinner
k k
We say that a variable z ∈ O is bad if neither τ(z ,T) selection algorithm is given in the following Algorithm 2.
k k
nor τ(zk,F) is covered by S′; otherwise it is good. We next Algorithm 2: MELON Double Auction Winner Selection
show that only a small number of the variables are bad.
Input:T, R, W, Γ, a, b, q, Q;
Observation 3. There are at most 1ǫ0n0 bad variables. Output:SR, SW, C;
Then,weconstructthefollowingassignmenttothevariables // Initialization
of O. If variable z ∈O is good,then there is a uniquevalue 1SR ←∅, SW ←∅;
k
// Find a feasible cover
γ ∈ {T,F}, such that task τ(z ,γ ) is covered by S′. We
k k k 2C ←FC(T,Γ,q,Q);
thenassignz thevalueγ .Ifz isbad,weassignitanyvalue
k k k 3foreach j s.t. τj ∈T do
arbitrarily. We now claim that the above assignment satisfies 4 Cj ←{wi|wi ∈C,τj ∈Γi};
more than (1−ǫ)m clauses. We say that a clause is bad if it
// Main loop
cainoren5taactilnamsuosasetsb,a1ǫtd0nh0evabnrauiadmbvlbeae,rriaaonbfdlebisat,diascnlgdaoueosadecshoitshveaarrtwiamibsolees.tpSǫ2ain0nrtci≤ecipt3h1aǫ0etm0eres. 567whiljSe∗Rm←←axajSr:rgRjm∈∪Ra{x(cid:0)rja:jrj∗j}−∈;RP(cid:0)ai:jwi−∈CPjbi:iw(cid:1)i≥∈Cj0bdi(cid:1)o;
So there are more than (1−ǫ)m good clauses. Let Cl be a 8 R←R\{rj∗};
good clause, and Γ(Cl,α) be the set correspondingto Cl that 9 SW ←SW ∪Cj∗;
belongs to S′. Then α is an assignment to the variables of 10 foreach j s.t. ri ∈R do
C that satisfies C , and each variable participating in C was 11 Cj ←Cj\Cj∗;
l l l
assigned a value consistent with α. So clause Cl is satisfied. 12 return SR,SW;
To conclude, we have assumed that ϕ is a No-Instance,
Algorithm 2 takes as inputs the task set T, the requester
andshowedthat,iftheMELON-SWMproblemhasasolution
set R, the worker set W, the profile of workers’ interested
withnon-negativesocialwelfare,thereisanassignmenttothe
task sets Γ, the requesters’ and workers’ bid profile a and b,
variables of ϕ satisfying more than (1−ǫ)m of its clauses,
the q matrix, as well as the Q vector. Firstly, it initializes
which is impossible for a No-Instance. Therefore, if ϕ is a
the winning requester and worker set as ∅ (line 1). Then, it
No-Instance, every solution has social welfare at most 0.
calculates a feasible cover, denoted by C, containing the set
Next, we describe Theorem 4 that states the NP-hardness of workers that make Constraint (16) feasible for each task
and inapproximability of the MELON-SWM problem. τ given that each y = 1, by calling another algorithm FC
j j
Theorem 4. The MELON-SWM problem is NP-hard, and for which takes the task set T, the profile of workers’ interested
any factor φ, there is no efficient φ-approximation algorithm task sets Γ, the q matrix, and the Q vector as inputs (line 2).
to the MELON-SWM problem. Algorithm FC can be easily implemented in time polynomial
in M and N. For example, FC could greedily select each
Proof. Based on Theorem 2, there exists a reduction from
worker w into the feasible cover in a decreasing order of
any 3SAT(5) problem instance ϕ to an instance I(ϕ) of the i
the value q until all constraints are satisfied. The
MELON-SWM problem. From Theorem 3, we have that the j:τj∈Γi i,j
computational complexity of such FC is O(N). We assume
optimalsolution to I(ϕ) also givesa solutionto ϕ. Thatis, if P
thatFCadoptssuchagreedyapproachintherestofthispaper.
theoptimalsocialwelfareofI(ϕ)ispositive,thenϕisaYes-
NotethatthespecificchoiceofFCisnotimportant,aslongas
Instance;otherwise,ϕisaNo-Instance.TogetherwithLemma
it returns a feasible cover in polynomial time. Next, for each
1 stating the NP-completeness of the 3SAT(5) problem, we
task τ , Algorithm 2 chooses from the feasible cover the set
conclude that the MELON-SWM problem is NP-hard. j
ofworkersC whoseinterestedtasksetscontainthistask(line
In fact, Theorem 2 and 3 give an inapproximability result j
3-4).
abouttheMELON-SWM,aswell.Supposethereisanefficient
Based on C, the main loop (line 5-11) of the algorithm
factor-φ approximation algorithm A for the MELON-SWM
selects the set of winning requesters and workers that give
problem. We can use it to distinguish Yes- and No-instances
non-negativesocialwelfare.Itexecutesuntilmax a −
of the 3SAT(5) problem on n ≫ φ variables. If ϕ is a Yes- j:rj∈R j
b , the maximum marginal social welfare of in-
Instance, then the algorithm has to return a solution with i:wi∈Cj i (cid:0)
cluding a new requester r and the set of workers C into,
positive social welfare for I(ϕ), and if ϕ is a No-Instance, P (cid:1) j j
respectively, the winning requester and worker set, becomes
then any solution has social welfare at most 0. So algorithm
negative (line 5). In each iteration of the main loop, the
A distinguishes the Yes- and the No-instances of 3SAT(5),
contradicting Lemma 1. Algorithm finds first the index j∗ of the requester rj∗ that
providesthe maximum marginalsocial welfare (line 6). Next,
3) Proposed Mechanism it includes rj∗ into the winning requester set SR (line 7),
Theorem4notonlyshowstheNP-hardnessoftheMELON- removes rj∗ from the requester set R (line 8), and includes
SWM problem, but also indicates that there is no efficient all workers in Cj∗ into the winning worker set SW (line 9).
algorithmwitha guaranteedapproximationratioforit. There- The last step of the main loopis to removeallworkersin Cj∗
fore, we relax the requirement of provable approximation fromC foreachtaskτ (line10).Finally,Algorithm2returns
j j
the winning requester and worker set S and S (line 12). C′ such that w belongs to C′ and r belongs to S′ , the
R W k i k k R
Next, we present the pricing algorithm of the MELON algorithm calculates the maximum bid bi,k for worker wi
double auction in Algorithm 3. to be selected as a winner at this point (line 12-18). The
calculation firstly sorts requesters in the decreasing order
Algorithm 3: MELON Double Auction Pricing
of their marginal social welfare, i.e., a − b (line
Input: T, R, W, Γ, a, b, q, Q, SR, SW; 13), and finds the index f of the fijrst thei:wrie∈qCuj′esiter in
Output: pr, pw; P
this order such that w does not belong to C′ (line 14). If
// Initialization i f
1pr ←0, pw ←0; rf is a winning requester in SR′ , then bi,k should satisfy
// Pricing for winning requesters a − b + b = a − b , which is
2foreach j s.t. rj ∈SR do eqkuivalentwtoh∈bCk′ h= a i−,k f b − wah∈C−f′ h b ;
3 run Algorithm 2 on R\{rj} and W; (cid:0)P i,k k (cid:1) wh∈Ck′ h P f wh∈Cf′ h
4567 SfifoR′rCej′←apc=rjhw←∅ikntnmshi.neti.ngnr(cid:8)rkepq∈rju,eSPsR′tewrid∈soeCtj′wbih+enalkin−e 3Pswtoip∈sC;k′ bi(cid:9); mwopwithahxiec(irlhmwinuiiessme1,e5vbq-aiu1,lkui8ve)as.hlaeFomniuntoladntloglsyat,bhtiiAe,skPsfleyg=obari,kiktah’−ksma(cid:0)−r3PePrcwehtwh(cid:0)ou∈hsr∈Cenk′nCsk′bathhbsPhe+t.hrebeTiqph,kuae(cid:1)eynsm,=teterhn0(cid:1)se’t,
8 pr ←min{pr,0}; and workers’ payment profile pr and pw (line 19).
j j
4) Analysis of the Proposed Mechanism
// Pricing for winning workers
In this section, we prove several desirable properties of
9foreach i s.t. wi ∈SW do
10 run Algorithm 2 on R and W\{wi}; our MELON double auction, described in Algorithm 2 and
11 S′ ←winning requester set when line 10 stops; 3. Firstly, we show its truthfulness in Theorem 5.
R
12 foreach k s.t. wi ∈Ck′ and rk ∈SR′ do Theorem5. TheproposedMELONdoubleauctionistruthful.
13 sort requesters according to the decreasing order of
14 afj←−inPdeix:wio∈fCtj′hebifi;rst requester with wi6∈Cf′; Ptioronobf.yWsheowprionvgeththaetittrustahtifsufilneesstsheofptrhoepeMrtEieLsOofNmdoonuobtloenaicuicty-
15 if rf ∈SR′ then and critical payment.
16 pwi ← • Monotonicity. The algorithm FC called by Algorithm 2
max pwi ,ak− wh∈Ck′ bh− af− wh∈Cf′ bh ; is independent of the requesters’ and workers’ bids, and
17 else (cid:8) P (cid:0) P (cid:1)(cid:9) winnersareselectedbasedonadecreasingorderofthevalue
18 pwi ←max(cid:8)pwi ,ak−Pwh∈Ck′ bh(cid:9); aj − i:wi∈Cjbi. Thus, if a requester rj wins by bidding
a , she will also win the auction by bidding any a′ > a .
19 return pr, pw; j P j j
Similarly, if a worker w wins by bidding b , she will win
i i
Apart from the same inputs to Algorithm 2, Algorithm 3 the auction, as well, if her bid takes any value b′i <bi.
also takes as inputs the winning requester and worker set • Critical payment. Algorithm 3 in fact pays every winning
S and S , outputted by Algorithm 2. Firstly, Algorithm 3 requesterandworkertheinfimumandsupremumofherbid,
R W
initializestherequesters’andworkers’paymentprofileaszero respectively, that can make her a winner.
vectors (line 1). Then, it calculates the payment pr charged As proved in [39], these two properties make an auction
j
from each winning requester (line 2-8). For each rj ∈ SR, truthful, i.e., each requester rj maximizes her utility by bid-
Algorithm2isexecutedontheworkersetW andrequesterset dingvj, and eachworkerwi maximizesher utility bybidding
Rexceptrequesterrj (line3).Next,itsetsSR′ asthewinning ci. Therefore, the MELON double auction is truthful.
requester set when line 3 stops (line 4). For each r ∈ S′ ,
k R Next, we show that the proposed MELON double auction
Algorithm 3 finds the minimum bid a for requester r to
j,k j satisfies individual rationality in Theorem 6.
replace r as the winner. To achieve this, a should satisfy
k j,k Theorem6. TheproposedMELONdoubleauctionisindivid-
a − b = a − b , which is equivalent to
j,k wi∈Cj′ i k wi∈Ck′ i ual rational.
a = b +a − b . Note that C′,··· ,C′
j,k P wi∈Cj′ i k Pwi∈Ck′ i 1 M Proof. By Definition 3 and 4, losers of the MELON double
denote the sets C ,··· ,C when the specific requester r is
selectedPintoS′ .1IfC′ isnMotPempty,theminimumvalueamkong auctionreceivezero utilities. FromTheorem5, everywinning
these a ’s isRchosenjas the paymentpr (line 5-6); otherwise, requester rj bids vj, and every winning worker wi bids ci
j,k j
to the platform. Moreover, they are paid, respectively, the
it is further compared with 0 (line 7-8), since requester r
j
infimumandsupremumofthebidforthemtowintheauction.
could win, in this case, as long as her bid is non-negative.
Therefore, it is guaranteed that all requesters and workers
Next, Algorithm3 derivesthe paymentpw to each winning
i receive non-negativeutilities, and thus the proposed MELON
worker w (line 9-18). Similar to line 3, Algorithm 2 is
i double auction is individual rational.
executed on the requester set R and worker set W except
worker w (line 10), and S′ is set as the winning requester In Theorem 7, we provethat the proposed MELON double
i R
set when line 10 stops (line 11). In the rest of the algorithm, auction has a polynomial-time computational complexity.
we also use C′,··· ,C′ to denote the sets C ,··· ,C when Theorem 7. The computational complexity of the proposed
1 M 1 M
the specific requester r is selected into S′ . For each set MELON double auction is O(M3N +M2N2).
k R
Proof. AsmentionedinSectionIV-A1,thealgorithmFC(line which we call AIR double auction. The rest of its winner
2) in Algorithm 2 takes a greedy approach, and has a com- selection, as well as the entire pricing algorithm is the same
putational complexity of O(N). Line 3-4 of Algorithm 2 that as those of our MELON double auction. It is easily provable
find the sets C ,··· ,C terminate at most after MN steps. that the AIR double auction is also truthful and individual
1 M
Next, the main loop (line 5-11) terminates after M iterations rational.
in worst case. Within each iteration, finding the index of the Furthermore, we compare our weighted data aggregation
requester that provides the maximum marginal social welfare mechanism with a mean aggregation mechanism, which out-
(line 6) takes O(M) time, and updating the sets C ,··· ,C puts +1 as the aggregated result for a task if the mean of
1 M
takes O(MN) time. Therefore,the computationalcomplexity workers’ labels about this task is non-negative, and outputs
of the main loop is O(MN), and thus, that of Algorithm 2 −1, otherwise. Another baseline aggregation mechanism that
is O(M2N) overall. After Algorithm 2, our MELON double we consider is the median aggregation that takes the median
auctionexecutesits pricingalgorithmdescribedby Algorithm of workers’ labels about a task as its aggregated result.
3, where the loop for requester pricing (line 1-8) terminates
B. Simulation Settings
in worst case after M iterations. Clearly, the computational
complexity of each iteration of the loop is dominated by the Setting vj ci θi,j βj |Γ∗i| N M
execution of Algorithm 2 in line 3. Therefore, the requester I [10,20] [5,15] [0,1] [0.05,0.1] [15,20] [90,150] 60
II [10,20] [5,15] [0,1] [0.05,0.1] [15,20] 60 [20,80]
pricing (line 1-8) in Algorithm 3 takes O(M3N) time. Fol-
TABLEI
lowinga similar methodofanalysis, we canconcludethatthe SIMULATIONSETTINGS
worker pricing in Algorithm 3 takes O(M2N2) time. Hence,
the computationcomplexityofAlgorithm3, aswellasthatof The parameter settings in our simulation are givenin Table
theoverallMELONdoubleauctionisO(M3N+M2N2). I.Specifically,parametersvj,ci,θi,j,βj,and|Γ∗i|aresampled
uniformly at random from the intervals given in Table I. The
Finally, we show in Theorem 8 that our MELON double workerw ’strueinterestedtasksetΓ∗ contains|Γ∗| tasksthat
i i i
auction guarantees non-negativesocial welfare, as required.
are randomlyselected fromthe task set T. Insetting I, we fix
Theorem 8. The MELON double auction guarantees non- thenumberofrequestersas60andvarythenumberofworkers
negative social welfare. from 90 to 150, whereas we fix the number of workers as 60
and varythe numberof requestersfrom20 to 80 in setting II.
Proof. Clearly,inthewinnerselectionalgorithmdescribedby
Algorithm 2, a requester r and the workers in C could be
j j C. Simulation Results
selected as winners, only if the correspondingmarginalsocial
welfare a − b is non-negative (line 5). Thus, as 950 500
j i:wi∈Cj i 900 450
the overall social welfare given by Algorithm 2 is the sum of 850 400 MMESWLO−NGreedy
twghuheaeraraefnotnreeeemwsenwnotininPo-nnneeerdgsamatirvaeergsseionlecacilatselodwc,ietahllfeawrMee.lEfaLreONofdeovuerbyleitaeuracttiioonn Social Welfare667780505000000 MMAIESRWLO−NGreedy Social Welfare223305050000 AIR
550 150
V. PERFORMANCE EVALUATION 500 100
450 50
In this section, we introduce the baseline methods, simula- 94 98102106110114118122126130134138142146150 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80
Number of Workers Number of Tasks
tionssettings,aswellassimulationresultsoftheperformance
Fig.2. Socialwelfare (settingI) Fig.3. Socialwelfare (setting II)
evaluation about our proposed CENTURION framework.
0.56 0.5
A. Baseline Methods 0.5 0.44
0.44 0.38
In our evaluation of the incentive mechanism, the first 0.38
0.32
baselineauctionistheMarginalSocialWelfaregreedy(MSW- AE0.32 AE0.26
M0.26 M
Greedy) double auction. As in Algorithm 2, it also initializes 0.2
0.2 Median Median
the winner sets as ∅, executes the algorithm FC to obtain a 0.14 MWeeaignhted Aggregation 0.14 MWeeaignhted Aggregation
feasible cover C, and chooses from C the set C containing 0.08 0.08
j
0.02 0.02
each worker wi such that τj ∈ Γi for each task τj. Different 94981021061N10u1m1b4e1r1 8of1 2W2o1r2k6e1r3s0134138142146150 24 28 32 36 40Nu4m4b4e8r o5f2 T5a6sks60 64 68 72 76 80
from the MELON double auction, it sorts requesters in a
Fig.4. MAE(settingI) Fig.5. MAE(settingII)
decreasing order of their marginal social welfare, i.e., the InFigure2 and3, we comparethesocialwelfaregenerated
value a − b for each requester r . Then, it selects byourMELONdoubleauctionwiththoseofthetwobaseline
j i:wi∈Cj i j
the requester r and the set of workers in C as winners auctions. These two figures show that our MELON double
j j
P
untilthemarginalsocialwelfarebecomesnegative.Itspricing auction generates social welfare far more than the MSW-
algorithm is the same as that of the MELON double auction. Greedy and AIR double auction under both setting I and II.
Clearly, the MSW-Greedy double auction is truthful and We evaluate CENTURION’s accuracy guarantee in setting
individual rational. Another baseline auction is the one that I and II with a minor change of the parameter β , i.e., β
j j
initiAlizes the feasIble cover C as the entire woRker set W, for each task τ is fixed as 0.05 to simplify presentation. We
j
0.5 0.5 [8] H.Xie,J.Lui,J.W.Jiang,andW.Chen,“Incentivemechanism
0.44 0.44 and protocol design for crowdsourcing systems,” in Allerton,
0.38 0.38 2014.
Probability000..23.262 MMeedanian Probability000..23.262 MMeedanian [9] tHMimo.ceH,-.s2eC0n1hs5eitu.ivneg,taRs.kSsoeulethcwtioenll,iFn.mHoobui,leancdroJw.Hdsueannsgin,g“,D”iisntriMbuotbeid-
Weighted Aggregation Weighted Aggregation
0.14 Required Upper Bound 0.14 Required Upper Bound [10] L. Gao, F. Hou, and J. Huang, “Providing long-term participa-
0.08 0.08 tion incentive in participatory sensing,” in INFOCOM, 2015.
0.02 0.02 [11] H. Jin, L. Su, D. Chen, K. Nahrstedt, and J. Xu, “Quality
9498102106110114118122126130134138142146150 24 28 32 36 40 44 48 52 56 60 64 68 72 76 80
Number of Workers Number of Tasks of information aware incentive mechanisms for mobile crowd
Fig.6. Errorprobability (settingI) Fig.7. Errorprobability (settingII) sensing systems,” in MobiHoc, 2015.
[12] Y.Wen,J.Shi,Q.Zhang,X.Tian,Z.Huang,H.Yu,Y.Cheng,
compare the mean absolute error (MAE) for all tasks, which
and X. Shen, “Quality-driven auction based incentive mecha-
is defined as MAE = 1 |ˆl −l |, of our weighted
M j:τj∈T j j nism for mobile crowd sensing,” in TVT,2014.
aggregation mechanism proposed in Algorithm 1 with those [13] D.Zhao,X.-Y.Li,andH.Ma,“Howtocrowdsourcetaskstruth-
P
of the mean d median aggregation. The simulation for each fully without sacrificing utility: Online incentive mechanisms
combination of worker and requester number is repeated for with budget constraint,” in INFOCOM, 2014.
[14] Y. Chen, B. Li, and Q. Zhang, “Incentivizing crowdsourcing
50000 times, and we plot the means and standard deviations
systems with network effects,” in INFOCOM, 2016.
of the MAEs in Figure 4 and 5. From these two figures, we
[15] S. He, D.-H. Shin, J. Zhang, and J. Chen, “Toward optimal
observethattheMAEofourweightedaggregationmechanism allocation of location dependent tasks in crowdsensing,” in
is far less than those of the mean and median aggregation. INFOCOM, 2014.
Then, we show our simulation results about Pr[|ˆl − l |], [16] T. Luo, S. S. Kanhere, H.-P. Tan, F. Wu, and H. Wu, “Crowd-
j j
sourcing with tullock contests: A new perspective,” in INFO-
referred to as task τ ’s error probability (EP). After 50000
j COM, 2015.
repetitions of the simulation for any given combination of
[17] L. Duan, T. Kubo, K. Sugiyama, J. Huang, T. Hasegawa,
worker and requester number, empirical values of the EPs and J. Walrand, “Incentive mechanisms for smartphone col-
are calculated, and the means and standard deviations of the laboration in data acquisition and distributed computing,” in
empiricalEPsareplottedinFigure6and7.Thesetwofigures INFOCOM, 2012.
[18] Q.Zhang,Y.Wen,X.Tian,,X.Gan,andX.Wang,“Incentivize
show that the empirical EPs are less than the required upper
crowd labeling under budget constraint,” in INFOCOM, 2015.
bound β and far less than those of the mean and median
j [19] X. Zhang, G. Xue, R. Yu, D. Yang, and J. Tang, “Truthful
aggregation. incentivemechanismsforcrowdsourcing,”inINFOCOM,2015.
[20] D. Yang, G. Xue, X. Fang, and J. Tang, “Crowdsourcing to
VI. CONCLUSION smartphones: Incentive mechanism design for mobile phone
In this paper,we proposeCENTURION, a novelintegrated sensing,” in Mobicom, 2012.
[21] X. Zhang, Z. Yang, Z. Zhou, H. Cai, L. Chen, and X. Li,
framework for multi-requester MCS systems, consisting of
“Free market of crowdsourcing: Incentive mechanism design
a double auction-based incentive mechanism that stimulates
for mobile sensing,” in TPDS, 2014.
the participation of both requesters and workers, and a data [22] H. Zhang, B. Liu, H. Susanto, G. Xue, and T. Sun, “Incentive
aggregation mechanism that aggregates workers data. Its in- mechanismforproximity-basedmobilecrowdservicesystems,”
centive mechanism bears many desirable properties including in INFOCOM, 2016.
[23] Y. Wei, Y. Zhu, H. Zhu, Q. Zhang, and G. Xue, “Truthful
truthfulness,individualrationality,computationalefficiency,as
online double auctions for dynamic mobile crowdsourcing,” in
well as non-negative social welfare, and its data aggregation
INFOCOM, 2015.
mechanism generates highly accurate aggregated results. [24] I.Koutsopoulos, “Optimal incentive-driven designof participa-
tory sensing systems,” in INFOCOM, 2013.
REFERENCES [25] H. Jin, L. Su, H. Xiao, and K. Nahrstedt, “Inception: incen-
[1] P. Mohan, V. N. Padmanabhan, and R. Ramjee, “Nericell: tivizing privacy-preserving data aggregation for mobile crowd
Rich monitoring of road and traffic conditions using mobile sensing systems,” in MobiHoc, 2016.
smartphones,” in SenSys, 2008. [26] H. Jin, L. Su, B. Ding, K. Nahrstedt, and N. Borisov, “En-
[2] A. Thiagarajan, L. Ravindranath, K. LaCurts, S. Madden, abling privacy-preserving incentives for mobile crowd sensing
H. Balakrishnan, S. Toledo, and J. Eriksson, “Vtrack: Accu- systems,” in ICDCS, 2016.
rate, energy-aware road traffic delay estimation using mobile [27] J.Wang,J.Tang,D.Yang,E.Wang,andG.Xue,“Quality-aware
phones,” in SenSys, 2009. and fine-grained incentive mechanisms for mobile crowdsens-
[3] J. Eriksson, L. Girod, B. Hull, R. Newton, S. Madden, and ing,” in ICDCS, 2016.
B. Hari, “The pothole patrol: using a mobile sensor network [28] Z. Feng, Y. Zhu, Q. Zhang, L. Ni, and A. Vasilakos, “Trac:
for road surface monitoring,” in MobiSys, 2008. Truthful auction for location-aware collaborative sensing in
[4] “Myheartmap,” http://www.med.upenn.edu/myheartmap/. mobile crowdsourcing,” in INFOCOM, 2014.
[5] S. Hu, L. Su, H. Liu, H. Wang, and T. F. Abdelzaher, [29] M.Karaliopoulos, I.Koutsopoulos, andM. Titsias,“Firstlearn
“Smartroad: Smartphone-based crowd sensing for traffic reg- thenearn:Optimizingmobilecrowdsensingcampaignsthrough
ulator detection and identification,” in TOSN, 2015. data-driven user profiling,” in MobiHoc, 2016.
[6] Y. Cheng, X. Li, Z. Li, S. Jiang, Y. Li, J. Jia, and X. Jiang, [30] L. Pu, X. Chen, J. Xu, and X. Fu, “Crowdlet: Optimal worker
“Aircloud: A cloud-based air-quality monitoring system for recruitmentforself-organizedmobilecrowdsourcing,”inINFO-
everyone,” in SenSys, 2014. COM, 2016.
[7] K. Han, H. Huang, and J. Luo, “Posted pricing for robust [31] D. Peng, F. Wu, and G. Chen, “Pay as how well you do:
crowdsensing,” in MobiHoc, 2016. A quality based incentive mechanism for crowdsensing,” in