Differential Privacy and the Fat-Shattering Dimension of Linear Queries

0.17 MB·English

by  Aaron Roth

Differential Privacy and the Fat-Shattering
Dimension of Linear Queries
Aaron Roth⋆
Computer Science Department
Carnegie Mellon University
and
Microsoft Research New England
1
1
0 Abstract. Inthispaper,weconsiderthetaskofansweringlinearqueries
2 under the constraint of differential privacy. This is a general and well-
studied class of queries that captures other commonly studied classes,
n
including predicate queries and histogram queries. We show that the
a
J accuracy to which a set of linear queries can be answered is closely re-
lated to its fat-shattering dimension, a property that characterizes the
9
1 learnability of real-valued functions in the agnostic-learning setting.

]
S 1 Introduction
D
.
s The administrator of a database consisting of sensitive, but valuable
c informationfacestwoconflictingobjectives.Becausethedataisvaluable,
[
she would like to make statistical information about it available to the
2 public.However,becausethedataissensitive,shemusttakecarenotto
v release information thatexposesthedataofanyparticularindividualin
5 thedataset. Thecentralquestion in thefield ofprivate data analysis is
0 how these two objectives can be traded off, and more specifically, how
2 manyqueriesofwhattypecanbeansweredtogivendegreesofaccuracy,
3 while still preservingprivacy.
.
4 Recent work on differential privacy providesa mathematical framework
0 to reason about such questions. Informally, a probabilistic function f
0 from a database D to some range is α-differentially private if adding
1 or removing a single individual froRm the dataset does not change the
v: probability that f(D) = r for any outcome r by more than an eα
∈ R
i factor.Theintuitionbehindthisdefinitionisthatanindividual’sprivacy
X
shouldnotbeconsideredtohavebeenviolatedbysomeeventr,ifrwould
r havebeen almost as likely tooccur evenwithout the individual’s data.
a
Inthispaper,weconsiderdatabasesDwhicharerealvaluedvectors,and
theclassofqueriesthatweconsidercorrespondtolinearcombinationsof
theentriesofD.Formally,weconsiderdatabasesD Rn,andqueriesof
theform q [0,1]n.Theanswer toqueryq on datab∈ase+D is simply the
∈
dot-productofthetwovectors:q(D)=q D.This modelhaspreviously
·
beenconsidered([DN03,DMT07,DY08,HT10]),andgeneralizestheclass
⋆ This work has been supported in part by an NSFGraduate Research Fellowship.
2 Aaron Roth
of count queries or predicate queries, which has also been well studied
([DMNS06,BLR08,DNRRV09,RR10,UV10]).
The fat-shattering dimension (FSD) of a class of real-valued functions
C over some domain is a generalization of the Vapnik-Chervonenkisdi-
mension, and characterizes a distribution-free convergence property of
the mean value of each f C to its expectation. The fat-shattering
∈
dimension of a class of functions C is known to characterize the sam-
ple complexity necessary to PAC learn C in the agnostic framework
[ABCH97,BLW94]:thatis,ignoringcomputation,thesamplecomplexity
thatisbothnecessaryandsufficienttolearnC intheagnosticframework
is polynomially related to thefat-shattering dimension of C.
Ourmainresultisasimilarinformationtheoreticcharacterizationofthe
magnitude of thenoise that must beaddedto theanswer toeach query
in some class C in terms of the fat-shattering dimension of C, FSD(C).
We show polynomially related information theoretic upper and lower
bounds on the noise that must be added to each queryin C in terms of
FSD(C).Thisgeneralizestheresultsof[BLR08]tolinearqueries,andto
ourknowledgegivesthefirstanalysisofgenericlinearqueriesusingsome
parameter other than their cardinality. This yields the first mechanism
capableofansweringapossiblyinfiniteset ofgenericlinearqueries,and
thefirstnon-triviallowerboundforinfiniteclassesofnon-booleanlinear
queries. As a consequence, we extend results of Kasiviswanathan et al.
and Blum etal. [KLNRS08,BLR08]relating thesample complexitynec-
essaryforagnosticPAClearningandprivateagnosticPAClearningfrom
classes of boolean valued functions to classes of real valued functions.
1.1 Related Work and Our Results
DinurandNissimstudiedthespecialcaseoflinearqueriesforwhichboth
thedatabaseandthequeryareelementsofthebooleanhypercube 0,1 n
{ }
[DN03].Evenin thisspecial case,theyshowedthattherecannotbeany
private mechanism that answers n queries with error o(√n), because
an adversary could use any such mechanism to reconstruct a 1 o(1)
−
fraction of the original database, a condition which they called blatant
non-privacy. This result was strengthened byseveral subsequentpapers
[DMT07,DY08,KRSU10].
Beimeletal.considertheclassofbasisvectorsonthebooleanhypercube,
andshowthateventhoughthisclasshasaconstantVC-dimension(and
hence fat-shattering dimension), it requires a superconstant number of
samples for useful private release [BKN10]. Specifically, they show that
the logn factor which appears in the upper bound in this paper and in
[BLR08], but not in thelower bound,is in fact necessary in some cases.
Dwork et al. gave the original definition of differential privacy, as well
as the Laplace mechanism, which is capable of answering any k “low
sensitivity” queries (including linear queries) up to error O(k). A more
refined analysis of the relationship between the laplace mechanism and
function sensitivity was later given by [NRS07].
In a different setting, Blum Ligett and Roth considered the question of
answering predicate queries over a database drawn from some domain
X [BLR08]. This can be viewed as a special case of linear queries in
Differential Privacy and theFat-ShatteringDimension of Linear Queries 3
whichthequeriesarerestrictedtolieonthebooleanhypercube,andthe
database must be integer valued: D Zn. They give a mechanism for
∈ +
answering every query in some class C with noise that depends linearly
ontheVC-dimensionoftheclassofqueries.Thisisaquantitythatisat
mostlog C forfiniteclassesC,andcanbefiniteevenforinfiniteclasses.
| |
Roth and Roughgarden later gave a mechanism which achieved similar
bounds in the online model, in which the mechanism does not know
the set of queries that must be answered ahead of time, and instead
must answer them as they arrive [RR10]. We generalize the technique
of [BLR08,RR10]toapply togeneral linearqueries.VC-dimension is no
longer an appropriate measure of query complexity in this setting, but
we show that a quantity known as Fat-Shattering dimension plays an
analogous role.
Dwork et al. [DNRRV09] also gave upper and lower bounds for pred-
icate queries, which are incomparable to the bounds of [BLR08,RR10]
(and those presented in this paper). The upper bounds of [DNRRV09]
are for an approximate form of differential privacy, and have a better
dependenceon α, butaworse dependenceon k.Their lower boundsare
computational, whereas the lower bounds presented in this paper are
information theoretic.
HardtandTalwar[HT10]givematchingupperandlowerboundsonthe
noisethatmustbeaddedforα-differentialprivacywhenansweringk n
≤
linear queriesofroughly Θ(√klog(n/k)).Incontrast, weproveboundsin
α
termsofdifferentparameters,andcanhandlearbitrarily(eveninfinitely)
large values of k. For finite sets of k queries, our mechanism adds noise
roughly
O D 2/3 logklogn 1/3 . Note that for some settings of the parame-
|| ||1 · α
ter(cid:16)s,thisissi(cid:0)gnificantly(cid:1)les(cid:17)snoisethantheboundsof[HT10]:specifically,
fork Ω( D 4/3).Toachievelowrelative errorη(i.e.errorǫ=η D ),
≥ || ||1 || ||1
ourmechanismrequiresonlythat D bepoly-logarithmicink,rather
1
|| ||
than polynomial in k. For infinite classes of queries C , the logk in our
| |
boundcan bereplaced with thefat shattering dimension of theclass C.
We also show a lower boundin terms of the fat shattering dimension of
theclass C,which is thefirst non-triviallower boundforinfiniteclasses
of non-boolean linear queries.
2 Preliminaries
A database is some vector D Rn, and a query is some vector q
∈ + ∈
[0,1]n.Wewritethattheevaluationofq onD isq(D)=q D.Wewrite
D = n D to denote the ℓ norm of D, and note·that for any
|| ||1 i=1 i 1
query q, q(D) [0, D ]. We let C denote a (possibly infinite) class
1
P ∈ || ||
of queries. We are interested in mechanisms that are able to provide
answers a for each q C so that the maximum error, defined to be
i i
∈
max q (D) a is as small as possible. Without loss of generality,
i C i i
∈ | − |
we restrict our attention to mechanisms which actually output some
synthetic database: mechanisms with range = Rn. That is, if our
R +
4 Aaron Roth
mechanismoutputssomesyntheticdatabaseD′,wetakeai tobeqi(D′)
for each i.1
We formalize our notion of utility and relative utility for a randomized
mechanism M:
Definition 1 (Usefulnessand RelativeUsefulness).Amechanism
M :Rn Rn is (ǫ,δ)-useful with respect to a class of queries C if with
+ → +
probability at least 1 δ (over the internal coins of the mechanism), it
−
outputs a synthetic database D′ such that:
sup qi(D) qi(D′) ǫ
| − |≤
qi∈C
For0<η 1,M is(η,δ)-relativelyusefulwithrespecttoC fordatabases
≤
ofsizesifitis(η D ,δ)-usefulwithrespecttoC forallinputdatabases
1
|| ||
D with D s.
1
|| || ≥
That is, useful mechanisms should have low error for each query in C.
We now definedifferential privacy:
Definition 2 (Differential Privacy [DMNS06]). A mechanism M :
Rn Rn is α-differentially private, if for any two databases D ,D
su+ch→that+D D 1, and for any S Rn: 1 2
|| 1− 2||1 ≤ ⊆ +
Pr[M(D ) S] eαPr[M(D ) S]
1 2
∈ ≤ ∈
The standard notion of differential privacy need only hold for mecha-
nisms defined over integer valued databases D ,D Nn, which is a
1 2
∈
weaker condition. Our upper bounds will hold for the stronger notion
of differential privacy, and our lower bounds for the weaker notion. A
useful observation is that arbitrary (database independent)functions of
differentially private mechanisms are also differentially private:
Fact 1 If M :Rn Rn is α-differentially private, and if f :Rn Rn
+ → + + → +
isa(possiblyrandomized)function,thenf(M)isα-differentiallyprivate.
2.1 Fat Shattering Dimension
Fat-shattering-dimension is a combinatorial property describing classes
of functions of the form f : X [0,1] for some domain X. It is a gen-
→
eralization of the Vapnik-Chervonenkis-dimension, which is a property
only of classes of boolean valued functions of the form f : X 0,1 .
→ { }
In this section, we generalize these concepts slightly to classes of linear
queries,whereweviewourlinearqueriesaslinearcombinationsoffunc-
tions f :X [0,1], wherewe let X betheset ofstandard basis vectors
of Rn. →
LetB= e n denotethesetofnstandardbasisvectorsofRn(e isthe
{ i}i=1 i
vector with a 1 in thei’th coordinate, and a 0 in all other coordinates).
1 This is without loss of generality, because given a different representation for each
answer ai toerror ǫ,it ispossible tocomputeasyntheticdatabase D′ with error at
most 2ǫ using the linear program of [DNRRV09].
Differential Privacy and theFat-ShatteringDimension of Linear Queries 5
ForanyS B ofsize S =d,wesaythatS isγ-shatteredbyC ifthere
exists a ve⊆ctor r [0,|1]d| such that for every b 0,1 d, there exists a
∈ ∈ { }
query q C such that for each e S:
b i
∈ ∈
r +γ,if b =1;
qb(ei) ≥ri γ,if bi=0.
i i
(cid:26)≤ −
Note that since the range of each query is [0,1], γ can range from 0 to
1/2.
Definition 3 (FatShatteringDimension[BLW94,KS94]).Theγ-
fat-shattering dimension of a class of linear queries C is:
FSD (C)=max d N:C γ shatters some S B with S =d
γ
{ ∈ − ⊆ | | }
Inthespecialcasewhenγ =r =1/2foralli,notethatthefatshattering
i
dimension of a class of boolean valued functions is equal to its VC-
dimension.
For finite classes C, we will let k = C denote the cardinality of C.
| |
Thefollowingobservationfollowsimmediatelyfromthedefinitionoffat-
shattering dimension:
Observation 1 For finite classes C, FSD (C) logk for all γ > 0,
γ
≤
where k= C .
| |
3 Lower Bound
Inthissection,weshowthatanyα-differentiallyprivatemechanismthat
answerseverylinearqueryinsomeclassC mustaddnoiseatleastlinear
in the fat-shattering dimension of C at any scale. The bound that we
prove in this section is in terms of the privacy parameter α and the fat
shatteringdimensionoftheclass.Itdiffersfromtheupperboundproved
in the next section by several important parameters, which include a
logn term and a term depending on the size of the database. Beimel
et al. [BKN10] have shown that the logn term in the upper bound is
necessaryinsomecontexts.Thedatabasethatweconstructinourlower
boundisofsizeO(γ FSD (C)).Therefore,inordertoproveanontrivial
γ
·
lower bound on the relative error achievable by a private mechanism, it
wouldbenecessarytoremoveafactorofγ fromourcurrentbound.This
is possible in the context of VC-dimension, and we conjecture that it
shouldalsobepossibleforaboundintermsoffat-shatteringdimension,
and is merely a limitation of our techniques as present. The problem of
provingatightlowerboundencapsulatingalloftherelevantparameters
remains an interesting open question. We now proceed with the lower
bound:
Theorem 2. For any δ bounded away from 1 by a constant, let M be
a mechanism M that is (ǫ,δ) useful with with respect to some class of
linear queries C. If M preserves α-differential privacy, then
γ2 FSD (C)
ǫ Ω sup · γ
≥ 0<γ 1/2 eα !
≤
6 Aaron Roth
We begin with some preliminaries which allow us to prove some useful
lemmas:
Given some class of linear queries C and any γ > 0, let S B be a
⊆
collection of basis-vectors of size FSD (C) that are γ-shattered by C,
γ
and let r [0,1]FSDγ(C) bethecorresponding vectorasin thedefinition
∈
offat-shatteringdimension.WenowpartitionSinto1/γ pieces.Foreach
j 1,...,1/γ , let:
∈{ }
Sj = e S :(j 1) γ <r j γ
i i
{ ∈ − · ≤ · }
Sincethesets S partitionS,Bythepigeon-holeprinciple,thereexists
j
Wsomeceojn∗sisduecrhsut{hbaste}t|sSTj∗|≥Sγj∗·|oSf|s=izeγT·FS=Ddγ/(2C.)F.oLreetadch=su|Scjh∗|s.ubset,we
⊂ | |
consider the database D = e . Let bT 0,1 d be the vector
guaranteed by thedefinitTion of faeit∈sThaittering dim∈e{nsion}such that:
P
1,e T;
bTi = 0,oith∈erwise.
(cid:26)
Let q C be the query that corresponds to bT as in the definition of
T
∈ ∗
fat shattering dimension, and let CSj∗ ={qT :T ⊆Sj ,|T|=d/2}.
We first show that each function q takes its highest value on D and
T T
cannot takelarge values on databases DT′ for sets T′ that differsignifi-
cantly from T.
∗
Lemma 1. For all qT ∈CSj∗ and for all T′⊆Sj with |T′|=d/2:
γ
qT(DT)−qT(DT′)≥ 2 ·|T △T′|
Proof.
qT(DT)−qT(DT′)= qT(ei)− qT(ei)
eXi∈T eXi∈T′
= q (e ) q (e ) + q (e )
T i T i T i
 − 
ei∈XT∩T′ ei∈XT\T′
 
q (e )
T i
−
ei∈XT′\T
r +γ r γ
i i
≥ − − 
ei∈XT\T′ ei∈XT′\T
   
≥2γ·|T \T′|− imTa′xTri−imTinT′ri ·|T \T′|
(cid:18)∈ \ ∈ \ (cid:19)
γ T T′
≥ ·| \ |
∗
where the last inequality follows from the fact that T,T′ Sj which
⊂
was constructed such that:
max r min r γ
(cid:18)i∈Sj∗ i−i∈Sj∗ i(cid:19)≤
holds. Observingthat T T′ =2T T′ completes theproof.
| △ | | \ |
Differential Privacy and theFat-ShatteringDimension of Linear Queries 7
With this lemma, we are ready to prove the main technical lemma for
our lower bound:
Lemma 2. For any δ bounded away from 1 by a constant, let M be an
(ǫ,δ)-useful mechanism with respect to class C. Given as input M(D ),
T
∗
where D is an unknown private database for some T Sj with T =
T
⊆ | |
d/2, with constant probability 1 δ, there is a procedure to reconstruct a
−
new database DT∗ such that |T △T∗|≤ 4γǫ.
Proof. Supposethat mechanism M is (ǫ,δ) useful with respect to C for
someconstantδboundedawayfrom1.Thenbydefinition,withconstant
probability, given input DT, it outputs some database D′ such that for
∗
all qi C, qi(DT) qi(D′) ǫ. For each T′ Sj with T′ =d/2 let:
∈ | − |≤ ⊆ | |
v(T′)=qT′(DT′)−qT′(D′)
and let T∗=argminT′v(T′). Therefore, we have:
v(T∗) v(T)=qT(DT) qT(D′) ǫ (1)
≤ − ≤
where the last inequality follows from the usefulness of the mechanism.
We also have:
v(T∗)=qT∗(DT∗) qT∗(D′)
−
qT∗(DT∗) qT∗(DT) ǫ
≥ − −
γ
T T∗ ǫ
≥ 2 ·| △ |−
where the first inequality follows from the usefulness of the mechanism,
and the second inequality follows from lemma 1. Combining this with
equation 1, we get:
4ǫ
T T∗
| △ |≤ γ
We are now ready to provethelower bound:
∗
Proof (Proof of Theorem). Let T Sj with T = d/2 be some ran-
⊂ | |
domlyselectedsubset.LetD = e bethecorrespondingdatabase.
Bylemma2,givenM(D ,ǫ)T,withepir∈oTbaibility1 δ thereisaprocedure
T
P −
P toreconstructadatabaseDT∗ suchthat T T∗ 4ǫ/γ.Throughout
| △ |≤
therestoftheargument,weassumethatthiseventoccurs.Letx T be
∈
anelementselectedfromT uniformlyatrandom,andlety S T bean
∈ \
element selected from S T uniformly at random. Let T′=T x y .
\ \ ∪{ }
Observethat:
d/2 2ǫ/γ 4ǫ
Pr[x∈P(M(DT,ǫ))]≥ d−/2 =1− γ d
·
2ǫ/γ 4ǫ
Pr[x∈P(M(DT′,ǫ))]≤ d/2 = γ d
·
8 Aaron Roth
Since||DT−DT′||1 ≤2,wehavebythedefinitionofα-differentialprivacy
and fact 1:
Pr[x P(M(D ,ǫ))]
eα ∈ T
≥ Pr[x∈P(M(DT′,ǫ))]
1 4ǫ
− γd
·
≥ 4ǫ
γd
·
γ d
= · 1
4ǫ −
Solving for ǫ, we findthat:
γ d
ǫ Ω ·
≥ eα
(cid:18) (cid:19)
Since this holds for all choices of γ, theclaim follows from thefact that
d γFSD (C).
γ
≥
4 Upper Bound
Wenowshowthat(ignoringtheotherimportantparameters),itissuffi-
cienttoaddnoiselinearinthefatshatteringdimensionofC tosimulta-
neously guarantee usefulness with respect to C and differential privacy.
Unlike our lower bound which was not quite strong enough to state in
terms of relative error, our upper bound is most naturally stated as a
bound on relative error.
We make use of a theorem of Bartlett and Long [BL95] (improving a
bound of Alon et al. [ABCH97]) concerning the rate of convergence of
uniformGlivenko-Cantelliclasseswithrespecttotheirfat-shatteringdi-
mension.
Theorem 3 ([BL95] Theorem 9). Let C be a class of functions from
some domain X into [0,1]. Then for all distributions P over X and for
all η,δ 0:
≥
m
1
Pr"fsupC(cid:12)m f(xi)−Ex∼P[f(x)](cid:12)≥η#≤δ
∈ (cid:12) Xi=1 (cid:12)
(cid:12) (cid:12)
where {xi}mi=1 are m i(cid:12)(cid:12)ndependent draws from P(cid:12)(cid:12)and
1 1 1
m=O d ln2 +ln
η2 η/5 η δ
(cid:18) (cid:18) (cid:19)(cid:19)
where d =FSD (C).
η/5 η/5
We usethis theorem to provethefollowing useful corollary:
Corollary 1. Let C be a class of linear functions with coefficients in
N[0n,1w]iftrhom Rn+ to R. For any database D∈Rn+, there is a database D′ ∈
||D′||1 =O dηη2/5 ·log2 η1
(cid:18) (cid:18) (cid:19)(cid:19)
Differential Privacy and theFat-ShatteringDimension of Linear Queries 9
such that for each q C,
∈
D
(cid:12)q(D)− ||||D′||||11q(D′)(cid:12)≤η||D||1
(cid:12) (cid:12)
where d =FSD (cid:12)(C). (cid:12)
η/5 η/5(cid:12) (cid:12)
Proof. Let B = e n denote the set of n standard basis vectors over
Rn.LetP beth{epi}roi=b1abilitydistributionoverBthatplacesprobability
D
D / D on e . Note that for any q C:
i 1 i
|| || ∈
n n
D 1 q(D)
Eei∼PD[q(ei)]= Di1q(ei)= D 1 q(Diei)= D 1
i=1 || || || || i=1 || ||
X X
Let x ,...,x be m = O 1 d ln2 1 +ln2 independent draws
1 m η2 η/5 η
from PD, and let D′ = mi=(cid:16)1xi.(cid:16)Then: (cid:17)(cid:17)
P n m
q(D′)= q(Di′ei)= q(xi)
i=1 i=1
X X
By lemma 3, we have:
m
q(D′) q(D) 1
Pr(cid:20)(cid:12)(cid:12)(cid:12)(cid:12) m − ||D||1(cid:12)(cid:12)(cid:12)(cid:12)≥η(cid:21)=≤P21r"(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)mXi=1q(xi)−Eei∼PD[q(ei)](cid:12)(cid:12)(cid:12)(cid:12)(cid:12)≥η#
In particular, there exists some D′ Nn with D′ 1 = m that satisfies
′ ∈ || ||
q(D) q(D) η. Multiplying through by D gives the desired
||D′||1 − ||D||1 ≤ || ||1
b(cid:12)ound. (cid:12)
(cid:12) (cid:12)
(cid:12) (cid:12)
Armed with Corollary 1, we may now proceed to instantiate the expo-
nentialmechanismoverasparsedomain,analogouslytotheinstantiation
of theexponential mechanism in [BLR08].
Definition 4 (The Exponential Mechanism [MT07]). Let be
some domain, and let s : Rn R be some quality score mapDping
+×D →
database/domain-element pairs to some real value. Let
∆ max sup s(D ,r) s(D ,r)
s 1 2
≥ r∈DD1,D2∈R+n:||D1−D2||1≤1| − |
be an upper bound on the ℓ sensitivity of s.The exponential mechanism
1
definedwithrespecttodomain andscoresistheprobabilitydistribution
D
(parameterized bytheprivatedatabaseD)whichoutputseachr with
∈D
probability proportional to:
s(D,r) α
r exp ·
∼ (cid:18) 2∆s (cid:19)
Theorem 4 (McSherryandTalwar[MT07]).Theexponentialmech-
anism preserves α-differential privacy.
10 Aaron Roth
We let m=O dη/5 log2 1 , and define the domain of our instanti-
η2 · η
ation of theexp(cid:16)onential m(cid:16)ech(cid:17)a(cid:17)nism to be:
= D′ Nn: D′ 1 =m
D { ∈ || || }
Wenotethat =nm.Finally,wesampleeachD′ withprobability
|D| ∈D
proportional to:
D′ ∼exp−supq∈C(cid:12)q(D)−4||||DD′||||11 ·q(D′)(cid:12)α (2)
(cid:12) (cid:12)
(cid:12) (cid:12)
 
adnadtabouastepsutDth,eDdastuacbhastehaDtouDt ≡ ||||DDD′||||11 ·D′12.wOebhsaevrev:e that for any two
1 2 1 2 1
|| − || ≤
D D
qs∈uCp|q(D1)− ||||D1′||||11 ·q(D′)|−qs∈uCp|q(D2)− ||||D2′||||11 ·q(D′)|≤
D D
||D1−D2||1+ ||| 1||1m−|| 2||1| ≤
1
1+
m
Therefore, the distribution defined in equation 2 is a valid instantiation
of the exponential mechanism, and by [MT07] preserves α-differential
privacy.Itremainstoshowthattheaboveinstantiation oftheexponen-
tial mechanism yields a useful mechanism with low error. In particular,
it gives us a relatively useful mechanisms with respect to classes C for
databases that have size linear in the fat shattering dimension of C, or
only logarithmic in C for finite classes C. This is in contrast to the
| |
boundsof [HT10] that require databases to beof size polynomial in C
| |
before giving relatively-useful mechanisms.
Theorem 5. For any constant δ and any query class C, there is an
(η,δ)-relativelyusefulmechanismthatpreservesα-differentialprivacyfor
any database of size at least:
FSD (C)logn
D Ω˜ 2η/5
|| ||1 ≥ αη3
(cid:18) (cid:19)
Proof (ProofofTheorem).Recallthatthedomain ofourinstantiation
of the exponential mechanism consists of all dataDbases D′ Nn with
∈
||D′||1 =mwithm=O dηη/25 ·log2 η1 }Inparticular,bycorollary1,
there exists a D∗ su(cid:16)ch that: (cid:16) (cid:17)(cid:17)
∈D
(cid:12)q(D)− ||||DD∗′||||11q(D∗)(cid:12)≤η||D||1
(cid:12) (cid:12)
(cid:12) (cid:12)
2 If D isnotpublicknow(cid:12) ledge,itcanbeestim(cid:12)atedtosmallconstanterrorusingthe
1
|| ||
Laplace mechanism [DMNS06], losing only additiveconstantsin theapproximation
parameter ǫ and privacy parameter α. This does not affect our results.