Binary adaptive embeddings from order statistics of random projections

0.56 MB·

by  Diego Valsesia

1
Binary adaptive embeddings from order statistics of
random projections
Diego Valsesia, and Enrico Magli
Abstract—We use some of the largest order statistics of the a similar idea by proposing to choose hash functions with
random projections of a reference signal to construct a binary a robustness criterion, that essentially measures how far a
embeddingthatisadaptedtosignalscorrelatedwithsuchsignal. random projection falls from the edges of a quantization
The embedding is characterized from the analytical standpoint interval. Our work presents a rigorous analytical treatment of
and shown to provide improved performance on tasks such as
abinaryembeddingobtainedfromtheselectionoftherandom
classification in a reduced-dimensionality space.
projections of a reference signal with largest magnitude. The
7
Keywords—Binary Embeddings, Random projections analysis of the embedding provides insights on its advantages,
1
particularly in mitigating the difficulty of low-contrast nearest
0
2 I. INTRODUCTION neighbor problems and superior performance on classification
tasks, e.g. in a neural network.
n The ever-increasing amount of information that is produced
a in the age of Big Data calls for efficient techniques for
II. PROPOSEDMETHOD
J storage and processing of a large number of high-dimensional A. Preliminaries

0 signals.Compactrepresentationscanbeobtainedwithdifferent
Definition 1. A mapping φ : X → Y of metric spaces,
3 methods depending whether the particular task requires signal
endowed with distances d and d is called an embedding
] raencdonvsitsruuacltiizoanti(oen.g).,oirmtahgeegaonadlviisdetoocionmfeprressosmioenifnofrodrmeliavtieoryn with distortion C > 0 ifXLdX(u,Yv) ≤ dY(φ(u),φ(v)) ≤
G CLd (u,v) for some constant L>0 and for all u,v∈X.
fromthesignals(e.g.,inclassification,regression,information X
L A well-known binary embedding is the sign random pro-
retrieval, etc.). Embeddings provide compact representations
s. of signals for the latter tasks. Formally, an embedding is a jections [3] where a random matrix Φ made of independent
and identically distributed (i.i.d.) Gaussian entries is used
c transformationthatmapsasetofsignalsinahighdimensional
[ space to a lower dimensional space, in such a way that the to compute some random projections, which are then quan-
1 geometry of the set is approximately preserved. The concept tized to a binary representation by keeping their sign. The
Hamming distance between the binary vectors approximately
v of embedding has been successfully used in the context of
preserves the angle between the signals in the original space
1 information retrieval [1], where it is usually called “hashing”.
[3], i.e., P(sign(Φ u)=sign(Φ v)) = 1 − θ, being θ =
1 An important class of signal embeddings are those preserving (cid:16) (cid:17) i i π
5 the distances among pairs of signals. Johnson and Linden- cos−1 uTv and Φ the i-th row of Φ.
(cid:107)u(cid:107)(cid:107)v(cid:107) i
8 strauss [2] famously stated that an embedding can be realized
B. Proposed adaptive embedding
0 withaLipschitzmappingtoapproximatelypreserveEuclidean
1. distances with a dimension of the embedding space that only A reference signal u is used to generate the adaptive
0 depends on the desired distortion and logarithmically in the embedding in the following way. A number mpool of random
7 number of signals to be embedded. Random projections have projections is computed by means of an i.i.d. Gaussian matrix
1 been shown to implement such embedding with high proba- Φ ∈ Rmpool×n as y = Φu. The m entries with largest
v: bility. Several extensions have later been proposed, allowing magnitude are identified and their locations stored in vector
i one to approximately preserve the angle between signals [3], l. The m-bit resulting binary code is:
X
[4], control the maximum distance that is embedded [5], or
p=sign(Φ u) , (1)
l
r preserve the Jaccard distance [6], [7]. Recently, some works
a
have studied learning embeddings [8]–[10] from training data where Φ is the matrix Φ restricted to the rows indexed by
l
to derive compact codes by exploiting the particular geometry l. The locations vector l is saved as side information of the
of the dataset (e.g., signals living close to a manifold). embedding and used as in (1) whenever a new signal is to be
In this paper we propose an approach to construct an embedded, i.e., q = sign(Φ v), where v is a generic signal
l
embeddingthatisnotbasedonlearninganddoesnotrequirea and q its embedding.
training set of data, but rather is adapted to a single reference The first theorem we prove confirms that the Hamming
signal. This choice maintains to some degree the universality distancedH(p,q)= m1 (cid:80)ipi⊕qi,i.e.thenumberofdiffering
of random projections and it is useful when the data present entries, between binary codes obtained with the adaptive
no particular structure. Jegou et al. [11] empirically explored embedding concentrates around its expected value.
Theorem 2. Let X ⊂ Rn be a set of N signals and u,v ∈
TheauthorsarewithPolitecnicodiTorino,Torino,Italy.Thisworkwassup-
portedbytheEuropeanResearchCouncilthroughtheEuropeanCommunity X. Let Φ ∈ Rmpool×n with Φi,j ∼ N(0,σ2), y = Φu and
SeventhFrameworkProgramme(FP7/2007-2013)underGrant279848. the locations l of the m ≤ mpool entries in y with largest
2
magnitudebeknown.Letp=sign(Φ u)andq=sign(Φ v). andthenrepeatthesamePoissonbinomialargumentasbefore.
l l
Then for 0<ε≤2e−1, However,thisiscumbersometocomputeandweinsteadderive
some bounds.
P(|dH(p,q)−µ|>(cid:15)µ)<e−µε22 +e−µε42 (2) Theorem 3. Under the same assumptions as Theorem 1, and
with 1 (cid:88)m being e2(mpool−m+1);2mpool the expected value of the 2(mpool−
µ=E[d (p,q)]= p m+1)-th order statistic of a sample of N(0,σ2(cid:107)u(cid:107)2) of size
H m i
2m :
i=1 pool
pi = 12 + 12erf(cid:32)−yli√2σ(cid:107)u(cid:107)(cid:112)(cid:107)uu(cid:107)2T(cid:107)vv(cid:107)2−(uTv)2(cid:33) E[dH(p,q)]≤ 12 + 12erf(cid:32)√2−σ(cid:107)eu2((cid:107)m(cid:112)poo(cid:107)l−um(cid:107)2+(cid:107)1v);2(cid:107)m2p−oolu(uTTvv)2(cid:33)
Proof: Let us consider a single measurement in the li Proof: We first notice that pi ≤ pm, ∀i = 1,...,m
location yi = Φliu and zi = Φliv. Then ζ = [yi,zi] is so that E[dH(p,q)] = m1 (cid:80)mi=1pi ≤ pm. Then, pm =
a bivariate Gaussian with zero mean and covariance Σ = E [g(τ)]≤g(E[τ])byapplyingJensen’sinequalitytog(τ)=
σ2(cid:20)(cid:107)uuT(cid:107)v2 (cid:107)uvT(cid:107)v2.(cid:21). Suppose that yi is observed to be yi =τi, PAτl(sηoτn≤ot0ic|eτ)t,hia.et.,thtehecsoanmveexGityauosfsiagnatlaloilwpsroubsabtoiliutyseasJebnesfeonre’s.
tηhτein=t(hzei|ycion=diτtiio)n∼alNdi(cid:16)sτtrii(cid:107)buuuT(cid:107)tvi2o,nσ2o(cid:16)f(cid:107)vzi(cid:107)2g−ive(nu(cid:107)Tuy(cid:107)v2i)2(cid:17)=(cid:17) .τi is tihneequ(maliptoyo.l E−[τm] =+e˜1()m-tphool−omrd+e1r);mstapotoilstiiscthoef eaxpseacmtepdlevaolfuesizoef
After quantization of the measurements, the probability of mpool from a half Gaussian (since we consider |y|). We then
mismatching bits in position li is notice that that is equivalent [13] to e2(mpool−m+1);2mpool, i.e.,
pi =P(ητ ≤0|τi >0) the2(mpool−m+1)-thorderstatisticofasampleofsize2mpool
(cid:32) (cid:33) of a full Gaussian with zero mean and σ2(cid:107)u(cid:107)2 variance.
1 1 uTv
As a further remark, according to [14] an approximation of
= + erf −τ √
2 2 i 2σ(cid:107)u(cid:107)(cid:112)(cid:107)u(cid:107)2(cid:107)v(cid:107)2−(uTv)2 the expected value of the desired order statistic is:
Define the following random variable (cid:18)2(m −m+1)−α(cid:19)
e ≈F−1 pool
E =(cid:26)0 with probability 1−pi . (3) 2(mpool−m+1);2mpool 2mpool−2α+1
i 1 with probability pi being α = 0.375 and F−1 the inverse CDF of a normal
Then, D = 1 (cid:80)m E is a Poisson Binomial random distribution with zero mean and variance σ2(cid:107)u(cid:107)2.
H m i=1 i Sofarweconsidereddistancesbetweenatestsignalandthe
variable measuring the Hamming distance between p and q.
reference used to adapt the embedding. We will now consider
Eq. (2) can be readily obtained using Chernoff bounds [12]
what happens to the distance between any arbitrary pair of
for the tails of D .
H
signals v and w. Qualitatively, we can say that the curve of
The previous theorem holds for a fixed pair of signals. It
the expected value of the Hamming distance in the embedded
is customary to derive an asymptotic result on the number
of measurements needed to provide a distortion δ around spaceasfunctionoftheoriginaldistancebetweenvandwwill
be somewhere between the one predicted by Theorem 2 and
the expectation when signals are drawn from a finite set of
cardinality N. Standard derivation using a union bound on the one of non-adaptive sign random projections depending
Eq. (2) yields m = O(δ−2logN), which is exactly the same on how much w is close to the reference u. The following
theorem formalizes this concept.
as classic results on non-adaptive random projections [2]. The
advantages of the proposed method are, in fact, due to the Theorem 4. LetX ⊂Rn beasetofN signalsandu,v,w∈
modified expected value rather than the variance. X. Let Φ ∈ Rmpool×n with Φi,j ∼ N(0,σ2), y = Φu and
Moreover, the previous theorem supposed we knew the the locations l of the m ≤ m entries in y with largest
pool
valuesoftheprojectionsofthereferencesignalatthelocations magnitude be known. Let p = sign(Φ u), q = sign(Φ v),
l l
kept as side information. This allows us to compute the exact and r=sign(Φ w). Then for 0<ε≤2e−1,
l
expected value of the Hamming distance in the embedded
space as function of the inner product (or correlation coeffi- P(|dH(q,r)−µ|>(cid:15)µ)<e−µε22 +e−µε42 (4)
cient)betweentheoriginalsignals.However,itmightbeuseful
tohavesomea-prioriknowledgeabouttheembeddingwithout withµ=E[d (q,r)]= 1 (cid:88)m p
the need to know the reference signal. The following theorem H m i
i=1
approximatelyboundstheexpectedvalueoftheembeddingby (cid:20) (cid:18)(cid:20) (cid:21)(cid:19) (cid:18)(cid:20) (cid:21)(cid:19)(cid:21)(cid:20) (cid:18)(cid:20) (cid:21)(cid:19)(cid:21)
0 0 +∞
characterizingtheorderstatisticsof|y|.Thek-thorderstatistic p = F −F 1−F
i +∞ 0 0
of a statistical sample is equal to its kth-smallest value.
(cid:20) (cid:18)(cid:20) (cid:21)(cid:19) (cid:18)(cid:20) (cid:21)(cid:19)(cid:21) (cid:18)(cid:20) (cid:21)(cid:19)
Letuscallf (τ)theprobabilitydensityfunctionofthek-th +∞ 0 +∞
k + F −F F ,
order statistic of |y|. We could then in principle compute the 0 0 0
probability of bit mismatch as
being F the CDF of a bivariate Gaussian
(cid:90) (cid:20) (cid:21)
pi = P(ητ ≤0|τ)fmpool−i+1(τ)dτ for i=1,2,...,m with mean µ(cid:48) = (cid:107)yul(cid:107)i2 uuTTwv and covariance
3
(cid:34) (cid:107)v(cid:107)2− (uTv)2 vTw− (uTv)(uTw)(cid:35)
Σ(cid:48) =σ2 (cid:107)u(cid:107)2 (cid:107)u(cid:107)2 .
vTw− (uTv)(uTw) (cid:107)w(cid:107)2− (uTw)2
(cid:107)u(cid:107)2 (cid:107)u(cid:107)2
Proof:TheproofissimilartotheproofofTheorem2.Let
usconsiderasinglemeasurementinthel locationy =Φ u,
i i li
z =Φ v, a =Φ w. Then ζ =[z ,a ,y ] is Gaussian with
i li i li i i i 
(cid:107)u(cid:107)2 uTv uTw
zero mean and covariance Σ = σ2uTv (cid:107)v(cid:107)2 vTw.
uTw vTw (cid:107)w(cid:107)2.
Supposethaty isobservedtobey =τ ,thentheconditional
i i i
distribution of [z ,a ] given y = τ is η = ([z ,a ]|y =
i i i i i i i
(cid:20) (cid:21)
τi)∼N (µ(cid:48),Σ(cid:48))withµ(cid:48) = (cid:107)yul(cid:107)i2 uuTTwv andcovarianceΣ(cid:48) =
(cid:34) (cid:107)v(cid:107)2− (uTv)2 vTw− (uTv)(uTw)(cid:35) Fig. 1. Hamming distance in the embedding a function of inner product
σ2 (cid:107)u(cid:107)2 (cid:107)u(cid:107)2 .
vTw− (uTv)(uTw) (cid:107)w(cid:107)2− (uTw)2 between reference and test signals. Unit norm signals, m = 800, mpool =
(cid:107)u(cid:107)2 (cid:107)u(cid:107)2 5000.Non-adaptivecurveusessignrandomprojections[3].
After quantization of the measurements, the probability of
mismatching bits in position l is
i
p =P(η <0|η >0)P(η >0)+P(η >0|η <0)P(η <0)
i 1 2 2 1 2 2
Define the random variable E as in (3), then D =
i H
1 (cid:80)m E is a Poisson Binomial random variable measuring
m i=1 i
theHammingdistancebetweenqandr.Eq.(4)canbereadily
obtained using Chernoff bounds [12] for the tails of D .
H
The key distinction between sign random projections [3]
and the method presented in this paper is that the former
provides a linear relationship between the Hamming distance
in the embedded space and the angle in the original space.
On the other hand, the binary adaptive embedding provides
a nonlinear relationship between the two distances, with the
Fig. 2. Hamming distance in the embedding a function of inner product
important property that the Hamming distances observed with
betweenreferenceandtestsignalsandbetweenbothtestsignals.Unitnorm
theadaptiveembeddingarealwayssmallerthanthoseobserved signals,m=800,mpool=5000.
with sign random projections. This property is at the core of
the improved performance of the embedding for tasks such as able to find approximate nearest neighbors in a time O(Nr)
binary classification, as discussed in Sec. III. Fig. 1 shows the that is sublinear in the database size N but that degenerates
Hamming distance in the embedded space as function of the to linear search as the contrast approaches 1. The curse of
inner product between the test signal and the reference signal dimensionality makes low contrast more probable when high-
in the original space. It can be noticed how the curve of the dimensional spaces are considered. The adaptive embedding
adaptive embedding always lies below the one for the non- presented in this paper can be used as a dimensionality
adaptive embedding. For a fixed value of m increasing the reduction technique to solve approximate nearest neighbor
pool
umber of measurements m will reduce the variance and move search more efficiently in low-contrast scenarios.
theexpectedvaluetowardsthatofthenon-adaptiveembedding. In order to show this we develop an experiment in a high-
Viceversa,forafixedmincreasingm willlowerthecurve. dimensional space where the goal is to find the nearest neigh-
pool
Fig.2showstheHammingdistancebetweenthefirsttestsignal bors of a given signal within a certain radius. True neighbors
and the second test signal in the embedded space as function are generated as standard Gaussian vectors with n = 8192
oftheinnerproductbetweenthefirsttestsignalandthesecond i.i.d. entries with an expected correlation coefficient equal to
test signal (vTw) and between the second test signal and the 0.07 to a reference that is also used as query. Disturbing
reference signal (uTw) in the original space. Notice how for signals are i.i.d. Gaussian with zero expected correlation.
decreasing uTw the shape of the embedding tends to the one Notice that they are almost orthogonal to each other but
of a non-adaptive embedding. the contrast is low because the true neighbours are weakly
correlated with our query. This problem is not unrealistic and,
in fact, as an example, it occurs in the detection of photo-
III. APPLICATIONSANDEXPERIMENTS
response non-uniformity artifacts from camera sensors [18]
A. Low-contrast approximate nearest neighbors
[19], used to attribute a given picture to a given camera
A measure of difficulty [15], [16] of a nearest neighbor sensor. In this experiment all the signals in the database are
search problem is the contrast r, which is defined as the ratio adaptively embedded, i.e., the locations of the m entries of
between the distance of the closest false neighbor and that of largest magnitude are identified and stored with the binary
the farthest true neighbor. Locality sensitive hashing [17] is code. The random projections of the query are computed and
4
1 TABLEI. CLASSIFICATIONACCURACY
0.9 Method Accuracy
Uncompressed 92.64%
0.8
m=32 m=64 m=128 m=256
0.7 Adaptive 92.30% 92.33% 92.40% 92.49%
etaR0.6 Sign(erqa.ndcoommppleroxjietyc)tions 75.49% 87.03% 91.09% 92.27%
evitisoP0.5 Un(ievqe.rscaolmEpmlebxeidtyd)ing 77.13% 87.81% 92.10% 92.30%
eurT0.4 m=233 m=405 m=680 m=1065
Signrandomprojections
0.3 91.90% 92.32% 92.38% 92.51%
(eq.storage)
0.2 UNnocnoamdappretivsese (deq. complexity) UniversalEmbedding 92.20% 92.32% 92.39% 92.49%
Nonadaptive (eq. storage) (eq.storage)
0.1 Adaptive
UUnniivveerrssaall ((eeqq.. csotomrapgleex)ity) large, this operation may require significant storage space for
0
0 0.2 0.4 0.6 0.8 1 the real-valued weights as well as computational resources to
False Positive Rate
computealltheinnerproducts.Thiscanbeovercomeusingan
Fig.3. ROCfornearestneighborsearch.Apointisdeclaredaneighborof
embeddingsuchassignrandomprojections.Afterthetraining
the query if the Hamming distance is below a given threshold. n = 8192,
mpool=8192,m=512.Thequeryisusedasreferencesignal. phaseiscompleted,theweightsareembeddedinacompactbi-
narycodeforeachclass.Duringpredictionsthefeaturevectors
appropriatelysubsampledaccordingtothelocationsstoredfor
are also embedded, the Hamming distance with the weights is
each database signal under test. The storage requirement for
computed and the class label corresponding to the minimum
each database entry is the sum of the bits needed by the
distance is selected, i.e. l = argmin d (ω ,y),
binary measurements and the overhead due to the adaptively i={1,...,k} H i
chosen locations and it amounts to m + log (cid:2)(cid:0)mpool(cid:1)(cid:3) bits. being ωi = sign(Φw) and y = sign(Φx). Replacing sign
2 m random projections with the proposed adaptive embedding
Fig. 3 shows the Receiver Operating Characteristic (ROC)
can improve the classification performance of the compressed
showingtheprobabilityofdetectionofatrueneighboragainst
system.Overall,thereareasmanyadaptiveembeddingsasthe
the probability of false alarm. We notice that the adaptive
numberofclasses.Therandomprojectionsofeachw areused
embeddingprovidesaperformanceclosertotheuncompressed i
tocomputeadifferentsetoflocationsl foreachclass.Attest
case. Two non-adaptive strategies are presented for a fair i
time, the feature vector is embedded k times to generate y
comparison.Anon-adaptivemethodusingthesamestorageas i
theadaptivemethodwouldusem(cid:48) =m+log (cid:2)(cid:0)mpool(cid:1)(cid:3)binary (thisamountstogeneratingmpool non-adaptiveprojectionsand
2 m then subsampling according to the corresponding l ). Hence,
random projections, with the drawback of increased compu- i
the class label is given by l = argmin d (ω ,y ),
tational complexity in the Hamming distance evaluation. The i={1,...,k} H i i
secondstrategyequalizescomputationalcomplexity,thususing being ωi =sign(Φliw) and yi =sign(Φlix).
m(cid:48) =m non-adaptive measurements. This is advantageous in The following experiment is a classification problem on
the CIFAR-10 dataset [22] comprising 10 classes. We im-
termsofstoragebutitperformssignificantlyworse.Finally,we
plemented the same convolutional neural network architecture
comparedtheproposedmethodwiththeUniversalEmbedding
presentedin[23].Thisnetworkiscomposedof8convolutional
ofBoufounosetal.[5]whichisakindofadaptiveembedding
layers followed by 2 fully connected layers all with ReLU
wherethequantizercanbeparametrizedinordertodistortthe
activation units [24] and a final linear layer. The last linear
expected value of signal distances, similarly to the embedding
layer outputs one of the k =10 class labels from a n=1024-
proposed in this paper. The Universal Embedding bounds the
dimensionalinputfeaturevector.Afterconventionaltrainingof
maximum distance that is embedded, beyond which points
the network, we replaced the layer weights with its embedded
becomeindistinguishable.Itisthereforeexpectedtohavepoor
codes as explained above. For the adaptive method we used
performance in low-contrast, low-correlation scenarios, as it
appears from Fig. 3., where the quantization step is ∆=2. mpool = 1024. Table I shows the classification accuracy as
function of the number of measurements used by the embed-
B. Multiclass linear classifiers
ding. It can be noticed that the adaptive embedding allows to
In this section we apply the adaptive embedding to a
achieve a significant dimensionality reduction at a negligible
multiclass linear classifier in order to improve its storage and
loss in terms of classification accuracy, with respect to both
computational efficiency. Linear classifiers are widely used
sign random projections [3] and the universal embedding [5].
in the context of deep neural networks, where the layers of
Thequantizationstepsizeoftheuniversalembeddinghasbeen
the network are trained to disentangle the features of each
optimized via cross validation to value ∆=180.
class and a simple linear classification layer provides the
class labels. A k-class linear classifier can be written as IV. CONCLUSIONS
l = argmax w x, being l the class label, w the This paper presented a technique to generate compact bi-
i={1,...,k} i i
weightsvectorsandxafeaturevector.Theweightsarelearned nary codes from high-dimensional signals adapting them to a
duringthetrainingphaseusingasuitablelossfunctionsuchas reference signal. The resulting embedding displays interesting
thehingeloss[20]forsupportvectormachinesorthesoftmax properties that allow to improve performance in classification
cross-entropy [21] for multinomial logistic regression which tasks when those are performed in the reduced-dimensionality
is more popular in deep neural networks. Since the feature domain. Future work will focus on generalizing the approach
vectors may be high dimensional and the number of classes to sub-Gaussian and structured sensing matrices.
5
REFERENCES [20] C.CortesandV.Vapnik,“Support-vectornetworks,”Machinelearning,
vol.20,no.3,pp.273–297,1995.
[1] D. Achlioptas, “Database-friendly random projections: Johnson- [21] C.M.Bishop,“Patternrecognition,”MachineLearning,vol.128,2006.
Lindenstrauss with binary coins,” Journal of computer and System [22] A. Krizhevsky and G. Hinton, “Learning multiple layers of features
Sciences,vol.66,no.4,pp.671–687,2003. fromtinyimages,”2009.
[2] W.B.JohnsonandJ.Lindenstrauss,“ExtensionsofLipschitzmappings [23] K.SimonyanandA.Zisserman,“Verydeepconvolutionalnetworksfor
intoaHilbertspace,”ContemporaryMathematics,vol.26,1984. large-scaleimagerecognition,”arXivpreprintarXiv:1409.1556,2014.
[3] M. S. Charikar, “Similarity estimation techniques from rounding [24] V. Nair and G. E. Hinton, “Rectified linear units improve restricted
algorithms,” in Proceedings of the Thiry-fourth Annual ACM Boltzmann machines,” in Proceedings of the 27th International Con-
Symposium on Theory of Computing, ser. STOC ’02. New ferenceonMachineLearning(ICML-10),2010,pp.807–814.
York, NY, USA: ACM, 2002, pp. 380–388. [Online]. Available:
http://doi.acm.org/10.1145/509907.509965
[4] L.Jacques,J.N.Laska,P.T.Boufounos,andR.G.Baraniuk,“Robust1-
bitcompressivesensingviabinarystableembeddingsofsparsevectors,”
IEEETrans.Inf.Theory,vol.59,no.4,pp.2082–2102,April2013.
[5] P.T.BoufounosandS.Rane,“Efficientcodingofsignaldistancesusing
universal quantized embeddings,” in Data Compression Conference
(DCC),2013,March2013,pp.251–260.
[6] A. Broder, “On the resemblance and containment of documents,” in
Proceedings of the Compression and Complexity of Sequences 1997.
Washington, DC, USA: IEEE Computer Society, 1997, pp. 21–29.
[Online].Available:http://dl.acm.org/citation.cfm?id=829502.830043
[7] D.Valsesia,S.Fosson,C.Ravazzi,T.Bianchi,andE.Magli,“Sparse-
Hash:EmbeddingJaccardCoefficientbetweenSupportsofSignals,”in
Proc.ofMM-SPARSEWorkshopatICME2016,2016.
[8] M.Norouzi,D.J.Fleet,andR.R.Salakhutdinov,“Hammingdistance
metriclearning,”inAdvancesinneuralinformationprocessingsystems,
2012,pp.1061–1069.
[9] Y. Weiss, A. Torralba, and R. Fergus, “Spectral hashing,” in
Advances in Neural Information Processing Systems 21, D. Koller,
D. Schuurmans, Y. Bengio, and L. Bottou, Eds. Curran Associates,
Inc., 2009, pp. 1753–1760. [Online]. Available: http://papers.nips.cc/
paper/3383-spectral-hashing.pdf
[10] K. He, F. Wen, and J. Sun, “K-means hashing: An affinity-preserving
quantizationmethodforlearningbinarycompactcodes,”inTheIEEE
ConferenceonComputerVisionandPatternRecognition(CVPR),June
2013.
[11] H. Jegou, L. Amsaleg, C. Schmid, and P. Gros, “Query adaptative
localitysensitivehashing,”in2008IEEEInternationalConferenceon
Acoustics,SpeechandSignalProcessing,March2008,pp.825–828.
[12] M.MitzenmacherandE.Upfal,Probabilityandcomputing:Random-
ized algorithms and probabilistic analysis. Cambridge University
Press,2005.
[13] N. Balakrishnan and A. C. Cohen, Order statistics & inference: esti-
mationmethods. Elsevier,2014.
[14] H.L.Harter,“Expectedvaluesofnormalorderstatistics,”Biometrika,
vol. 48, no. 1/2, pp. 151–165, 1961. [Online]. Available: http:
//www.jstor.org/stable/2333139
[15] J.He,S.Kumar,andS.-F.Chang,“Onthedifficultyofnearestneighbor
search,”inICML2012.
[16] K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, When
Is “Nearest Neighbor” Meaningful? Berlin, Heidelberg: Springer
Berlin Heidelberg, 1999, pp. 217–235. [Online]. Available: http:
//dx.doi.org/10.1007/3-540-49257-7 15
[17] A.AndoniandP.Indyk,“Near-optimalhashingalgorithmsforapprox-
imate nearest neighbor in high dimensions,” Commun. ACM, vol. 51,
no.1,pp.117–122,Jan.2008.
[18] D. Valsesia, G. Coluccia, T. Bianchi, and E. Magli, “Compressed
fingerprintmatchingandcameraidentificationviarandomprojections,”
IEEE Transactions on Information Forensics and Security, vol. 10,
no.7,pp.1472–1485,July2015.
[19] ——,“Large-scaleimageretrievalbasedoncompressedcameraiden-
tification,”IEEETransactionsonMultimedia,vol.17,no.9,pp.1439–
1449,Sept2015.