Table Of ContentThe Quantified City: Sensing Dynamics in Urban
Setting
Tahar Zanouda, Noora Al Emadi, Sofiane Abbar, and Jaideep Srivastava
Qatar Computing Research Institute
Hamed Bin Khalifa University
Doha, Qatar
7
1 {tzanouda, nalemadi, sabbar, jsrivastava}@hbku.edu.qa
0
2
n
a ABSTRACT ders a set of map-centric widgets that summarize the
J underlying related dynamics. This paperhighlights the
Theworldiswitnessingaperiodofextremegrowthand
6 needforsuchaholisticplatform,identifiesthestrengths
urbanization; cities in the 21st century became nerve
1 ofthe”QuantifiedCity”concept,andshowcasesawork-
centers creating economic opportunities and cultural
values which make cities grow exponentially. With this ing demo through a real-life scenario.
]
I rapid urban population growth, city infrastructure is
S
facing major problems, from the need to scale urban 1. INTRODUCTION
.
s systems to sustaining the quality of services for citizen
c ’We shape our buildings, and afterwards, our build-
atscale. Understandingthedynamicsofcitiesiscritical
[ ings shape us.’ wrote Winston Churchill [].
towards informed strategic urban planning. This paper
After the revolutionary growth of the modern indus-
1 showcases QuantifiedCity , a system aimed at under-
try in the late 18th century, cities have became a niche
v
standing the complex dynamics taking place in cities.
3 ofnewopportunitiesthatattractedhugenumbersoflo-
Often,thesedynamicsinvolvehumans,services,andin-
5 calandinternationalmigrants. In2015,wewere53.89%
frastructuresandareobservedindifferentspaces: phys-
2 to live in urban areas and this number is projected to
4 ical(IoT-based)sensingandhuman(social-based)sens- reach 70% by the end of 2050. This fast urbaniza-
0 ing. The main challenges the system strives to address
tion has exposed different services and infrastructures
. are related to data integration and fusion to enable an
1 in cities to an increasing stress, leading to a plethora of
0 effective and semantically relevant data grouping. This urbanchallengesrelatedtohumanmobility(e.g.,traffic
7 is achieved by considering the spatio-temporal space as congestion),publichealth(e.g,airpollution),neighbor-
1 a blocking function for any data generated in the city.
hood deprivation (e.g., lack of parks), and economical
: Our system consists of three layer for data acquisition,
v difficulties (e.g., unemployment). The situation is such
i dataanalysis,anddatavisualization;eachofwhichem- that the United Nation has declared managing urban
X
beds a variety of modules to better achieve its purpose
areas as one of the most important development chal-
r (e.g.,datacrawling,datacleaning,topicmodeling,sen-
a lenges of the 21st century.
timent analysis, named entity recognition, event detec-
Luckily, the new cities that we are building are big
tion, time series analysis, etc.) End users can browse
data generators, which unleashed many initiatives to
thedynamicsthroughthreemaindimensions: location,
use data-driven methods in order to tackle the afore-
time, and event. For each dimension, the system ren-
mentioned challenges and understand more generally
how cities work and operate [2]. A new body of lit-
erature related to Urban Computing has been enabled
by the availability of data at large scales [8]. On one
hand, the advent of the Internet of Things [1] made it
easier to collect real-time data about the city dynamics
including: infrastructureusage,powerconsumption,en-
vironment, health related issues, human behavior, and
commutes. At the other hand, the high penetration
of smart connected devices (e.g., smartphones) and the
rapidadoptionofsocialmediaplatforms,hasturnedhu-
Copyrightisheldbytheauthor/owner(s).
mans into mobile and real-time data generators, lead-
ing to what is known as social sensing [7]. Thus, re-
Data Acquisition Data Analysis Data Visualization
searchers have studied different types of dynamics by
mining spatio-temporal data generated either by users
or sensors. Examples include using call data records Human Insights
mobility
(CDR) to understand mobility and commute patterns
Social Data
in cities [3, 4], using crowd-sourced taxi GPS data to Event
Detection Recomm-
estimate gas consumption and air pollution [6], and fi- es nedations
nally using check-ins data from location-based social aliti Traffic
networks (LSBNs) to infer traffic congestion [7]. tion Inference Statistics
Using IoT-based sensing data has long time tradi- c
un Location
tion in building smart city applications such as parking F identificat
ion
management systems, whereas using social-based sens- Mapping
ing data has been extensively used in building life style Content
Analysis
applications such as recommender engines for restau-
Internet of Things
rants. We aim in this paper to bridge the gap between
these two spaces, by designing QuantifiedCity – a
es Stream Spark
comprehensiveplatformthatmakesitpossibletolever- gi acquisition
age data from different spaces to build smarter appli- olo Giraph
n
h
cations. QuantifiedCity aims at quantifying the dy- ec Rheem HTML5 CSS3 NodeJS
namics involving humans, services and infrastructures, T SQL Mongo
taking place in cities. A good example to showcase the
relevanceofthiscombinationareroadtrafficmonitoring
systems. Indeedtraditionalsystemsrelyonthedeploy- Figure 1: Overview of data extraction and data
ment of physical sensors on road segments and inter- fusion Approach
sections to count the number of cars passing by. This
allows a nice ”real-time”visualization of the traffic sta-
2. THE QUANTIFIED CITY
tusindifferentpartsofthecity. However,relyingsolely
onphysicalsensors,makesitimpossibletoproceedwith The general architecture of the QuantifiedCity is
the diagnosis of the traffic. A red congested segment, composed of three major components: (1) data acqui-
canbesoformanyreasons(e.g.,accident,malfunction- sition, (2) data analysis, and (3) data visualization.
ing traffic light, weather condition, etc.) that are com- The data acquisition component contains the needed
pletelyhiddentothesystem. Thesocialspaceprovides modules to scrap and aggregate streams of data, as
a nice complementary as many users may have posted well as static geographical data sets such as census.
online in a completely unsolicited way about the event Data streams are processed concurrently in real-time
causing the traffic. Thus, combining the physical space and stored in one database.
with the social space, modulo the deployment of full- The data analysis component contains modules that
fledged intelligence to link data in the two spaces, may analyzethedata. Theresultsoftheanalysisarevisual-
results in semantically richer monitoring dashboards. ized through in a map-centric dashboard. The system
QuantifiedCity consists of three layers for data is designed as a suite of independently, small, scalable
acquisition, data analysis, and data visualization. Its modules tomakeiteasierandsimplerfordevelopment.
maincontributionistheuseofthespatio-temporalspace In what follows, we provide the details of each com-
as a blocking function to fuse data that is generated in ponent.
different spaces (e.g., physical, social). This is inspired
from the work of Jaro [5] in which he introduced the 2.1 Data Acquisition
use of blocking methods for record linkage. The intu-
This module allows the collection and fusion of sev-
ition behind our adaptation is to say that two pieces of
eral data sets. As a first step, we acquire static geo-
information are related if they are generated in same graphical and census data 1, this later is used to spa-
spatiallocationandwithinthesametimeinterval. The
tially index the city’s data. In order to fuse the knowl-
second big contribution of QuantifiedCity is the use
edgefromdifferentdatasources,weusespatio-temporal
of cross-modal data mining and machine learning tech-
units to conduct our studies. We divide the city into
niques in order to better combine features from the dif-
highresolutionspatialunits,andthedayonfine-grained
ferent spaces of data in the knowledge discovery pro-
timeunitstomakeiteasiertostudyanurbanphenom-
cesses [8].
ena from different angels. As an example: when we
Wedescribeinthefollowingsectionstheglobalarchi-
study the effects of on-road traffic on urban air pollu-
tecture of QuantifiedCity and provide some details
tion, it’s important to collect data on air quality from
about each of its layers. We also include a demonstra-
tion of its capabilities in a real-life scenario. 1http://www.mdps.gov.qa/ar/Pages/default.aspx
Current Event
Qatar ExxonMobil Open
Figure 2: Main GUI of QuantifiedCity . This showcases some statistics around ”Qatar ExxonMobil
Open”tennis tournament such as overall sentiment per location (map) and time (time line), density
of interactions on the map, traffic status, etc.
environmental sensors, number of cars from proximity to effectively manage big events.
sensors, and events happening in the same neighbor-
2.3 Data visualization
hoodfromsocialstreams. Thecross-referenceddatasets
can provide insights on relationship between the choice Aswehavedifferentmediumsofinformationtoshow-
of local events’ places and long-term air quality level in case, a highly interactive system is needed to allow ef-
the city. This later can lead urban planners to conclu- ficient and smooth interactions with complex intercon-
sions that can not be identified easily. nected data. The visualization explorer enables users
Our system captures, concurrently and continuously, to explore the data easily and intuitively. It is mainly
data streams on the city’s infrastructure and on the composed of two sub-views, namely, (1) the dashboard,
city’s social interactions. Once captured, data streams (2) map view.
are indexed based on time, and geographical informa- Dashboard View: The dashboard view shows the re-
tion. Then, data is loaded to centralized geo-database. sultsofthestatsinformation. Theviewshowsdifferent
pages, categorized in main categories and can be ac-
2.2 Data analysis
cessed by scrolling from top to bottom. Every page
This module consists of a set of data mining and has different panels, and every panel contain a chart to
machine learning algorithms to enable knowledge dis- visually communicate results of the different analysis.
covery from cross-modal fused data. This set of intel- Map view: Using Map-centred exploratory approach
ligence techniques vary from content-based techniques allows users to easily select spatial choices, as it intu-
(sentiment,NER,topicmodeling,eventdetection,etc.), itively provides contextual insights. In addition to the
to structure-based techniques such as network analysis maplayer, we use coloredzones shapes to showcase the
(community detection, etc. ). different results. The zones layer enables users to spec-
Asanexample,thesystemisabletocaptureandmine ify the targeted zone.
socialcontent,detectevents,andsensethesentimentof
people. By mining social networks, and photo-sharing
3. DEMONSTRATION
platforms,thesystemcanrevealplacesthatattractpeo-
plemost. Usingcontent-analysis,wecanidentifyplaces In this section, we showcase a detailed demonstra-
from text, link activity to places and verify whether tion plan and describe how different users (monitors,
people complain about traffic using sentiment analysis. urban planners, decision makers, police, etc.) can in-
Our results can optimize traffic light system in the city teract with QuantifiedCity .
Thesystemprovidescontextualinsightsbasedonthree ingfromdifferentsources. Theintuitionbeingthatdata
dimensions: i. time,ii. location,andiii. event. Asdata generatedinthesamegeographicalspaceandtimewin-
inoursystemisindexedtoaparticulartimespan,users dow can be grouped together and used for knowledge
can select the beginning and the end of the time range, discoveryrelatedtowhatishappeninginthatlocation.
and the granularity of the time analysis (e.g., group Bybridgingthegapbetweenthephysicalandthesocial
data by 20 secs or 5 minutes.) Users are able to nar- sides of cities, and through an interactive and intuitive
rowfocustoasubsetofdatabasedontheselectedtime map-centricinterface,oursystemallowsurbanplanners
bounds. Similarly, users can select specific regions or a and decision makers to visually inspect problems, and
particular location to filter the data and have limited interactively analyze and track in real-time the devel-
views to their selections. More interestingly, users can opment of events.
tap into the wealth of data around one event. Subse-
quently, users can combine different geographical and 5. REFERENCES
temporalcriteriatogainaccesstointuitive,yetinsight-
[1] L. Atzori, A. Iera, and G. Morabito. The internet
ful reports.
of things: A survey. Computer networks,
As an example to illustrate the results, we will fo-
54(15):2787–2805, 2010.
cus on some big Tennis tournaments that took place
[2] M. Batty, K. W. Axhausen, F. Giannotti,
in Qatar this January and show how QuantifiedCity
A. Pozdnoukhov, A. Bazzani, M. Wachowicz,
capturestheurbandynamics(socialactivity,sentiment
G. Ouzounis, and Y. Portugali. Smart cities of the
analysis, fans mobility, traffic congestion, etc.) around
future. The European Physical Journal Special
these events. With the use of content-analysis meth-
Topics, 214(1):481–518, 2012.
ods, QuantifiedCity can identify and distinguish be-
[3] M. C. Gonzalez, C. A. Hidalgo, and A.-L. Barabasi.
tween events in the same context (i.e., “Tennis tour-
Understanding individual human mobility
nament”will be automatically associated with QATAR
patterns. Nature, 453(7196):779–782, 2008.
EXXONMOBIL OPEN for men during the first week
[4] S. Grauwin, S. Sobolevsky, S. Moritz, I. G´odor, and
of January, and will be linked to Qatar Total Tourna-
C. Ratti. Towards a comparative science of cities:
ment for women in the last week, thanks to the spatio-
Using mobile traffic records in new york, london,
temporal blocking function). The system is able to
and hong kong. In Computational approaches for
fuse the knowledge from different contextual data to
urban environments, pages 363–387. Springer, 2015.
estimate crowd movement patterns (from sensor data
aboutcarsmovingaroundthestadiums,andsocialme- [5] M. A. Jaro. Advances in record-linkage
dia activity and engagement), and evaluate the com- methodology as applied to matching the 1985
plaints to the city managers during the event (using census of tampa, florida. Journal of the American
topic modeling and sentiment analysis). The informa- Statistical Association, 84(406):414–420, 1989.
tion can be used to fuel a recommendation engine to [6] J. Shang, Y. Zheng, W. Tong, E. Chang, and
help re-routing people from-to the game venues, it can Y. Yu. Inferring gas consumption and pollution
also notify the authorities about the need of an evacu- emission of vehicles throughout a city. In
ation plan if needed. The system can be connected to Proceedings of the 20th ACM SIGKDD
theroadtrafficlightsystemtoadjustcrowdmovement. International Conference on Knowledge Discovery
Figure 2 shows QuantifiedCity system visualizing and Data Mining, KDD ’14, pages 1027–1036, New
results of Doha city. On the right side of the dash- York, NY, USA, 2014. ACM.
board, various panels allow users to see the different [7] S. Wang, L. He, L. Stenneth, P. S. Yu, and Z. Li.
results of data analysis modules. On the left side, a Citywide traffic congestion estimation with social
map with zones polygons are visualized. By clicking on media. In Proceedings of the 23rd SIGSPATIAL
every zone, the system will generate statistics that will International Conference on Advances in
take the same form, but projected on one zone. Geographic Information Systems, SIGSPATIAL
’15, pages 34:1–34:10, New York, NY, USA, 2015.
ACM.
4. CONCLUSION
[8] Y. Zheng, L. Capra, O. Wolfson, and H. Yang.
In this paper, we presented QuantifiedCity , a sys- Urban computing: concepts, methodologies, and
tem that captures the complex dynamics taking place applications. ACM Transactions on Intelligent
in fast growing cities. The system relies on a compre- Systems and Technology (TIST), 5(3):38, 2014.
hensive set of data sources coming from different lay-
ers: physicaloverlaythroughIoT-basedsensingandhu-
manoverlaythroughsocial-basedsensingwhichenables
a holistic understanding of how cities work and oper-
ate. QuantifiedCity brings a novel perspective into
dataintegrationparadigmbyusingthespatio-temporal
spaceasablockingfunctiontolinkparcelsofdatacom-
