Managing Data Visualization Pipeline with Backbone.js and D3.js

87 Pages·2012·2.44 MB·English

Aalto University
School of Science
Master's Programme in ICT Innovation

Taha Kachwala

Managing Data Visualization Pipeline
with Backbone.js and D3.js
Improving overall software efficiency using
Automated Build Pipeline with Gulp

Master's Thesis
Espoo, 10.10.2016

Supervisor: Prof. Petri Vuorimaa
Instructor: Pertti Lounamaa
Aalto University
School of Science ABSTRACT OF
Master's Programme in ICT Innovation MASTER'S THESIS

Author: Taha Kachwala
Title:
Managing Data Visualization Pipeline with Backbone.js and D3.js.
Improving overall software efficiency using Automated Build Pipeline with Gulp
Date: 10.10.2016 Pages: 87
Professorship: Digital Media Technology Code: T-111
Supervisor: Prof. Petri Vuorimaa
Instructor: Petri Lounamaa Ph.D. (Economic Systems)
This thesis studies how a Model-View-Controller (MV*) framework can be
integrated into a Data Visualization Pipeline. Specifically, this thesis aims to
cover the challenges related in integrating an MV* framework like Backbone.js
with D3.js, which is a popular JavaScript based visualization library. Additionally,
it also evaluates another sub-topic regarding task runners, which are tools
claiming to automate manual tasks as well as streamlining the build process.

Data Visualization has become an important aspect for many web applications.
Developers need to employ sophisticated mechanisms to provide interactive
visualizations. This requires separation of concerns within the visualization
pipeline which is achieved with the combination of Backbone.js and D3.js.
Evaluation of this methodology indicates that such a combination enables the
application to be more versatile and robust while also improving performance.

In recent years, client-side web applications have become increasingly complex.
A typical web application on average requires between 10-60 external
open-source JavaScript libraries. Application development also requires the use
of several other tools and performing manual tasks. Managing all these libraries
and tools can create a bottleneck, and task-runners like Gulp aim to address
these issues. This thesis implements an Automated Build Pipeline that can
streamline the build process and automates all the manual tasks. The results
prove significant performance and overall efficiency benefits. However, it also
highlights a few serious drawbacks. Moreover, this thesis also covers some of
the best practices employed by skilled front-end web developers.
Keywords: Data Visualization, JavaScript, MVC, Web Application, Gulp
Language: English

1
Acknowledgements

First, I would like to thank my supervisor, professor Petri Vuorimaa, to provide me
guidance throughout my internship and thesis.

Furthermore, I would also like to thank my employer Pajat Solutions to provide me
with a challenging internship and an interesting thesis topic. I would especially like
to thank the CEO Pertti Lounamaa, for providing me with the flexibility and freedom
to make decisions regarding the product development of Poimapper.

Espoo, October 10, 2016

Taha Kachwala

2
Abbreviations & Acronyms

API Application Programming Interface
CSS Cascading Style Sheets
HTML HyperText Markup Language
InfoVis Information Visualization
JS JavaScript or JavaScript file
JSON JavaScript Object Notation
JSP Java Server Pages
MVC, MV* Model View Controller Framework
NPM Node Package Manager
POI Point of Interest
REST Representational State Transfer
SLOC Source Lines of Code
UI User Interface

3
Contents

1. Introduction 7
​
1.1 Motive for the Research 7
1.2 Research Objectives 9
1.3 Target Group 9
1.4 Poimapper 9
1.5 Structure of Thesis 10
​
2. Product Description - Poimapper 11
​ ​
2.1 Form Builder 12
​
2.2 Data Viewer 13
​
2.3 Hierarchy Editor 15
​
2.4 Admin and Scheduler 15
​
3. Data Visualization and D3.js 16
​ ​
3.1 Definition 16
​
3.2 Branches of Visualization 17
​
3.3 InfoVis 18
​
3.3.1 Attributes of Visualization 18
​
3.3.2 Principles of Visualization 18
​
3.3.3 The Visualization Pipeline 27
​
3.4 Creating Visualizations using D3.js 27
​
3.4.1 Chart Object 27
​
3.4.2 Scalable Vector Graphics (SVG) 28
3.4.3 D3-Scales 29
3.4.4 D3-Axes 30
​
3.4.5 Legends and Tooltips 31
​
3.4.6 Bar Charts 31
​
3.4.7 Pie Charts 34
​
4
3.4.8 Line Charts 35
​
3.4.9 Area Charts 35
​
4. Backbone.js - An MV* Framework 37
​
4.1 Need for MV* Frameworks 37
4.2 Model-View-Controller 38
4.2.1 Model 39
4.2.2 Controller 39
4.2.3 Views 39
4.2.4 Routers 40
​
4.3 More about Backbone.js 40
​
4.3.1 Underscore.js 41
​
4.3.2 Agnostic Templating 42
​
4.3.3 MV* 42
​
4.3.4 Clean HTML 46
​
4.3.5 Extensions 46
5. Implementing the Data Visualization Pipeline 47
​
5.1 Specifications of a Web Report 47
5.2 Architecture of Web Reports Module 48
5.2.1 Models 49
5.2.2 Collections 50
​
5.2.3 Views 51
​
5.3 Adding New Features 54
​
6. Automated Build Pipeline using Gulp 55
​
6.1 Miscellaneous Tools 56
6.1.1 CSS Preprocessors 56
6.1.2 CoffeeScript 56
6.1.3 Bower 59
6.2 Gulp 60
​
6.2.1 Installation and Setup 60
​
6.2.2 Gulp Workflow 60
​
6.2.3 Helper Plugins 61
​
6.2.4 Build Pipeline 64
6.2.6 Gulp Tasks 66
7. Evaluation 74
​
7.1 Evaluation of Data Visualization Pipeline with Backbone.js and D3.js 74
7.1.1 Data-Binding 74
7.1.2 Performance Analysis 75
7.1.3 Loose Coupling and High Cohesion 76
7.2 Evaluation of Automated Build Pipeline using Gulp 77
5
7.2.1 Improved Developer Efficiency 77
7.2.2 Performance Gains using Gulp 77
7.2.3 Page Speed Comparisons 79
7.2.4 Disadvantages of Gulp 79
8. Discussion and Conclusion 81
​ ​
8.1 Future Work 82
​
Appendix A 86

6
Chapter 1

Introduction

1.1 Motive for the Research
The client-side applications using JavaScript are becoming increasingly complex.
One factor contributing to this fact is that there are thousands of libraries available
through open-source contributions that help in solving a variety of issues and few
even introducing new ones. These libraries depend on other libraries, and in no
time a web application might become an explosive cocktail that is impossible to
contain. In recent years, Node.js has enabled JavaScript to be used as a
server-side language as well, and some libraries like Browserify1 propose to write
even the front-end applications with server-side Node.js logic. Furthermore, there
are different families of open-source libraries and tools that claim to solve various
problems in various stages of software development cycle. One such such family is
of JavaScript Task Runners. This family consists of several libraries to choose
from, like GRUNT 2, Gulp 3, Cake 4 and Broccoli.js 5. They all claim superiority over
others in certain aspects, however the leaders in this race are Grunt and Gulp.
These task runners aim to automate various manual tasks like compiling scripts,
minification, creating build pipelines, testing and several more and help tackle
various issues surrounding JavaScript. Also, a different set of files needs to served
for development and production purposes. They claim to handle all theses tasks
seamlessly and ease the software development cycle, but the question is, is it
really required?

In addition, JavaScript front-end MVCs (Model-View-Controller) is another family of
open-source libraries and tools. They aim to address the logical complexities of the
Web applications. MVCs is a very powerful software architecture concept and have
1 "Browserify." 2013. 8 Sep. 2016 <http://browserify.org/>
2 "Grunt: The JavaScript Task Runner." 2012. 8 Sep. 2016 <http://gruntjs.com/>
3 "gulp.js - the streaming build system." 2013. 8 Sep. 2016 <http://gulpjs.com/>
4 "cake.coffee - CoffeeScript." 2010. 8 Sep. 2016
<http://coffeescript.org/documentation/docs/cake.html>
5 "Broccoli.js - The asset pipeline for ambitious applications." 2014. 8 Sep. 2016
<http://broccolijs.com/>
7
effectively helped solving a variety of issues on server-side programming
languages. Some of the most powerful ones being Spring6 for Java, Django7 for
Python and Express.js8 for Node. In the past few years, increasing amounts of
application logic is being implemented on the client-side rather than on the
server-side9. This has led to the development of many front-end JavaScript based
MVC frameworks like Backbone.js, Knockout.js, Ember.js, Angular.js and React.js.
They too claim to be superior over others in some or the other aspect, and there
have been several studies comparing the benefits and downsides of using these
frameworks. However, I would like to find out if a MVC can help in tackling the
challenges of Data Visualization, that have become an important aspect for
front-end web applications. I planned to test this idea while I was working as an
intern at Poimapper (see chapter 2). I was given the responsibility to create
interactive Web Reports containing visualizations for different types of data. In the
past, I have created visualizations and dashboards (figure 1.1) using purely
JavaScript with D3.js- A JavaScript based visualization library. However, I soon
realised that handling the entire Visualization Pipeline (see section 3.3.3) with such
a complex set of data is extremely difficult if not impossible by using pure
JavaScript.

6 "22. Web MVC framework - Spring." 2013. 8 Sep. 2016
<http://docs.spring.io/spring/docs/current/spring-framework-reference/html/mvc.html>
7 "Django: The Web framework for perfectionists with deadlines." 2011. 8 Sep. 2016
<https://www.djangoproject.com/>
8 "Express - Node.js web application framework." 2015. 8 Sep. 2016 <https://expressjs.com/>
9 Runeberg, Joakim. "To-Do with JavaScript MV*: A study into the differences between
Backbone. js and AngularJS." (2013).
8
1.2 Research Objectives
The main purpose of creating Web Reports for Poimapper is to provide our users
with the answer of a simple question, “What is happening?”. The user can then
dive into more details by interacting with the charts. I knew from my past
experience that the process can become extremely complicated when the logic of
creating charts (Chart Logic) gets intertwined with the actual application logic.
Previously, I have ended in making several flavors of the same type of charts for
different types of interaction instead of reusing the code. It would be interesting to
test if an MV* framework can effectively handle the challenge of separation of
concerns within the Data Visualization Pipeline as described in section 3.3.3.

Furthermore, there are several important practices that also need to be followed to
develop and manage a web application. JavaScript task runners claim to be an
easy to use tool to conveniently implement these practices. Since Poimapper was
missing a task runner, I planned to experiment with Gulp and test if it really helps.

Therefore, to test these hypotheses, I would like to experiment with two most
important families of the JavaScript open-source libraries, namely, Task Runners
and MV* frameworks:

RQ1: Does a JavaScript task runner help improve overall software
efficiency and productivity?
RQ2: Can a MV* framework like Backbone.js help in separation of
concerns within the Data Visualization Pipeline?

1.3 Target Group
The target audiences of this thesis are developers and researchers already familiar
to some concepts of Data Visualization, Model-View-Controller frameworks and
JavaScript. However, I have provided sufficient background for novice JavaScript
developers interested in developing web applications and visualizations with
minimal boilerplate.

1.4 Poimapper
During my internship at Pajat Solutions, I was working on their core product,
Poimapper 10. Poimapper is a fairly sophisticated product, but to put it simply it is a
Mobile data collection platform to cater the needs of on-field teams such as
industrial inspection, facility management, market research, inspections, audit, and
sales reporting. It allows mobile users to collect, share, and visualize geotagged
data in real-time.
10 "Poimapper." 2010. 12 May. 2016 <http://www.poimapper.com/>
9