Big Data Analytics Made Easy

170 Pages·2016·10.79 MB·English

by  Y. Lakshmi Prasad

Notion Press
Old No. 38, New No. 6
McNichols Road, Chetpet
Chennai - 600 031
First Published by Notion Press 2016
Copyright © Y. Lakshmi Prasad 2016
All Rights Reserved.
ISBN 978-1-946390-72-1
This book has been published with all efforts taken to make the material error-free after the
consent of the authors. However, the authors and the publisher do not assume and hereby
disclaim any liability to any party for any loss, damage, or disruption caused by errors or
omissions, whether such errors or omissions result from negligence, accident, or any other
cause.
No part of this book may be used, reproduced in any manner whatsoever without written
permission from the authors, except in the case of brief quotations embodied in critical
articles and reviews.
This book is dedicated to
A.P.J. Abdul Kalam
(Thinking should become your capital asset, no matter whatever ups and downs you come
across in your life.)
To download the data files used in this book,
please use the below link:
www.praanalytix.com/Bigdata-Analytics-MadeEasy-Datafiles.rar
Contents
Preface
Author’s Note
Acknoweldgements
STEP 1 Introduction to Big Data Analytics
STEP 2 Getting started with R
STEP 3 Data Exploration
STEP 4 Data Preparation
STEP 5 Statistical Thinking
STEP 6 Introduction to Machine Learning
STEP 7 Dimensionality Reduction
STEP 8 Clustering
STEP 9 Market Basket Analysis
STEP 10 Kernel Density Estimation
STEP 11 Regression
STEP 12 Logistic Regression
STEP 13 Decision Trees
STEP 14 K-Nearest Neighbor Classification
STEP 15 Bayesian Classifiers
STEP 16 Neural Networks
STEP 17 Support Vector Machines
STEP 18 Ensemble Learning
Preface
This book is an indispensable guide focuses on Machine Learning and R Programming, in an
instructive and conversational tone which helps them who want to make their career in Big
Data Analytics/ Data Science and entry level Data Scientist for their day to day tasks with
practical examples, detailed description, Issues, Resolutions, key techniques and many more.
This book is like your personal trainer, explains the art of Big data Analytics/ Data Science
with R Programming in 18 steps which covers from Statistics, Unsupervised Learning,
Supervised Learning as well as Ensemble Learning. Many Machine Learning Concepts are
explained in an easy way so that you feel confident while using them in Programming. If you
are already working as a Data Analyst, still you need this book to sharpen your skills. This
book will be an asset to you and your career by making you a better Data Scientist.
Author’s Note
One interesting thing in Big Data Analytics, it is the career Option for people with various
study backgrounds. I have seen Data Analyst/Business Analyst/Data Scientists with different
qualifications like M.B.A, Statistics, M.C.A, M. Tech, M.sc Mathematics and many more. It is
wonderful to see people with different backgrounds working on the same project, but how can
we expect Machine Learning and Domain knowledge from a person with technical
qualification.
Every person might be strong in their own subject but Data Scientist needs to know more
than one subject (Programming, Machine Learning, Mathematics, Business Acumen and
Statistics). This might be the reason I thought it would be beneficial to have a resource that
brings together all these aspects in one volume so that it would help everybody who wants to
make Big Data Analytics/ Data Science as their career Option.
This book was written to assist learners in getting started, while at the same time providing
techniques that I have found to be useful to Entry level Data Analyst and R programmers. This
book is aimed more at the R programmer who is responsible for providing insights on both
structured and unstructured data.
This book assumes that the reader has no prior knowledge of Machine Learning and R
programming. Each one of us has our own style of approach to an issue; it is likely that
others will find alternate solutions for many of the issues discussed in this book. The sample
data that appears in a number of examples throughout this book was just an imaginary, any
resemblance was simply accidental.
This book was organized in 18 Steps from introduction to Ensemble Learning, which
offers the different thinking patterns in Data Scientist work environment. The solutions to
some of the questions are not written fully but only some steps of hints are mentioned. It is
just for the sake of recalling the memory involving important facts in common practice.
Y. Lakshmi Prasad
Acknoweldgements
A great deal of information was received from the numerous people who offered their time. I
would like to thank each and every person who helped me in creating this book.
I heartily express my gratitude to all of my peers, ISB colleagues, friends and students
whose sincere response geared up to meet the exigent way of expressing the contents. I am
very much grateful to our Press, editors and designers whose scrupulous assistance
completed this work to reach your hands.
Finally, I am personally indebted to my wonderful partner Prajwala, and my kid Prakhyath,
for their support, enthusiasm, and tolerance without which this book would have never been
completed.
Y. Lakshmi Prasad
STEP 1
Introduction to Big Data Analytics
1.1 WHAT BIG DATA?
Big Data is any voluminous amount of Structured, Semi-structured and Unstructured data
that has the potential to be mined for information where the Individual records stop mattering
and only aggregates matter. Data becomes Big data when it is difficult to process using
traditional techniques.
1.2 CHARACTERISTICS OF BIG DATA:
There are many characteristics of Big data. Let me discuss a few here.
1. Volume: Big data implies enormous volumes of data generated by Sensors, Machines
combined with internet explosion, social media, e-commerce, GPS devices etc.
2. Velocity: It implies to the rate at which the data is pouring in like Facebook users
generate 3 million likes per day and around 450 million of tweets are created per day by
users.
3. Variety: It implies to the type of formats and they can be classified into 3 types:
Structured – RDBMS like Oracle, MySQL, Legacy systems like Excel, Access
Semi- Structured – Emails, Tweets, Log files, User reviews
Un-Structured – Photos, Video, Audio files.
4. Veracity: It refers to the biases, noise, and abnormality in data. If we want meaningful
insight from this data we need to cleanse it initially.
5. Validity: It refers to appropriateness and precision of the data since the validity of the
data is very important to make decisions.
6. Volatility: It refers to how long the data is valid since the data which is valid right now
might not be valid just a few minutes or fewer days later.
1.3 WHY BIG DATA IMPORTANT?
The success of the organization not just lies in how good there are in doing their business but
also on how well they can analyze their data and derive insights about their company, their
competitors etc. Big data can help you in taking the right decision at right time.
Why not RDBMS? Scalability is the major problem in RDBMS, it is very difficult to
manage RDBMS when the requirements or the number of users change. One more problem
with RDBMS is that we need to decide the structure of the database at the start and making any
changes later might be a huge task. While dealing with Big data we need flexibility and
unfortunately, RDBMS cannot provide that.