Overview

 What is big data infrastructure?

 Computers and computing approaches
 Networks and the Internet

Lecture 9: Big Data

Infrastructure I
HD0102 Making Sense of Big Data

What is Big Data Infrastructure? What is Big Data Infrastructure?

 Big data analytics requires information  Hardware, software, data management
technology support for technology, networking technology, and
 Capturing data people to support big data analytics
 Storing data  Computer hardware
 Managing data  Equipment for processing, data storage, input and
 Retrieving data output
 Processing/analyzing data  Computer software (or “programs”)
 Known as big data (information  Instructions for computer hardware to perform
technology) infrastructure automatically
What is Big Data Infrastructure? What is Big Data Infrastructure?
 Data management technology
 Stores and manages data
Network
 E.g. inventory, HR, customers, sales transactions
 Networking Data
Management
Hardware
 Hardware and software that provides data
connectivity User (People)

 People Software
 Employees, consultants and vendors for acquiring,
creating and/or maintaining hardware and
Supporting Services
software (People)

Consequences of Poor Infrastructure Consequences of Poor Infrastructure

 Slow hardware  Slow network
 Unable to analyze data efficiently  Time consuming to transfer or obtain data over
 Wrong analytics software the Internet
 Cannot analyze data to fulfil project requirements  Poorly trained people
 Incorrect data management technology  Cannot perform the required analysis of big data
 Cannot support users who analyze data
 Cannot handle volume and variety of big data
Consequences (Examples) Consequences (Examples)

Consequences (Examples)

Computers and Computing Approaches

Types of Computers Types of Computers
 What are computers?  Mobile devices
 Devices that can run software  Portable devices for communication and
 Major components computing (e.g. mobile phones, tablets)
 Central processing unit  Personal computers and laptops
 Memory  General purpose computers for individuals
 Storage  Workstations
 Input and output devices  Similar to PCs but with more powerful CPU and
other components
 For technical, maths or scientific applications

Types of Computers Wooclap Activity (Graded)

 Mainframes  Switch to your Wooclap Activities tab
 Large-capacity, high-performance computers  Answer two questions about infrastructure and data storage
for processing large amounts of data rapidly
 Designed to support multiple-users
 Supercomputer
 Highest end of computing spectrum
 High performance computer for tasks
requiring rapid and complex calculations with
thousands of variables, millions of
measurements
 E.g. scientific simulations, weather forecasting
Client/Server Computing Client/Server Computing
 The hotel concierge  Common approach for performing
computing tasks
 E.g. accessing Web sites, checking mail
 Splits computing between “clients” and “servers”
 Clients
 Computers used by users
 E.g. mobiles, PCs, workstations
 Also refers to software used by users!
 Requests servers to perform computing tasks
 Updates the user interface
 E.g. Web browser, Microsoft Outlook, etc.

Client/Server Computing Client/Server Computing

 Servers  Basic setup
 Computers that fulfil client requests
 E.g. mainframes, supercomputers, specially
customised workstations
 Also refer to software that runs on server
computers!
1. Requests 2. Data
 Processes requests and
 Performs computations The services
Client Server
 Communication is via a network (e.g. Internet) Internet
Client/Server Computing Client/Server Computing
 Example: Reading email  Advantages
 Computing resources can be shared
1. Requests  Anytime, anywhere access
email message
 Less expertise required of end-users
2. Sends
requested  Disadvantages
message
 Disruptions due to server or network failures
Microsoft Outlook  Complex hardware and software may be needed
(email client) The Internet
 IT expertise needed

Email server

Cloud Computing Cloud Computing

 Model of computing where organizations  Example: Amazon Web Services
and individuals obtain computing resources
over the Internet
 A form of “rented” client/server computing
 Types of computing resources offered
 Infrastructure (e.g. Amazon Web Services)
 Offers processing, storage, and other computing
resources – hardware
 Software (e.g. Gmail)
 Offers various types of software accessible using
Web browsers
Cloud Computing Cloud Computing
 Characteristics  Advantages
 Computing resources available on-demand  Ownership of complex IT infrastructure not
 Computing resources can be scaled as needed needed
 Anytime, anywhere access via the Internet  IT infrastructure accessible even with little IT
 Cost is typically based on usage expertise
 Pay only for usage
 Hardware, software, data maintained by provider
 Disadvantages
 Data and services under control of provider
 Security and privacy risks
 High costs if usage is unexpectedly high

Wooclap Activity (Graded)

 Switch to your Wooclap Activities tab
 Answer the question on WhatsApp

Networks and the Internet

Computer Networks Computer Networks
 What is a network?  Major parts of a basic network
 Consists of two or more connected computing  Clients and servers
devices  Computing devices to be connected
 PCs, mobile phones, mainframes, printers, etc.  Network operating system
 Uses  Software that manages communications on the
 Exchange of data between devices network
 Communication between users  Network interfaces
 Sharing of hardware resources  Component that connects a device to the network
 Sharing of files, information, and other data

Computer Networks Computer Networks

 Major parts of a basic network  Local Area Network (LAN)
 Connection medium  Common type of network
 Means with which data is transmitted in the  Computing devices are connected within a small area (~100m)
network
 Wired or wireless
 Hub/switch
 Device that sends data to computing devices within
the same network using the connection medium
 Router
 Device used to send data to computing devices
across different networks
The Internet The Internet
Network B
 A global collection of interconnected Network A

networks Router
 “Network of networks” Router

 Characteristics
 Each network comprises computing devices
 Networks connected by routers
 Not all networks are directly connected
 Routers find a suitable route across networks for
devices to communicate
 All devices “speak” a common language known as
Networks…
TCP/IP Network D Network C

The Internet The World Wide Web

 Internet-based Applications  Set of documents (Web pages) connected
 Primarily client/server based together by references called (hyper)links
 E.g. E-mail, Web, instant messaging, streaming video  Web pages
 Each contains a collection of multimedia content
(e.g. text, images, audio, video)
 Stored on Web servers (servers)
 Accessed by Web browsers (clients)
1. Requests 2. Data and
Client services  Rich source of data used in analytics
The Internet Server
• Web browser, other apps
• Web, email, others
The World Wide Web Wooclap Activity (Graded)
 Switch to your Wooclap Activities tab
The Internet  Answer the question on what is true about the Internet

1. Request
(Send me web page named xxxxx)

Web browser Web server

2. Response
(OK, here it is…….)

The Internet of Things The Internet of Things

 The Internet – connects people (via
devices)
 Internet of Things (IoT)
 Connects physical devices to Internet
 Purposes
 Collect data about the state of the device and/or
its surroundings
 Exchange data with other devices and applications
 Immense source of big data
The Internet of Things The Internet of Things
 Data collection accomplished through  Example: Water-level sensors in Singapore
sensors  Can be found along drains and canals
 Part of the device or the entire device that  Determine water-level
can detect changes in the environment  Alerts and other actions taken
 Examples
 Fitness trackers
 Smart appliances (e.g. fridges)
 Agriculture
 Security and surveillance

The Internet of Things The Internet of Things

 Yet another example: Smart homes  Benefits
 Appliances and other devices can be  Access to more data
automatically monitored and controlled remotely  Through sensors (potentially) far from you
via the Internet  Automation
 Automatically act on the IoT-sensed data
 Efficiency and convenience
 Through automation and access to data
 Personalization
 Devices learn about you and your surroundings to
offer personalized services
The Internet of Things The Internet of Things
 Problem – security and privacy  IoT technology components can be divided
into
 IoT devices
 Physical devices and sensors
 Cloud storage and computing
 Storing and sending IoT data
 Integrating and processing data (analytics)
 Applications
 Creating alerts, reports or other actions
Straits Times
12 Oct 2020

Wooclap Activity (Graded) Questions?

 Switch to your Wooclap Activities tab  Post on Wooclap during the lecture
 Answer the question on IoT devices  Post on the course discussion forum in
NTULearn after the lecture