Cut Video

Course 1: Exploring and Preparing your Data

with BigQuery

Module 1: Introduction

Lesson Title: Introduction

Format: Talking head with slides

Video Name: xxx

 2

From Data to Insights with

Google Cloud Platform

v1.0

© 2017 Google Inc. All rights reserved. Google

and the Google logo are trademarks of Google Inc.
All other company and product names may be
trademarks of the respective companies with
which they are associated.
© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 3

Facilities

Parking Facilities Food

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 4

Course etiquette

Please silence Recording Ask questions

your phone and this class interactively or
take calls outside is prohibited via chat (online)

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 5

Agenda

1- Introduction to Data on the Google Cloud Platform

2 - Big Data Tools

3 - Exploring your Data with SQL in BigQuery

4 - Pricing

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 6

Agenda

5 - Cleaning and Transforming Data

6 - Storing and Exporting Data

7 - Ingesting New Datasets

8 - Visualization Basics with Data Studio

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 7

Agenda

9 - Joining and Merging Datasets

10 - Advanced Clauses and Functions

11 - Schema Design and Nested Data Structures

12 - Advanced Visualization with Google Data Studio

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 8

Agenda

13 - Optimizing for Performance

14 - Advanced Insights with Cloud Datalab

15 - Data Access

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 9

Audience and Prerequisites

Target Audiences Prerequisites

1. Data Analysts, Business Analysts, 1. Basic Knowledge of SQL

Business Intelligence professionals

2. Data Engineers who will be

partnering with Data Analysts to
build scalable data solutions on
Google Cloud Platform

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 10

Introductions

Your instructor
• Organization
• Background
• Course goals

You
• Name
• Organization
• Job role
• Course goals

© 2017 Google Inc. All rights reserved. Google and the Google logo
are trademarks of Google Inc. All other company and product names may
be2017
© trademarks of rights
Google Inc. All the respective companies
reserved. Google with
and the Google which
logo they areofassociated.
are trademarks Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 11

Pay special attention to slides with key messages or pitfalls

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 12

Module 1 In this module we will:

• Highlight Analytics Challenges Faced by Data
Analysts
Introduction to Data on
• Compare Big Data On-Premise vs on the Cloud
the Google Cloud • Learn from Real-World Use Cases of
Platform Companies Transformed through Analytics on
the Cloud
• Navigate Google Cloud Platform Project Basics

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Data analysts face query, infrastructure, and storage challenges

“We’re a data department, not

“My queries are taking “We can only afford
an infrastructure department.
way too long to run to store a subset of
Maintaining and upgrading
and is stalling my the data our business
our own servers is
analysis.” generates”
unsustainable.”

“We don’t have a

“My on premise central data analytics
“I have no easy way to clusters aren’t scaling warehouse or set of
combine and query all with my analysis” tools”
the data I’ve collected”

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 14

Module 1 In this module we will:

• Highlight Analytics Challenges Faced by Data
Analysts
Introduction to Data on
• Compare Big Data On-Premise vs
the Google Cloud on the Cloud
Platform • Learn from Real-World Use Cases of
Companies Transformed through Analytics on
the Cloud
• Navigate Google Cloud Platform Project Basics

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 15

Reasons why Google Cloud Platform

is used for Data Analysis

● Storage is Cheap

● Focus on Queries,
not Infrastructure

● Massive Scalability

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 16

The cost of 1GB of storage has dropped dramatically

Cost of 1 GB from 1980 to 2017 drops exponentially
Cost per 1 GB

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 17

Traditional big data platforms require an investment in

infrastructure

1 2 3 4

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Monitoring Insights

Time to Understanding Performance Resource

tuning provisioning

Typical Big Data

Processing
Utilization Handling
improvements growing scale

Deployment &
Reliability
configuration

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Insights

Time to Understanding

Big Data with Google: Writing code

Focus on insights,
not infrastructure.

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Proprietary + Confidential

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
Training and Certification 20
 21

“[Google's] ability to build, organize, and

operate a huge network of servers and
fiber-optic cables with an efficiency and
speed that rocks physics on its heels.
This is what makes Google Google: its
physical network, its thousands of fiber
miles, and those many thousands of
servers that, in aggregate, add up to the
mother of all clouds.”
- Wired

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 22

Google Cloud Platform opens Google-scale big data analysis

Global data
1
centers

2 Global
network

Edge locations
3 in 30+
countries

Software-
4
defined
networking
(why this matters)

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 23

Before After
Assembly required True On-Demand Cloud

On Premise In the Cloud

You Manage An actual, global
Hardware elastic cloud
Your kit, someone Invest your time in query
else’s building. writing, not infrastructure
Yours to manage.

Ad Hoc Querying
and Scalable Storage
Storage Processing Memory Network

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 24

Google Cloud Platform enables on-demand scalability

On-Premise Google Cloud Platform

Underprovisioned
(demand > capacity)
Query Processing Time = 1 Cloud Virtual Machine

(120s, 1 Machine)

Overprovisioned
(demand < capacity)

Query Processing Time

(10s, 120 Machines) Query Processing Time

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 25

Separation of storage and computing power enables efficient

resource allocation
On-Premise Google Cloud Platform

Pay for ability to use processing power Pay for only the resources you
even when no queries running are using and no more
© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 26

BigQuery scales automatically and you only pay for what you use

Fully-Managed
Infrastructure Scales
to Process Faster

Query Processing Time

.. and you only pay for

bytes processed +
$ storage
© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 27

Module 1 In this module we will:

• Highlight Analytics Challenges Faced by Data
Analysts
Introduction to Data on
• Compare Big Data On-Premise vs on the Cloud
the Google Cloud • Learn from Real-World Use Cases of
Platform Companies Transformed through Analytics on
the Cloud
• Navigate Google Cloud Platform Project Basics

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 28

Store Petabytes of Data

“[Our mission is] to make our data so
intelligent it has the answer before the
question is even asked. It was a stretch
goal but essentially one that means we
have to capture all the data we produce
- both now and in the future.”
Dan Nelson - Head of Data
Ocado

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 29

Focus on your Business, not Hardware

“The less time that we can spend

solving problems that are already
solved, like scaling,... the more time and
energy we can spend on turning our
data into value”
Nicholas Harteau - VP Infrastructure
Spotify

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 30

Module 1 In this module we will:

• Highlight Analytics Challenges Faced by Data
Analysts
Introduction to Data on
• Compare Big Data On-Premise vs on the Cloud
the Google Cloud • Learn from Real-World Use Cases of
Platform Companies Transformed through Analytics on
the Cloud
• Navigate Google Cloud Platform
Project Basics

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 31

Navigate the Google Cloud Platform using the dashboard

1. Projects

2. Resources

3. Billing

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 32

1. Projects organize and govern your activities in the cloud

● Navigate and launch cloud tools for

your project by exploring the Products
and Services menu

● Work collaboratively by adding project

users through IAM (Identity and
Access Management)

● Authorize Tools and Apps through the

API manager

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 33

2. Resources are what you are using in the cloud

Commonly used by data analysts:

● Storage in Google Cloud Storage

○ Example: You use a Bucket for
uploading large CSV files to ingest
later for analysis

● Datasets in Google BigQuery

○ Example: You perform analysis on
raw data and create a brand new
dataset

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 34

2. The Cloud Storage Bucket is your goto for scalable storage

● Buckets are scalable containers that

hold your data.

● You can create and upload files to your

buckets within your Cloud Console

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 35

3. You are billed for the resources you use

Commonly used by data analysts:

● Storage in Google Cloud Storage

○ Billed for Bucket Storage

● Datasets in Google BigQuery

○ Billed for Query processing
○ Billed for Table Storage

After this course, try exporting BigQuery logs using this

tutorial to recreate the above Data Studio billing dashboard

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 36

Summary: GCP Scale

Module Summary: offers you
with thethe ability
Google Cloudto:
Platform

Overcome query Efficiently scale Manage and Evangelize data

speed, your compute monitor your analysis in your
infrastructure, and storage project resources organization
and cost needs in one place
challenges

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 37

Lab 0
Getting Started with
Google Cloud Platform and Qwiklabs

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Getting started with Google Cloud Platform and Qwiklabs

● Open an incognito window

● Navigate to:
googlecloud.qwiklabs.com

● Create a new account with the

email address you used when
you registered for this course

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 39

What you get

For each lab, Qwiklabs offers:

• A free set of resources for a fixed

amount of time

• A clean environment with

permissions

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 40

Qwiklabs sign-in process

From the incognito browser,
sign in to Qwiklabs

Open an incognito browser

Sign in to the GCP

console using the
provided credentials

Select the lab and

click Start Lab

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 41

Open Qwiklabs

Open an incognito window

1
(or private/anonymous window).

Go to the Qwiklabs URL your Username

2
instructor provides.
Password

Sign in and launch the course

3 (with credentials you used to register
for the course).

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 42

View your labs

Lab completed

Active lab

Not yet available

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 43

Select a lab

You cannot pause and restart

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 Run a lab

1. Click

2. Note the following

3. Click and sign in

4. Accept terms and note the project

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 45

End a lab

● When done, click to free your resources.

Some labs may require you to NOT end the lab; the instructor will
inform you.

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.
 46

Course materials: End of class

1 Click Materials on 2 Select the class from the

the top navigation bar. Course Materials list.

From Data to Insights on Google Cloud Platform

© 2017 Google Inc. All rights reserved. Google and the Google logo are trademarks of Google Inc. All other
company and product names may be trademarks of the respective companies with which they are associated.