The future of data analytics

Data Science and The Future
of Analytics
How Data Science and Big Data are Evolving
By Edward Chenard

A little history of the Data Ecosystems
2011
• Hadoop goes mainstream
• Big data teams start to form
2012
• Data Science starts to form around stats and coding
2014
• Data Science goes mainstream
• Spark starts to take off
2015
• Machine Learning and Deep Learning go mainstream
2016
• Data Strategy starts to take off
Despite all these advancements, failure and frustration still runs rampant.

How Our Tools and Play Get Expressed

Welcome to the Dark Ages of Knowledge
According to Gartner, 73% of executives believe big data/ data
science will revolutionize their business – yet only 8% describe
their own big data/ data science projects as “successful.”
Believed common reasons for failed data science efforts are
more obvious and include:
• Cost: including tools, skills, infrastructure
• Dependence on legacy systems
• Siloed organizational information
• Lack of strong executive sponsorship
• Absence of clear business case
At the end of the day, data science has failed to live up to the promise. It never can deliver on that promise on its own.

How do we Understand The New Changes?
• Data Philosophy
• Data Concepts
• Team Dynamics
• Putting it All Together
• What’s Next

Data Philosophy, The foundation to Analytics
The real way to get success from data science is to help solve its short coming by applying other disciplines to problem.
Data
Philosophy
Identity
Expert
Intuition
Networks
(People)
Aspect-to-
Aspect
Transitions
Atemporality
Heterotopias
Connections and linked data, we are getting really good
at, but we are failing at understanding the meaning in
the data. It causes us to get good at knowing what is
wrong not at knowing what to do about it.
History is often called a study of humanity. Where as data
is a study of human activity

Expertise Governance (Expert Intuition)
We often don’t quantify our technology use. We just assume new is good.
There is little evidence that digital accounts for most our changes in business.
Regularity improves intuition intelligence. The rules of the
environment provide feedback that allows us to gain expert
level intuition with enough stimuli. Stock brokers don’t have
intuition due to the chaotic nature of the market. Short
success is attainable but never proven to work long term.
Intuition without expertise often come with the same level of
confidence as expert based intuition, but are often wrong.
Being a good data scientist is a lot like learning to be a grand chess master, it takes a lot of time and a lot of
learning of regularities to develop intuition.

Frequency Illusion
Leads of to believe a greater synchronicity than there actually is.
Our brains are patterns recognizing super
heroes. Combined with the recency effect
and confirmation bias, we are often fooled
into thinking something is important when in
fact it is not.
Expert Intuition is the only current method
to counter this illusion

Networks
The network does not respect history.
Extremely fluid, often poorly organized but seemingly persuasive even when wrong.
The networks we have today currently lack the ability to create a master narrative (maybe we will in a decade) but that
form is still yet undefined
Networks are human groups of knowledge all
Sharing
Networks have replaced traditional knowledge
sharing methods
We still lack a strong knowledge of how networks
influence our ability to solve problems

Diderot Effect
We often fall for this effect when it comes to data science, when a company starts on the journey and it catches on, all
of a sudden nothing from the past is good enough
Identity is believed to be uniformed and this
drives the Diderot Effect, which is often in
play in DS, the desire to out class takes over
decision making

Atemporality
One of the key characteristics of our time is the inability to define itself with a key set of intellectual ideas
Atemporality is an unmooring from historical methods and a transition period to find a new normal. We are in
such a time period now.
Increased chaos and an over abundance of information and view points have made it difficult for any area to truly stand the
test of contemporary ideas.
Atemporality allows for a blending of the past and potential future.
A clear vision is near impossible during Atemporality.
Too often teams are choosing between past and
present methods when in fact it is really about
blending to create something new entirely.

Aspect to Aspect Transition
Space is more important than action and time. Our culture is very goal oriented but aspect is often more important than
action when it comes to analysis work.
Space is often over looked as a component of data science. The environment often influences our decision making.
Our space is not created by the user but by networks or external players, our work is always limited by the way our space
is produced.
Being there over getting there.
Abadon time for the exploration
of space

Heterotopias
Places that exist in a dynamic space of layers and meaning. Margin spaces to explore non-standard methods
- Norms are suspended
- Precise and determined function
- Always have a system of opening an closing, not always open to everyone
Heteroptopias are spaces that are required for data
science, it allows for the different view points and
methods of exploration to take place. Often a singular
view of the world is required by leadership, this
creates a lot of failure because discovery is about
finding the new, not repeating the known.

Identity
All things change in an dynamic environment, including self.
Increased chaos and an over abundance of information and view points have made it difficult for any discussion
Identity and space are now merging. The data scientist
is often defined by the space he or she works in.
Spaces like identity are constructed. Our abilities are
limited by the space in which it is produced.
How to define identity was as is still an important
question for any data science team. You can’t define
based off of old terms like stats and coding, but a new
definition is still being defined.
Heterotopias allow for the exploration of space and
identity to be refined.

Wisdom
Collective application
of knowledge into
action
Knowledge
Experience, values, context
applied to a message
Information
A message meant to change receiver’s
perception
Data
Discrete, objective facts about an event
Experience
Grounded Truth
Complexity
Judgement
Heuristics
Values & Beliefs
Quantitative
Contextual
Evaluative
Qualitative
Intuitive
Informative
Quantitative
Connectivity
Transactions
Informative
Usefulness
Quantitative
Cost, Speed
Capacity
Timeliness
Relevance,
Clarity
Adding Value:
Action-oriented
Measurable efficiency
Wiser decisions
Adding Value:
Contextualized
Categorized
Calculated
Corrected
Condensed
Adding Value:
Comparison
Consequence
Connections
Conversations
Transitioning to emerging technologies
+
Human/Machine
=
Transformation
Establish a culture that allows the team to drive from data to wisdom. A combination of both machine and
human wisdom is needed to out perform competitors

Data ConceptsThe Strange Ways Data Behaves

From Concepts to Practical
Governance
Management
RunBuildPlan
MonitorDirect
Evaluate
Business Needs /
Strategy
Monitor
Data ScienceEngineeringDesign
IT
Architecture
Development

Elements of our Various Selves
The Basic Selves
Each person brings several aspects to the table of any team, of themselves. An understanding of the
various selves helps leaders understand how to engage teams
Our Various Selves that Playout
• Combine the strengths of Google
and Facebooks methods with
psychograph techniques.
• Listen, Adapt, Respond
• Services co-created with customers
and are interpedently with wider
service networks.
Psychograph
Self
Facebook Self
Google Self
Clash between
Today and Future
Aspirational
You
Present You
1-1

Various Aspects of you
Google Edward
(Public online self)

Facebook Edward
(Aspirational Self)

Psychograph Edward (Offline Self -
External)

Psychograph Edward (Offline Self - Internal)

The Distortion Problem
• The virtual equivalent of smoking addiction
• Technology and group think can create a dependency that distorts one’s
world view and actually encourages the dependency with false facts

Filter Bubbles
• A filter bubble is the restriction of a user’s
perspective that can be created by
personalized search technologies. (Haughn
2015)
• Information pluralism in the media refers to
the fair and diverse representation of and
expression by various political and ideological
groups, including minorities, in the media.
(Leuven et al. 2009, p. 12)

Open Vs Closed Systems
Most natural systems are open systems. An open system is a system that
exchanges information with its environment.
Most processes that are customer facing are closed systems, with limited
exchanges.
A resilient team ecosystem by it’s nature needs to be an open system, sharing
information with customers or any data that a customer/team member wishes
to bring into the system.
Team Ecosystems needs to not only be adaptive, but are often complex
systems that are open. These are the models that survive, the more closed the
model, the less use it will have by customers. (Think city vs corp systems)
Three criteria of a good system: Distributed control, strong identity, resilient
(not robust).

The Uncanny Valley
• Creepiness Factor: This term is often used in personalization to talk
about how unsettling an experience is to the user. If it creeps a
person out, it tends to have a high creepiness factor.
• Why does it exist, speculation is that AI that replicates us is seen as a
threat to our own individual uniqueness. The less human like, the
more accepting we tend to be of AI.

What is Personalization
how to disrupt the market – Persuasion Profiling
Persuasion Profiling
Persuasion profiling: Suggests that the kind of arguments you respond to are highly transferrable from one
product category to another.
i.e.: If you like discounts you will respond well to them for shoes or TV’s. If you want the most
popular and trendy product, you will want them for clothes and tablets.
Combining persuasion profiling with new methods of sentiment analysis, it is now possible to guess
someone’s mood and target the right message to them based on their mood, time of day and the kind of
argument they best respond to.
Ex: People use substantially more positive words when they feel happy. Analysis of twitter or FB can show
this via sentiment analysis, based on the time of day, we can know what your typical aspirations are then
coupled with the argument you tend to like, we can send an email to you about a product that will feel just
right for you that is timely and relevant to your mood. Amazon can’t do this.
In research, this has seen a 30-40% increase in sales.

The Doppelganger Effect
Brand Preference

Second Order Simulacra
Distinctions between representation and reality break down due to the proliferation of mass-reproducible copies of items,
turning them into commodities. The commodity's ability to imitate reality threatens to replace the authority of the
original version, because the copy is just as "real" as its prototype.

What is Personalization
Day Parting
Breakfast
Lunch
Dinner
Choices are often affected by the time of day in which we make those choices

Walled Gardens
• Designed to lock a user into the ecosystem
of a specific company. Think Facebook or
Amazon or Google. Once the user is
locked it, psychological nudges are used to
keep them from going else where.
• Economic drive of walled gardens is often
to increase ad revenue.

The Red Queen Effect
Innovation Theatre
Robust vs resilient. Robust systems are efficient. Resilient system can handle many unexpected challenges and be affective

Relevance
• Two metrics can be defined for news stories:
• Importance: intrinsic “value” of a story with respect to society
• Relevance: probability that a story will be “liked” by the user;
performance index of the recommender system
• Recommender systems (personalizing filters) are relevance maximizers
Example
• “A squirrel dying in front of your house may be more relevant to your
interests right now than people dying in Africa.”
– Mark Zuckerberg (Facebook CEO)

Team or Group?
• Group example: yoga class is a “group”?
• Groups of people who play hockey as a “team”?
• A collection of people are not necessarily a group and a group is not
necessarily a team

How a group becomes a team
• Evolutionary process
• Teams are constantly changing and developing
• Groups go through four stages of development (Tuckman, 1965)
• Vary in duration and sequence for different groups

Cohesion
A team is not the sum of its parts. Trust and purpose is what separates good teams
and great teams. Builds a special bond that gets things done even when the
environment is changing. The effectiveness of the team is all about the bonds the
team forms with one another, also known as cohesion.
In theory this is all nice and can be done, but do people have the will to have it be
done?
Getting a perfectly efficient system is elusive in the new environment. People often carry the baggage of doing
things right. If you follow the process often you won’t be criticized even if you fail. But the current processes will
give you failure, which is unsustainable.

Social Loafing
• Ringelmann (1913,
1927) observed that men
pulling on a rope
attached to a
dynamometer exerted
less force in proportion
to the number of people
in the group:
The Ringelmann effect
0
10
20
30
40
50
60
70
1 2 3 4 5 6 7 8
Group size (persons)Forceperperson(kg)
Expected performance
Actual performance
BASIC PRINCIPLE
The larger the number of individuals whose work is combined on a group task, the smaller is each individual’s
contribution.

Winning or Not Losing
Are you Interested in Winning or Not Losing?
Teams fail when there is not good alignment. Not losing is the not the same
as winning.
Not losing personalities can be toxic in a team environment.
Not losing is often shown when we overvalue what we have
(the endowment effect).
The positivity ratio is the # of positive statements to the # of negative
statements. High positive ratios help teams focus on winning, not on not
losing
High performing teams average 6:1; low performing teams average 1:1

Data Ethics – Ethical Analytics
▶With more granular insights comes greater
responsibility
▶Just because you can, doesn’t mean you
should
▶A culture of “ethical” analytics
Data science is a field filled with legal and compliance pitfalls. A culture of ethical analytics must be
instilled in the team to ensure we don’t run foul of any legal, ethical or social norms with our data
collecting or insights.

A Hybrid Structure
Hybrid Team
Model
Business
Data
Science
Developers
Designers
Social
Science
Engineers
Increase adaptability
Most teams are still operating like the 20th century
For now

Hub and Spoke Embedded Model
aka Dandelion Model
• Pod teams are embedded in various delivery and business teams to work on projects for a portion
of their time. When not working those teams, they are in a hub and spoke model, in a more
centralized area working as a larger group to work on enterprise wide problems.
• This model is becoming the most common approach to data science teams.
• Decentralization means let people be doers and thinkers, not just push responsibility down the
down the chain.
• Team players need to be more intrapreneurs than just an employee

Core Teams / DS Pods
•Coding
•Mock ups
•Wire frames
•Cluster
Management
•Distributed Tech
Management
•ETL
•Data Application
Management
•Model
•Math
•Algorithms
Data Science Data Engineer
Developer
Infrastructure
Engineer
Each team is assigned to work with a specific
part of the business. There is 1 primary
point of contact for each area (i.e. there is a
primary data scientist working on Inventory
projects)
The team can call on others with specialized
skills to help them work on projects as
needed.
As work grows in an areas, more pods are
spun up to manage the work. These pods
are typically an additional data scientists and
developer with the engineers helping all
pods related to that part of the business.

Big Data Science
Group
Structure IT Business
Strategy
DS
Big
Data
DS DS
Data Science and Big Data Become
Two Disciplines
Strategy manages the overlap and
compliance areas along with
roadmaps and schedules.
Big Data Science Group a combo of all
three parts drives innovation

Maturing of Data Science and Big Data
Present Needs
Future Value
Alignment
As these disciplines grow from our current start up
phase, they will need to change how they are
structured and managed in order to meeting their
ability to create value in the near future.
Data Science is not IT or Business but a hybrid role
and a way of running and building these disciplines
needs to also address this. Making Services and
Solutions that address the needs of customers to
apply big data and data science in a more tangible
form is what will get the most future value.
A future value focus: Technological revolutions tend
to involve some important activity becoming cheaper,
like the cost of communication or finding information.
Machine intelligence is, in its essence, a prediction
technology, so the economic shift will center around
a drop in the cost of prediction. Our future value
must focus on machine intelligence i.e. machine
learning

New team models are supported by:
Decentralization, networked self-
organization
Open Collaboration, hyper-
competition, crowd
input/delegation, resource
sharing
DIY, maker culture, passion driven
Radical transparency,
transdisciplinary, community-
directed, network oversight
Responding to opportunity,
value-centric, let network do the
work
Examples:
Stakeholder councils
Hackathons
Open APIs
Open Source
Ambassador programs
Tech incubators
P&L focus
Design thinking
Communication means providing wisdom

Skills Sharing and Building
Outside
Experts
Teach Outs
Lunch and
Learn
Code Reviews
Not all data scientists are alike. To help ensure we have well
rounded team players in the BDSG, we need to have various
opportunities for members to share out what they are working on
so that other members of the group can learn.
Code Reviews: Basic way to help everyone understand the work in
progress.
Lunch and Learn: Informal teach outs such as a brain dump if
someone attended a conference.
Teach Outs: Help others learn new skills such as doing a session on
how to use Pig or Cassandra.
Outside Experts: Bring in people to share their experience to the
team to get an outside perspective on how to do work.

Communication Process
There are various levels of communication processes which are
managed by the strategy arm.
These include:
- Team Level Communications
- Group Level Communications
- Corporate Communications

http://upload.wikimedia.org/wikipedia/commons/thumb/6/66/Einstein_1921_by_F_Schmutzer.jpg

Design of Experiments (DoE) Otherwise known as
Lean Innovation
Learn
Compare
CompleteShare
Frame
Empathize
Hypothesize
Dollarize
Document
Build
Design
ImplementDeploy
Measure
Collect
AnalyzeOrganize

Rumsfeld Analytics
Things We
Don’t
know
Facts – could be wrong
We don’t
know
Intuition – quantify to
improve
Know
We don’t
know
Questions – do reporting
Exploration – unfair advantages
We know
We know
Innovation should be
focused on exploring the
unknown unknowns to give
us an unfair advantage

Give the team a purpose
Break Even
Break Through
Break Away
Efficient or Effective, you can’t be effective if you are more
interested in the efficiency of the process than on the goals of
why the process is there in the first place. To be effective,
adjust processes to adapt to the rapid changes

Transformation Strategy
Focus on
Productivity
Focus on Customers
Enhancement
Focus on a
Platform
HowtoPlay
HowtoWin
Capabilities and
Operating Model
Innovation Business
Model
Talent and Culture
Partner Ecosystem
Model
Data and Connected
Infrastructure
Change the Game Harness the Platform Go Together, Go Far Building Data,
Insights, Action
into our DNA
All About Outcomes

Creating an Insight and Action Driven Team
Invest in the Foundations
Culture: Create a culture which expects decisions are informed by data and experience.
e.g. Determining strategy, goal setting, impact estimates of initiatives
Process: Consciously map how you use data and arrive at insights and actions
e.g. product strategy reviews, design discussions, testing and documentation but not
just templating someone’s methods
Tools: Invest in the data ecosystem
e.g. Specialized skills, data quality, pipeline, access tools.
Culture
Process
Tools

What Problems are We Solving
Quality
• Consistent and Repeatable
Organization
• Keeping track of artifacts in a distributed environment
Collaboration
• Across teams and resources
Knowledge Accumulation
• Effective sharing, preventing reinvention of wheels
Agility
• Get going fast, execute efficiently
Globally
Focused
Diverse Skill
Sets
Varied
Clients,
internal and
external

Brainswarming
Solutions
DS: Stats
DS:
Machine
Learning
DS: NLP
Big Data
When a Core team runs into a problem they have difficulty
solving, all teams come together in a brainswarm model to help
come up with a solution. Because data scientists are not all the
same coming with different skills, applying skills from other
teams can help the core team come up with a solution faster
than on their own. Brainswarms can be called by any team and
it is the Big Data Science Group leader to come form the swarm.

Data Science and Big Data as a Product
Offering
Creating a portfolio of products/services to engage customers with Data Science and Big Data
i.e.
Moving our competitive position from the past, to the future

Services
Data Science Lab
Consulting Services
Deployment Services
Data Science as a Product Offering
Currently we have data science as a few people working to support projects.
However, our ultimate goal should be a way to use data science as a
product/service offering to internal and external customers alike. This
approach helps to market what we do as not just another team, but a real
value add to the organization or customers.
It creates structure that we can then capitalize upon to bring in new streams
of revenue to the organization by a truly new service offering that allows us to
engage our customers in new ways that we have not before.
Done correctly, this can add an additional $100M in 3-4 years to top line
growth with current staffing levels. Higher levels can be obtained if staffed
and managed appropriately.

Signals Hubs Are Emerging As A Core Offering
•Signals Hubs are a new and critically important capability for enterprises – allowing them a never-before-available capability to rapidly maximize
value from big data flows
Enterprise Operations
Enterprise
Data
Warehouse
Hadoop
Internal /
External Data
Enterprise
Signals Hub
Specific Applications
Only data with predictive value
Applications
operationalize
signals and bring
them to the front
lines to drive
productivity and
profit
CONTINUAL
SIGNAL
EXTRACTION
Signals Hubs
continually
extract relevant
data from inside
and outside the
enterprise… and
turn this data into
predictive signals

Signal Types
Signals have attributes depending on their representation in time or frequency domain can
also be categorized into multiple classes
All signal types have certain qualities that describe how quickly signals can be generated
(frequency), how often the signals vary (rate of change), whether they are forward
looking (quality), and how responsive they are to stimulus (sensitivity)
Rate of Change
(Slow or Fast)
Quality
(Predictive or Descriptive)
Sensitivity
(Sensitive or Insensitive)
Frequency
(High or Low)
Sentiment
Expressed as
positive, neutral,
or negative, the
prevailing
attitude towards
and entity
Behavior
These signals
identify
persistent
trends or
patterns in
behavior over
time
Event/Alert
A discrete signal
generated when
certain
threshold
conditions are
met
Clusters
Signals based on
an entity’s
cohort
characteristics
Correlation
Measures the
correlation of
entities against
their prescribed
attributes over
time

Going Forward, The Human Frontier
• The most important frontier today is our understanding of being
human
• Neuroscience
• Positive Psychology
• Behavioral Economics

Psychology & Social Science
ResponseStimulus
Brain
Neuroscience

Your offer
+
neuro-nudge(s)
=
Personalization
Neuro-Nudges

New Model for Success
Data
Strategy
Engineering
Knowledge
Analytics
Knowledge
Psychology
Knowledge
Business
Expert
Legal
Knowledge
UX
Knowledge

The future of data analytics

Recommended

More Related Content

What's hot (20)

Similar to The future of data analytics (20)

More from Edward Chenard (10)

Recently uploaded (20)

The future of data analytics