Applied Analytics in an Organizational Context

Module 11

November 14, 2024

Applied Analytics Program

 In the News…
Supply chains are facing ongoing stress from geopolitical
tension, labor disruption and natural disasters.

“But in the past, we’ve got that balance wrong. Most

organizations have spent too much time driving costs
out of the supply chain and going for more and more
efficiency. The consequence being that in a world that is
more uncertain, you are less protected from
disruption.”

Businesses that have invested in supply chain resilience

saw revenue grow an average of 23% from 2018 to
2023, compared with 15% growth experienced by their
peers

Vendors are mining public import-export data, such as

shipping records, to try to better understand the inputs
of a given product down to the raw materials

https://www.wsj.com/articles/supply-chain-woes-carry-high-risks-big-rewards-for-some-companies-28229749?mod=hp_minor_pos10
Decision Intelligence
How Decision Intelligence Improves Business Outcomes

For more insights on decision intelligence, visit: https://www.gartner.com/en/conferences/hub/data-analytics-conferences

In a recent Gartner survey, 65% of respondents said the decisions they make are more complex than just two years ago, and 53% said they face more pressure to explain or justify their decisions. In this video, Gareth Herschel, VP Analyst at Gartner, discusses the role of analytics and psychology in smarter decision-making and helps organizations intentionally design the way they make decisions.

Subscribe: https://www.youtube.com/user/Gartnervideo/

LEARN MORE ABOUT GARTNER

Gartner delivers actionable, objective insight to executives and their teams. Our expert guidance and tools enable faster, smarter decisions and stronger performance on an organization’s mission-critical priorities.

Website: https://www.gartner.com/
LinkedIn: https://www.linkedin.com/company/gartner
Twitter: https://twitter.com/Gartner_inc
Instagram: https://www.instagram.com/gartner_inc/
Podcast: https://www.gartner.com/en/podcasts/thinkcast

https://www.youtube.com/embed/OFAxIPbUMyI?start=203&rel=0&end=290
Agenda and Learning Objectives

IT Analytics
Analytics in Legal Function

• Explain different roles of IT and analytics

• Explain how big data and advanced analytics are changing the practice of law
• Use structured decision analysis to understand risk and outcomes in the context of legal
decisions
IT Analytics

Collaboration between IT and Analytics teams

Agile Processes

Automation
• IT Service Management
Collaboration across roles

Analytic processes are highly iterative

Roles do not work sequentially – they must collaborate closely

Skills, processes and culture must support effective collaboration

 Roles within Analytics
Demystifying Data Science

Data Engineer: Business Analyst: Translator

• Architecting distributed systems • Business acumen
• Creating reliable pipelines • Can communicate well with both the
• Combining data sources data scientist and C-suite to help drive
• Architecting data stores data-driven decisions faster.
• Collaborating with data science teams • Typically work across sales and
and building the right solutions for them marketing teams to make data-driven
decisions.

Data Scientist:
• Largely rooted in statistics, data modeling, analytics and
algorithms.
• Data mining - one of the most in-demand skills on LinkedIn
• Extracting value from data, using techniques like pattern
recognition, algorithm design and clustering, to better predict
behavior
http://blog.hackerrank.com/the-biggest-misconception-about-data-scientists/
Clarity of End-to-End Process

• Analytic process typically involves detailed sub-steps driven by specific team

members

• If knowledge is not shared with others, high risk of process disruption if

team member leaves

• “Best practice” considered to be documentation

However: documentation alone does not guarantee smooth

transitions or reproducibility
Analytic Workflows
From pages of code ….. ….. to modular workflows

• Bespoke (custom-developed) • Standardized

• Intensive manual effort • Rapidly developed
• Difficult to adapt • Reusable
• Error-prone • Robust
Examples of Platforms for Creating Reproducible
Workflows

Enable live connections to source databases

Embed and integrate different tools and Image result for microsoft azure

programming languages

Enable process automation Image result for dataiku

 Alteryx
Alteryx – End to end

Automate every step of analytics, including data prep, blending, reporting, predictive analytics, and data science.
With augmented machine learning, you can quickly build predictive models without coding or performing complex statistics.
Remove the barriers to organization-wide data analysis and insights by converging the capabilities of multiple tools into one platform.
Prep, blend, analyze, and interpret data faster with hundreds of no-code, low-code analytic building blocks.

https://www.youtube.com/watch?v=yCKef0Z1Dlc
Dataiku
Dataiku 3-Minute Demo

Meet Dataiku, the universal AI platform.

To explore more about Dataiku, check out the rest of our content:
CHECK OUT DATAIKU: https://bit.ly/36XBlpK
AIUs: https://www.dataiku.com/stories/ai-and-us/
ROAI: https://www.dataiku.com/return-on-ai/
Linkedin: https://bit.ly/3lXyRMb
Twitter: @dataiku
Instagram: @dataiku

https://www.youtube.com/watch?v=0GXvaHg5s-w
Automation
 Automation and the Future of Work

• Less than 5 percent of occupations can be fully automated using current

technology
• 60 percent of all occupations have at least 30 percent of activities that
are technically automatable based on current technologies
• Current automation technologies could affect 50 percent of the world
economy, or 1.2 billion employees and $14.6 trillion in wages.
• China, India, Japan, and the United States account for ~ half

• Growing role of big data in the economy and business will create a significant
need for statisticians and data analysts
• Estimate a shortfall of up to 250,000 data scientists in the United States in a decade

http://www.mckinsey.com/global-themes/employment-and-growth/technology-jobs-and-the-future-of-work#section 2
 Even in industries that are less
susceptible to automation, the
repeatable tasks associated
with data collection and
process are highly automatable.

https://www.mckinsey.com/business-functions/operations/our-insights/making-healthcare-more-affordable-through-scalable-automation
 Robotic Process Automation (RPA)

RPA is software designed to be used by business experts to

automate manual, time-consuming, rules-based tasks

PwC estimates that 45% of work activities can be automated

with a global workforce cost reduction of $2 trillion

Typical ROIs are >30% during the first year of RPA

implementation

https://www.pwc.lu/en/rpa/docs/robotics-process-automation.pdf
Potential RPA Usage
Data entry with input from Data entry within same system
multiple systems • Complete data entry with
• Access multiple programs (e.g. navigation through a series of
Excel and Oracle) screens
Excel • Find record information from
Excel based on a set of criteria
• Copy and paste information
into Oracle
Oracle Oracle

PDF
Email notification Data validation between
• Send email notification with multiple systems / OCR
the proper attachment after • Identify fields in multiple
an activity is completed based systems and conduct data
SFDC
on pre-defined rules and time validation
• Leverage Optical Character
Recognition (OCR) technology
Email notification to extract fields from PDF
XSM

Data manipulation / calculation Access databases via APIs

/ formatting • At pre-set time, the program
• Perform data clean up based Databases APIs is kicked off to generate
on pre-defined rules which reports based on pre-defined
Original include getting input from rules by accessing APIs
report
various systems
Set of rules • Calculate and format final
financial report
Final report
Report
RPA Value Drivers
Speed to Implement – RPA works through the UI layer and avoids a traditional process
integration (e.g., IT resource investment, business requirements documents, significant
development time)

The Key to Scale – a virtual workforce can respond to growth events (e.g., organic,
acquisitive) with speed, agility, and resiliency

Robots Never Sleep – many of today’s digitally-enabled processes can be orchestrated

to operate autonomously 24x7, driving real-time transactions

Value-Focused Talent – priorities of the employee workforce will shift to solving

complex challenges, innovation, strategy, and other business development activities

Improved Quality & Control – RPA reduces error, produces detailed activity logs, and
enables programmable controls
“Automation” Covers a Range of Capabilities
“Automation” Covers a Range of Capabilities
In-Class Discussion (Breakout)

You have been asked by a business client to develop a business case

for an investment in Robotic Process Automation (RPA).

How would you approach addressing the request?

How would you know if the investment realized the benefits of the
business case?
Future of Work
Future of work strategies and the benefits derived are a function of
costs, value, and meaning creation.

• Focus is on Company • Focus is on Customer, Workforce and Company

• Work of tomorrow • Expand notions of value beyond just cost
same as work of today • Explore new sources of value and meaning in order to remain
competitive amid rapidly changing market dynamics
Schwartz, J., et al. (2019) MIT Sloan Management Review 60, 1-6.
 Data-Driven Excellence and Empowered
Algorithms
Autonomous Advisor Autonomous Autonomous Full Autonomy
Outsourcer Employee
Algorithms review and make Company trains people to Full commitment to
Evolve from business process
recommendations hire and work with the best algorithmic autonomy
outsourcing to algorithmic
autonomy and brightest algorithms
Human oversight committee Human leadership defers to
approves what becomes Smart software is treated as algorithmic power
Process and decision owners
autonomous and how it is determine resource a valuable colleague that
implemented – Top Down comes up with the right Autonomy is the
allocation and whether
answer organizational and
autonomy should lead to
Data Scientists intermediate operational center of gravity
greater innovation,
the oversight committee Culture of cocreation and for innovation and growth
optimization or both
and the targets of collaboration is critical to
implementation Data Scientists are project success
managers and support the
process and decision owners Data Scientists facilitate
collaboration

Michael Schrage, HBR, January 27, 2017

New Jobs in an AI Era

Trainers Explainers Sustainers

Teach AI systems how they should Bridge the gap between Ensure that AI systems are
perform technologists and business leaders operating as designed and that
unintended consequences are
addressed with appropriate urgency

MIT Sloan Management Review, 2017

Analytics and IT Service Management

Minimize the impact of business downtime - Anticipate service outages by

monitoring metrics like frequency of downtime and mean time to repair
Optimize resource management - Develop optimal staffing model to suit volume of
customer demands and improve service desk efficiency.

Improve service quality - Correlate ticket resolution and re-opening rates to accurately
determine work efficiency.

Maximize ROI on software purchases - Track licenses (identifying over-licensed or under-

licensed software), predict service request trends and measure software utilization rates to
calculate unnecessary software expenditures.

Ensure high levels of end-user satisfaction - Maintain SLA levels. Set realistic SLA goals,
automate and route ticket assignments, communicate risks of SLA violations and set up
escalations proactively.
Automation and IT Service Management

Efficient handling of level 1 incidents

Auto-approvals and work flow

Predict issues and problems proactively

Classify and route issues with greater ease

Asset lifecycle management

https://www.manageengine.com/products/service-desk/itsm-help-desk-machine-learning.html
Legal Function
 General Counsel and Corporate Legal Function

Strategic business partner Chief Legal officer

Understand the business and the Focus on law-related matters

practical aspects of the client’s issues
• Spot problems Staff counsel’s work often
• Help design responsive and compartmentalized into discrete
creative solutions specialty areas of legal practice
• Provide legal perspective during
planning processes Functions like a law firm
• Educate on legal issues affecting
the business

Legal departments must balance the inherent tension

between these two visions
Civil Example – J&J

J&J facing more than 38,000 lawsuits involving body powders with talc

J&J was ordered to pay $4.69 billion in damages by a Missouri court in May 2019. The case brought by 22
women who blamed their ovarian cancer on its baby powder and other talc products.
• Reduced to $2.12 billion in June 2020
• Missouri’s highest court refused to consider Johnson & Johnson’s appeal

New Jersey state appeals court recently overturned a $223 million jury verdict blaming talc for cancer based
on the trial court accepting low-quality evidence from the plaintiffs’ expert witnesses.

Bloomberg Intelligence estimated that settling all the outstanding cases could cost J&J as much as $10 billion.

https://www.forbes.com/sites/korihale/2020/10/14/johnson--johnsons-100-million-baby-powder-lawsuit-settlement-is-
overdue-for-black--hispanic-women/?sh=5b792aac63b3
https://www.reuters.com/article/us-johnson-johnson-talc-ruling/johnson-johnson-to-appeal-2-12-billion-talc-verdict-to-
u-s-supreme-court-idUKKBN27J2N4
https://www.wsj.com/articles/johnson-and-johnson-court-verdict-overturned-talcum-powder-9fd6262c
Criminal Example – Arthur Andersen and Enron

Federal prosecutors criminally prosecuted Arthur Andersen in association with

the collapse of Enron. Andersen was Enron’s auditor, and Enron was Andersen’s
biggest client

Andersen was charged with obstruction of justice because of document

shredding and deleting by Andersen personnel who audited Enron

Andersen collapsed

https://www.corporatecrimereporter.com/news/200/arthur-andersen-corporate-criminal-liability-and-the-rise-of-deferred-and-non-
prosecution-agreements/
Discussion
The following events were part of the Andersen litigation:
• Andersen was offered, but would not agree to a plea agreement.
• Justice Department offered a deferred prosecution agreement that would require no guilty
plea. Andersen said no, demanding that the Justice Department clear the firm of any
wrongdoing.
• The government indicted. Prosecutors again suggested a deferred prosecution agreement
and again Andersen said no.
• The case went to trial and the firm was convicted of obstruction of justice.
• Two years later, the Supreme Court reversed the decision, but it was too late. Andersen had
collapsed.

How could analytics have been used by Andersen to guide their legal strategy?
Probabilistic Analysis – Decision Trees

Structured process for informing a decision based on analysis of cost, risk and
outcome at each phase of the decision process
Litigate or Settle?

GlaxoSmithKline (GSK) settled the biggest ever healthcare lawsuit in history with
a $3B payment in 2012.

The company pleaded guilty to three counts and paid $1B as a criminal penalty and $2B to
resolve civil liabilities.
• GSK was found guilty of unlawfully promoting certain prescription drugs.
• It failed to report certain safety data to the FDA
• It reported false drug prices to underpay rebates owed under the Medicaid Drug Rebate
Program.

https://www.pharmaceutical-technology.com/features/biggest-pharmaceutical-lawsuits/
Example - Settle at
$500,000 or Litigate?
1. On the basis of expected value, would you settle or litigate?

1. What other considerations might influence your decision?

1. What is the probability of incurring an award equal to or less than the cost of settling ($500,000)?

1. What is the probability of incurring an award greater than the cost of settling?

1. What major factor(s) might be missing from this analysis?

AI impact on legal profession

Source: Goldman Sachs Economic Research: The Potentially Large Effects of Artificial Intelligence on Economic Growth (Briggs/Kodnani)
Impact of AI on US employment

Source: Goldman Sachs Economic Research: The Potentially Large Effects of Artificial Intelligence on Economic Growth (Briggs/Kodnani)
Challenges driving the use of e-discovery and
analytics
e-discovery – Use of technology in identifying, collecting and producing
electronically stored information (ESI) in response to a request for
production in a law suit or investigation.
• ESI includes, but not limited to, emails, documents, presentations,
databases, voicemail, audio and video files, social media, and web
sites.

Amount of evidentiary information – ESI

Large number of civil and criminal cases
Size of potential settlements
Tight timelines for cases and fines for missing deadlines
e-discovery process

e-discovery is as much a business process as it is a legal one

• concerted effort to move beyond reactive and fragmented e-discovery processes to an
orchestrated e-discovery process that is more standardized, repeatable and defensible
• project management principles should be applied to e-discovery

https://searchsecurity.techtarget.com/definition/electronic-discovery-e-discovery-or-ediscovery
Document Discovery Process
Challenge – locate all documents useful to a case

Keyword search terms are useful for locating appropriate documents but are limited to terms
that are obvious to the attorneys

Use analytics and data visualization to identify concepts in body of documents

• Occurrence of seemingly inexplicable (and, therefore, attention worthy) words and phrases
e.g., Star Wars code words in Enron documents.
• Occurrence and use of code words in a specific time frame to identify other parties that
may have had knowledge of the suspected wrongdoing

Predictive Coding
• Use algorithms to select a subset of all case documents e.g., 1,000 out of a universe of
10,000, that an attorney reviews to determine their relevancy to the case. The algorithm
then learns from the set of 1,000 to identify and code key trends or patterns in the other
9,000 documents automatically.
In-Class Discussion

How would you describe the kind of analytics used to evaluate a subset of
1,000 documents to code trends and patterns in the universe of 10,000
documents?

What kind of analytic technology would you deploy?

Is the learning model supervised or unsupervised?

GPT4
GPT-4 is the first AI to pass the bar exam, scoring in the 90th percentile on both the
multiple-choice and written portions of the Uniform Bar Examination

CoCounsel – uses ChatGPT 4

• document review
• legal research memos
• deposition preparation
• contract analysis

“CoCounsel gives lawyers more time for things a machine can’t do—like thinking creatively, devising
and applying strategies, and building and strengthening relationships with colleagues and clients—in
the service of deepening their expertise, growing their practice, and serving more people’s legal
needs..”
https://casetext.com/blog/casetext-coauthors-study-on-gpt-4-bar-exam-score/?utm_medium=paidsearch&utm_source=google&utm_campaign=brand-
research&utm_content=_&utm_term=casetext
In the News…
The case involved a man named Roberto Mata, who had sued the airline
Avianca, claiming he was injured during an August 2019 flight from El
Salvador to New York.

Avianca asked the judge to dismiss the lawsuit because the statute of
limitations had expired. Mata’s lawyers responded with a 10-page brief
citing more than half a dozen court decisions in support of their argument
that the suit should be allowed to proceed.

Avianca’s lawyers could not locate the cases, so the judge ordered Mata’s
lawyers to provide copies, and they submitted a compendium of decisions
- the cases were not real.

Mata’s lawyer told the judge that he had believed ChatGPT had greater
reach than standard databases. “I heard about this new site, which I
falsely assumed was, like, a super search engine,”
https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html
Harvey

Generative AI trained with legal sector-specific data

Allen & Overy piloting Harvey

PwC signed a 12-month deal with Harvey that will give

its lawyers access the tool

https://www.harvey.ai/
Next Class

No class on Thanksgiving

Next Class - HR and Talent Analytics

Cappelli, P. (2019). "Your Approach to Hiring Is All Wrong: Outsourcing and

algorithms won’t get you the people you need." Harvard Business Review 97, 48-58.
Lauret, J. (2019). "Amazon's Sexist AI Tool: How Did It Go So Wrong?
Lalwani, P. (2021) "What Is HR Analytics? Definition, Importance, Key Metrics, Data
Requirements, and Implementation"