Anatomy of a data science project

Dec 7, 2018Download as PPTX, PDF4 likes265 views

Slides for the presentation given at the Data Science Scotland Meetup (https://www.meetup.com/Scotland-Data-Science-Technology-Meetup/events/256269263/). This talk aimed to give some general advice, tips, and tricks about how to run a successful data science project. Hosted by: Incremental Group - https://www.linkedin.com/company/incremental-group/ MBN Solutions - https://www.linkedin.com/company/mbn-recruitment-solutions/ The Datalab - https://www.linkedin.com/company/the-data-lab-innovation-centre/

‘ANATOMY OF A DATA
SCIENCE PROJECT’
ADAM SROKA, SENIOR DATA SCIENTIST
WELCOME TO THE DECEMBER SCOTLAND DATA SCIENCE & TECHNOLOGY MEETUP
RICARDO ANTUNES, DATA SCIENTIST
INCREMENTAL GROUP
- with
&

MBN ACADEMY ARE THE OFFICIAL PARTNERS OF THE
DATA LAB MSC. PLACEMENT PROGRAMME 2018/19.
IF YOU ARE PART OF THE SCOTTISH BUSINESS
COMMUNITY AND INTERESTED IN BECOMING A
POTENTIAL HOST ORGANISATION, PLEASE EMAIL ROB
AT ACADEMY@MBNSOLUTIONS.COM

Anatomy of a Data Science
Project
Adam Sroka, Senior Data Scientist
adam.sroka@incrementalgroup.co.uk
@adzsroka

Why we exist
Digital technology
is changing
everything
Sustainable
success comes
from incremental
improvements
Our mission is to
enable
government and
industry to digitally
transform, one
step at a time

What makes a good project?
A few points to consider before you start

Ask yourself…
1. Will this be easy to deploy & use?
2. Will it be considerably better than existing solutions?
3. Can parts be automated, reproduced, and reused?
4. Is it easy to understand, explain, and test?
5. Is it technically interesting?

Data
Making your raw materials go further

 This is always the
first step
 Build a reusable
set of tools for
measuring quality
 DataExplorer is
great start
Quality

“Never use a long word
where a short one will do.”
George Orwell

 It’s easy to get excited about the new big thing
 Sometimes seems like expressing intelligence has
taken priority over delivering value
 Marginal gains at the cost of understanding and
interoperability aren’t gains at all
Complexity

 What are other people
doing?
 Are there similar problems
to yours on Kaggle?
 Do you have any biases?
 Take a quick first pass with
everything and review
What works?

Tools
A few things to make your life easier

Templates
 Figure out a template that
works for you – then stick to it
 It makes moving between and
sharing projects tolerable
 https://drivendata.github.io/co
okiecutter-data-science/

 For longer lasting projects,
strongly consider build
automation
 This will manage rebuilding
what’s needed when you
make a change
 Tools like Azure Pipelines,
AWS CodeBuild, Luigi, or
even Makefiles
Build Tools

Containers
 Package your entire workspace
into easily manageable
containers
 Makes reproducibility and sharing
simple
 Many cloud platforms allow
automatic distribution of
containers to clusters

Thanks!
Adam Sroka, Senior Data Scientist
adam.sroka@incrementalgroup.co.uk
@adzsroka

This document outlines the methodology for a data science project using the Cross-Industry Standard Process for Data Mining (CRISP-DM). It describes the 6 phases of the project - business understanding, data understanding, data preparation, modeling, evaluation, and deployment. For each phase, it provides an overview of the key steps and asks questions to determine readiness to move to the next phase of the project. The overall goal is to successfully apply a standard data science methodology to gain business value from data.

20151016 Data Science For Project ManagersTze-Yiu Yong

This document discusses data science projects from the perspective of a project manager. It begins with an overview of the large and growing amount of data being generated every day. It then discusses the skills needed for data science, including the data science workflow. The implications for project managers managing data science projects are that they will have many stakeholders to consider, need to focus on quality of data and understand where data is coming from and going, and internalize privacy concerns. Unique considerations for data science projects include agreeing on required data quality and understanding and communicating about the data.

A Hybrid Approach to Data Science Project ManagementElaine K. Lee

Planning Your Data Science ProjectsSpotle.ai

1440 track 2 boire_using our laptopRising Media, Inc.

This document discusses feature engineering and machine learning approaches for predicting customer behavior. It begins with an overview of feature engineering, including how it is used for image recognition, text mining, and generating new variables from existing data. The document then discusses challenges with artificial intelligence and machine learning models, particularly around explainability. It concludes that for smaller datasets, feature engineering can improve predictive performance more than complex machine learning models, while large datasets are better suited to machine learning approaches. Testing on a small travel acquisition dataset confirmed that traditional models with feature engineering outperformed neural networks.

Big Data Agile Analytics by Ken Collier - Director Agile Analytics, Thoughtwo...Thoughtworks

We are in the midst of an exciting time. There is an explosion of very interesting data, and emergence of powerful new technologies for harnessing data, and devices that enable humans to receive tremendous benefits from it. What is required are innovative processes that enable the creation and delivery of value from all of that data. More often than not, it is the predictive (what will happen?) and prescriptive (how to make it happen!) analytics that produces this value, not the raw data itself. Agile software teams are continuously involved in projects that involve rich, complex, and messy data. Often this data represents innovative analytics opportunities. Being analytics-aware gives these teams the opportunity to collaborate with stakeholders to innovate by creating additional value from the data. This session is aimed at making Agile software teams more analytics-aware so that they will recognize these innovation opportunities. The trouble with conventional analytics (like conventional software development) is that it involves long, phased, sequential steps that take too long and fail to deliver actionable results. This talk will examine the convergence of the following elements of an exciting emerging field called Agile Analytics: •sophisticated analytics techniques, plus •lean learning principles, plus •agile delivery methods, plus •so-called "big data" technologies Learn: •The analytical modeling process and techniques •How analytical models are deployed using modern technologies •The complexities of data discovery, harvesting, and preparation •How to apply agile techniques to shorten the analytics development cycle •How to apply lean learning principles to develop actionable and valuable analytics •How to apply continuous delivery techniques to operationalize analytical models

H2O World - Migrating from Proprietary Analytics Software - Fonda IngramSri Ambati

This document discusses strategies for moving an organization from proprietary software to open source software. It notes that open source provides freedom from vendor control and can reduce costs through evaluating the total cost of ownership. Open source also allows organizations to innovate in real time and regain internal expertise. The document recommends starting small with a proof of concept, gaining user buy-in through education and support, and realizing that open source may not be identical but can coexist with previous proprietary software during a transition. It also acknowledges that security issues can exist with both open source and commercial software.

Agile Data ScienceVolodymyr Kazantsev

Leveraged Analytics at ScaleDomino Data Lab

Most of analytics modeling work today focuses on the production of single-purpose "artisanal" models for predictions. This approach to analytics is fragile with respect to model consistency, reorganization, and resource availability. This talk will argue that instead the focus of analytics modeling should be toward the production of analytics interchangeable parts, which can be combined in creative ways to produce a wide variety of analytics results. This "nuts and bolts" approach allows analytics groups to produce results in an agile way where the time between ask and answer is determined by the right combination of analytics, rather than the modeling.

Data Science Salon: Introduction to Machine Learning - Marketing Use CaseFormulatedby

This document provides an introduction and agenda for a machine learning marketing use case presentation. It includes an overview of the data science process, machine learning algorithms, and examples of machine learning in marketing. It discusses data preparation, feature selection, preprocessing, transformation, and algorithm selection. It also provides a primer on deep learning, the benefits of deep learning for feature extraction, and examples of innovations using deep learning. The presentation aims to help understand how to apply machine learning and deep learning techniques to optimize marketing.

Managing Data Science | Lessons from the Field Domino Data Lab

The document discusses best practices for managing data science teams based on lessons learned. It outlines common pitfalls such as solving the wrong problem, having the wrong tools, or results being used incorrectly. Issues include data science being different from software development and forgetting other stakeholders. Recommendations include establishing processes for the full lifecycle from ideation to monitoring, using modular systems thinking, and defining roles like data scientists, managers, and product owners to address organizational challenges. The goal is to deliver measurable, reliable, and scalable insights.

Data Quality Analytics: Understanding what is in your data, before using itDomino Data Lab

Analytics and data science are ever growing fields, as business decision makers continue to use data to drive decisions. The pinnacle of these fields are the models and their accuracy/fit,; what about the data? Is your data clean, and how do you know that? Our discussion will focus on best practices for data preprocessing for analytic uses. Beginning with essential distributional checks of a dataset to a propose method for automated data validation process during ETL for transactional data.

Agile Analytics: The Secret to Test, Improve, Fail & Succeed Quickly.Venveo

The document discusses agile analytics and its benefits for businesses. Agile analytics involves rapidly testing hypotheses, analyzing results, and making improvements to gain a better understanding of customers, get results more quickly, and reduce risks. It recommends businesses focus on a single problem, develop small testable hypotheses, iterate testing every 2-4 weeks with specific changes, and use findings to direct the next round of improvements. Practicing agile analytics allows organizations to test, improve, fail, and succeed quickly.

1645 track 3 porterRising Media, Inc.

This document discusses how organizations can harness data and experiments to build engaging individual outreach campaigns. It recommends incorporating holdout groups into testing, constantly iterating tests, and measuring efficacy at the individual level. BlueLabs uses techniques like uplift modeling and predictive analytics to identify those most likely to take action from outreach and engage in multistage processes. Relevant inputs like behavior, circumstances, community data, and creative modeling approaches are important to success.

H2O World - What you need before doing predictive analysis - Keen.ioSri Ambati

This document provides guidance on setting up predictive analytics. It recommends being proactive rather than reactive, understanding how to acquire and analyze the right data, and creating a diverse team that includes various domain expertise. It also stresses the importance of performance, knowing your tools, and addressing challenges like deciding which analytics tools to use, combining data sources, and collecting data while maintaining privacy and integrity. The goal is to figure out what the existing data reveals, agree on business problems, identify useful predictions, and build an iterative pipeline to feed predictive algorithms.

Evaluation of big data analysisΚαρολίνα Κάτι

The speaker discusses the importance of evaluating big data analysis to improve projects. They recommend getting a second opinion on methodology or using another data source for verification. A case study on estimating traffic congestion is presented where ground truth sensor data was collected and compared to estimates using metrics to provide feedback. Lessons include ground truth not being easy to obtain, using the right tools like Python and pandas for evaluation, and having an iterative workflow for timely feedback.

H2O World - Intro to Data Science with Erin LedellSri Ambati

This document provides an introduction to data science. It defines data science as using data to solve problems through the scientific method. The roles of data scientists, data analysts, and data engineers on a data science team are discussed. Popular tools for data science include Python, R, and APIs that connect data processing engines. Machine learning algorithms are used to perform tasks like classification, regression, and clustering by learning from data rather than being explicitly programmed. Deep learning and ensemble methods are also introduced. Resources for learning more about data science and machine learning are provided.

Reproducible Dashboards and other great things to do with JupyterDomino Data Lab

Putting data science in your business a first utility feedbackPeculium Crypto

Licensed to Analyze? Strata Data NY 2019 IADSS Session - Usama Fayyad, Hamit ...IADSS

The latest insight into IADSS Research was shared with analytics community at Strata Data NY 2019 by O'reilly. IADSS Co-founders Usama Fayyad and Hamit Hamutcu talked about the current status of data science job market, the wasted cost of data science recruitment and role definitions & required skill-sets for most common roles in data science. Please check out IADSSglobal on Twitter and visit www.iadss.org for more information.

Operationalizing Machine Learning in the Enterprisemark madsen

TDWI Munich 2019 What does it take to operationalize machine learning and AI in an enterprise setting? Machine learning in an enterprise setting is difficult, but it seems easy. All you need is some smart people, some tools, and some data. It’s a long way from the environment needed to build ML applications to the environment to run them in an enterprise. Most of what we know about production ML and AI come from the world of web and digital startups and consumer services, where ML is a core part of the services they provide. These companies have fewer constraints than most enterprises do. This session describes the nature of ML and AI applications and the overall environment they operate in, explains some important concepts about production operations, and offers some observations and advice for anyone trying to build and deploy such systems.

Operational analytics overviewpallavi pentapati

1555 track 1 huang_using his macRising Media, Inc.

1) The document discusses challenges in predictive analytics projects including finding ideas, assessing impact, prioritizing projects, developing and deploying models, and post-implementation monitoring. 2) It describes moving from manual processes to automating workflows to using decision-making and machine learning to drive innovation. New data sources like prescriptions and driving records can help leverage new predictors for better customer experiences. 3) A framework is presented for evaluating analytics projects based on their value, data and process integration requirements, urgency, and ease of implementation to identify high priority projects that maximize return. The top priority projects identified were predictive underwriting, pricing models, and customer journey analytics.

Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...Ali Alkan

Supporting innovation in insurance with randomized experimentationDomino Data Lab

Recent technological advances, a dynamic competitive landscape, and an evolving regulatory environment have led to a period of rapid innovation for many insurance providers. Here, we’ll explore how data scientists may use randomized experiments to rigorously assess the causal impact of innovations on business outcomes. Particular emphasis will be placed on experimentation in “offline” channels, with some of the challenges and mitigation strategies highlighted.

H2O World - Advanced Analytics at Macys.com - Daqing ZhaoSri Ambati

The document discusses advanced analytics at Macys.com. It outlines the challenges of big data predictive modeling such as scaling models, ensuring timely models, integrating models, and testing models. It describes Macys.com's advanced analytics team which includes data scientists with backgrounds in quantitative fields. The team works on projects such as personalized site recommendations, response propensity models, customer acquisition/retention modeling, and experimentation platforms. It provides examples of Macys.com's real-time site personalization and customer segmentation work.

CRISP-DM - Agile Approach To Data Mining ProjectsMichał Łopuszyński

This document summarizes Michał Łopuszyński's presentation on using an agile approach based on the CRISP-DM methodology for data mining projects. It discusses the key phases of CRISP-DM including business understanding, data understanding, data preparation, modelling, evaluation, and deployment. For each phase, it provides examples of best practices and challenges, with an emphasis on spending sufficient time on data understanding and preparation, developing models with the deployment context in mind, and carefully evaluating results against business objectives.

Data science tools of the tradeFangda Wang

This document discusses best practices for developing data science products at Philip Morris International (PMI). It covers: - PMI's data science team of over 40 people across four hubs working on fraud prevention and other problems. - Key principles for PMI's data science work, including being business-driven, investing in people, self-organizing, iterating to improve, and co-creating solutions. - Challenges in data product development involving integrating work between data scientists and other teams, and practices like continuous integration/delivery to overcome these challenges. - The role of data scientists in contributing code that is readable, testable, reusable, reproducible, and usable by other teams to integrate into

Online productivity tools - SILS20090is20090

More Related Content

What's hot (20)

Agile Data ScienceVolodymyr Kazantsev

Leveraged Analytics at ScaleDomino Data Lab

Data Science Salon: Introduction to Machine Learning - Marketing Use CaseFormulatedby

Managing Data Science | Lessons from the Field Domino Data Lab

Data Quality Analytics: Understanding what is in your data, before using itDomino Data Lab

Agile Analytics: The Secret to Test, Improve, Fail & Succeed Quickly.Venveo

1645 track 3 porterRising Media, Inc.

H2O World - What you need before doing predictive analysis - Keen.ioSri Ambati

Evaluation of big data analysisΚαρολίνα Κάτι

H2O World - Intro to Data Science with Erin LedellSri Ambati

Reproducible Dashboards and other great things to do with JupyterDomino Data Lab

Putting data science in your business a first utility feedbackPeculium Crypto

Licensed to Analyze? Strata Data NY 2019 IADSS Session - Usama Fayyad, Hamit ...IADSS

Operationalizing Machine Learning in the Enterprisemark madsen

Operational analytics overviewpallavi pentapati

1555 track 1 huang_using his macRising Media, Inc.

Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...Ali Alkan

Supporting innovation in insurance with randomized experimentationDomino Data Lab

H2O World - Advanced Analytics at Macys.com - Daqing ZhaoSri Ambati

CRISP-DM - Agile Approach To Data Mining ProjectsMichał Łopuszyński

Agile Data ScienceVolodymyr Kazantsev

Leveraged Analytics at ScaleDomino Data Lab

Data Science Salon: Introduction to Machine Learning - Marketing Use CaseFormulatedby

Managing Data Science | Lessons from the Field Domino Data Lab

Data Quality Analytics: Understanding what is in your data, before using itDomino Data Lab

Agile Analytics: The Secret to Test, Improve, Fail & Succeed Quickly.Venveo

1645 track 3 porterRising Media, Inc.

H2O World - What you need before doing predictive analysis - Keen.ioSri Ambati

Evaluation of big data analysisΚαρολίνα Κάτι

H2O World - Intro to Data Science with Erin LedellSri Ambati

Reproducible Dashboards and other great things to do with JupyterDomino Data Lab

Putting data science in your business a first utility feedbackPeculium Crypto

Licensed to Analyze? Strata Data NY 2019 IADSS Session - Usama Fayyad, Hamit ...IADSS

Operationalizing Machine Learning in the Enterprisemark madsen

Operational analytics overviewpallavi pentapati

1555 track 1 huang_using his macRising Media, Inc.

Intelligently Automating Machine Learning, Artificial Intelligence, and Data ...Ali Alkan

Supporting innovation in insurance with randomized experimentationDomino Data Lab

H2O World - Advanced Analytics at Macys.com - Daqing ZhaoSri Ambati

CRISP-DM - Agile Approach To Data Mining ProjectsMichał Łopuszyński

Similar to Anatomy of a data science project (20)

Data science tools of the tradeFangda Wang

Online productivity tools - SILS20090is20090

Course 8 : How to start your big data project by Eric Rodriguez Betacowork

For more info about our Big Data courses, check out our website ➡️ https://www.betacowork.com/big-data/ --------- "Data is the new oil" - Many companies and professionals do not know how to use their data or are not aware of the added value they could gain from it. It is in response to these problems that the project “Brussels: The Beating Heart of Big Data” was born. This project, financed by the Region of Brussels Capital and organised by Betacowork, offers 3 training cycles of 10 courses on big data, at both beginner and advanced levels. These 3 cycles will be followed by a Hackathon weekend. No prerequisites are required to start these courses. The aim of these courses is to familiarize participants with the principles of Big Data. ------ For more info about our Big Data courses, check out our website ➡️ https://www.betacowork.com/big-data/

Enabling Data centric TeamsData Con LA

Data Con LA 2020 Description Coming from a grand belief of data democratization, I believe that in order for any team to be successful collaborators, it has to be data centric and data should be accessible to all. *To ensure that your non software or software engineering centric team has maximum efficiency, data should be visible, data lake should be accessible. *Form a database for analytics summaries, talk about the different technologies(SQL, NoSQL) cost of deployment, need, team driven structure. Build an API for this database for external/inter team crosstalk. *Build analytics and visual layer on top of it. Flask/Django/Node, etc.., to enable the team to have high visibility in their analysis, and to ensure a higher turnaround of data. *Talk about an easy way of enabling the team to run code, could be local/cloud, JupyterHub is a great way of doing so, talk about the tremendous value added in that and the potential it enables *Talk about the common tools user for version control/CICD/Coding technologies, etc.. *Finally summarize the value of the mixture of all these tools and technologies in order to ensure the maximum efficiency. Speaker Nawar Khabbaz, Rivian, Data Engineer

Forms 2 Future - the ongoing journey into the future for Oracle based organiz...Lucas Jellema

Many organizations around the world have adopted Oracle technology for developing custom applications. Over the past two decades, they may have used PL/SQL, Reports, Forms, Designer, Portal or the Web PL/SQL Toolkit. Many of these organizations have come to face new challenges: more agility or functionality, new user groups or channels or more efficient maintenance. Or they fear getting stuck in the past, running out of support or qualified and motivated resources. What is the right way to approach the future? What mix of tools, how and when to adopt which new technology, how to build a business case? This session recounts various more and less successful warstories of organizations that embarked on a journey into the future. We will discuss SOA, ADF, OSB, WebCenter, SaaS, Forms, WebLogic, .NET, Java and much more.

How Cloud is Affecting Data Scientists CCG

This document discusses how data science models have transitioned to the cloud to take advantage of greater computing resources. It notes that data science models are resource-intensive and traditionally required powerful local machines. The cloud allows data scientists to run models on cloud infrastructure for lower costs than high-end laptops and with access to many GPUs. Several major cloud platforms - Azure, AWS, and Google Cloud - are discussed and compared in terms of their machine learning offerings. The document also introduces Microsoft's Team Data Science Process, which aims to help data science teams collaborate more effectively on projects in the cloud.

EclipseDay Milano 2017 - How to make Data Science appealing with open source ...SpagoWorld

A Methodology for Building the Internet of ThingsThe Internet of Things Methodology

This document proposes an iterative methodology for building the Internet of Things (IoT). The methodology aims to allow everyone to experience the IoT through prototyping common use cases with a "lean startup" mentality. It outlines steps in the methodology including co-creating solutions with end users, ideating ideas, conducting questions and analysis, using an IoT architecture model, prototyping, and deploying solutions. It provides examples and influences at each step. The overall goal is to enable individuals, communities and organizations to explore the possibilities of the IoT through an open, collaborative approach.

Pausefest: Solve your own damn problemMike Ojo

RightScale Roadtrip Boston: Accelerate to CloudRightScale

OpenSistemas Corporate PresentationOpenSistemas

Data technology can help companies predict outcomes through simulations, find unexpected relationships in large data sets, and monitor situations in real-time. OpenSistemas is a company that specializes in data management, analysis, storage and visualization using technologies like Apache Spark, machine learning, and cloud integration. They provide services in areas like data processing, analytics, visualization and cloud to help clients strengthen the strategic value of their information.

Digital Transformation: A Case for Modern WorkplaceSani Garba Consulting

Digital transformation involves integrating digital technologies into all areas of a business to fundamentally change operations and deliver value to customers. This requires cultural change and continual challenging of the status quo. The 4th industrial revolution is an era of connectivity, globalization, cloud computing, and opportunity. Digital transformation impacts work through automation substituting repetitive jobs with robotic workers by 2025 and evolving market demands focused on customer experience. Data and processes are better integrated to create value rather than focus on activities. Skills will be preferred over academic qualifications. Cloud computing delivers computing services like storage, servers, and software via the internet. It provides flexibility and pay-as-you-go pricing models. Digital solutions like Microsoft 365 provide consolidated apps, security, collaboration and

Azure - The Best Cloud for DevelopersInovar Tech

Why Should Nonprofits Care About Cloud ComputingTechSoup Global

The silent project disruptor: Building AI solutionsAssociation for Project Management

Kai-Fu Lee predicted that AI would change the world more than anything in the history of humanity – even electricity. It would disrupt how we live and work, how we operate our businesses, the core products and services on offer and the way in which we build technology. However, in 2024 the impact of AI can no longer be discussed in future tense. With Microsoft copilot now publicly available, the change is already upon us. There is no consultation period or ‘unsubscribe’ button. Project management professionals are likely to be asked to manage AI projects - and we are expected to skilfully use AI in our daily work lives. While overwhelming, this is not the first time we’ve had to adapt. Sarah helps her audience sharpen their cutting-edge skills by answering: What do I need to know about AI right now? If I’m asked to work on an AI project, what techniques do I need to be successful? Where do I start my own learning journey to upskill and prepare? Sarah’s expertise in advanced agile and experience in highly regulated Finance environments give her a unique perspective into balancing governance with technical innovation. She uses her own experience building an AI solution in 2023 to share practical, widely applicable concepts in an “AI for project managers” 101 style session.

Maciej Marek (Philip Morris International) - The Tools of The TradeCodiax

This document discusses best practices for developing and deploying data science products at PMI. It recommends using Docker containers to ensure reproducibility, project templates for organization, continuous integration/delivery (CI/CD) pipelines for testing and deployment, and monitoring tools like Kibana and Airflow. Adopting these practices can help data scientists, data engineers, and devops teams work more collaboratively and rapidly deliver machine learning models to production.

Cloud Computing WebinarTechSoup

The term cloud computing is being used more and more, but what is it and why should you understand it? In this free webinar we will explain what cloud computing means, define the different types, discuss how it is impacting nonprofits and libraries, and outline some criteria for use. The challenges of using the “cloud” will be discussed, as well as whether cloud computing will simplify your life and reduce software and IT staffing costs. Hear from Anna Jaeger, Co-Director, GreenTech at TechSoup Global, and Peter Campbell, Nonprofit Technologist at Earthjustice, who will help you understand this topic in order to better communicate with your consultants, staff and board. This webinar is applicable for any size organization and ideal for decision makers who need to communicate about cloud computing with tech consultants, and who are interested in making more informed technology decisions.

Capgemini Ron Tolido - the 3rd Platform and InsuranceEDGEteam

1) The document discusses digital transformation in the insurance industry and outlines several frameworks for how insurance companies can progress in their digital capabilities and mastery. 2) It presents different "levels" of digital capability that insurance companies may fall into, from "beginners" to "digital masters", and suggests that most insurers currently rank as "conservatives". 3) Several technology trends and drivers are introduced that can help insurance companies advance their digital transformation, such as social, mobile, analytics/big data and cloud computing. Combining these drivers is seen as particularly powerful.

Where the Warehouse Ends: A New Age of Information AccessInside Analysis

The document provides information about an upcoming webinar hosted by The Briefing Room. The webinar will feature David Besemer, CTO of Composite Software, who will discuss how Composite addresses the challenges of data integration and providing data for analytics. The webinar aims to explain how Composite's data virtualization platform can help analysts more easily access and work with data from various sources through self-service analytic sandboxes and data hubs. The webinar also hopes to demonstrate how Composite can help organizations gain business insights faster while reducing costs compared to traditional data integration and warehousing approaches.

How Data Virtualization Puts Enterprise Machine Learning Programs into Produc...Denodo

Watch full webinar here: https://bit.ly/3offv7G Presented at AI Live APAC Advanced data science techniques, like machine learning, have proven an extremely useful tool to derive valuable insights from existing data. Platforms like Spark, and complex libraries for R, Python and Scala put advanced techniques at the fingertips of the data scientists. However, these data scientists spend most of their time looking for the right data and massaging it into a usable format. Data virtualization offers a new alternative to address these issues in a more efficient and agile way. Watch this on-demand session to learn how companies can use data virtualization to: - Create a logical architecture to make all enterprise data available for advanced analytics exercise - Accelerate data acquisition and massaging, providing the data scientist with a powerful tool to complement their practice - Integrate popular tools from the data science ecosystem: Spark, Python, Zeppelin, Jupyter, etc.

Data science tools of the tradeFangda Wang

Online productivity tools - SILS20090is20090

Course 8 : How to start your big data project by Eric Rodriguez Betacowork

Enabling Data centric TeamsData Con LA

Forms 2 Future - the ongoing journey into the future for Oracle based organiz...Lucas Jellema

How Cloud is Affecting Data Scientists CCG

EclipseDay Milano 2017 - How to make Data Science appealing with open source ...SpagoWorld

A Methodology for Building the Internet of ThingsThe Internet of Things Methodology

Pausefest: Solve your own damn problemMike Ojo

RightScale Roadtrip Boston: Accelerate to CloudRightScale

OpenSistemas Corporate PresentationOpenSistemas

Digital Transformation: A Case for Modern WorkplaceSani Garba Consulting

Azure - The Best Cloud for DevelopersInovar Tech

Why Should Nonprofits Care About Cloud ComputingTechSoup Global

The silent project disruptor: Building AI solutionsAssociation for Project Management

Maciej Marek (Philip Morris International) - The Tools of The TradeCodiax

Cloud Computing WebinarTechSoup

Capgemini Ron Tolido - the 3rd Platform and InsuranceEDGEteam

Where the Warehouse Ends: A New Age of Information AccessInside Analysis

How Data Virtualization Puts Enterprise Machine Learning Programs into Produc...Denodo

Recently uploaded (20)

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

Most consumers believe they’re making informed decisions about their personal data—adjusting privacy settings, blocking trackers, and opting out where they can. However, our new research reveals that while awareness is high, taking meaningful action is still lacking. On the corporate side, many organizations report strong policies for managing third-party data and consumer consent yet fall short when it comes to consistency, accountability and transparency. This session will explore the research findings from TrustArc’s Privacy Pulse Survey, examining consumer attitudes toward personal data collection and practical suggestions for corporate practices around purchasing third-party data. Attendees will learn: - Consumer awareness around data brokers and what consumers are doing to limit data collection - How businesses assess third-party vendors and their consent management operations - Where business preparedness needs improvement - What these trends mean for the future of privacy governance and public trust This discussion is essential for privacy, risk, and compliance professionals who want to ground their strategies in current data and prepare for what’s next in the privacy landscape.

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell

Heap, Types of Heap, Insertion and DeletionJaydeep Kale

Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul

Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.

Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity

This session is designed to equip developers with the skills needed to build mission-critical, end-to-end processes that seamlessly orchestrate agents, people, and robots. 📕 Here's what you can expect: - Modeling: Build end-to-end processes using BPMN. - Implementing: Integrate agentic tasks, RPA, APIs, and advanced decisioning into processes. - Operating: Control process instances with rewind, replay, pause, and stop functions. - Monitoring: Use dashboards and embedded analytics for real-time insights into process instances. This webinar is a must-attend for developers looking to enhance their agentic automation skills and orchestrate robust, mission-critical processes. 👨‍🏫 Speaker: Andrei Vintila, Principal Product Manager @UiPath This session streamed live on April 29, 2025, 16:00 CET. Check out all our upcoming Dev Dives sessions at https://community.uipath.com/dev-dives-automation-developer-2025/.

AI and Data Privacy in 2025: Global TrendsInData Labs

In this infographic, we explore how businesses can implement effective governance frameworks to address AI data privacy. Understanding it is crucial for developing effective strategies that ensure compliance, safeguard customer trust, and leverage AI responsibly. Equip yourself with insights that can drive informed decision-making and position your organization for success in the future of data privacy. This infographic contains: -AI and data privacy: Key findings -Statistics on AI data privacy in the today’s world -Tips on how to overcome data privacy challenges -Benefits of AI data security investments. Keep up-to-date on how AI is reshaping privacy standards and what this entails for both individuals and organizations.

AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB

I started my online journey with several hosting services before stumbling upon Ai EngineHost. At first, the idea of paying one fee and getting lifetime access seemed too good to pass up. The platform is built on reliable US-based servers, ensuring your projects run at high speeds and remain safe. Let me take you step by step through its benefits and features as I explain why this hosting solution is a perfect fit for digital entrepreneurs.

Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55

Linux Support for SMARC: How Toradex Empowers Embedded DevelopersToradex

Toradex brings robust Linux support to SMARC (Smart Mobility Architecture), ensuring high performance and long-term reliability for embedded applications. Here’s how: • Optimized Torizon OS & Yocto Support – Toradex provides Torizon OS, a Debian-based easy-to-use platform, and Yocto BSPs for customized Linux images on SMARC modules. • Seamless Integration with i.MX 8M Plus and i.MX 95 – Toradex SMARC solutions leverage NXP’s i.MX 8 M Plus and i.MX 95 SoCs, delivering power efficiency and AI-ready performance. • Secure and Reliable – With Secure Boot, over-the-air (OTA) updates, and LTS kernel support, Toradex ensures industrial-grade security and longevity. • Containerized Workflows for AI & IoT – Support for Docker, ROS, and real-time Linux enables scalable AI, ML, and IoT applications. • Strong Ecosystem & Developer Support – Toradex offers comprehensive documentation, developer tools, and dedicated support, accelerating time-to-market. With Toradex’s Linux support for SMARC, developers get a scalable, secure, and high-performance solution for industrial, medical, and AI-driven applications. Do you have a specific project or application in mind where you're considering SMARC? We can help with Free Compatibility Check and help you with quick time-to-market For more information: https://www.toradex.com/computer-on-modules/smarc-arm-family

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Cyber Awareness overview for 2025 month of securityriccardosl1

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Want to learn practical tips for designing systems that can scale efficiently without compromising speed? Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development. As you explore key principles of designing low-latency systems with Rust, you will learn how to: - Create and compile a real-world app with Rust - Connect the application to ScyllaDB (NoSQL data store) - Negotiate tradeoffs related to data modeling and querying - Manage and monitor the database for consistently low latencies

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Build Your Own Copilot & Agents For DevsBrian McKeiver

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

TrustArc Webinar: Consumer Expectations vs Corporate Realities on Data Broker...TrustArc

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell

Heap, Types of Heap, Insertion and DeletionJaydeep Kale

Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul

Dev Dives: Automate and orchestrate your processes with UiPath MaestroUiPathCommunity

AI and Data Privacy in 2025: Global TrendsInData Labs

AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...SOFTTECHHUB

Special Meetup Edition - TDX Bengaluru Meetup #52.pptxshyamraj55

Linux Support for SMARC: How Toradex Empowers Embedded DevelopersToradex

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

Semantic Cultivators : The Critical Future Role to Enable AIartmondano

Cyber Awareness overview for 2025 month of securityriccardosl1

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Build Your Own Copilot & Agents For DevsBrian McKeiver

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Drupalcamp Finland – Measuring Front-end Energy ConsumptionExove

Anatomy of a data science project

1. ‘ANATOMY OF A DATA SCIENCE PROJECT’ ADAM SROKA, SENIOR DATA SCIENTIST WELCOME TO THE DECEMBER SCOTLAND DATA SCIENCE & TECHNOLOGY MEETUP RICARDO ANTUNES, DATA SCIENTIST INCREMENTAL GROUP - with &

2. MBN ACADEMY ARE THE OFFICIAL PARTNERS OF THE DATA LAB MSC. PLACEMENT PROGRAMME 2018/19. IF YOU ARE PART OF THE SCOTTISH BUSINESS COMMUNITY AND INTERESTED IN BECOMING A POTENTIAL HOST ORGANISATION, PLEASE EMAIL ROB AT [email protected]

3. Anatomy of a Data Science Project Adam Sroka, Senior Data Scientist [email protected] @adzsroka

4. Why we exist Digital technology is changing everything Sustainable success comes from incremental improvements Our mission is to enable government and industry to digitally transform, one step at a time

5. THE DATA SCIENCE HIERARCHY OF NEEDS

6. THE DATA SCIENCE HIERARCHY OF NEEDS

7. COMPLEXITY FROM SIMPLICITY

8. What makes a good project? A few points to consider before you start

13. Ask yourself… 1. Will this be easy to deploy & use? 2. Will it be considerably better than existing solutions? 3. Can parts be automated, reproduced, and reused? 4. Is it easy to understand, explain, and test? 5. Is it technically interesting?

14. Data Making your raw materials go further

16.  This is always the first step  Build a reusable set of tools for measuring quality  DataExplorer is great start Quality

17. Build pipelines

18. Models Mastering the tools of the trade

19. “Never use a long word where a short one will do.” George Orwell

20.  It’s easy to get excited about the new big thing  Sometimes seems like expressing intelligence has taken priority over delivering value  Marginal gains at the cost of understanding and interoperability aren’t gains at all Complexity

21.  What are other people doing?  Are there similar problems to yours on Kaggle?  Do you have any biases?  Take a quick first pass with everything and review What works?

22. Tools A few things to make your life easier

23. Templates  Figure out a template that works for you – then stick to it  It makes moving between and sharing projects tolerable  https://drivendata.github.io/co okiecutter-data-science/

24.  For longer lasting projects, strongly consider build automation  This will manage rebuilding what’s needed when you make a change  Tools like Azure Pipelines, AWS CodeBuild, Luigi, or even Makefiles Build Tools

25. Containers  Package your entire workspace into easily manageable containers  Makes reproducibility and sharing simple  Many cloud platforms allow automatic distribution of containers to clusters

26. Resources reddit.com/user/adzsroka/m/d atascience datatau.com kagglenoobs.herokuapp.com dataelixir.com getrevue.co/profile/ datamachina/

27. Thanks! Adam Sroka, Senior Data Scientist [email protected] @adzsroka

Editor's Notes

#10: Ease of adoption – awful interfaces
#11: Is this an improvement https://medium.com/@EvanSinar/7-data-visualization-types-you-should-be-using-more-and-how-to-start-4015b5d4adf2
#12: https://aeon.co/essays/is-technology-making-the-world-indecipherable
#13: Reusability https://www.freeimageslive.co.uk/free_stock_image/building-concept-jpg
#16: https://gengo.ai/datasets/the-50-best-free-datasets-for-machine-learning/ https://digital.nhs.uk/ https://registry.opendata.aws/ https://data.europa.eu/euodp/en/data/ https://data.gov.uk/ https://www.data.gov/
#17: https://cran.r-project.org/web/packages/DataExplorer/vignettes/dataexplorer-intro.html
#18: https://icons8.com/icon/2297/gis

Anatomy of a data science project

Recommended

More Related Content

What's hot (20)

Similar to Anatomy of a data science project (20)

Recently uploaded (20)

Anatomy of a data science project

Editor's Notes