How mentoring programs can help newcomers get started with open source

Nov 5, 20100 likes799 views

As adoption of Open Source code and development practices continues to gain momentum, more newcomers have become interested in getting involved and contributing to Open Source. However, it's usually not easy for newcomers to start contributing to open source projects. This session will discuss how mentoring programs can ease the way for newcomers to get started with open source, and will provide an overview of existing mentoring programs such as Google Summer of Code and our in-house Apache Mentoring Programme.

How mentoring programs
can help newcomers
get started with
open source
Luciano Resende
lresende@apache.org
http://lresende.blogspot.com

Agenda
•  Apache Community Development
(ComDev) PMC
•  Why Mentoring ?
•  Available Programs
– Google Summer of Code
– Google Code-In
– Apache Mentoring Programme
•  Finding Project Ideas
•  Finding Mentors

Community Development
Why Apache is so successful ?
The Apache Way
Community over code

Community Development
•  Created to foster greater community
and help others understand how the
ASF works and give you the
confidence to work more directly

Why Mentoring
Open Source Project

Mentor

Why Mentoring ?
•  What to expect from the mentor
– Will help you get familiar with the open
source community, and help you interact
with the existent community
– Your mentor might also help with
technical questions/issues, but this might
not be a complete requirement

Google Summer of Code
•  Goals
– Inspire students to engage in open
source development and also give them
more exposure to real-world software
development scenarios
•  Eligibility Requirements
– Must be 18 years old by program
announcement/start
– Must be a student.

Google Summer of Code
•  Restrictions
– Code only
– Summer only
– Students only

Google Summer of Code
•  How it works ?
Project Ideas

Project Proposals

Mentoring

Evaluations

Google Summer of Code
•  Google Summer of Code web page
–  http://code.google.com/soc/
•  Google Summer of Code FAQ
–  http://code.google.com/soc/2009/faqs.html
•  Google Summer of Code discussion
group
–  http://groups.google.com/group/google-summer-of-code-discuss
•  IRC Channel
–  IRC Freenode #gsoc channel

Google Code-In
•  Goals
– Inspire young students to engage and
contribute to open source communities
•  Eligibility Requirements
– Must be between 13-18 years old
– Must be a student.

Google Code-In
•  Google Code-In web page
–  http://code.google.com/opensource/gci
•  Google Code-In FAQ
–  http://code.google.com/opensource/gci/2010-11/faqs.html

Apache Mentoring
Programme
•  Goals
– Foster a greater community around the
various Apache Projects
– Introduce The Apache Way to people
new to Apache and open source in
general

Apache Mentoring
Programme
•  Restrictions ?
NO RESTRICTIONS …
– Any contribution
– Any time
– Anyone

Apache Mentoring
Programme
•  How it works ?
Project Ideas

Project Proposals

Mentoring

Evaluations

Apache Mentoring
Programme
•  Finding a local mentor
– http://community.zones.apache.org/

Apache Mentoring
Programme
•  Community Development web page
–  http://community.apache.org
•  Mentoring Programme web page
–  http://community.apache.org/mentoringprogramme.html
•  FAQ
–  http://community.apache.org/newbiefaq.html

Finding Project Ideas
https://openhatch.org

Resources
•  Apache Community Development
–  http://community.apache.org
•  The Apache Way
–  http://www.apache.org/foundation/how-it-works.html

WebFWD is Mozilla's three month global business creation program aimed at open source innovators. It offers weekly training sessions and exercises, coaching from industry experts, and potential funding to help teams transform projects into sustainable businesses. Teams participate remotely from locations worldwide. The program seeks entrepreneurs interested in building the open web future, as well as scouts to share information about WebFWD and communicators to promote it on social media. Interested individuals are encouraged to apply now.

Tuscany : Applying OSGi After The FactLuciano Resende

Building apps with tuscanyLuciano Resende

Building and composing the components of a distributed application can be a challenge and complex bespoke solutions are commonplace. The Apache Tuscany runtime, and the Service Component Architecture (SCA) on which the runtime is based, simplifies the process be presenting a component based application assembly model. In this talk we look at the Tuscany travel booking application and explain how the individual components of the application are constructed using a variety of technologies including Java, BPEL, Python and Javascript. We also look at how these services are wired together using a different communication protocols such as SOAP/HTTP and JSON-RPC. The complete model can then be deployed to both stand-alone and distributed runtimes without changes to the application itself.

SCA Reaches the CloudLuciano Resende

This document provides an overview of developing composite applications for the cloud using Apache Tuscany. It discusses cloud computing goals and challenges, introduces Service Component Architecture (SCA) and its goals, and shows typical SCA scenarios. It also demonstrates Apache Tuscany and Apache Nuvem, describing how SCA components can be rewired in the cloud and how Nuvem provides cloud-friendly components. The document encourages audience participation to provide input on desired cloud components.

Building Asynchronous Services With ScaLuciano Resende

This document discusses building asynchronous services with Service Component Architecture (SCA). It begins with an overview of SCA and why asynchronous services are important. It then covers facilities that support asynchronous services like concurrency classes, JAX-WS, JMS, and BPEL. The document explains how SCA supports asynchronous service interactions through callbacks and bidirectional interfaces. It provides an example of an asynchronous service, client, and composite. Finally, it compares asynchronous services to using events and provides a demo scenario of an asynchronous travel search application built with SCA.

S314011 - Developing Composite Applications for the Cloud with Apache TuscanyLuciano Resende

Today's cloud environments pose new challenges for application developers: hiding cloud infrastructure from business logic, assembling components on heterogeneous and distributed cloud environments, and optimizing the provisioning of the required cloud resources. This session will demonstrate how to use Apache Tuscany and the Service Component Architecture (SCA) to develop, build, and run an application composed of several service components in a distributed cloud environment. We'll illustrate how to encapsulate cloud infrastructure services as SCA components to simplify the construction and assembly of the application and how to move components around and rewire the application to adjust to new business and cloud deployment conditions.

Building RESTful services using SCA and JAX-RSLuciano Resende

This document provides an overview of RESTful SCA using Apache Tuscany. It discusses what Apache Tuscany and Apache Wink are, and how they enable RESTful SCA. It then covers several usage scenarios: exposing an SCA service as REST, allowing HTTP GET access to RPC services, accessing external REST services from SCA, and using JAX-RS applications within SCA. The document concludes with a summary and discussion of future plans.

Data access layer and schema definitionsLuciano Resende

How mentoring can help you start contributing to open sourceLuciano Resende

This document discusses various mentoring programs that can help people start contributing to open source projects, including Google Summer of Code, Google Code-In, GNOME Outreach Program for Women, and the Apache Mentoring Programme. It provides details on the goals, eligibility requirements, and processes for these programs. It also discusses how to find mentors and project ideas within Apache projects and more generally.

Software management plans in research softwareShoaib Sufi

Slides from the 14th August 2019 webinar presentation as part of the Best Practices for HPC Software Developers (Webinar) series - https://ideas-productivity.org/events/hpc-best-practices-webinars/ - more info at https://www.exascaleproject.org/event/smp-rp/ and a recording on YouTube is at - https://www.youtube.com/watch?v=7sELeZStzdY&feature=youtu.be Abstract: Software is a necessary by-product of research. Software in this context can range from small shell scripts to complex and layered software ecosystems. Dealing with software as a first class citizen at the time of grant formulation is aided by the development of a Software Management Plan (SMP). An SMP can help to formalize a set of structures and goals that ensure your software is accessible and reusable in the short, medium and long term. SMP’s aim at becoming for software what Data Management Plans (DMP’s) have become for research data (DMP’s are mandatory for National Science Foundation grants). This webinar takes you through the questions you should consider when developing a Software Management Plan, how to manage the implementation of the plan, and some of the current motivation driving discussion in this area of research management.

How Open Source Helps to Bring Back Product ObsessionSauce Labs

This document discusses how open source can help organizations bring back product obsession. It outlines Sauce Labs' approach to open source which includes establishing an Open Source Program Office (OSPO) to manage open source consumption and contributions. The OSPO aims to increase customer engagement, shape industry standards, and build Sauce Labs' reputation through open source. Guidelines are provided for how employees can participate in open source through code and non-code contributions. The benefits of open source for customers, engineering culture, and bringing product obsession are discussed.

How to get in GSoC , DevFest Algiers 2018Assem CHELLI

This document provides information and advice about applying for Google Summer of Code (GSoC). It outlines eligibility requirements, key dates, advantages of participating, and tips for choosing an organization and project idea. The author recommends starting open source contributions now, improving communication skills, and contacting mentors early for guidance. The document emphasizes submitting prototype code and fixes to prove abilities. It notes the application process is competitive and stresses persistence.

Effective Tech Community Engagement - Best PracticesRuncy Oommen

This document provides best practices for effective community engagement in 3 stages: 1) Build your brand and recognition through small contributions, 2) Open up your work by presenting at meetups and publishing materials, and 3) Achieve "community nirvana" by becoming a meetup organizer. It also outlines plans for engaging the Google Cloud Community, including maximizing diverse membership, joint events, and activities like extended sessions, study groups, and webinars.

G so c_2015_presentationgdgjss

Google Summer of Code is a program that provides university students stipends to write code for open source software projects over the summer. The goals are to inspire students to participate in open source development, provide real-world development experience, and create more open source code. Students submit project proposals to organizations and are paired with mentors if accepted. Those accepted receive stipends for completing milestones throughout the summer coding period from May to August. Both students and organizations benefit from increased skills, contributions to open source projects, and experience.

G so c_2015_presentationSuyash Tilhari

Google Summer of Code is a program that provides university students stipends to write code for open source software projects over the summer. The goals are to inspire students to participate in open source development, provide real-world development experience, and create more open source code. Students submit project proposals to organizations and are paired with mentors if accepted. Those accepted receive stipends for completing milestones by evaluation deadlines in May, July, and August. The program benefits both students, who gain skills and experience, and organizations, which gain new contributors and exposure.

Info Session - Presentation, Google Developer Groups on Campus - HITAM.pptxrapoluav12

Quest_Slack_Academy that talks about business and educationdeepak725769

Google Summer of Code (GSoC) 2020Pradeeban Kathiravelu, Ph.D.

Software Engineering and Social mediaJorge Melegati

This document discusses the use of social media in software engineering. It explores several social media channels used in software development, including Q&A websites like Stack Overflow, microblogging platforms like Twitter, and developer blogs. For each channel, it summarizes relevant research that looks at how developers use these tools, what kinds of content and discussions they facilitate, and their potential benefits and risks. The document also considers how social media could enhance tasks like collaboration and learning, but also reverse trends like reliance on formal documentation or obsolesce tools like email lists. It concludes that while software development is becoming more collaborative, the impacts of social media are still unknown and require more research.

OSS Community Meeting - OSS Community Management for DummyWong Hoi Sing Edison

The document summarizes Wong Hoi Sing, Edison's presentation on online community management. It discusses the basics of online community management, how to build a productive online community, grow a community, assess community health, and take a community offline. It also provides case studies of the Hong Kong Web Developer Group, Codeaholics, and Hong Kong Drupal User Group communities. The presentation aims to teach attendees how to effectively manage online communities.

Accelerate the ROI of PHP in your EnterpriseEnterprise PHP Center

The Philosophy in Programming Society is a Singapore-based organization that was started in 2006 to promote PHP programming. It holds regular monthly meetups with companies like Microsoft, Oracle, and Yahoo where it has had over 700 members participate. It provides PHP training and certification courses to help developers gain skills and has partnerships with schools and technology companies. The document discusses recommendations for developers to get a fast ROI with PHP, including building a team with PHP knowledge, using the right tools like frameworks and version control, and following good processes with modeling, design patterns, and methodologies.

Final - dec. 11 hkl4 presentation (1)Hack the Hood

“Hack the Hood: Building Character through Building Competency” Learn how Hack the Hood uses project-based learning as a strategy to create new behaviors that transform youth, as well as the perceptions of youth by local neighborhoods. Through a curriculum focused on building youth leadership skills, an entrepreneurial mindset, and cultural competency, youth move from being passive consumers of digital tech to being knowledgeable workers and tech producers as they become valued resources to local small businesses. Come hear about character development and SEL in action from the youth themselves and their adult leaders. Workshop will be led by Jackie Shonerd, Susan Mernit, and Damon Packwood.

Info session developer student club telkom universityRaihan Hamid Suraperwata

The document provides information about Developer Student Clubs (DSC) at Telkom University, including what DSC is, upcoming events, how to join, and benefits of joining. DSC helps students develop technology skills and bridges the gap between theory and practical application. Details are given about various event types like info sessions, workshops, and showcases. Contact information is listed for the DSC core team and social media accounts.

How Do You Know that Gal Knows Drupal? Towards an Open Source Curriculum and ...Dominik Lukes

An open source curriculum and community-based certification scheme for Drupal was proposed. The proposal involved developing competencies organized around key areas, with maintainers responsible for updates. Training providers could share compatible materials and assess individuals' competencies through portfolios. Certification would be community-based and representatively democratic. Benefits included easier entry for new members, easier hiring for clients, and ensuring the direction of certification. Challenges and a discussion of practical implementation were discussed. Participants were invited to join further discussion on the topic.

Avoiding a CMS misfitPerttu Tolvanen

Creating Digital PortfoliosDesiree Caskey

Creating Digital Portfolios The document discusses creating digital portfolios for both students and teachers. It defines a digital portfolio as a collection of work, ideas and artifacts that can be used to assess student learning and guide instruction. The document outlines a five stage process for creating digital portfolios, including defining the context and audience, gathering content, adding reflections, linking items, and publishing the portfolio online. It provides examples of tools like Evernote, Weebly, Google Docs, YouTube and SoundCloud that can be used to embed multimedia content into the portfolio.

Experiences with the Apache Software Foundation marpierc

Mobile media module part 6 - app development rev-mfMichelle Ferrier

The Mobile Media Module is designed as a two-week, broad-based study on the mobile landscape that can be applied in many courses. The program was implemented at Ohio University’s Scripps College of Communication to support our Scripps Innovation Challenge and to build knowledge of the mobile landscape across our communication curricula. For implementation, we brought in an expert in mobile development to teach in four existing classes over two weeks in Spring 2013. Faculty teaching those classes became the students and built their capacity to teach the material in subsequent semesters. By “hacking the curriculum” using the “module method,” we were able to reach more than 500 students in one semester with new material. For more information, contact Dr. Michelle Ferrier, associate dean for innovation, Scripps College of Communication, [email protected].

A Jupyter kernel for Scala and Apache Spark.pdfLuciano Resende

Many data scientists are already making heavy usage of the Jupyter ecosystem for analyzing data using interactive notebooks. Apache Toree (incubating) is a Jupyter kernel designed that enables data scientists and data engineers to easily connect and leverage Apache Spark and its powerful APIs from a standard Jupyter notebook to execute their analytics workloads. In this talk, we will go over what's new with the most recent Apache Toree release. We will cover available magics and visualizations extensions that can be integrated with Toree to enable better data exploration and data visualizations. We will also describe some high-level designs of Toree and how users can extend the functionality of Apache Toree powerful plugin system. And all of these with multiple live demos that demonstrate how Toree can help with your analytics workloads in an Apache Spark environment.

Using Elyra for COVID-19 AnalyticsLuciano Resende

In this session, Luciano will be walking you through a real use case pipeline that uses Elyra features to help analyze COVID-19 related datasets. He will introduce Elyra, a project built to extend JupyterLab with AI-centric capabilities. He'll showcase the extensions that allow you to build Notebook Pipelines and execute these in a Kubeflow environment, execute notebooks as batch jobs, the ability to create, edit and execute Python scripts directly from JupyterLab

More Related Content

Similar to How mentoring programs can help newcomers get started with open source (20)

How mentoring can help you start contributing to open sourceLuciano Resende

Software management plans in research softwareShoaib Sufi

How Open Source Helps to Bring Back Product ObsessionSauce Labs

How to get in GSoC , DevFest Algiers 2018Assem CHELLI

Effective Tech Community Engagement - Best PracticesRuncy Oommen

G so c_2015_presentationgdgjss

G so c_2015_presentationSuyash Tilhari

Google Summer of Code is a program that provides university students stipends to write code for open source software projects over the summer. The goals are to inspire students to participate in open source development, provide real-world development experience, and create more open source code. Students submit project proposals to organizations and are paired with mentors if accepted. Those accepted receive stipends for completing milestones by evaluation deadlines in May, July, and August. The program benefits both students, who gain skills and experience, and organizations, which gain new contributors and exposure.

Info Session - Presentation, Google Developer Groups on Campus - HITAM.pptxrapoluav12

Quest_Slack_Academy that talks about business and educationdeepak725769

Google Summer of Code (GSoC) 2020Pradeeban Kathiravelu, Ph.D.

Software Engineering and Social mediaJorge Melegati

OSS Community Meeting - OSS Community Management for DummyWong Hoi Sing Edison

Accelerate the ROI of PHP in your EnterpriseEnterprise PHP Center

Final - dec. 11 hkl4 presentation (1)Hack the Hood

Info session developer student club telkom universityRaihan Hamid Suraperwata

How Do You Know that Gal Knows Drupal? Towards an Open Source Curriculum and ...Dominik Lukes

Avoiding a CMS misfitPerttu Tolvanen

Creating Digital PortfoliosDesiree Caskey

Experiences with the Apache Software Foundation marpierc

Mobile media module part 6 - app development rev-mfMichelle Ferrier

How mentoring can help you start contributing to open sourceLuciano Resende

Software management plans in research softwareShoaib Sufi

How Open Source Helps to Bring Back Product ObsessionSauce Labs

How to get in GSoC , DevFest Algiers 2018Assem CHELLI

Effective Tech Community Engagement - Best PracticesRuncy Oommen

G so c_2015_presentationgdgjss

G so c_2015_presentationSuyash Tilhari

Info Session - Presentation, Google Developer Groups on Campus - HITAM.pptxrapoluav12

Quest_Slack_Academy that talks about business and educationdeepak725769

Google Summer of Code (GSoC) 2020Pradeeban Kathiravelu, Ph.D.

Software Engineering and Social mediaJorge Melegati

OSS Community Meeting - OSS Community Management for DummyWong Hoi Sing Edison

Accelerate the ROI of PHP in your EnterpriseEnterprise PHP Center

Final - dec. 11 hkl4 presentation (1)Hack the Hood

Info session developer student club telkom universityRaihan Hamid Suraperwata

How Do You Know that Gal Knows Drupal? Towards an Open Source Curriculum and ...Dominik Lukes

Avoiding a CMS misfitPerttu Tolvanen

Creating Digital PortfoliosDesiree Caskey

Experiences with the Apache Software Foundation marpierc

Mobile media module part 6 - app development rev-mfMichelle Ferrier

More from Luciano Resende (20)

A Jupyter kernel for Scala and Apache Spark.pdfLuciano Resende

Using Elyra for COVID-19 AnalyticsLuciano Resende

Elyra - a set of AI-centric extensions to JupyterLab Notebooks.Luciano Resende

In this session Luciano will explore the different projects that compose the Jupyter ecosystem; including Jupyter Notebooks, JupyterLab, JupyterHub and Jupyter Enterprise Gateway. Jupyter Notebooks are the current open standard for data science and AI model development, and IBM is dedicated to contributing to their success and adoption. Continuing the trend of building out the Jupyter ecosystem, Luciano will introduce Elyra. It's a project built to extend JupyterLab with AI-centric capabilities. He'll showcase the extensions that allow you to build Notebook Pipelines, execute notebooks as batch jobs, navigate and execute Python scripts, and tie neatly into Notebook versioning.

From Data to AI - Silicon Valley Open Source projects come to you - Madrid me...Luciano Resende

Ai pipelines powered by jupyter notebooksLuciano Resende

The Jupyter Notebook has become the de facto platform used by data scientists and AI engineers to build interactive applications and develop their AI/ML models. In this scenario, it’s very common to decompose various phases of the development into multiple notebooks to simplify the development and management of the model lifecycle. Luciano Resende details how to schedule together these multiple notebooks that correspond to different phases of the model lifecycle into notebook-based AI pipelines and walk you through scenarios that demonstrate how to reuse notebooks via parameterization.

Strata - Scaling Jupyter with Jupyter Enterprise GatewayLuciano Resende

Born in academia, Jupyter notebooks are prevalent in both learning and research environments throughout the scientific community. Due to the widespread adoption of big data, AI, and deep learning frameworks, notebooks are also finding their way into the enterprise, which introduces a different set of requirements. Alan Chin and Luciano Resende explain how to introduce Jupyter Enterprise Gateway into new and existing notebook environments to enable a “bring your own notebook” model while simultaneously optimizing resources consumed by the notebook kernels running across managed clusters within the enterprise. Along the way, they detail how to use different frameworks with Enterprise Gateway to meet the needs of data scientists operating within the AI and deep learning ecosystems.

Scaling notebooks for Deep Learning workloadsLuciano Resende

Deep learning workloads are computing intensive, and training these type of models is better done with specialized hardware like GPUs. Luciano Resende outlines a pattern for building deep learning models using the Jupyter Notebook’s interactive development in commodity hardware and leveraging platforms and services such as Fabric for Deep Learning (FfDL) for cost-effective full dataset training of deep learning models.

Jupyter Enterprise Gateway OverviewLuciano Resende

This document discusses Jupyter notebooks and the Jupyter Enterprise Gateway. It provides an overview of Jupyter notebooks, their functionality, and architecture. It then describes the limitations of running Jupyter notebooks at an enterprise scale. The Jupyter Enterprise Gateway is introduced as a solution that enables sharing of resources across clusters in a scalable, secure, and multi-tenant way. The document outlines the key features and architecture of the Gateway when used with YARN and Kubernetes clusters. Finally, it provides information on deploying and connecting to the Jupyter Enterprise Gateway.

Inteligencia artificial, open source e IBM Call for CodeLuciano Resende

Nesta palestra vamos abordar algumas das tendências em Inteligência Artificial e as dificuldades na uso da Inteligência Artificial. Por isso, também apresentaremos algumas ferramentas disponíveis em código livre que podem ajudar a simplificar a adoção da IA. E faremos uma breve introdução ao “Call for Code” que é uma iniciativa da IBM para construir soluções na prevenção e reação a desastres naturais.

IoT Applications and Patterns using Apache Spark & Apache BahirLuciano Resende

The Internet of Things (IoT) is all about connected devices that produce and exchange data, and building applications that produce insights from these high volumes of data is very challenging and require a understanding of multiple protocols, platforms and other components. On this session, we will start by providing a quick introduction to IoT, some of the common analytic patterns used on IoT, and also touch on the MQTT protocol and how it is used by IoT solutions some of the quality of services tradeoffs to be considered when building an IoT application. We will also discuss some of the Apache Spark platform components, the ones utilized by IoT applications to process devices streaming data. We will also talk about Apache Bahir and some of its IoT connectors available for the Apache Spark platform. We will also go over the details on how to build, test and deploy an IoT application for Apache Spark using the MQTT data source for the new Apache Spark Structure Streaming functionality.

Getting insights from IoT data with Apache Spark and Apache BahirLuciano Resende

The Internet of Things (IoT) is all about connected devices that produce and exchange data, and producing insights from these high volumes of data is challenging. On this session, we will start by providing a quick introduction to the MQTT protocol, and focus on using AI and machine learning techniques to provide insights from data collected from IoT devices. We will present some common AI concepts and techniques used by the industry to deploy state-of-the-art smart IoT systems. These techniques allow systems to determined patterns from the data, predict and prevent failures as well as suggest actions that can be used to minimize or avoid IoT device breakdowns on an intelligent way beyond rule-based and database search approaches. We will finish with a demo that puts together all the techniques discussed in an application that uses Apache Spark and Apache Bahir support for MQTT.

Open Source AI - News and examplesLuciano Resende

Building analytical microservices powered by jupyter kernelsLuciano Resende

This document discusses building analytical microservices powered by Jupyter kernels. It provides an overview of Jupyter notebooks and their architecture. It then introduces the Jupyter Enterprise Gateway, which allows running Jupyter kernels on a distributed cluster for improved scalability and security. Finally, it demonstrates a use case of a sentiment analysis microservice that leverages PySpark on a Hadoop cluster via Jupyter kernels.

Building iot applications with Apache Spark and Apache BahirLuciano Resende

We leave in a connected world where connected devices are becoming part of our day to day and are providing invaluable streams of data. In this talk, we will introduce you to Apache Bahir and some of its IoT connectors available for Apache Spark. We will also go over the details on how to build, test and deploy an IoT application for Apache Spark using the MQTT data source for the new Apache Spark Structure Streaming functionality.

An Enterprise Analytics Platform with Jupyter Notebooks and Apache SparkLuciano Resende

IBM has built a “Data Science Experience” cloud service that exposes Notebook services at web scale. Behind this service, there are various components that power this platform, including Jupyter Notebooks, an enterprise gateway that manages the execution of the Jupyter Kernels and an Apache Spark cluster that power the computation. In this session we will describe our experience and best practices putting together this analytical platform as a service based on Jupyter Notebooks and Apache Spark, in particular how we built the Enterprise Gateway that enables all the Notebooks to share the Spark cluster computational resources.

The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017Luciano Resende

What's new in Apache SystemML - Declarative Machine LearningLuciano Resende

SystemML was designed with the main goal of lowering the complexity required to maintain and scale Machine Learning algorithms. It provides a declarative machine learning (DML) that simplify the specification of machine learning algorithms using an R-like and Python-like that significantly increases the productivity of data scientist as it provides flexibility on how the custom analytics are expressed and also provides data independence from the underlying input formats and physical custom analytics. This presentation gives a quick introduction to Apache SystemML, provides an updated on the recent areas that are being developed by the project community, and go over a tutorial that enables one to quickly get up to speed in SystemML.

Big analytics meetup - Extended Jupyter Kernel GatewayLuciano Resende

Luciano Resende from IBM's Spark Technology Center presented on building an enterprise/cloud analytics platform with Jupyter Notebooks and Apache Spark. The Spark Technology Center focuses on contributions to open source Apache Spark projects. Resende discussed limitations of the current Jupyter Notebook setup for multi-user shared clusters and demonstrated an Extended Jupyter Kernel Gateway that allows running kernels remotely in a cluster with enhanced security, resource optimization, and multi-user support through user impersonation. The Extended Jupyter Kernel Gateway is planned for open source release.

Jupyter con meetup extended jupyter kernel gatewayLuciano Resende

Writing Apache Spark and Apache Flink Applications Using Apache BahirLuciano Resende

Big Data is all about being to access and process data in various formats, and from various sources. Apache Bahir provides extensions to distributed analytic platforms providing them access to different data sources. In this talk we will introduce you to Apache Bahir and its various connectors that are available for Apache Spark and Apache Flink. We will also go over the details of how to build, test and deploy an Spark Application using the MQTT data source for the new Apache Spark 2.0 Structure Streaming functionality.