When Python meets

GraphQL
Managing contributor identities
in your Open-source project

FOSDEM 2020 Python DevRoom

share this slide! @mghfdez

About me
My name is Miguel-Ángel Fernández

Working at Bitergia, part of the Engineering team

Software developer...

… also involved in stuff related with data and

metrics

share this slide! @mghfdez

share this slide! @mghfdez
How can I measure
my project?

How many contributors do we have ?

How many companies are contributing to

my project?

share this slide! @mghfdez

It’s all about identities

Tom Riddle

Affiliated to Slytherin, Hogwarts

Photo credit: juliooliveiraa

share this slide! @mghfdez

It’s all about identities

Lord Voldemort

Working as a freelance (dark) wizard

Photo credit: James Seattle

share this slide! @mghfdez

Wait… they are the same person!

Photo credit: juliooliveiraa Photo credit: James Seattle

share this slide! @mghfdez

A little bit more complex
Manrique López <[email protected]>
Jose Manrique López de la Fuente <[email protected]>
Manrique López <[email protected]>

jsmanrique

[email protected]
[email protected]
[email protected]

[email protected]

jsmanrique
02/2005 - 12/2010 CTIC
01/2010 - 12/2012 Andago
01/2013 - 06/2013 TapQuo
07/2013 - 12/2015 freelance (ASOLIF, CENATIC)
07/2013 - now Bitergia

share this slide! @mghfdez

Who is who?

Project manager

share this slide! @mghfdez

“For I'm the famous Sorting Hat.
(...)
So put me on and you will know
Which house you should be in... ”

share this slide! @mghfdez SortingHat: Wizardry on Software Project Members

 Lord Voldemort
Merge identities!
Tom Riddle

Affiliate this person!

Name: Tom
Complete the profile!
Gender: Male
Photo credit: James Seattle

Email: [email protected]

share this slide! @mghfdez

Boosting SH integration

Main idea: building a robust API

Easy to integrate with external apps

Flexible, easy to adapt Hatstall

Ensure consistency

Python module

share this slide! @mghfdez

GraphQL is...
… A query language, transport-agnostic
but typically served over HTTP.

… A specification for client-server communication:

It doesn’t dictate which language to use, how the data
should be stored or which clients to support.

… Based on graph theory: nodes, edges and connections.

share this slide! @mghfdez

REST vs GraphQL
query {
unique_identities(uuid:“<uuid>”) {
identities {
uid
}
profile {
/unique_identities/<uuid>/identities email
gender
/unique_identities/<uuid>/profile }
enrollments {
/unique_identities/<uuid>/enrollments organization
end_date
/organizations/<org_name>/domains }
domains {
domain_name
}
}
}

share this slide! @mghfdez

Comparing approaches: REST
Convention between server and client

Overfetching / Underfetching

API Documentation is not tied to development

Multiple requests per view

share this slide! @mghfdez

Comparing approaches: GraphQL
Strongly typed language

The client defines what it receives

The server only sends what is needed

One single request per view

share this slide! @mghfdez

Summarizing ...

share this slide! @mghfdez

Implementing process

Define data model & Support paginated Up next...

schema results

Implement basic
Authentication
queries & mutations

share this slide! @mghfdez

Implementation:
Graphene-Django
Graphene-Django is built on top of Graphene.

It provides some additional abstractions

that help to add GraphQL functionality to
your Django project.

share this slide! @mghfdez Picture credit: Snippedia

Schema
Types

GraphQL
Schema

s
ion
Qu
eri

tat
es

Mu
share this slide! @mghfdez
Schema.py
Models

GraphQL
Schema:

Re

ns
era D
Graphene-Django
so

tio
U
CR
lve
rs

op
share this slide! @mghfdez
It is already a graph Name: Tom
Gender: Male

Email: [email protected]

Profile

Lord Voldemort

Identities

UUID Tom Riddle

Affiliations

slytherin.edu

share this slide! @mghfdez

(Basic) Recipe for building queries
class Organization(EntityBase): class OrganizationType(DjangoObjectType):
name = CharField(max_length=MAX_SIZE) class Meta:
model = Organization
class Meta:
db_table = 'organizations'
unique_together = ('name',)
class SortingHatQuery:
def __str__(self):
organizations = graphene.List(OrganizationType)
return self.name
def resolve_organizations(self, info, **kwargs):
return Organization.objects.order_by('name')
models.py
schema.py

share this slide! @mghfdez

Documentation is already updated!

share this slide! @mghfdez

(Basic) Recipe for building mutations
class AddOrganization(graphene.Mutation):
class Arguments:
name = graphene.String()

organization = graphene.Field(lambda: OrganizationType)

class SortingHatMutation(graphene.ObjectType):
def mutate(self, info, name): add_organization = AddOrganization.Field()
org = add_organization(name)

return AddOrganization(
organization=org
)

schema.py
share this slide! @mghfdez
(Basic) Recipe for building mutations
@django.db.transaction.atomic def add_organization(name):
def add_organization(name):
validate_field('name', name)
try: organization = Organization(name=name)
org = add_organization_db(name=name)
except ValueError as e: try:
raise InvalidValueError(msg=str(e)) organization.save()
except AlreadyExistsError as exc: except django.db.utils.IntegrityError as exc:
raise exc _handle_integrity_error(Organization, exc)

return org return organization

api.py db.py

share this slide! @mghfdez

Documentation is already updated… again!

share this slide! @mghfdez

About pagination

How are we getting the cursor? identities(first:2 offset:2)

identities(first:2 after:$uuid)
It is a property of the connection,
not of the object. identities(first:2 after:$uuidCursor)

share this slide! @mghfdez

Edges and connections
Friend A

Information that is specific to the edge,

rather than to one of the objects.
Friendship
time
There are specifications like Relay

Friend B

share this slide! @mghfdez

Implementing pagination

We are taking our own approach without

reinventing the wheel

It is a hybrid approach based on offsets and

limits, using Paginator Django objects

Also benefiting from edges & connections

share this slide! @mghfdez

Query Result

share this slide! @mghfdez

share this slide! @mghfdez
 class AbstractPaginatedType(graphene.ObjectType):

@classmethod
def create_paginated_result(cls, query, page=1,
page_size=DEFAULT_SIZE):
Django objects paginator = Paginator(query, page_size)
result = paginator.page(page)
Query results
entities = result.object_list

page_info = PaginationType(
page=result.number,
page_size=page_size,
num_pages=paginator.num_pages,
Pagination info has_next=result.has_next(),
has_prev=result.has_previous(),
start_index=result.start_index(),
end_index=result.end_index(),
total_results=len(query)
)

return cls(entities=entities, page_info=page_info) share this slide! @mghfdez

Returning paginated results
class OrganizationPaginatedType(AbstractPaginatedType):
entities = graphene.List(OrganizationType)
page_info = graphene.Field(PaginationType)

class SortingHatQuery:

def resolve_organizations(...)

(...)

return OrganizationPaginatedType.create_paginated_result(query,
page,
page_size=page_size)

share this slide! @mghfdez

Authenticated queries
It is based on JSON Web Tokens (JWT)

An existing user must generate a token

which has to be included in the Authorization
header with the HTTP request

This token is generated using a mutation

which comes defined by the graphene-jwt
module

share this slide! @mghfdez

Testing authentication
Use an application capable of setting up headers to the HTTP requests

Heads-up!
Configuring the Django CSRF token properly was not trivial

Insomnia app

share this slide! @mghfdez

Testing authentication
from django.test import RequestFactory

def setUp(self):

self.user = get_user_model().objects.create(username='test')
self.context_value = RequestFactory().get(GRAPHQL_ENDPOINT)
self.context_value.user = self.user

def test_add_organization(self):

client = graphene.test.Client(schema)
executed = client.execute(self.SH_ADD_ORG, context_value=self.context_value)

share this slide! @mghfdez

Bonus: filtering
class OrganizationFilterType(graphene.InputObjectType):
name = graphene.String(required=False)

class SortingHatQuery:

organizations = graphene.Field(
OrganizationPaginatedType,
page_size=graphene.Int(),
page=graphene.Int(),
filters=OrganizationFilterType(required=False)
)

def resolve_organizations(...):
# Modified resolver

share this slide! @mghfdez

(some) Future work

Implementing a command line & web Client

Limiting nested queries

Feedback is welcome!

share this slide! @mghfdez

GrimoireLab architecture

share this slide! @mghfdez

Let’s go for some
questions
Twitter @mghfdez

Email [email protected]

GitHub mafesan

FLOSS enthusiast & Data nerd

Software Developer @ Bitergia
speaker pic
Contributing to
CHAOSS-GrimoireLab project

share this slide! @mghfdez