A P2PU API story

We've had some thoughts, talk and implementation for a P2PU API. It seems like everyone agrees that it's a good idea and something that we need, but we are lacking concrete use cases. Maybe because we don't have the complete API yet - the classic chicken and egg problem.

So I propose that we start by building a few prototypes of what we would like to use an API for. A prototype doesn't need to be a running program, although a running prototype would be great! We just need something a little bit more tangible to help us think about the ideal API that we want for P2PU.

A prototype may look something like this:

Lets say you want to build something to run a MOOC, you will need to use the following parts of P2PU:
  • Authentication
  • User profiles
  • Activity
  • Notifications
  • Messaging
  • Badges

User story

So lets think up a little about a user story:

You go to mooc.org and see a mooc that you like - "The study of geographic and temporal art as encountered in urban environments - SOGTA". You decide to sign up for the MOOC. When you click on sign-up you get redirected to P2PU to sign-in to your P2PU profile. Once you signed in, you are asked by P2PU if you want to allow mooc.org to send you notifications and publish activities to your activity feed?

Once you confirm you are redirected back to mooc.org. You am now signed up for SOGTA with your P2PU profile. On mooc.org you can participate in group discussions. Whenever you get a reply to a message that you posted in a discussion, you receive an email in your inbox. When you reply to the email, your message is also posted in the group discussion thread.

Every week you receive an email from the MOOC organizer that informs you what's happening with SOGTA.

On mooc.org there is a list of courses that is relevant to SOGTA. You can participate in these courses together with other people that is also signed up for SOGTA. There is also lots of references to other learning resources like coursera.

Finally, after participating in the MOOC, you receive a SOGTA participation badge to show that you completed the MOOC. This badge is shown on your profile at P2PU, but it's stored in your open badge backpack.


Prototype

Interaction between mooc.org and p2pu.org may look like this.

When someone at mooc.org creates a new MOOC, the MOOC gets published to the course register at p2pu.org:

https://api.p2pu.org/courses PUT {'title': 'The study of geographic and temporal art as encountered in urban environments', 'shortcode': 'SOGTA', 'signup-url':'mooc.org/sogta/sign-up', 'course-style':'MOOC',  'tags':['art','geo-art'], ... }

Now the "The study of geographic and temporal art as encountered in urban environments" will show up when someone searches for "geographic art" on the P2PU course register!

https://api.p2pu.org/courses also support GET,POST and DELETE to query, update and delete courses from the course register

When a user signs up for a MOOC on mooc.org, the browser will be redirected to https://p2pu.org/login. After the user successfully logged in, they get a token and user profile from p2pu.org and that is passed onto mooc.org (handled by javascript without the user knowing anything about it). mooc.org then does a post to

https://api.p2pu.org/user/verify-login POST {'user-profile':'https://p2pu.org/username', 'token':'bd46c283195c0aefaffb179f6197f18d184d38b8'}

which respons with 200 OK to indicate that the user is logged in and that the token is valid.

mooc.org can now safely add https://p2pu.org/username to the list of users that signed up for the MOOC. mooc.org also sends a message to https://p2pu.org/username to give the user instructions about SOGTA

https://api.p2pu.org/send-message POST {'user-profile':'https://p2pu.org/username', 'subject':'Welcome to SOGTA', 'content':'We are glad that you joined SOGTA ...' }

p2pu.org automatically add 'app':'mooc.org' to the data and depending on the user's preferences is sent a message to the email address registered with p2pu.org

Whenever a significant action is performed in SOGTA by a user, mooc.org does the following:

https://api.p2pu.org/activity PUT {'subject':'https://p2pu.org/username', 'verb':'post', 'object':'https://mooc.org/sogta', 'verb-url':'https://mooc.org/sogta/comment/135', 'message':'user posted a comment in the MOOC SOGTA'}

This is only allowed if user gave permission to mooc.org to post activity. If this failed mooc.org may choose to do

https://api.p2pu.org/activity PUT {'verb':'post', 'object':'https://mooc.org/sogta', 'verb-url':'https://mooc.org/sogta/comment/135', 'message':'A new comment was posted in the MOOC SOGTA'}

https://api.p2pu.org/activity GET {'noun':'https://p2pu.org/user'} - will return all activities involving user

https://api.p2pu.org/activity GET {'noun':'https://mooc.org/sogta'} - will return all activities involving SOGTA

When a user makes a comment and receives a reply, something like this can happen:

https://api.p2pu.org/notification PUT {'profile':'https://p2pu.org/username', 'subject':'[SOGTA] reply to comment', 'message':'Someone replied to your comment', 'email-reply-possible':'True'}

This call returns {'reply-token':'005f035e7d1b1c2ff6de0d4a73cc2f040d5b4127'} to mooc.org

The user receives an email if notifications are allowed for mooc.org. The email look like this:

From: reply+005f035e7d1b1c2ff6de0d4a73cc2f040d5b4127@reply.p2pu.org
Subject: [mooc.org] [SOGTA] reply to comment
Message:
Someone replied to your comment.

You can respond by replying to this email with your text:


When the user replies to the email, mooc.org gets a post from p2pu.org at the URL that mooc.org had to supply when registering to use the P2PU API:

https://mooc.org/api/message-in PUT {'from':'https://p2pu.org/user', 'reply-token':'005f035e7d1b1c2ff6de0d4a73cc2f040d5b4127', 'message':'???'}

I haven't yet thought through the badge API for this scenario.

So start thinking about how you would like to use an API to interact with P2PU and what you will build and send it to us at p2pu-dev@lists.p2pu.org

Learning Lernanta


I recently took a dive into the source code of a project called Lernanta. Lernanta is the Django based software platform that is used by Peer 2 Peer University to facilitate peer learning on the web! You can see it in action at p2pu.org

While trying to figure out how the code base comes together to form the final web application, I was presented with many challenges. One of those challenges was that I didn't know about everything that Lernanta does. Another challenge was (and still is) that terminology used in the source code differs from the terminology used on p2pu.org.

In this post I intend to outline the relevant top level entities of Lernanta. I will also try to explain how these entities corresponds to the source code.

 

Users

Users are probably the most important entity on P2PU. Without users no peer learning will be possible and everyone working on P2PU will probably get very lonely and depressed.

Lernanta uses django.contrib.auth for authentication and profiles are managed by the users app.

 

Courses

On the P2PU website a course may be know as either a course, challenge or a study group. Courses, challenges and study groups provides different ways for users to interact with learning material, but they have a lot of features and functionality in common.

On the source code side of things we have projects. Projects stores the category of the course. The category indicates if a given project is a course, challenge or a study group.

Users can participate in a project. If the project is a course or challenge the uses is taking the challenge and if it's a study group, the user joins the group.

Users can also follow a project. In this case the user does not participate, but rather observers.

One or more users will also be course facilitators. These users are responsible for running the course.

On the software side of things participation is indicated by a Participation entity. Participation is also used to indicate if a participant is also a course facilitator using the organizing property.

Users following courses are indicated using a Relationship entity. A relationship entity is automatically create for an participant, thus a participant is also a follower.

Course content is created by adding tasks to a course. In the source code tasks are represented by Pages that are part of the content application in Lernanta. Pages are versioned to preserve their history.


Schools

Schools are used to group together courses that are about a similar topic. Currently there are the School of Webcraft, the School of Education, the School of Social Innovation and the School of Mathematical Future. As I am speaking, the School of Data is being launched!

Schools each have a dedicated page with more information about the school and sets of courses associated with the school. Schools can also have specific sets of courses that they want to be displayed on their landing page.

In Lernanta, schools are implemented using the schools app. Courses can be associated with a school using the school property. Sets of courses are managed by schools.models.ProjectSet. Users doesn't need to belong to a school in order to participate in a course offered by a school.


Badges

Users can earn badges to show what they have learned. Badges are OBI compliant and users can push a badge to their Open Badge Backpack.

On p2pu.org there are several different types of badges. The first type of badge is a project completion badge. This badge gets automatically awarded to a user whenever they complete the challenge associated with a badge.

Another type of badge is a skill badge. To get this badge, a user must apply for the badge and submit proof that they satisfied all the criteria for the badge. The submission then needs to be reviewed by other users and assessed according to the rubrics associated with the badge. Once enough users assessed the submission, the badge can be awarded to the applicant

Finally there are community badges. Community badges can be awarded by any user doing a course by another user doing the course.

In Lernanta, the badges application is used to implement the above mentioned badges. Badges are represented by badges.models.Badge. Associated with a badge is badges.models.Rubrics and badges.models.Logic. Rubrics are used to indicate what rubrics should be considered for a skill badge, while Logic indicates how many assessments are necessary and what the average rating for a rubric should be.

The relevant entities for badges

When a user apply for a badge, badges.models.Submission is used. When a user reviews a badge, badges.models.Assessment is used. badges.models.Rating is associated with an assessment and a rubric to indicate the rating that the user gave corresponding to the assessment and the rubric.

 

A final word

If you would like to get involved with the development of Lernanta, I recommend that you proceed by setting up your development environment by following this guide.

backing up your delicious bookmarks

As some may have heard, delicious is being acquired by AVOS. You can transfer your bookmarks, but I decided to make a backup of my bookmarks, just in case.

The quickest way I could think of, was using the RSS feed.

Log into your account on the web and get the URL for the "Private RSS Feed" at the bottom of the page. Change the last part of the URL to be larger than the number of bookmarks you have saved on delicious.

Ex. change
"http://feeds.delicious.com/v2/rss/user_bob?count=15"
to
"http://feeds.delicious.com/v2/rss/user_bob?count=1000"
in the URL.

You can safe the page your browser displays when you enter the URL, or you can use wget or another utility to download the feed.

The data is now in your hands and it's up to you to keep it safe.

High resolution dependency graph

I generated a high resolution Gentoo Linux package dependency graph.

Packages are ordered according to their complete Gentoo package name. The result is that the graph shows groupings of packages with lots of dependencies or reverse dependencies. These groupings typically fall under dev-*, kde-*, gnome-*, x11-libs or media-*.

Text labels are only generated for packages with more that 10 dependencies or reverse dependencies. The numbers in brackets after the package name, is the number of dependencies followed by the number of reverse dependencies.

Packages grouped by categories (1600x1600)

A package of particular interest is app-text/sword. It has a total of 165 reverse dependencies!! This is very large number of packages that depends on app-text/sword! All of the packages that depends on app-text/sword seems to be different modules specifically for sword.

A nice picture of (dependency) hell

"Dependency hell" refers the difficulty that arise when installing a software package that requires a lot of other software packages to be installed. The required software packages (or dependencies) may themselves require other software packages to be installed.

To give a better idea of why this can be difficult, I created the following graph.
circle graph of software dependencies
63988 Dependencies between 14319 software packages
This graph shows the dependencies between software packages in the Gentoo Linux operating system. In total there are 14319 packages with 63988 dependencies between them!

Packages are drawn on the circumference of the circle and a dependency is indicated by a line draw from one package to another. The color of a package is randomly generated, while the color of a line is determined by the package that is required by the other packages.

Every package has a line starting on the rim of the circle drawn radially outwards. The length of the line is determined by the amount of other packages that depend on the package.

Here is a closeup of a small part of the graph:
circle graph of software dependenciesThe most common dependencies are development tools. This is a list of the 15 packages with the most reverse dependencies:

Package# of reverse dependencies
dev-lang/perl1559
dev-util/pkgconfig1195
dev-lang/python1047
x11-libs/gtk+1042
sys-devel/libtool950
app-arch/unzip878
sys-devel/automake818
sys-devel/autoconf766
dev-libs/glib652
x11-libs/qt-gui612
virtual/jdk588
sys-apps/sed575
x11-libs/libX11545
app-admin/eselect-python496
dev-util/cmake455

The graph was created with data obtained from the Gentoo Portage tree and drawn using the Python Image Libray (PIL).