Turning Europeana into a source for educational content

Together with my colleague Matthias Palmér I participated in the “Hack4Europe!” competition in Stockholm which has been arranged by the Swedish National Heritage Board in the context of the Europeana project. The goal of the competition was to show the potentials of the Europeana content and APIs by building applications to showcase the social and business value of open cultural data.

Hannes Ebner and Matthias Palmér at Hack4Europe! in Stockholm. (BY-NC-SA by Kristina Alexanderson)

During the hack day we worked in a team of two (representing our newly founded spin-off MetaSolutions) and developed a showcase to demonstrate how heterogeneous metadata can be managed using an approach that we have developed in research projects both at the Royal Institute of Technology and Uppsala University. During Hack4Europe! a combination of the SCAM framework (EntryStore) and Confolio (EntryScape) is used, and both applications are extended so that they can search in Europeana and extract Europeana metadata from the search results. This allows for adding resources directly from a Europeana search result to a user’s personal portfolio for further annotation with educational metadata (or, depending on the use case, some other metadata in a non-educational context).

Screenshot of the demonstrated Europeana showcase.

The demonstrated use case was to search for resources which are suitable to be used in an educational context and to turn them into learning resources by annotating them with educational metadata. Technically this means searching and caching metadata described using the Europeana Data Model and adding educational metadata (e.g. in LOM/DCAM) on top of it. Everything is integrated into the Confolio interface and the end user does not have to know anything about where the metadata originates from, which standards are used, etc.

See the following video for a demonstration of our use case at the end of the competition:

Technical details

The annotation process of Europeana content with educational metadata as implemented in our showcase is greatly supported by the Resource and Metadata Management Model (ReM³) which is described in a paper submitted to the Semantic Web Journal (not yet accepted). ReM³ is an information model for keeping track of resources and their metadata. It is based on the concepts of Contexts and Entries where each context manages a set of entries. An entry contains a resource, descriptive metadata about the resource and some administrative information about the entry.

Schematic overview of an ReM3 entry

The information model is RDF-oriented and relies on the concept of Named Graphs (NG).  This is used to express that NGs are related, as it is the
case when the same resource is described in different contexts. In a typical situation where already existing metadata is built upon – such as in our Europeana showcase – the referenced resource is described by an additional metadata graph, resulting in at least two metadata graphs per resource: the original one from Europeana and the additional one with educational metadata in our system.

The relation between the original external metadata and the newly created metadata is stored in the entry information, where it is indicated that there are multiple descriptions for the same resource. The use of NGs makes contextualisation of metadata possible. Without the fourth piece of information in the quadruple it would be hard to differentiate between triples from different sources.

The information structure of ReM³ is represented by a hierarchical URI model which has been implemented as a REST-ful interface using HTTP. An easy-to-use and REST interface which exposes the functionality of ReM³ allows for enrichment of metadata as the protocol makes communication in both ways possible (read/write). Resources in other systems can be described by linking to them and building a connection between the metadata and the resource. Such connections are in turn exposed using Linked Data which integrates heterogeneous information sources.

Links

• Our demonstrated use case described at hack4europe.se
• The demo installation and outcome of our session
• Our spin-off MetaSolutions, information management using Semantic Web technologies
• “Hack4Europe!” competition in Stockholm
• The Europeana project

Educational metadata in RDF with SPARQL endpoint

Some time ago (within the Organic.Edunet project that ended in late 2010) I implemented a mapping of IEEE LOM to the Dublin Core Abstract Model (thanks to my colleague Mikael Nilsson who did this work) and also applied it to some bigger data sources such as the repository of the ARIADNE foundation. This post is basically a write-up of what I produced and in which ways it can be used/exploited by others. This is about public data (namely metadata describing educational resources) and a contribution to the LOD cloud.

The software used for harvesting, converting, and exposing are the SCAM framework and the Confolio frontend (I’ll post links to these applications as soon as we have some usable documentation online). Organic.Edunet uses the SCAM/Confolio combination natively so a conversion as with ARIADNE was not necessary in the case of this project.

Harvested repositories exported to RDF files

One RDF file per harvested repository is exported every Monday  morning and can be downloaded from: http://knowone.csc.kth.se/rdf/.

Some quick facts about the exports:

• This is a direct and metadata-only export from SCAM which means that only educational metadata is included here. I.e. I assume that the ACL, provenance, etc information from SCAM is not of general interest so I don’t include it in the export.
• SCAM supports contextualized annotation of resources, that’s why all metadata is stored in separate graphs with a URI. This basically means that the same resource (identified by a URI which you can use to download/refer to/view the resource) can be described by different graphs, e.g. in different educational contexts, or simply with some generic metadata and educational metadata on top of it.
• The format it TriG, mainly because it has the needed support for including information about Named Graphs (and because it is a nice format ).
• Of course you can import the RDF data into your own triple store and provide/use your own SPARQL endpoint (or use it in a completely different way). If you want to play around before you do this you can also access the public SPARQL endpoint as described below.

SPARQL endpoint to query educational metadata

Using Sesame and Snorql I created a public SPARQL endpoint with a simple interface which you can use to play around. If you intend to use this more heavily in own applications you are kindly asked to setup your own endpoint to avoid hitting the server too hard. The endpoint currently supports SPARQL 1.0.

• SPARQL endpoint (for machines): http://knowone.csc.kth.se/sparql/ariadne-big
• A simple interface to it (for humans): http://knowone.csc.kth.se/snorql/

The endpoint currently only contains quadruples generated from the ARIADNE repository. I am thinking of providing also separate endpoints for other repositories, I only have to find a way of maintaining this in a way which it not too much effort. The graph URIs you can request in SPARQL (using the GRAPH keyword, see some examples below) can also be queried using plain HTTP. With content negotiation (either through the HTTP header) you can request different RDF serializations, JSON, and LOM/XML.

Examples for such URIs for a GET request look like this:

• http://knowone.csc.kth.se/scam/9/cached-external-metadata/41081 (the default returns RDF/XML and is the same as requesting it with “?format=application/rdf+xml”)
• http://knowone.csc.kth.se/scam/9/cached-external-metadata/41081?format=application/json
• http://knowone.csc.kth.se/scam/9/cached-external-metadata/41081?format=application/lom+xml

The response of these requests is a full metadata instance describing a resource, the root of the graph is the resource URI. You can get some administrative information by replacing “cached-external-metadata” by “entry”. If you had username/password an access rights to the SCAM installation you could also perform a PUT in order to update the metadata.

Enough about this, let’s get the hands dirty with some…

SPARQL query examples

Because it is always easier to build upon and modify existing things you can have a look at the following queries that have been shown to work. Preceding the query is a short description what it actually does. Depending on the query you will also need some of the following prefixes. They are predefined in the web interface, but you have to provide them yourself if you send the SPARQL query using an different client:

Prefixes

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX dcterms: <http://purl.org/dc/terms/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX vcard: <http://www.w3.org/2001/vcard-rdf/3.0#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX lom: <http://ltsc.ieee.org/rdf/lomv1p0/lom#>
PREFIX lomvoc: <http://ltsc.ieee.org/rdf/lomv1p0/vocabulary#>
PREFIX lomterms: <http://ltsc.ieee.org/rdf/lomv1p0/terms#>
PREFIX lre: <http://organic-edunet.eu/LOM/rdf/voc#>
PREFIX oe: <http://organic-edunet.eu/LOM/rdf/>

Find all contexts describing a resource

SELECT DISTINCT ?g WHERE {
  GRAPH ?g { <http://www.orgfoodfed.com/> ?p ?o }
}

Find all resources from FAO in English language

SELECT DISTINCT ?uri ?title ?g WHERE {
  GRAPH ?g {
    ?uri dcterms:title ?title ;
      dcterms:language ?b .
      ?b rdf:value "en"^^dcterms:RFC4646 .
    FILTER(regex(str(?uri), "^http://www.fao.org/", "i"))
  }
} LIMIT 10

Get all available types from the repository

SELECT DISTINCT ?types WHERE {
  ?s rdf:type ?types
} ORDER BY ?types

Search including various LOM elements, language, and regexp

SELECT ?context ?uri ?title ?desc WHERE {
  GRAPH ?context {
    ?uri dcterms:title ?title ;
      dcterms:audience lomvoc:IntendedEndUserRole-learner ;
      lom:copyrightAndOtherRestrictions "false"^^xsd:boolean ;
      lom:status lomvoc:Status-final ;
      dcterms:description ?b .
      ?b rdf:value ?desc .
    FILTER(
      langMatches(lang(?title),"en") &&
      langMatches(lang(?desc),"en") &&
      regex(str(?title), "^organic", "i")
    )
  }
}

I’ll probably post some more examples soon, I provided the examples above only to get the interested ones started. I also hope to be able to upgrade to SPARQL 1.1 soon (the Sesame guys are working on it; looking forward to the next release!), which provides a more complex query syntax/logic.

Long time no blog post

It feels as if it is time again to start blogging, there are some things to write about which do not fit into the traditional scientific publication format. I guess blog posts are an appropriate complementary medium. I moved my blog from electricbum.wordpress.com (including all old posts) over to this one and will only post here in the future.

Stay tuned for some upcoming blog posts on Linked Data in the fields of Educational Metadata and Sustainability (mainly environmental).

Host Networking with VirtualBox 2.1

Finally. The new version of Sun’s VirtualBox makes the fiddling with tunnels obsolete. It basically works the same way as in VMware now.

New release of Java 1.5

Sun released Java 1.5.0_17 today. This release fixes (among other things) the regression bug of Java 1.5.0_16 which made it impossible to view and edit metadata in Conzilla. I wrote about this in a previous blog post.

Generally I recommend to use Java 1.6, but if you have to use 1.5 for some reason you can get the new version directly from Sun.

Sun xVM Host Interface Networking on Ubuntu

I just struggled a bit with getting xVM’s host interface networking up and running on Ubuntu. It works now and maybe somebody else finds the following short how to useful.

Host Setup

Run “sudo aptitude install bridge-utils”.

Add the following lines to /etc/network/interfaces:

auto tap0
iface tap0 inet manual
  tunctl_user user # replace "user" with the user name running xVM
  uml_proxy_arp sativex-ub

auto br0
iface br0 inet static
  address 10.10.1.1
  netmask 255.255.255.0
  bridge_ports tap0
  bridge_maxwait 0

Run “service networking restart”. (or “/etc/init.d/networking restart” if your Ubuntu doesn’t have the service command yet)

Run “sudo VBoxAddIF vbox0 user br0″. (like above, change “user” to your xVM user)

In your VM settings choose “Attached to: Host Interface”, as “Interface Name” in the field below enter “vbox0″. If you want to have Internet access in your VM you should also configure a second network interface, pick “Attached to: NAT” there.

VM Setup

Start up your VM, and add the following lines to /etc/network/interfaces:

auto eth0
iface eth0 inet static
  address 10.10.1.2
  netmask 255.255.255.0

auto eth1
iface eth1 inet dhcp

Run “service networking restart”.

That’s it, your VM should now be accessible (from your host) via 10.10.1.2. Of course you can adjust the IP addresses/networks as you like.

WOD-PD 2008 | Web of Data Practitioners Days

On Wednesday and Thursday I attended the Web of Data Practitioners Days (WOD-PD 2008) at “Altes AKH” in Vienna. AFAIK it was the first event of such a kind, it was something in between academic presentations and hands-on sessions, everything about Semantic Web and its related technologies.

I liked this format of a workshop very much, especially the “hacking session” in the evening of the first day with its “lightning talks”, and of course the socializing in between the talks. I finally had the chance to talk to some guys I already knew from various blogs and publications, but never have met in real life.

The Semantic Web Company (an austrian company who is pushing the Semantic Web topic really hard forward) was represented by Jana Herwig and published a few blog posts summarizing the talks of the event:

• Web of Data Practitioners Days, 1st Session: Tweaking Turtles
• Semantic Desktop, Lifting and Human Language Technology
• Bringing (Legacy) Data to the Web
• Session 4: Using the Web of Data
• Multimedia in the Web of Data – Annotating and Interlinking Photos, Music, Multimedia

The proceedings are online with almost all presentations and links to the web sites which were used for the hands-on sessions.

Oh yes, of course there are pictures online on Flickr tagged with wodpd2008.