A 2007 overview of JISC projects in terms of openness

by Ramón Casero Cañas on 7 June 2007 , last updated 1 February 2010

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

This report summarises the review of 104 JISC-funded projects that took place from 6 March to 1 April 2007. While its initial goal was to simply identify projects that produce open source software, from an early stage it became evident that it would be interesting to assess “openness” in a wider sense, and identify potential open communities and points of collaboration between projects.

This has provided OSS Watch with an opportunity to evaluate the extent to which open source and open community practices can be found in JISC funded projects. This helps OSS Watch to focus its support efforts.

Methodology

The JISC list of projects sorts all active JISC-funded projects into alphabetical order. Each entry has a short description of the project and a link to a “JISC project page” with a general description of its background, goals, deliverables, team, stakeholders, contact information etc. While the content varies in the level of detail, sectioning, and type of information, all project pages have a similar appearance and structure up to a certain point. Automatic extraction of information would probably be feasible, but the level of detail so obtained would be insufficient for most practical purposes.

Most projects also have an external link to a “project website”. The collection of project websites is completely heterogeneous. It often provides more in-depth information about the project, but this information cannot be compiled in an automated way. Evaluating projects in this way is tedious and time-consuming.

These difficulties are being addressed by the open source project Simal, that aims to provide tools for building project catalogues in an automated way. Simal was started at OSS Watch following open development and open community guidelines, and its requirements are partly informed by this study.

For the present review, each JISC project page and external project website were browsed, and notes were taken regarding the following categories:

Governance and community: Is the project carried out by a single team/institution? Are several institutions collaborating? Is it an open project where anybody can join?
Visibility: Does the project have a website? Is it good? Is it possible to contact the project team?
Deliverables: What is the project going to deliver? Software? Data? Documentation? Web services? Under what conditions or licence?
Standards: What standards are projects using to encode, store, and transmit data and resources?
Relationship between projects: Is the project using outputs from or collaborating with other projects (JISC or not)? In some cases, there is a fine line between partnership and third party. Partnership is more about teams of people working for the same project, and belongs in the “Governance and community section”. Third party is more about taking on the results from a previous project, being collaborators to an open source project led by another group, or using a piece of software that is developed externally.

The categories above may seem too broad, and in fact they are so by design, as this study started without much previous knowledge of the wide range of types of JISC projects, or what issues regarding “openness” they would raise.

But thanks to this study we have been able to learn how to better characterise JISC projects. Combined with the experience gained doing the review of online survey tools, we are producing templates for the review of JISC projects that will allow future studies to be more focused.

Governance and community

Projects were classified into 3 categories according to the type of teams working in them. This is a rough classification, as it is very difficult to learn from project websites who is actually doing what for each project.

Single institution or team: A small closed team of people, usually from a single lab or office in an institution.
Closed community projects: Teams split between different institutions working on the same project; only team members can contribute to the project. Drawing the line between this category and the former has been almost random sometimes. Many projects nominally have a lead institution plus partner institutions, but this says nothing about the degree of involvement of each party. I have the feeling that in some cases, partners are active contributors to the project, and thus belong in this category, but in others they may be institutions where the project deliverables are tested.
Open community projects: While all JISC projects have a team leader and team members, projects in this category allow or even encourage input from outside collaborators, in particular direct involvement of the user.

Of the open community projects, none is truly open in governance, as all project teams are closed. The JISC Guide to Bidding states: “Do not underestimate the problems in recruiting suitable staff to work on the project. Staff with suitable qualifications in areas where the JISC is interested can be in short supply or expensive. If you have appropriate people to be seconded to the work this should be stated in your bid. Alternatively you should provide contingency plans in the event that you experience problems with recruitment.”

The responsibility of outcomes (“value for money” ) is placed on the bidders, who have limited budget and time to achieve the objectives of the project. Therefore, all JISC projects can be expected to have decided on the project team beforehand, and be unwilling to move to an open governance model.

An argument that could support open governance is better sustainability, but this is only tangentially mentioned in the Guide to Bidding “Comment on sustainability issues when project funding ceases;”

and the review of projects shows that it is not a big concern in practice.

In addition, open governance seems counterintuitive for some projects that just aim at producing some data or installing an in-house system, because it is understood that the data or system will be maintained within the institution, so only employees shall be granted administrative access and information.

Therefore, lack of open governance is arguably a result of both JISC’s bidding process and project idiosyncrasy.

But while full open governance can be unrealistic or even inapplicable, many projects could benefit from becoming involved with existing communities or creating new open communities where none currently exist. This is particularly true for projects

that produce academic output, e.g. Virtual Learning Objects (VLOs)
with software outcomes
that produce documentation (HowTo papers, evaluations, requirements…) on the implementation and usage of third-party software, e.g. Virtual Learning Environments (VLEs), Virtual Research Environments (VREs), ePortfolios, etc.

The main benefit of creating open communities for JISC projects would arguably be sustainability, mostly survival of the project once the JISC funding has ended. For a wider discussion on sustainability, see Sustainable Open Source and the JISC sustainability study.

Visibility

Visibility is both a consequence of and a requisite for project openness. Specific issues of visibility, like access to source code and public documentation will be dealt with in the following sections. This section focuses on project websites and contact channels.

There are several types of JISC projects in terms of websites:

Very good quality in terms of layout and content, but not specific to the project. For example, projects that link to the Archival Sound Recordings website, copyright of The British Library Board, or to Histpop, The Online Historical Population Reports website, copyright of the University of Essex
Good websites rich in information, offering project resources, directly or indirectly. They offer a good overview of the project, contact channels, the tools used and developed, documentation and links to related projects. They sometimes do not look sophisticated (mostly plain HTML), but provide useful content, links and a development version of the system
Good looking site, but quite empty of content. They provide no access to learning materials, no information about the evolution of the project, and almost no documentation. Sometimes the project website is a copy of the JISC project page, with no extra content
Minimal website. Websites with only a presentation paragraph
Half built websites with, for example, blogs that need to be configured
No website
The link to the website is broken

Websites are the main entry point for any potential user/contributor to the project, and the lack thereof nips the building of communities in the bud because projects are simply invisible to the outside world. The same can be said about contact channels; without the means to get in touch with project members, no community can be built.

Contact channels come in different shapes and sizes. Almost all JISC projects provide at least a contact address for the project manager, and quite often to other team members, but in order to build communities, projects need to provide means for users and collaborators to communicate with each other too (mailing lists, forums, wikis, etc).

Finally, websites and contact channels are not only entry points to the project, but the living memory of the project and the core around which the community is built. Therefore, it is paramount that they are set up right at the beginning and frequently updated.

Deliverables

JISC projects belong to different programmes and cover a wide range of aims and goals, and consequently produce a variety of outputs. This section identifies the main types of deliverables (each type identifies potential communities) and the features that can make them more or less open.

Digitisation of documents and/or creation of digital data libraries

These projects aim at building collections of digital data enriched with metadata from different sources:

scanning and OCR of paper pamphlets, books and journals, cartoons, documents of historic interest, newspapers, theses
art works
sound recordings
images
theatre resources: playbills, programmes, press cuttings, photographs
video tape
combinations of images, text, audio and video

Shibboleth

Shibboleth is an architecture for (and open-source implementation of) federated identity-based authentication and authorization infrastructure. There seems to be a trend towards using Shibboleth’s Web Single SignOn (SSO) to manage access to UK academic resources. The “SDSS: Shibboleth Development and Support Services” project aimed to build a federation of institutions, but it has been now absorbed into the UK Access Management Federation. Shibboleth is open source software (Apache 2.0) developed by the Internet2/MACE community. The specific contexts that JISC projects address for the use of Shibboleth are:

HE institutions
digital repositories
geospatial services

eLearning

Given that one of JISC’s focus is ICT in education, it is only natural that a large quantity of projects engage with eLearning systems. There is a whole vocabulary around eLearning:

Virtual Learning Environments (VLEs), Learning Objects (LOs), Units of Learning (UoLs), learning nugget: VLEs are computing systems that allow students to follow an online course. They consist of LOs, UoLs or learning nuggets, i.e. the course materials. Examples of LOs, UoLs or nuggets include papers, flash presentations, multi-choice online tests, slides etc.
Learning Design: The patterns or recipes that describe learning activities, i.e. how students learn (e.g. by reading a paper, holding a discussion over it and sitting a test). Learning Design is a general term, but it often refers in particular to the IMS Learning Design Specification, a standard model originally developed at the Open University of the Netherlands (OUNL)
blended learning: Combining different methodologies for teaching a course, e.g. face to face seminars with online resources
eAssessment: Systems and materials to allow teachers and students themselves to evaluate their progress in a course. ePortfolios, Personal Development Planning (PDP): Computer systems to create a demonstration of the user’s skills and knowledge. They may include notes, blogs, images, pdf files, multimedia etc.
course evaluation: Assessment of the course quality done by the students

Projects that aim to provide a web service in one or more institutions are quite closed in terms of the information they offer. There is a significant use of open source solutions, but they remain mixed with proprietary software.

Examples of software used or considered for use by JISC projects for eLearning are:

Moodle: An open source (GPL) VLE used to create online courses. Development is led by the Moodle company, Australia
LAMS: An open source (GPL) tool for the design of learning activities (http://www.lamsfoundation.org/). Development is led by Macquarie University, Australia
DialogPlus Toolkit: An online environment to create, modify and share learning activities and resources. Everybody can use it, but the source code or binaries are not available
RELOAD: Collection of open source (MIT licence) tools for packaging content (IMS CP), editing IMS Metadata, playing SCORM packages and editing and playing IMS Learning Designs. One of the projects aims to develop a plugin for to connect RELOAD to the UoLs repository OpenDocument.net (not online yet)
WCKER: Open source (MIT licence) plugin to create wizards for creation of online courses in RELOAD
Blackboard/WebCT: A suit of proprietary eLearning applications
Bodington: An open source (Apache License 2.0) VLE developed at the University of Leeds
SPIDER: Proprietary VLE developed by the University of Strathclyde
RLO-CETL Evaluation Toolkit: A questionnaire (CC attribution noncommercial sharealike) to evaluate LOs
Blogsom: an open source (BSD licence) multi-blog system
TikiWiki: Open source (LGPL) CMS
SIMPLE: Gaming technology being developed for teaching courses
WebPA: An online peer moderated marking system in development that will be released under the GPL
Xmarks: Middleware to link the staff and students record database (Oracle 9i) with Moodle
PebblePad: A proprietary ePortfolio system
Progress: A proprietary PDP service developed at Canterbury Christ Church University
‘Generic’ ePortfolio: Developed at Newcastle University to provide a baseline open source ePortfolio that can be customized for specific institutions. I could not find information about the licence, or the source code
ePET: A web service interface for the ‘Generic’ ePortfolio
ePARs: A proprietary PDP system developed at the University of Nottingham, and based on Microsoft products
LUSID: A proprietary PDP system developed at the University of Liverpool
Reflective Learning Diary: A proprietary PDP system developed at the University of Southampton
PETAL: In theory, an open source general e-portfolio tool Open Source Portfolio Initiative (OSPI) and the Certified Member of ALT scheme (CMALT). Development finished in March 2005, but there are no source code or binaries available
L4All: An ePortfolio Portal; I could not find information about the licence, source code or a link to the portal
ioPortal: A web application for PDP. I could not find any link to the source code or the binaries
ioNode: Middleware for exchange of Individual Learner Record data. I could not find any link to the source code or the binaries
EFSCE: e-Framework Services for Course Evaluation. Proprietary system developed by the University of Southampton
Horus: An open source (LGPL) course evaluation system, that is being deprecated

Virtual Research Environments (VREs)

Another focus of the JISC in education is research. VREs are “frameworks that offer tools, services and resources for research”. The JISC has 15 projects in the Virtual Research Environments Programme; at the time of this review, only 3 were active, but a further £2 million have been allocated until March 2009 for this programme [http://www.jisc.ac.uk/whatwedo/programmes/vre1].

There is little information on how the VREs are set up, especially in terms of hardware. Software includes a mix of open source and proprietary software, but there is not a clear use of open source development techniques.

The projects cover:

Support for the Virtual Research Environment Programme, in the sense of reviewing tools, “information on interoperability and associated standards and specifications”
A VRE for the Integrative Biology (IB) Research Consortium, in particular the user community’s requirements in the areas of graphical user interfaces and collaborative tools
A VRE for an MA in the History of Political Discourse 1500-1800, based on the Sakai Collaborative Environment [http://www.sakaiproject.org/]. This MA provided the “excuse” to build the VRE that is now used in the broader context of Early Modern Texts [http://www.earlymoderntexts.org/vre/]

Digital repositories

A digital repository is a collection of data or publications in digital format. The JISC has a specific programme for digital repositories (pdf link).

Projects in this category offer little or no information about the hardware they use, and those that provide a web service remain quite closed. However, there is a significant number of projects that have considered and/or are using open source solutions.

Projects reviewed included:

LOs
Statistical data of downloads from repositories
Policy, infrastructure, legal studies, DRM, hardware procurement
Search engines for engineering resources, institutional repositories
Development of digital repositories
Integration of research data with corresponding publications. Experimental physics, UK Data Archive
Implementation of digital repositories software, e.g. FEDORA, GNU ePrints, LionShare
Add features to digital repositories, e.g. tagging
Automated curation of robot generated data
Peer-reviewed papers
Shared clinical high stakes assessment

Products used or explored by these projects include:

GNU ePrints: An open source (GPL) digital repository system initially developed by the University of Southampton. It is a very mature open source project closely involved with Open Access (OA) and offers consultation, installation and support services
Curator: A proprietary digital repository system. I could not find much information online. Apparently it was developed by Endeavor Information Systems, part of the Ex Libris group
Digitool: A proprietary digital repository system also developed by the Ex Libris group
Fedora: An open source (Educational Community License) digital repository system
DSpace: An open source (BSD licence) digital repository system initially developed by the MIT Libraries and Hewlett-Packard (HP)
Greenstone: An open source (GPL) digital library system developed by the University of Waikato in collaboration with UNESCO and the Human Info NGO
PerX: A search engine pilot for engineering resources
LionShare: An open source (GPL) digital repository system based on P2P. Only registered users can download the binaries (source code available for everybody), and the system will only be used by registered users at partner universities of the “SPIRE” project
JHOVE: An open source (LGPL) framework for automatic identification and validation of digital content. “RepoMMan: Repository Metadata and Management”
ActiveWebflow Professional: A proprietary plugin for Eclipse to write Business Process Execution Language (BPEL)
The Depot: A UK digital repository for peer-reviewed papers, articles, and book chapters, developed by “The Depot” project and based on ePrints
Jorum: An online digital repository service for learning and teaching materials developed as a JISC project. The software behind the service is proprietary and it does not seem to be possible to obtain it, although access to the service is free
Contextual Learning Activity Repository (CLARe): A service developed at the University of Southampton. Proprietary software, and there seems to be no public access to the service either

It is also worth mentioning Open Access (OA), an initiative for immediate, free and unrestricted online access to digital scholarly material that is in the focus of several digital repository projects. This is the scientific publication equivalent to open source in software, although there are some differences. For example, in scientific literature peer-review is usually enforced before publication, and the ability to reuse and modify source code is a main characteristic of open source, but not a big concern in scientific papers.

The “IRIScotland” project concluded that populating repositories now presents a greater challenge than setting them up, and that there are 3 factors that hinder making them actually useful in Scotland [http://www.iriscotland.lib.ed.ac.uk/about/index.html]:

Lack of policies, procedures and mechanisms for OA in institutions
Lack of a wide institutional digital repository infrastructure
Lack of understanding of what functions should be undertaken at an institution level, and at a national level

Technical support

Some projects aim at providing technical support for other projects. This review found 2 instances:

Pedagogic and technical support to all projects in the Design for Learning Programme, hosting and/or searching services for learning designs
Provide a Shibboleth infrastructure for projects supported by the Core Middleware Development programmes

Software

The main reason for this category, that is quite wide in scope and overlaps with others, is to have a starting point to consider projects that are potentially interested in OSS Watch advisory services.

Projects were grouped in 3 subcategories:

Proprietary

When there is evidence of a commercial licence, or lack of evidence of an open source licence, combined with a lack of open development practices:

A web service is publicly available, but not the source code, binaries, and there is no indication of what licence they plan to use, if any
Binaries are provided, but no source code with an open source licence
Development is behind closed doors, and there is no indication of the type of licence they will use if they decide to release the software

Semi-open

When the project aims to produce open source software, but they are lacking a licence, or follow proprietary software development methods:

The project has manifested the intention to use open development and an open source licence, but has failed for the moment to do so
Source code is available, but there is no licence information
Source code is available, and an open source-like licence provided, but it is not an OSI-approved licence
There is no stated intention to develop open source software, no available binaries, source code or licence information, but the project builds on top of software with a viral licence (e.g. the GPL); thus the project will be forced to release their software under the same licence

Open

When the project produces open source software following open development methods: InterLoc

An open source (Apache v2.0, LGPL and Mozilla) groupware tool for educational dialogues. The source code is available from SourceForge [http://sourceforge.net/projects/interloc]. Based on jabber

D4LD

A front-end user interface to run Learning Designs. Builds on 2 previous open source projects, SleD (LGPL) and the CopperCore LD Engine (GPL) and improves them, also doing integration with Moodle

LauLima

An open source (LGPL) customised TikiWiki-based system comprising a learning environment and digital library

Integrative Biology VRE

An open source (BSD licence) VRE for the Integrative Biology project. The source code is available from SourceForge [http://sourceforge.net/projects/ibvre/]

Some tools used for software development:

Trac: An advanced wiki and issue tracking system with a modified BSD licence
Mantis: An open source (GPL) bug tracking system
MediaWiki: An open source (GPL) Wiki system
Joomla!: An open source (GPL) CMS
SourceForge: A proprietary collaborative development environment
LISTSERV: A proprietary mailing list server
Jabber: A set of free XML protocols used for instant messaging
WordPress: An open source blogging system
Confluence: A proprietary wiki system

National Centre

One project in the review dealt with building a new “National Centre for Text Mining”. Of the reviewed projects, this is one of the most ambitious in terms of the broad scope of its deliverables (software tools, services, documentation, training) and sustainability. It is important for OSS Watch to acknowledge the complexity of this kind of project when providing support and developing tools like Simal.

Data

As well as software or infrastructure, several projects produce data (documentation will not be considered data in this section). The availability and conditions under which data is released reflect on the openness of the project. Similarly to software, project outputs can be grouped in 3 categories:

Semi-open data: Data with some access restrictions, for instance only academic institutions
Open data: Data publicly available, for example through OA policies

Some limitations found in proprietary data:

It is necessary to ask the copyright holder for permission to reproduce, display, store, etc. the material
Only thumbnails of pictures are provided; to obtain full size reproductions it is necessary to contact the picture library
Materials only available for the college where the project takes place

Characteristics of semi-open data:

Materials are publicly available, but I could not find information about the licence
- icons
- video
- video and audio
- maps and georeferenced data
Materials available for private, research and student use
- Only UK FE and HE licensed institutions can download and play the recordings
- Free registration access to database is required. Copyright grants private or student use, but not publication
- Crown copyright, material can be downloaded from DocumentsOnline, and reproduced, but there is a fee for images, and use is only for research, non-commercial or education
- VLOs only available for students
Materials freely available within the UK only
Materials available only or mostly in proprietary formats
The project shows interest in OA, but there is no data and/or enough details
- laboratory robot data
- theses
- images and documents of everyday life in the UK
- drawings
No available information on how data will be released

Finally, for open data:

Public access through JSTOR and Google Scholar search
OA content; the website with the project outcomes has publicly available data

Documentation

Projects produce documentation for many reasons: To describe the project as a condition of the bid for funding, as deliverables of projects that perform reviews or produce standards or policies, to document software application or network architectures, etc.

Availability and licensing of this documentation is important in terms of openness. JISC projects typically offer online documentation, but do not release it under copyleft licences.

This section follows a similar classification of projects as the “Data” section.

Proprietary documentation:

Username and password required to access documentation, and no public registration
No website, or website with no documentation
Website provides little documentation, and no licence information

Semi-open documentation:

Some documentation is available, but it cannot be redistributed
Public documentation with open formats, but there is no licence information. It is interesting to comment on the “RepoMMan: Repository Metadata and Management” project. They investigated the development of a workflow tool for the Fedora open-source repository software. As they found no proper documentation, they wrote their own manual and made it publicly available
Public documentation using proprietary formats
Public documentation in a mix of open and proprietary formats
- PDF and MS Office
- Scientific journals
Public wiki but no registration possible
Promise of making contents “open” in the future

Open documentation:

Public wiki that anyone can edit, although there is no licence information
Public documentation that anyone can edit, with a Creative Commons licence

Standards

A measure of openness for projects is the use of standards. Standards avoid vendor lock-ins, improve interoperability and sustainability. The standards most often used by JISC projects are:

Metadata Encoding Transmission Standard (METS): An XML standard for encoding descriptive, administrative, and structural metadata regarding objects within a digital library
eXchanging Course-Related Information (XCRI): A vocabulary and appropriate technology bindings (e.g. XML, RDF) to describe UK course-related information that encompasses course marketing, course quality assurance, enrolment and reporting requirements
Open Archive Initiative (OAI): Interoperability standards for dissemination of content
L2O metadata scheme: Standard for metadata in language skills LOs
MathML: A W3C ‘low-level’ specification for describing mathematics as a basis for machine to machine communication’
IMS: ‘Open technical specifications for interoperable learning technology’ defined by the IMS Global Learning Consortium, Inc. For example, ‘the IMS Question & Test Interoperability (QTI) specification describes a data model for the representation of question (assessmentItem) and test (assessmentTest) data and their corresponding results reports’ [http://www.imsglobal.org/question/]
SCORM: A collection of standards and specifications for ‘interoperability, accessibility and reusability of Web-based learning content’
Chemical Markup Language (CML): An SGML-based standard to manage molecular information
Business Process Execution Language (BPEL): A language for the formal specification of business processes and business interaction protocols
Security Assertion Markup Language (SAML): An XML standard for exchanging authentication and authorization data between security domains developed by the OASIS Security Services Technical Committee and used by Shibboleth

Relationship between projects

There are strong connections between JISC and non-JISC projects, present and past. These relationships take many different forms, e.g.

Providing support
Providing sustainability
Adding features or functionality
Combining projects
Using outputs from other projects
Creating open source software versions of proprietary software products
Continuation of previous projects
Massively collaborative projects
A JISC service manages and works for a JISC project

This not only points out that JISC projects can naturally build communities, but it is of particular importance to OSS Watch when considering how to describe and make catalogues of projects.

Conclusions

While the only condition for software to be considered open source by OSS Watch is that it is released under an OSI certified licence, the openness of a JISC project needs to be evaluated more widely.

This review has identified several categories of interest for openness: Governance and community, visibility, deliverables, standards and relationship between projects, and found out that many JISC projects are either following proprietary or semi-open practices, but there is noteworthy interest in the usage of open source products, open standards and documentation too.

Moreover, the ability of a project to integrate within and/or build a community is vital in terms of sustainability.

OSS Watch can help existing and new projects identify and work with their communities of interest as well as assisting with the engagement of openness in each category described above.

Regarding the methodology, this study had to be made visiting each project website one by one, a very time consuming process that does not allow for good quantitative analysis. But it is to be expected that Simal, the new project started at OSS Watch to provide catalogue building tools will greatly improve future reviews.