The agenda for all expert workshops is settled on the basis of input from network members. An initial list of issues to give priority was established in the first strategic seminar in Naples, June 2007.
The three issues with highest priority for the first workshop in the series concerning Engagement of producers and users were:

• Ensuring ease of use (usability, wizards, help and support)
• Web 2.0 and repositories? (shared queries, social bookmarking, tagging…)
• My repository – the need for personalization (profiles, reviews, collections…)
  

These rather broad topics resulted in an agenda, with a focus on the user interface of repositories. At the first strategic seminar network members stressed the importance of interactivity during workshops. A number of groups sessions were therefore devoted to exploring common design patterns of educational repositories. To narrow down the issues to be further discussed and explored during group sessions participants were asked to fill in a survey prioer to the workshop. The results of this survey were used to suggest the initial patterns to include in the draft repository design pattern catalog worked in during group sessions.

 

As this was the very first workshop within the network it also served as a testbed for finding the best form and organization method of future workshops. One clear conclusion from the evaluation session was that discussions and interactivity within smaller groups was a very important component.
Workshop participants confirmed the need and value of having peers to discuss these issues with and being able to connect directly with projects/repositories in other countries currently dealing with the same developments or already having experiences in a particular area. This conclusion will be important for setting agendas for future workshops. At a later stage of the project it should however also produce more tangible results.

As a supplement to the group discussions members indicated that use of for example survey tools could help provide an overview of current status/implementation of different functionality.

Discussions in connection to presentations also gave input for issues to be further explored in upcoming workshops:

• (Central) services for curriculum mapping / vocabulary management - a mapping of status and strategies
• Community building in connection with educational repositories as a strong incentive for educators to share materials (more examples needed)
• Clear and unambigous presentation of usage rigths, and advice to licensors at the time of depositing (workshops on rights issues)
• Share and remix between professionally produced content and that produced by educators (workshops on rights issues)

 

Tuesday, October 2nd 2007
09.00	Welcome to UNI•C, Aarhus (Tommy Byskov Lund, UNI•C)
09.15	Presentation and discussion of EdReNe web site and Members Zone (it’s learning) (Tommy Byskov Lund, UNI•C)
09.45	Ensuring ease of use – identifying repository design patterns  (Julia Gardner, UNI•C)
10.30	Coffe break
10.45	Repository patterns: Group work session 1 Introduction, UNI•C EdReNe members
13.00	Lunch
14.00	Repository patterns: Group work session 2 Introduction, UNI•C EdReNe members
16.00	Coffe break
16.15	Show and tell session - Lessons learned: Vocabulary Management Service (Will Ellis, Becta) The need for repository editors and possible solutions (Iztok Kavlker, UNI-LJ-FMF) Screening teacher material - lessons on metadata quality (Annette Graae, UNI•C) EAPC - new EdReNe member (Orland Cardona, EACP) Analysis of browse and search strategies of teachers (Riina Vuorikari, EUN)
18.00	First day finishes
20.00	EdReNe dinner at restaurant L'escale in Aarhus

Wednesday, October 3rd 2007
09.45	Administrative issues and questions (Leo Højsholt-Poulsen, UNI•C EdReNe members)
09.45	Coffee break
10.00	Show and tell session: Teachers’ Multilingual Tagging behaviour (Riina Vuorikari, EUN) Drupal.org as a framework for publishing and meta-tagging of learning resources (Trond Hanssen, utdanning.no) Vetamix - a new Finnish repository (Christian Komonen, YLE) Online content metadata generation (Christine Champion-Bibard, CNDP)
11.45	Repository Patterns: Group work session 3 (EdReNe members)
12.30	Pattern presentation and question session (EdReNe members)
13.00	Lunch
14.00	lektion.se - Swedish teachers sharing lesson plans  (Per Malkert, lektion.se)
14.45	Pattern presentation and question session, continued  (EdReNe members)
16.00	Coffe break
16.30	Workshop evaluation – Summing up and input for next workshops (Tommy Byskov Lund, UNI•C)
17.30	Workshop finishes

The workshop started with a brief introduction of how the agenda came about - mentioning both input from the strategic seminar in Naples and the pre-workshop survey filled in by most of the participants. In addition there was an introduction to the EdRene Website including the Members Zone - especially the tools and functionalities that were to be used intensively during this workshop.

• Download presentation
    
  

Will Ellis, Becta, did an online demonstration of the Vocabulary Management Services described below. This was followed by discussions on how curriculum mapping of learning content currently was and could be approached in the future in other countries. No consensus was evident from current initiatives, and a conclusion was that a deeper analysis of the situation in individual countries was worth exploring. 

Description of Becta's VMS

Becta’s Vocabulary Management Services have been in operation since mid-2006. The services are used by stakeholder agencies in UK education which operate and use controlled vocabularies, and by industry which uses those vocabularies to tag digital learning content so that it can be discovered by those who need to use it. Controlled vocabularies are also used within learning platforms, portals and repositories. The core aims of the services are to foster a common vocabulary throughout UK education, both technically and semantically. The services were introduced to address significant problems with quality, inconsistency and lack of cohesion in the area of controlled vocabularies resulting in inaccuracy and inappropriateness in content discovery.

The Vocabulary Management Services (VMS) comprise two core elements – Studio and Bank. Studio is an online tool which is used in the development and management of controlled vocabularies and their export in Zthes version 1.0 XML format. Its unique feature is a central Spine of Concepts which can be used to map terms within and between vocabularies. Bank is an online vocabulary repository, to which Studio publishes vocabularies. Bank can be used to browse, compare and download stored vocabularies: through its web services, systems can directly connect to Bank and interrogate its contents. This latter aspect is critical in the drive towards distributed search and retrieval of content.

The Bank currently provides access to over 45 controlled vocabularies, and this service was successfully launched to industry in October 2006.
Brief history of Vocabulary Management Services

Becta commissioned the Vocabulary Management Group (“Vocman”) in late 2004 to design and develop the Studio application. It was originally conceived as a tool to support the development and management of controlled vocabularies used in education, with the ability to map between and within vocabularies, and as a way of introducing a standard technical specification for published vocabularies. Its aim was to make vocabularies available through a single point of entry for use in the tagging of digital content for educational use.

In June 2005, a formal launch of Studio was held with key stakeholders in the education and culture sectors. QCA was seen as the key stakeholder, without which the Studio and the wider VMS would hold little value. The initial focus of the efforts was therefore QCA and the National Curriculum Vocabularies, while at the same time, undertaking a rolling programme of Studio demonstrations and presentations. Since this point there has been an ongoing strategy to engage with the content industry and curriculum authorities to coordinate the development of the service.

Benefits

The existence of the VMS brings many tangible benefits to stakeholders, industry and, most importantly, those that need to search for and discover relevant and accurate content. Some of the key benefits are listed here:

Benefits to Stakeholders

• For the first time, a common tool used through the education sector for the management of vocabularies, with visibility of other authorities’ vocabularies.
• Automation of many traditional “hand made” processes
• Consistent technical and quality assurance
• Security
• Reduced risk of duplication and / or error
• Ability to publish immediately to one single place
• Ability to map vocabularies to other vocabularies through the Spine
• Fully supported
• A free service

Benefits to Industry

• One single place where all significant controlled vocabularies can be found
• Assurance of version management
• Consistency of technical format
• Ability to resolve terms
• Ability to download and use immediately
• Up to date information available to all
• Vocabularies mapped to one another via the Spine: tag once, classify many times

Benefits to content users

• Mapped vocabularies will allow the discovery of content tagged to more than one vocabulary
• Content discovery will become more accurate and appropriate to needs
  

Relevant links:

• Becta's Industry and developers' site 
• Vocabulary Management Group 

Iztok Kavlker, UNI-LJ-FMF, Slovenia led a presentation of strategies to ensure a sufficient level of metadata quality and update rates. A suggested model of user level based checking is presented as a strategy to finding a good balance between metadata quality and necessary resources for central metadata editing by experts.

Presentation summary

One of the most difficult problems of the Slovene national repository (SIO) -- and the publicly maintained repositories in general -- is the problem of metadata quality and the problem of resource appropriateness. The repository without a constant supervision tends to become an attic of many good resources lost in the sea of useless ones, with insufficient metadata to tell them apart and find what you need.
It appears that constant screening of inserted metadata is needed to achieve the desired quality. Why not? Every library with about 5000 books has a librarian, educated in the craft of maintaining book catalog entries. Repositories usually contain much more resources than that, especially when resources are divided in small, reusable units.
However, in the case of non-commercial repositories it seems to be very difficult to provide permanent financing for the adequate number of repository supervisors. So far, we had a practice of occasional complete checks of the repositories. This assured quality to some extent but there were a number of difficulties:
- newly entered data had to wait to be approved or become public without any screening
- it was hard to maintain a consistent policy across several checks
- most likely it wouldn’t work at all with a larger repository
Based on the experience, the new SIO will use a quality assurance scheme, based on the level of user trust. Initially there will be four levels of users, but this may change if the future experience indicates the need:
- basic user: all additions / changes to the repository must be approved by an editor
- trusted user: can add / modify (his own?) entries without additional approval
- editor: approves changes and additions
- administrator: upholds the repository policy, promotes (and if needed demotes) users
Note that in this context, trust means mostly “knowledge of repository policy”. After a number of successful contributions of a user, the repository notifies the administrator that she is eligible for promotion. If the user agrees, she obtains the new responsibility. When a user becomes an editor, she is assigned a field (or more fields), based on her own entries; only contributions from the assigned fields are sent to her for approval. As the number of editors will be small at the beginning, we plan to start with a small number of fields (<10), increasing the number once there is a lot of repository activity.
We believe this scheme guarantees reasonable level of quality, without overburdening the administrator. A single administrator should be enough for a perceived Slovene sized repository.
Unfortunately, there are possible problems. Above all, this scheme puts the majority of the burden on the editors. Even though they are volunteers, they have to screen data carefully and regularly, otherwise the contributions get stalled. Experience we have with our faculty’s wiki pages show that there are people willing to assume such responsibility. Moreover, we plan to assign each field to several editors, with only one opinion needed to accept a change (if there are several different opinions, the administrator has the right of arbitration). In this way, the temporary unavailability of one editor doesn’t stall the repository.

Relevant links:

• Download presentation
  

The interactive group sessions focused on repository user interface and in particular web 2.0 aspects relating to this. This was based on the input given from members at the first strategic seminar and in the pre-workshop survey filled in by most participants.

The strategy chosen was to begin building a repository pattern catalog, describing best practice design solutions to common repository features. While work was initiated during the workshop, it is a dynamic collection which will hopefully evolve and expand during the EdReNe project. All members have access to it, and can create new patterns / pattern groups.

The work was split in three parts:

• Introduction to design patterns.The group sessions were introduced by giving an introduction presentation on to design patterns (Julia Gardner, UNI-C)
• Three group sessions: The first session was used for doing initial work on two patterns; the second session evaluated/revisited the pattern the group should present on the second day of the workshop (members from different groups that worked on the same patterns in session 1 now formed a group); third session asked participants to employ a pattern on their own repository and in this way prepare questions to the group responsible for presenting the pattern on day two.
• Pattern presentation and discussion. The draft patterns were presented on the second day of the workshop - with questions from all participants including the group that had tried to employ the ideas sketched in the pattern on their "own" repositories.

Brief introduction to participants on working with design patterns

Working with design patterns is an efficient way of translating the very general rules of interaction design into specific design. Design patterns are particularly relevant to web design because usability is so critical to web applications. A number of existing collections of design patterns can be found on the web, some of the the most known examples being the Welie collection and the Yahoo design patterns. It is important to realize that a design pattern is not a finished design that can be transformed directly into code. It is a description or template for how to solve a problem that can be used in many different situations.

Quoting Brad Appleton, the main benefits of patterns include: 

• They solve problems: Patterns capture solutions, not just abstract principles or strategies.
• Concrete: They resist what seems like needless abstraction
• They represent proven concepts: Patterns capture solutions with a track record, not theories or speculation.
• The solution isn't obvious: Many problem-solving techniques (such as software design paradigms or methods) try to derive solutions from first principles. The best patterns generate a solution to a problem indirectly -- a necessary approach for the most difficult problems of design.
• They describe relationships: Patterns don't just describe modules, but describe deeper system structures and mechanisms.

As a base for establishing a repository pattern catalogue, participants were presented with an initial set of 31 suggestions for patterns, organized in the following pattern groups:

• Navigation
  This pattern group describes multiple ways that support users navigating a repository in order to find resources. Users navigate in many ways and if any key navigation tools are missing or hard to find, the web site will be regarded as hard to use. 
• Collecting resources
  Once a user finds relevant results during searching or browsing a repository the need to share or save these relevant results in some form arises.
• Metadata page
  The metadata page is the page that describes a single resource that users have found either through searching or by browsing. The central goal of the page is to provide an easy overview of information about the resource, and present possibilities to use/interact with the resource. Furthermore social information such as ratings, download popularity etc may be displayed. 
• Repository homepage
  First impressions are important – also for websites. Patterns included in this section offer clues to design elements relevant to present on the entry page of educational repositories. A homepage must satisfy the needs of all potential and current users by establishing the identity of the organisation and at the same time providing multiple ways to navigate. 
• Stay updated
  Maintaining contact with users is a central goal of all websites. Users on the other hand may have a need to stay updated within a specific topic area covered by your web site. A central way to reach business goals and establish a lasting relationship with users is providing services which keep users informed of new and interesting content.
• Metadata editors
  Content providers need to be able to register their resources in a way that makes the resources findable for end-users and includes enough information for the end-user to decide whether the material is relevant. An important aspect is for example linking the resources to the relevant curriculum. The registration process needs to be as simple and quick as possible to encourage content providers to submit their resources and to avoid mistakes in the metadata that might confuse the end-user.
• Profile page / personal settings
  Registered users need to see and manage the information that a web site keeps about them. A repository that lets certain users register resources, need a way to let these users manage personal information which the end-user will be able to read, and manage the resources they have registered.
  Users that have logged in can be offered additional functionality such as default values for searches, rating/reviewing resources, statistics on the use of resources they have provided,

During the group sessions, participants were asked to develop patterns based on a template. Each pattern was discussed/worked upon by two different groups, and all participants worked on at least two patterns. Participants were responsible for presenting “final” patterns during the workshop. Focus was on developing patterns related to web 2.0. A total of 12 of the 31 suggested patterns reached the stage of “final drafts” during the workshop. They are of course still available for members to fine tune and elaborate with new/better examples.The repository pattern catalogue is available to all members in the EdReNe Members Zone.

Relevant links:

• Introductory presentation of design patterns 
• Link to Members Zone (parts of draft pattern catalog) 

Annette Graae, UNI•C, lead a discussion on how to obtain quality metadata from non-specialist editors (educators).

The national repository in Denmark encourages all teachers to share their own material. They register and upload material using the web based tagging tool included in the repository. All teacher developed material is subsequently screened for copyright problems and metadata quality by central editors. The presentation focused on a discussion of the problems connected to finding the balance between a simple and intuitive user interface and obtaining sufficiently good metadata quality for efficient retrieval.

The discussion revealed similar problems/considerations in most repositories, and spurred a debate on how (semi-) automatic metadata creation could help alleviate the need for extensive central editing. Workshop participants expressed the wish, that a future workshop could provide hands-on/practical experiences using automatic metadata tools.
Relevant links:

• Info about Materialeplatformen (Danish national repository)

Orland Cardona did a brief presentation of EAPC - Public Administration School of Catalunya (Spain) and how they expect to contribute to the network. A more thorough presentation of the Merlí project which was briefly described was scheduled for an upcoming workshop.

Relevant links:

• EAPC presentation 
• Merlí project (later supplement to above) 

Riina Vuorikari (EUN - European Schoolnet) did a presentation on the interactions of end users (teachers) with the Calibrate portal based on extensive logging of user behaviour. The Contextual Attention Metadata framework (http://ariadne.cs.kuleuven.ac.be/cama2007/) has been used to generate the data analysed. The presentation is based on data collected in a three month period.

Relevant links:

• Download presentation 

 

A short session devoted to project administrative issues - describing current status and answering questions relating to financial and/or other project related matters.
The presentation s available to network members from the Members Zone.

Riina Vuorikari followed up the presentation on browse and search strategies, by a discussion on mulitilingual tagging. Although social, collaborative classification through tagging has been the focus of recent research, the effect of multilingual tags is often overlooked. This presentation is on an early exploratory study of the production and consumption of multilingual tags in a European educational K-12 context. The data, produced by teachers bookmarking and tagging learning resources during a three month period was analysed. Thereafter, a focus group of teachers evaluated a sample of learning resources with metadata records containing both thesaurus terms and multilingual tags. The results of this early study suggest that some tags are found as useful as thesaurus terms and that users are divided about the benefits of multilingual tags.

Relevant links:

• Download presentation

Trond Hanssen, utdanning.no, Norway presented some of the considerations underlying the choice of using drupal and fedora as the underlying platform for building:

• A national centralised repository with Course descriptions of almost all educations in
  Norway (course description metadata)
• A national centralised repository over learning object metadata
• A framework for learning content publishing

The presentation was done while showing the services online. Read more in the supplied background paper.
Subsequent discussions where focused on pros and cons of open source strategies versus proprietary software solutions. It was agreed that this topic could be pursued at subsequent workshop on repository strategies.

Christian Komonen explained the reasoning behind the design and content of a new educational repository - Vetamix - initiated in cooperation between the Finnish National Board of Education and Finnish Broadcasting Company.
The repository will feature both professionally produced sound and video from the archives (also texts and still images) of the national broadcasting company but will have a strong focus on user generated content and exploring possibilities of sharing and remixing content. An online prototype was demonstrated, illustrating the major features.

Relevant links:

• Vetamix (to launch autumn 2008)
• Presentation of Vetamix (on SlideShare)

Christine Champion-Bibard, CNDP, gave a presentation on prototyping automatic metadata creation of educational content with a focus on designing friendly screens with simplified input procedures and computerized routines allowing to create metadata automatically.

Presentation summary
In order to improve access to educational resources, the use of metadata is needed, but to create accurate metadata is a difficult task, especially for non-specialists of classifying and indexing. While standardisation offers appropriate formats such as LOM application profiles, metadata creation procedures pose problems. Considerable amounts of public and private resources offered for pedagogical usages are produced by teachers, researchers, ICT trainers, administrative staff, public and private publishers. These resources constitute a burgeoning and heterogeneous corpus.
Metadata creators fall into (at leat) two broad and very different groups:
- specialists (such as librarians and information specialists)
- non-specialists (such as teachers, authors, webmasters)

For librarians as for non-specialists of metadata, metadata creation procedures are time consuming. A librarian needs about 15 minutes to fill in a IEEE LOM metadata instance, twice as much for a non-specialist.

Metadata editing tools are offered to fill in corresponding fields with some online help. Some features of these tools could meet the needs of metadata specialists such as indexers but often fail to help content authors, teachers, resource administrators and webmasters. Non-specialists often have difficulties understanding the meaning of some elements. They are also not used to thesaurus and other controlled vocabularies; they are not used to index-link resources.

The success of automatic or semi-automatic metadata generation procedures could permit to gain time and possibly also help both specialists and non-specialists to select/create better metadata.

The French national Educational Resource Centre (CNDP) and CRIS lab of the University of Paris X are engaged in the French project PPRE (Production Platform of Educational Resources) financed by the Paris Region and headed by Editis publishing house. Within this project, the task devoted to CNDP and CRIS lab is to design and develop a friendly tool for authors of online educational content so that they have to enter only few data through one input screen while the rest of the metadata creation procedures is performed at least semi-automatically.

The project aims:

• To define templates based on document types and personal profiles
• To create mapping between elements (for example: technical format to installation remarks; educational context to intended end user role…)
• To extract metadata from file properties and META tags in html documents (Dublin Core)
• To automatically retrieve keywords and classifications using the extraction methods of search engines (Sinequa Search, Exalead) and to compare the automatically created content with data created manually by information professionals.

Relevant links:

• Download presentation 

 

Per Malkert, one of the founders of the succesful swedish repository gave an introduction of this portal initiated by teachers for teachers which currently holds more than 115.000 users, and a rapidly expanding base of shared lesson plans. The presentation was in the form of an online demonstration of lektion.se including background information on some of the features/design choices made during the implementation. The subsequent discussion focused on what was most likely to be the key factors in attracting such a high percentage of teachers to the site. Many issues were touched upon including: timing and lack of competition; low barrier to depositing (few metadata; relaxed quality criteria); strong community part (including necessity to login before use); mixing with other relevant offers (for example job section)
Relevant links:

• lektion.se 
• A study of user behavior on lektion.se – how teachers share knowledge through lektion.se (thesis by Ann-Louise Olausson, KTH, 2008 - in Swedish (summary in English))

The workshop ended with a short informal evaluation session. Discussions centered around the form of the workshop. Participants expressed that interactivity within smaller groups was a very important component.
They also confirmed the need and value of having peers to discuss opreational issues with and being able to connect directly with projects/repositories in other countries currently dealing with the same developments or already having experiences in a particular area.

At a later stage of the project it should however also produce more tangible results. As a supplement to the group discussions members indicated that use of for example survey tools could help provide an overview of current status/implementation of different functionality.

• Download presentation