Excitement at the OR11 Developer Challenge Show-and-Tell (Photo by @sparrowbarley)

An event that asked developers to demonstrate the Future of Repositories can only be considered a great success when it receives entries that include:

• Multiple real-time examples of using “Repositories As A Service (RaaS)”, not only exchanging data but also sharing sophisticated functionality between EPrints and DSpace – and even including an Android application
• A tool for bundling and depositing a whole raft of research related outputs from the Web via RDF
• A tactile repository search interface with dynamic search suggestions, specifically designed for tablets and smartphones
• A complete gesture and voice-driven system for depositing and searching in repositories

All these – and other great entries too – were achieved in a couple of days’ work during the course of the conference, for the annual OR Developer Challenge, and presented at a packed Show-and-Tell session on Thursday afternoon (true, there was free beer).

Stuart Lewis’s team were worthy winners with their RaaS project, particularly as they showed a genuine commitment to a cross-platform approach – something which, sensibly, backgrounds the individual software platforms, that often receive too much attention, and focuses on the Repository as an application and entity in its own right.

We were also really pleased to see a prize go to Patrick McSweeney and Matt Taylor. And enjoyed seeing Dave Tarrant stealing the show (again) with his live demonstration of using a Microsoft Xbox Kinect to submit items to a repository.

Our own entry may not have won, but several people liked it, and you may see more of it in future. For the second year running, the Developer Challenge was a great opportunity for Rory and me to concentrate on an idea that we’ve been kicking around, without having found a home for it in existing work (yet). This was true for our Semantic Metadata popup tools that won the challenge with last year.

This year we set about achieving our  longstanding desire to take the very tactile and dynamic search interface that Rory created for the MERLIN project, and turn it into a touchscreen app, for smartphones and tablets.  The results were pretty effective.

The MERLIN Mobile interface ("TEXAS") as demonstrated at OR11

The MERLIN interface on LASSO is quite complex, but at the heart of it is the tag cloud of related terms that the Termine text-mining suggests. This always looked like it might be good on a touchscreen, so we stripped it down, rearranged and tweaked it to make it viable on an Ipad screen. If you’ve got an Ipad, you can try it out by pointing your Safari browser at http://is.gd/texasweb. (It will work on desktop browsers too, but it looks best on a portrait oriented screen.)

If you’ve got an Android device, you can even download an app-based version of it from http://is.gd/texasapp. (It’s a bit cramped in a smartphone display, but still essentially working.)

If you’ve got neither we created this page to give you a rough idea what it looks like.

It’s worth mentioning that this is a real, live, working application: enter a search term (“logical positivism”, “climate change”, “Jeremy Bentham”,  …) and it searches over the full-text corpus of all articles in University of London Open Access repositories (SHERPA-LEAP consortium members), and makes suggestions about additional or alternative search terms, based on the results of the text-mining analysis of the articles

The hackathon approach of working closely together to create something quickly worked well again: Rory hacks, and I test and review each time he hits ‘Save’! All very agile and iterative.

In honor of our hosts in Austin, we decided to call the new interface Touchscreen Enhanced Cross-Search with Augmented Serendipity – or TEXAS for short.

Kudos to Mahendra Mahey, Peter Sefton and the DevCSI project for putting (and keeping) it all together, and to the awesome panel of judges (even though they didn’t pick us)!

Friday’s CeDAAME dissemination event was also an opportunity to be reminded that ULCC’s Digital Archives team has contributed in other ways to the success of SOAS’s team, directly and indirectly. Julie Makinson described how SOAS used the AIDA digital asset assessment toolkit in developing their strategic approach; and many of the SOAS team are alumni of the DPTP: so Ed and Patricia have also had their part to play in supporting SOAS’s digitisation efforts.

The presentations at SOAS were extremely interesting, describing the full range of activities of a multi-faceted digitisation programme, from the development of the strategy (using the aforementioned AIDA) to the many challenges of digitising Islamic manuscripts and related materials.

How, for example, do you reliably OCR pages of centuries-old text with mixtures of Arabic and Latin/English/French? The answer is that sometimes rekeying is unavoidable. We learned, too, that Yale used UKOLN’s DC Dot Dublin Core editor to create their metadata for Islamic collections (and then convert to TEI). Thanks to the native DC and Unicode support in EPrints, SOAS metadata (in English and Arabic) was created and managed directly in the repository. Metadata exchange between Yale’s Fedora-based system and SOAS’s EPrints system seems to have been achieved effectively – I know Rory worked closely with SOAS and Yale on this.

And I sensed genuine excitement in the room when the page-turning interfaces for viewing the books online were unveiled: both very impressive. (For SOAS Rory has been working long and hard on adapting the open source book viewer used by the Internet Archive, and ensuring that the right-to-left reading and page-turning functionality works smoothly.) We also learned about a variety of different approaches to the issues of managing and funding digitisation and cataloguing activities: with my work on the Mediawiki-based Transcribe Bentham project in mind, it was particularly interesting to hear about University of Michigan’s Collaborative Cataloguing initiative.

All in all an exciting day, and particularly satisfying to see close-up the kind of synergies that exist across all of the activities of ULCC’s Digital Archives and Repositories Team. In addition to further enhancing the SOAS Digital Archives system, we are also looking forward to working with them on their JISC-funded Engaging Overseas Communities project, which is going to involve hooking EPrints up to mobile phones in Africa and Asia.

As if that wasn’t enough, at lunchtime I also dashed over to the School of Pharmacy, where Jean, Neroli and Michelle had kindly organised a lunchtime meeting for the University of London repository managers in the LEAP consortium. It was an opportunity for me to unveil a preview of the new SHERPA-LEAP website (with added social networking goodness, courtesy of WordPress/BuddyPress) that we expect to launch very shortly.

It was a nice way to round off a week in which the Team also achieved significant milestones in our work on preservation metadata for the Parliamentary Archives and strategic development for The Women’s Library, began planning for the next DPTP course, and we received news that the FP7 BlogForever project, which will see us collaborating with Warwick, HATII, CERN and others until 2013, has received its final sign-off from the European Commission.

William Kilbride from the DPC asked the audience what were their main concerns and some of the answers were: obsolescence and migration issues; partners links; storage requirements; business cases and funding development. He explained the challenges for preserving digital data; the value and opportunities that preservation creates and the key approaches to achieve digital preservation (migration; emulation; hardware preservation and exhumation) as well as the risk management approach.

Our own Patricia Sleeman from ULCC explained clearly and with a very interesting example how to use the OAIS model for preserving our personal archive of photos, giving us some light on how to start and how to avoid losing crucial data.

She described the OAIS model as a tool that develops consensus from different sectors providing shared vocabulary, bringing everybody together. We were shown how an information package (digital object, metadata, packaging information which related digital object and metadata) travels through the OAIS model using a photographic archive example for the SIP, AIP and DIP stages. We had the opportunity to see other models from Portico and DCC

Bram Van Der Werf from Open PLANETS Foundation presented and demonstrated the Plato tool . He raised the need for more technical background connected to archival training paths. He welcomed the attendees to participate on the content community at Open Planets Foundation.

Caroline Peach from BLPAC gave us the opportunity to use a preservation plan. We spent time with a working example to identify the plan, its status and triggers, description of the institutional setting and the collection, requirements for preservation, evidence of decision for a preservation strategy, costs, roles and responsibilities. We had to think about what is important about the digital access we want to preserve. We used the PLATO tool to assess the preservation plan and its collection.

Ed Fay from LSE explained in detail how they got to preserve the born digital and digitized collections; and how they maintain the Institutional Repository. Initially, they established user requirements and security analysis to establish where they were in the digital preservation process. They had to investigate all the formats and their backups at the LSE data centre and he praised the robust service and infrastructure from the LSE Information Services Department to maintain their secure digital library. He briefly explained the use of checksum creation and verification with MD5. In conclusion, we saw that the LSE digital Library is flexible extensible and modular. They have a transparent process for decision making so if any changes in the technical infrastructure, everything is well documented and there is an excellent engagement throughout the institution for the purpose of digital preservation.

Other Resources and Training:

DPC events

AIDA

DPTP

PRONOM

National Library of New Zealand metadata extraction tool

BagIt

JHOVE

DPC

PADI

Library of Congress

Gloucestershire Archives

Archivematica

UKLON

ULCC

Repositories Support Project

#starting_dp at Twitter

It was great to take part in last week’s Repositories Support Project event at Sheffield Cathedral. The theme of the day, organised by Jackie Wickham and the RSP team, was “Doing It Differently” and it covered a wide range of repository-related themes. I took along an updated and expanded version of the presentation I made to SHERPA-LEAP repository managers. I covered the same topics, but in preparing the presentation, I was amazed how many more things there were to talk about a year on.

Stephanie Taylor gave an excellent overview of the repository scene, and I hope I followed it up with useful ideas about making repositories more user-friendly or just generally useful to users. Other talks went off into less well trodden areas, though no less interesting: Pat Lockley impressed again with his enthusiastic description of Xpert; Joss Winn described his further adventures in WordPress land; and Stephanie Meece described the challenges of non-textual repositories at UAL. My ears pricked up when Jason Hoyt of Mendeley mentioned that an imminent upgrade to Mendeley will be able to identify OA sources for papers, which might signal it’s time for me to finally catch up with Mendeley (dissertation starts next year!). I didn’t catch the final speakers as I had to catch my train, but I commend to you Vicki McGarvey’s post on the SHARE project blog at Nottingham Trent University.

I tried to keep things simple by steering clear of all the complicated issues in repository management – OA, OAI-PMH, copyright, advocacy, REF, RIM, etc – and just focus on simple UI enhancements that might improve a user’s experience of the repository, and effective use of features like RSS feeds and statistics, with examples from all over the world of institutional and specialist repositories. Which features a repository manager might choose, if any, is up to them and their own circumstances, but my aim was to ensure they are at least aware of what’s possible – as evidenced by what’s been done in many repositories around the country.

Although I focused on EPrints installations, I think nearly everything I demonstrated ought to be feasible in other platforms. Overloading an abstract page with features like “Share this on Facebook/Twitter”, QR Codes, or metadata export in RSS/JSON/CSV and more, should be a very easy way to enhance the user experience of repositories. As I suggested, adding buttons to support “the latest thing” users may be finding useful, is generally not difficult. A “Send This Paper To My Kindle” button, for example, seems so trivial I might even try it myself.

I had a long list of ideas/examples to show: for anyone who didn’t have time to copy down the small print, they were:

• SNEEP
• Lincoln EPrints
• Language Box
• MePrints
• Hum Box
• ULCC Publications Archive
• UCL EPrints
• IRStats
• Repository Stats using Google Analytics (presentation by Graham Triggs at OR10)
• E-Space at Manchester Metropolitan University
• Framework for Linking Inline Semantic Metadata
• Oxford University Research Archive
• MERLIN

As an outsider sometimes it feels that reading/hearing about digital preservation research is a bit like reading an article on particle physics.  In fact sometimes lectures on particle physics might be clearer.  The number of acronyms, the amount of jargon per project and also the amount of assumed knowledge can, at times make our subject inpenetrable to many, not to mention boring.

So with this in mind I steeled myself for a few days at the PLANETS workshop in London recently.   At the DPTP we talk a lot about OAIS and I was keen to hear about PLANETS’  ‘suite’ of tools developed with the OAIS model and digital preservation in mind. The OAIS model implies a number of flexible preservation workflows so I was looking forward to hearing about what the  PLANETS team had done.

Ross King’s  wonderfully clear session really drew the crowd in.  He set out the business case for digital preservation, provided explanations of the technical elements that make up digital preservation, the technical problems behind preservation; the options available and the decisions that need to be taken. While not taking away from the sheer volume of ‘stuff’ being created and which will be created as well as the challenges, it was a very motivational session. Next Clive Billeness drew our attention to the fact that risk is the only real motivating factor when it comes to getting the powers that be in organisations to take digital preaservation seriously. He says that we need to talk to financial planners, CEOs in terms of risk, using the language of risk. Companies are now being asked to keep a ‘living will’ so that they can garantee that they can operate in case of a disaster, ensuring that records needed to operate are kept in a manner which can ensure operations.

So what are these PLANETS tools? I am not going to cover them all but Hannes Kulovits introduced us to PLATO,  PLANETS preservation planning tool.  Preservation planning for digital preservation includes preservation policies, legal obligations, organisational and technical constraints, user requirements and preservation goals. It also describes the preservation context, ‘the evaluated alternative preservation strategies and the resulting decision for one strategy, including the rationale of the decision.’ During the workshop we looked in depth at how to decide on preservation strategies and factors which influence decision making.  Input is needed from a wide range of persons, depending on the institutional context and the collection. PLANETS have also pinned down some essential characteristics for digital objects with regard to their preservation.

I was a bit narked that I didn’t get to actually play with it on my laptop there and then but I have done so since the workshop.  How long this planning process takes place is another issue. This was a bit of a drop for us all, yes planning takes time but that much time?

A quick survey of the crowd by me during and after revealed that some people felt that while they enjoyed initial content they felt that the workshop  wasn’t for them, too technical or too highbrow. I consider myself to be a low to middle brow kind of girl (dp-wise I hasten to add) so was suprised by this as I felt that it was one the clearest events I had been to on digital preservation.  I felt this response was a real pity, perhaps there needs to be more interaction with the groups attending these workshops (one remaining) to gauge receptivity.  I struggle with projects who really cannot communicate their products. I didn’t feel this case here. The Planets Framework is really good and may need definining but I really enjoyed it and look forward to next developments.

Beacon cited through fog

Who would have thought that reference management could be so interesting? We spent a  very informative and enjoyable Thursday in snowy Milton Keynes, at the Innovations in Reference Management (#IRM10) event (part of the OU/JISC TELSTAR project). All thoroughly blogged by Owen Stephens, and tweeted by many.

Owen Stephens and Jason Platts of OU described the outputs of the TELSTAR project, which integrates the OU’s Moodle VLE with Refworks. This means that students using the VLE can move seamlessly between their reading lists and Refworks, locating resources, maintaining consistency of style and generating bibliographies easily.

Paul Stainthorp of Lincoln University described some exciting, bleeding-edge uses of Yahoo Pipes to mashup data from Refworks, OPAC, and Amazon. Arguably even more bleeding-edge was the presentation by Euan Adie from Nature Publishing, who showed us Help Me Igor, a reference manager plugin for Google Wave. Speakers from CiteULike and Mendeley also gave us fascinating insights into their respective social-tinged bibliographic management offerings.

Perhaps unsurprisingly, Kevin and I brought to the table the theme of web preservation. With reference to our work with JISC-PoWR, UKWAC and ArchivePress, we reminded anyone who hasn’t heard our spiel already that there are many important, valuable and eminently citable web resources, notably blogs by academic researchers, that are at risk of disappearing – making references to them virtually useless.

Authors may not be responsible for ensuring their readers can access the resources they reference, but we think they should at least give them a fighting chance of doing so! We  therefore proposed that students and researchers should be encouraged to locate and cite copies of web resources in stable web archives (such as the UK Web Archive) rather than “in the wild”.

We also discussed the idea that persistent collections of web resources could be created at the institutional level, whether that were an open archive of blog posts by a university’s researchers, or a closed repository where researchers can store copies of the web resources they cite.

One of the strong themes that emerged in discussion was the need for information literacy/digital skills training at all levels to address current tools and trends in reference management; and to re-assert the purpose, value and nature of citation in online digital environments

An interesting suggestion also made was that reference management tools are becoming a natural part of the environment, just as email has: is provision of specialised applications by universities an “aberration”?

I’m inclined to think not, after all it was clear from the workshop that there’s still a need to support ongoing study and research effectively, and scope to develop and validate new approaches.  Microsoft Word may now include reference management features, but that doesn’t obviate the need to educate people in how to use them effectively, and why.

We’re very grateful to Owen for including us in his programme: this is a fascinating area, where e-learning, libraries, preservation and publishing collide, and I’m sure we haven’t heard the last of it.

My naïve thinking in the area has been characterised by the assumption that the process is rather static or linear, and that the problem we’re facing is broadly the same every time; migrate data from a format that’s about to become obsolete or unsupported, onto another format that’s stable, supported, and open. MS Word document to PDF or PDF/A…now that, I can understand!

In fact, I learned at least two ways of thinking about formats that hadn’t occurred to me before. One simple one is costs; some formats can cost more to preserve than others. This can be calculated in terms of storage costs, multiplied over time, and the costs associated with migrations to new versions of that format. For example, we’ve tended to pin our faith on the TIFF format for images for many reasons, but there’s a high storage price to be paid for all that wonderful losslessness. This may be one reason why the DP world is looking with more favour on the JPEG2000 format, which is ‘virtually’ lossless and smaller in size.

Secondly, the problems of preserving digital data which doesn’t actually have a specified stable preservation format. Chris Puttick of Oxford Archaeology gave a vivid description of the problems he’s facing with CAD and GIS files, where the data can’t easily be tied to a single format in the first place (nor can a stable format for migration be identified). As the NLA put it on their PADI page, “At present there is little dealing specifically or comprehensively with the preservation of this particular type of data, although some aspects of database preservation are applicable to GIS. Some long term preservation issues include a lack of open source formats and metadata standards, large data volume and complex data objects.” Puttick suggests that his data doesn’t really perform at all unless it’s operated within a very specific environment of hardware and software. How do we preserve an environment? This appears to be quite a distinct preservation problem and much harder to solve than Word to PDF, to put it mildly.

William Kilbride suggested that such cases (and websites too, arguably, because they are time-based) are more like a stream of data – a handy image which conveys something about the dynamic of such information packages, and showing us that it’s much harder to nail them down into a single format. You can never step into the same river twice.

Following up his excellent interview with William Kilbride at DPTP in May, this time Frank Steiner conducted video interviews with some of the DPC students and scholarship winners – hopefully you can see them below:

This is just the first phase of the rollout, not only of Fürer-Haimendorf’s pictures, but also of many other valuable collections at SOAS. We are pleased and excited to have been able to assist with this endeavour, by customising EPrints to meet the extensive requirements set out for the system by Susannah Rayner and Malcolm Raggett, who are leading the project at SOAS. No less than with Linnean Online, it is a rare privilege to be associated with a project giving a new impetus, and worldwide access, to such invaluable historically important, archival collections.

SOAS organised a fascinating series of lectures on Friday to launch the collection. In one talk, Albert Alban von Stockhausen spoke passionately about the importance of sharing the collection not only with researchers, but also with the communities that Haimendorf visited. Surviving subjects in Nagaland, and their descendants, have been deeply moved on seeing these photos again. Some even refer to their own age in terms of how old they were when Fürer-Haimendorf visited.

Stuart Blackburn talked about Fürer-Haimendorf’s work in the Apatani valley in the 1940s, where he got a rapturous reception, if not as the first westerner to visit them, then probably as only the 5th or 6th, when he acted as an official representative of the British government in India, trying (not always successfully) to resolve tribal disputes. In the photos we can see valuable records of rice-growing, forest-clearance, village-settlement patterns, ritual and warrior practices, and the Apatani villages, with their densely populated, crowded lanes. There are also many individual portraits.

We are particularly pleased with the results. It is an EPrints-based repository which implements the very exacting requirements set out by the SOAS team. It has been an opportunity to develop even further the work Rory has done with EPrints, drawing on our experience adding plugins for bookmarking, tagging and commenting, which began with Linnean Online and continued with SNEEP. We also particularly enjoyed, and benefitted from, working closely with the team from 9Web, who provided graphic designs and meticulous usability testing.

I hope to enumerate the new features and describe the development work in a future post; meanwhile you can see it in action at digital.info.soas.ac.uk.