The JISC funded SNEEP project (Social Networking Extensions for EPrints) – part of the original JISC rapid innovation programme – aimed to provide a set of social networking tools for EPrints repositories. It ran for 6 months and ended in May 2008. Since the rather low key publication of the resultant EPrints plugin interest and uptake has been slowly but surely gathering momentum.

Today I am pleased to announce a couple of significant SNEEP related developments. Firstly , thanks to my colleague Ben Wheeler here at ULCC, SNEEP 0.3.2 released this week offers an automagic installer. This does away with the (slightly tortuous) manual install procedure that we suspect discouraged all but the hardier EPrints hac… I mean administrators.

You can download SNEEP 0.3.2 and/or read Ben’s post to the EP-tech mailling list. The download page is also a good place to see SNEEP in action.

PICT

I am also pleased to announce a new project (funded as part of the 2009 JISC rapid innovation programme) that aims to build on the SNEEP work to provide SNEEP-ish services to a broader range of web resources. The goal of the PICT project (Platform Independent Community Toolbox) is a lightweight javascript tool that can be deployed across an number of web resources (not just a repository) to encompass the web-based real estate of a given research community and provide that community with collaborative tools available at the on-line research coalface.

Effectively PICT will allow resource owners to offer

• tags
• comments
• notes
• other goodies

from their web page. The data gathered by these tools will be managed by a PICT server (probably run by a community-minded resource owner) and be available for cross referencing with other resources in a PICT community.

If all that is a bit difficult to picture, rest assured that demos will appear throughout the course of the project that should help to clear the murk.

Firstly, we don’t want to gather absolutely everything that’s presented as a web page in the wiki, since the wiki contains not only the user-input content but also a large number of automatically generated pages (versioning, indexing, admin and login forms, etc). This stems from the underlying assumption about doing digital preservation, mainly that it costs money to capture and store digital content, and it goes on costing money to keep on storing it. (Managing this could be seen as good housekeeping. The British Library Life and Life2 projects have devised ingenious and elaborate formulae for costing digital preservation, taking all the factors into account to enable you to figure out if you can really afford to do it.) In my case, there are two pressing concerns: (a) I don’t want to waste time and resource in the shared gather queue while Web Curator Tool gathers hundreds of pages from DigiRep, and (b) I don’t want to commit the JISC to paying for expensive server space, storing a bloated gather which they don’t really want.

Secondly, the above assumptions have led to me making a form of selection decision, i.e. to exclude from capture those parts of the wiki I don’t want to preserve. The parts I don’t want are the edit history and the discussion pages. The reason I don’t want them is because UKWAC users, the target audience for the archived copy – or the designated user community, as OAIS calls it – probably don’t want to see them either. All they will want is to look at the finished content, the abiding record of what it was that DigiRep actually did.

This selection aspect led to Maureen Pennock’s reply, which is a very valid point – there are some instances where people would want to look at the edit history. Who wrote what, when…and why did it change? If that change-history is retrievable from the wiki, should we not archive it? My thinking is that yes, it is valuable, but only to a certain audience. I would think the change history is massively important to the current owner-operators of DigiRep, and that as its administrators they would certainly want to access that data. But then I put on my Institutional records management hat, and start to ask them how long they really want to have access to that change history, and whether they really need to commit the Institution to its long-term (or even permanent) preservation. Indeed, could their access requirement be satisfied merely by allowing the wiki (presuming it is reasonably secure, backed-up etc.) to go on operating the way it is, as a self-documenting collaborative editing tool?

All of the above raises some interesting questions which you may want to consider if undertaking to archive a wiki in your own Institution. Who needs it, how long for, do we need to keep every bit of it, and if not then which bits can we exclude? Note that they are principally questions of policy and decision-making, and don’t involve a technology-driven solution; the technology comes in later, when you want to implement the decisions.

Since it is blogging applications that provide this stucture and behaviour (usually from simple database tables of Posts, Comments, Users, etc), perhaps we should consider making blogging software behave more like an archive. How difficult would that be? Do we need to hire a developer?

One interesting thing about Wordpress is the number of uses its simple blog model has been put to. Under-the-hood it is based on a remarkably simple data base schema of about 10 tables and a suite of PHP scripts, functions and libraries that provide the interface to that data. Its huge user-base has contributed a wide variety of themes and additional functions. It can be turned into a Twitter-like microblog (P2 and Prologue) or a fully-fledged social network (Wordpress MU, Buddypress).

Another possibility exploited by a 3rd-party plugin is that of using Wordpress as an aggregating blog, collecting posts automatically via RSS from other blogs: this seems like a promising basis for starting to develop an archive of blogs, in a blog.

The plugin in question is called FeedWordPress. It uses the Links feature of Wordpress as the basis of a list of feeds which it checks regularly, importing new content when it finds it, as Posts within Wordpress.

I installed FeedWordPress a while ago on ULCC’s DA Blog, and set it up to import all of the ULCC-contributed posts to JISC-PoWR, i.e. those by Ed Pinsent, Kevin Ashley and myself. I did this because I felt that these contributions warrant being part of ULCC’s insitutional record of its activities, and that DA Blog was the best to place to address this, as things stand.

JISC-PoWR also runs on Wordpress, therefore I knew that, thanks to Wordpress’s REST-like interface and Cool URIs, it is easy not only to select an individual author’s posts (/author/kevinashley) but also the RSS feed thereof (/author/kevinashley/feed). This, for each of the three author accounts, was all I needed to start setting up FeedWordPress in DA Blog to take an automatic copy each time any of us contributed to JISC-PoWR.The “author” on the original post has been mapped to an author in DA Blog, so posts are automatically (and correctly) attributed. The import also preserves, in custom fields, a considerable amount of contextual information about the posts in their original location.

In many cases, I’ve kept the imported post private in DA Blog. “Introductory” posts for the JISC-PoWR project blog, for example: as editor of DA Blog, I didn’t feel we needed to trouble our readers there with them; nevertheless they are stored in the blog database, as part of “the record” of our activities.

This is, admittedly, a very small-scale test of this approach, but the kind of system I’ve described is unquestionably a rudimentary blog archive, that can be set up relatively easily using WordPress and FeedWordPress – no coding necessary. Content is then searchable, sortable, exportable (SQL, RSS, etc). (Note, by the way, what happens when you use the Search box on the JISC-PoWR blog copy in UKWAC: this won’t happen with this approach!)

For organisations with many staff blogging on diverse public platforms this would be one approach to ensuring that these activities are recorded and preserved. UKOLN, for example, manages its own blog farm, while Brian and Marieke have blogs at Wordpress.com (as well as contributing to this one), and Paul Walk appears to manage his own blog and web space. This kind of arrangement is not uncommon, nor the problem of how an institution get a grasp on material in all these different locations (it’s been at the heart of many JISC-PoWR workshop discussions). A single, central, self-hosted, aggregating blog, automatically harvesting the news feeds of all these blogs, might be a low-cost, quick-start approach to securing data in The Cloud, and safeguarding the corporate memory.

There are more issues to address. What of comments or embedded images? Can it handle Twitter tweets as well as blog posts? Does it scale? What of look-and-feel, individual themes, etc? Now we start needing some more robust tests and decisions, maybe even a developer or two to build a dedicated suite of “ArchivePress” plugins. But thanks to the power and Open-ness of  Wordpress, and the endless creativity of its many users, we have a promising and viable short-term solution, and a compelling place to start further exploration.

Calls in the literature have advocated that blogs, as potentially valuable additions to the human record, are worthy of stewardship and long-term preservation. It’s hard to argue with this, in the light of recent widely publicised use of blogging by President Obama, the Number Ten Downing Street blog, the blogging and tweeting activities of certain celebrities, and so on. But of course anyone can have a blog (and it sometimes seems that almost everyone does). Blogs are phenomenally useful, and have a wide variety of applications: individual or corporate announcements and publications; time-stamped records of activities; public or private figures writing as themselves or pseudonymously; public and private journal and discussion spaces for students and a wide range of communities. A blog may well be just for Christmas (or any kind of project) no less than a traditional diary or journal. It is in essence, after all, just a sequential log (“blog” is merely a geeky contraction of “web log”). Blogging is therefore not so fundamentally alien: Captain Cook kept a log (and so will Captain Kirk). Rather than make it sound very technical and complicated, let’s start by taking some comfort in what’s familiar.

Should all blogs be preserved? Which blogs should be preserved? The DPE Briefing Paper highlights (as all good preservation advice must) the issue of selection. How do we decide which blogs to preserve? To start with, the remit of any archival body (national, municipal, corporate, specialist, etc.) is itself self-selecting. It’s unlikely any archivist at NARA, TNA or any other archive would ever have to begin their deliberations with “all blogs”. Of matters of authenticity and attribution, the nature of the archive undertaking the preservation will, equally, dictate whether knowing the true identity is essential or not. Common sense should prevail. Clearly for records of official activities, such as the Downing Street and Obama blogs, reliable provenance and authenticity are critical. On the other hand, satirical blogs by “Gordon Brown” or “Vladimir Putin“  are clearly not records of official activities, but they still may merit preservation as publications and/or as interesting artefacts: as with many fictional or satirical diaries or letters, the true identity of the author may or may not be material (Beachcomber, Henry Root, Primary Colors).

Is preservation of content, regardless of context, sufficient? This is one of the most significant questions raised by the briefing paper. Has the Web changed our expectations of context? Once a descriptive catalogue or historical account seemed sufficient to establish context. Digital media now tempt us into thinking that, for any object, we should automatically gather (a copy of)  everything linked-to, and perhaps everything that links to, and so on. We can leave a computer to follow the endlessly forking paths, and create a virtual machine environment that will always render the gather materials faithfully – all without any manual intervention. Will that provide context, or make the preserved content understandable? Can we do without expert researchers and information professionals to scope, select and provide commentary where necessary?

The inability to capture and preserve the design and features of the blog contradicts defining attributes of a blog. In fact it’s not impossible to capture and preserve many if not all of the design features of a blog. The JISC-PoWR blog has been harvested by UKWAC with most of its features intact (experiments at ULCC have suggested that, with a little care, WordPress blogs are particularly well-suited to harvesting). But in general, let’s accept (as the Blogbackuponline people do) the overriding significance of the textual content over and above the bells and whistles of the web presentation. If there is one particularly interesting feature of blogs, that does set them apart from conventional web content, it is that they are generally intended for delivery and consumption in two distinct formats: by HTML web page, and by RSS/Atom XML newsfeed. The newsfeed version is bare-bones hypertext stripped of all the styling, bells and whistles of the web version, and will be read in many contexts, from online newsreaders and aggregators, like Netvibes or Bloglines, to desktop clients, such as Mozilla Thunderbird, as well as on mobile phones, and by automatic syndication to other sites.

Blogs, including posts and comments, may be intentionally edited or deleted. Looked at in isolation, editability is a significant property of blogs in everyday use. But, as Chris Rusbridge observed , “editability, while vital for some kinds of re-use, is not essential for conveying the information essence of the objects”. Any preserved object we have only in the state it was when taken into archival custody (though we may also capture and preserve records of how it changed). Preservation schedules for blogs will need to decide how important this really is: when is it and when is it not important to preserve an edit that no one ever saw? (A transactional approach – only capturing page impressions users request – is one way to address this without excessive redundancy.) However, if anything, this volatiliy of blog content is a more serious issue for those citing an active blog (who may find their quotations no longer match): a copy in an archive ought to be fixed, and reliably citable. If anything, this would suggest that creating stable archives of valuable blogs for the record is essential.

James Currall rightly warned us against making an undue fetish of digital media, at the expense of the information (and entertainment) which they convey; arguably as dangerous is the fetishisation of specific manifestations of digital media. How different, really, are blogs from other publications or applications built on the Web platform? Blogs are essentially web pages and don’t exist in a void. The “problem” of hypertext links to possibly unstable objects in remote locations is the bête-noire of all efforts to tame the Web. “Not in archive” is, and will ever be, a familiar message to regular users of archive.org. But we can perhaps take heart that many archives of correspondence tend to be one-sided too.

Sometimes thinking about these things in the abstract is no substitute for actually doing them for real. Perhaps the best advice for anyone tasked with preserving blogs is, first of all, to try keeping one yourself for a few days or weeks. Then it only takes a few minutes to try out BlogBackupOnline; or check how your favourite blog is faring in the Internet Archive , the European Web Archive, or the UK Web Archive; or even download HTTrack or Wget and try harvesting something yourself.

Then it’s time to get Twittering…

In the Handbook we’ve tried to cover a huge, and sometimes controversial, area in as accessible a way as possible. The workshops, attended by both web-management and records-management professionals from HE institutions, brought  a wide range of concerns and issues to light. It’s been quite a job fitting it all in.

Even as the project progressed, we became aware of new developments in thinking about how to approach the special issues of managing web resources, including everybody’s favourite new fast automatic Web 2.0 applications. We saw the publication of Steve Bailey’s Records Management 2.0 book, TNA’s Web Continuity project, and further web archiving developments at UKWAC. We’ve even heard it whispered in some quarters that approaches to preservation may need a more profound reassessment in the context of the Web and the Cloud. Many of these issues were recorded on the PoWR blog, and we tried to reflect as much of this in the Handbook as possible.

Another recent JISC publication, The Digital Preservation Policies Study by Charles Beagrie Ltd, published at the same time, is complementary in many ways, and reassured us that many of the conclusions we groped towards in the Handbook were not so wide of the mark! Like PoWR, the  Digital Preservation Policies Study identified the necessity of high-level policy engagement as the sine qua non of effective digital preservation.

Digital preservation solutions are undoubtedly partly technical, and the tools being created will enhance digital longevity, but these solutions are also equally dependent on organisational issues. It is important to remember that digital preservation relies on the interaction between the digital preservation environment and wider organisational objectives and procedural issues. These could be financial and staffing issues, collection management, legal obligations, auditing requirements, and other strategies and policies. In this respect, recognition by organisational divisions that digital data is important and key to the successful running of an organisation is crucial.

The Digital Preservation Policies Study, p.11

Among the other recommendations the Study shares with PoWR include:

• Analysis of existing policies and strategies, and how our work can support them even if said polices don’t explicitly refer to preservation or digital assets
• Taking a phased approach – nothing happens all at once. (PoWR recommends pilot projects and working with supportive departments.)
• Careful scoping of preservation requirements. (With regard to web resources, PoWR suggests not everything, not every version, and not forever.)
• Identifying if and where existing systems will do the job
• Consideration of lifecycle, publication, and retention schedules.

The Charles Beagrie survey is a very concise and accessible contribution to the field, and we hope the PoWR Handbook, with its specific focus on established and emerging Web issues, and attention to the detailed and everyday concerns of our many contributors and correspondents, will be similarly useful. We also hope that the work of PoWR will continue in some form, on the blog and perhaps in the form of new projects and workshops, to fill in the gaps we left, and deal with the constantly emerging Web developments. Anyone for PoWR 2.0?

LCACE is a university collaboration promoting the exchange of knowledge and expertise with the capital’s arts and cultural sectors. The partners are: University of the Arts, London; Birkbeck, University of London; City University; Courtauld Institute of Art; Goldsmiths, University of London; King’s College London; The Guildhall School of Music & Drama, Queen Mary, University of London and Royal Holloway, University of London.

I was particularly interested by Simon Tanner’s presentation Collaboration: Working Together to Build the Digital Future, and Paul Vetch’s presentation Cost Effective Content Management and Web2 (in 2008). Simon shared some very helpful tips aimed at engendering successful collaboration on digital projects. Also of note, he highlighted the pivotal importance of digital projects and programmes becoming increasingly user/community-driven rather than purely technology-driven for technology’s sake; and stressed the funders’ need for a clearly defined vision and user benefits! Paul Vetch offered some thought-provoking findings on Web 2.0, adding further fuel to our earlier discussions about the perceived benefits of Web 2.0 for MLAs .

Flow Associates delivered a presentation on ‘Planning digital projects for learning outcomes’, advising us on how to get the most out of our assets, making our collections more accessible to learners , as well as suggesting ways that we can encourage people to interact more with collections. Interestingly, an organisational challenge commonly lies with trying to get the learning team and web team together in the same room; which is considered vital once it’s achieved! He emphasized the importance of collaborating early and becoming more user-focussed, as well as exploring new places to make your collections visible.

Paul Spence’s (CCH) presentation titled ‘Can XML be your friend?’ revealed not only some of XML’s commonly accepted advantages (e.g. platform/software independent, international/open/interoperable, facilitates greater access/data preservation etc) but also showed some of the innovative ways in which XML can be used e.g. enhances existing or creates new forms of scholarship; and it can be connected with 3D visualisation, allowing users to navigate 3D space to see connections between people.

The seminar was also a great opportunity to find out more about Richard Beacham’s exciting work at the King’s Visualisation Lab. He beautifully demonstrated how technology can be ‘used to evoke all elements of theatre’ and commented on how it allows people to ask and communicate new questions, and explore new areas of knowledge.

Along with the informative and interesting talks, the smorgasbord of lunch and patisserie delicacies also inspired more fruitful exchange and networking – all making for a very successful and stimulating day!

The presentations are available from the LCACE website.

I just spotted a posting from a fellow Antipodean, made to the Australian Archivists (aus-archivists) listserv, which has certainly raised some interesting questions surrounding web 2.0 technologies and their impact on the Archive sector…. Perhaps a debate well worth monitoring, and further exploring here, within the realm of web 2.0 itself?

See Australian Archivists listserv posting below:

Archival institutions are increasingly using social networking sites, tagging (folksonomies), blogs, wikis, and Flickr to promote their collections. Does anyone know any studies evaluating this phenomenon in the archival setting? I don’t mean the this-is-how-we-did-it, isn’t-it-exciting or look-how-many-hits-we’re-getting articles. I mean thoughtful consideration of the value of these tools and the effects they are having on our work. I’d also love some discussion on the list.

Why we have decided to use these tools? What benefits have they brought? What kind of new audiences are they attracting? How long do these audiences stick around? Is the resource taken to sustain these ‘relationships’ worth it? Do these audiences engage with us beyond the social network stuff? Do they use our databases? come in and use our collections? order quality copies? Does it matter if they don’t? How is our adoption of these tools affecting what material we choose to process and promote? With user-generated content and tagging are our formal documentation skills, cataloguing standards, thesauri, etc passé? Does mashing trivialise our research collections? Any observations or leads to articles welcome. [ Helen Yoxall, Archives Manager, Registration and Collection Management, Powerhouse Museum, PO Box K346, Haymarket NSW 1238, Australia, URL: www.powerhousemuseum.com/archives/index.asp]“

After a quick (and rudimentary) search, I stumbled across a few sources that may be of interest:

• UKOLN have a presentation “Web 2.0: Implications For The Cultural Heritage Sector”
• The Archives & Museum Informatics Museums and the Web website, includes a paper “Towards Community Contribution: Empowering Community Voices On-Line” (Angèle Alain, Library and Archives Canada, Canada; Michelle Foggett, The National Archives of England and Wales, UK). It refers to Web 2.0 and community involvement in museums, libraries and archives e.g. the Moving Here project which has sought to “break down barriers to the direct involvement of minority ethnic groups in sharing their history on-line” and is among other projects keen to “embrace social networking in future to give users a higher profile voice to enable their knowledge to be passed down to the next generation”. (However, “specialised and appropriate training” was identified as crucial to tackling the barriers,[such as the 'digital divide' itself]). On the related MuseumsComputerGroup website is the article abstract “Museums and Web 2.0: Connections + Community” (by Jennifer Trant, Archives and Museum Informatics), noting both the possibilities, and challenges, surrounding the adoption of web 2.0.
• Also of interest are the web 2.0 blogs: Library 2.0 network, Library 2.0 gang, Museum 2.0; and while I couldn’t spot an Archives 2.0-specific blog anywhere, there was an interesting posting on the Archives Hub blog and on ArchivesNext (which, as one of its aims, invites bloggers to explore “Web 2.0 applications and discussing their applicability to archival institutions”).
• Our own dablog has highlighted one dimension of the web 2.0 impact (inspired by UCL’s Dr Andrew Flinn) i.e. in relation to urgent calls to preserve the heritage outputs of web 2.0, due to “the transient history of the increasing number of minority/dissenting voices, whose heritage is only documented via websites, blogs, wikis and social software”.
• In true folksonomic Web 2.0 style, we can also see what’s been tagged as Archives2.0 in del.icio.us.

Know of any other sources that discuss the impact of Web 2.0 on the Archives Sector? Or would you like to share your opinions in response to the questions posed by Helen Yoxall (above)?

I’ve been an admirer of WordPress for a while now. I think we first looked at it circa 2004, for an internal news management system, when we needed an alternative to falling foul of Movable Type’s new licensing arrangements. (I think MT’s now reverted to free licensing, but their paid-for interim probably did WordPress a huge favour.)

Since then, I’ve used it successfully for DA Blog and several personal projects – not just for blogging, but also as a lightweight CMS. We’ve also seen the rise of WordPress.com, Edublogs, and, closer to home, JISC Involve. It’s favoured by many web illuminati, such as Brian Kelly and Jill Walker. WordPress is definitely on a roll.

As for Number 10, it certainly looks fresher than of old. Earlier news entries have all been imported – not always flawlessly. The earliest entry I can find is from 17th Feb 2000 (about 3 years before the first widely available version of WordPress) – and it’s not “Hello World”. I imagine there is somewhere behind it all a bigger news machine than the usual WordPress Admin interface – but, of course things like that are easy enough to integrate.

With JISC-PoWR in mind, and discussions about web archiving and continuity, it’s interesting to note that, as a result of the switchover, some versions of the Number 10 website in the Internet Archive are now well and truly broken, stylewise at least. Versions collected in The National Archives’ UK Web Archive seem to have fared better. Some of these seem to be based on Internet Archive material, others are part of the European Archive collection: not having looked for a while, I have to say it’s not quite clear where TNA’s UK Government Web Archive ends and the European Archive content begins. Also, a tad confusingly the “Prime Minister’s Office” website seems to be archived as both pm.gov.uk and number10.gov.uk. (The former now resolves to the latter, which may be a small step for Web Continuity.)

It will be interesting to see if the WordPress switch improves the quality of the website in its ongoing archived form, and general preservability (at least, once they sort out a raft of validation errors). Although it’s not without pitfalls for the unwary, I recently found it fairly easy to create an offline snapshot of a WordPress site (complete with valid HTML and CSS too).

I wonder whether The White House or The Kremlin will be next to embrace the WordPress way?

But what’s often given less serious consideration, in the excitement of using a third-party provider of wikis, blogs, Ning, etc., to get your collaborative hypertext project off the ground so quickly and easily – and without having to go cap or cheque in-hand to whoever guards your web space – is this key preservation issue: what happens when you want to get your painstakingly intricate web of hyperlinked pages out?

There are many good reasons why you might want to do this: you might want to migrate to another wiki system or CMS, as the shape and nature of your content evolves; or put it on a permanent, persistent footing by moving it into your own domain; you might simply want to back it up or take a snapshot; or you might want to pull out information for publication in a different form. When you had one or two pages, it might have seemed trivial; but what if you now have hundreds?

Unfortunately, just as exporting the information is often a secondary consideration for wiki content creators, so it also is for the wiki farm systems. The Wetpaint Wiki discussion boards indicate that an export feature was a long time in coming (and its absence quite a blocker to adoption by a number of serious would-be users). And what was eventually provided leaves a lot to be desired.

Wetpaint’s backup option “lets” you download your wiki content as a set of HTML files. Well, not really HTML files: text files with some embedded HTML-like markup. (Which version? Not declared.) Don’t expect to open these files locally in your browser and carry on surfing your wiki hypertext (even links between wiki pages need fixing). The export doesn’t include comment threads or old versions. Restoring it to your online wiki is not possible. But, for what it’s worth, you have at least salvaged some sort of raw content, that might be transformed into something like the wiki it came from, if hit with a bit of Perl script or similar.

I checked out Wikidot – another impressively-specced, free “wiki farm”. Wikidot’s backup option will deliver you a zip file containing each wiki page as a separate text file, containing your wiki markup as entered, as well as all uploaded file attachments. However, according to Wikidot support:

you can not restore from it automatically, it does not include all page revisions, only current (latest), it does not include forum discussion or page comments.

To reconstruct your wiki locally, you’ll, again, need some scripting, including using the Wikidot code libraries to reconvert its non-standard wiki-markup into standard HTML.

A third approach can be seen with a self-hosted copy of Mediawiki. Here you can select one or more pages by name, and have them exported as an XML file, which also contains revisions and assorted other metadata. Within the XML framework, the page text is stored as original wiki markup, raising the same conversion issues as with Wikidot. However, the XML file can be imported fairly easily into a different or blank instance of Mediawiki, recreating both hypertext and functionality more or less instantly.

In contrast to all these approaches, if you set a spidering engine like HTTrack or Wget to work “remotely harvesting” the site, you would get a working local copy of your wiki looking pretty much as it does on the web. This might be an attractive option if you simply want to preserve a record of what you created, a snapshot of how it looked on a certain date; or just in case a day should come when Wetpaint.com Inc., and the rest, no longer exist.

However, this will only result in something like a preservation copy – not a backup that can be easily restored to the wiki, and further edited – in the event, say, the wiki is hacked/cracked, or otherwise disfigured. For security alone, it may be sufficient to depend on regular backups of the underlying database, files and scripts: but you still ought to reassure yourself exactly what backup regime your host is operating, and whether they can restore them in a timely fashion. (Notwithstanding the versioning features of most wikis, rolling back a raft of abusive changes across a whole site is not usually a quick, easy or particularly enjoyable task.)

All this suggests some basic questions that one needs to ask when setting up a wiki for a project:

• How long do we need it for?
• Will it need preserving at intervals, or at a completion date?
• Is it more important to preserve its text content, or its complete look?
• Should we back it up? If so, what should we back up?
• Does the wiki provide backup features? If so, what does it back up (e.g. attachments, discussions, revisions)?
• Once “backed up”, how easily can it be restored?
• Will the links still work in our preservation or backup copy?
• If the backup includes raw wiki markup, do you have the capabilities to re-render this as HTML?

And questions like these are no less relevant when considering your uses of blogs and other social software: I hope we’ll be able to look at them more closely in another post.

As Richard noted in Part I, digital preservation is a “series of managed activities necessary to ensure continued access to digital materials for as long as necessary.” But what sort of digital materials might be in scope for the PoWR project?

We think it extremely likely that institutional web resources are going to include digital materials such as “records created during the day-to-day business of an organisation” and “born-digital materials created for a specific purpose”.

What we want is to “maintain access to these digital materials beyond the limits of media failure or technological change”. This leads us to consider the longevity of certain file formats, the changes undergone by proprietary software, technological obsolescence, and the migration or emulation strategies we’ll use to overcome these problems.

By migration we mean “a means of overcoming technological obsolescence by transferring digital resources from one hardware/software generation to the next.” In contrast, emulation is “a means of overcoming technological obsolescence of hardware and software by developing techniques for imitating obsolete systems on future generations of computers.”

Note also that when we talk about preserving anything, “for as long as necessary” doesn’t always mean “forever”. For the purposes of the PoWR project, it may be worth us considering medium-term preservation for example, which allows “continued access to digital materials beyond changes in technology for a defined period of time, but not indefinitely.”

We also hope to consider the idea of life-cycle management. According to DPC, “The major implications for life-cycle management of digital resources is the need actively to manage the resource at each stage of its life-cycle and to recognise the inter-dependencies between each stage and commence preservation activities as early as practicable.”

From these definitions alone, it should be apparent that success in the preservation of web resources will potentially involve the participation and co-operation of a wide range of experts: information managers, asset managers, webmasters, IT specialists, system administrators, records managers, and archivists.

(All the quotations and definitions above are taken from the DPC’s online handbook.)

This release also includes the Notes feature: this creates private comments visible only to the logged-in user who created them.

For more information about the SNEEP project, see the SNEEP wiki.

The conference was organised with gusto by Les Carr and the Southampton team, who kept things moving at a rapid pace, seemingly unphased by any of the inevitable challenges of staging an event like this for over 300 people. Encouragingly, as well as the usual crowd from western Europe and anglophone countries, other countries represented this time included Serbia, Ukraine and Sri Lanka.

One interesting innovation was using Crowdvine to create an online community of delgates, which proved very simple and effective. And of course there’s an Eprints repository of all the conference papers and proceedings.

My own presentation came in a plenary session on Tuesday morning which was a bit like a rerun of our Web 2.0 Pow-Wow in December. Ian Mulvany was there to talk about Connotea’s latest ideas (David Kane from Waterford Institute of Technology was on the bill too, but was unable to come); Daniel Alexander Smith effused about Rich Tags. In a few spare minutes at the end, Patrick McSweeney, one of Les’s 3rd year undergraduates, impressed everyone hugely with a spontaneous demonstration of a Flash-based Cover Flow visualiser he is developing for Eprints.

Sandwiched between Ian and Daniel, I shared my thirty-minute thoughts on the history of annotations, from Linnaeus and Carlyle, to Flickr and SNEEP. If you’d been there you would probably have seen something like this, and your notes might look a bit like this (unless you preferred to Twitter like this).

From the encouraging feedback I got afterwards, it looks like quite a few Eprints repositories, running or planned, will be interested in one or more of our SNEEP plugins – including our bonus “Easter Egg” Notes (private comments) feature, which Rory has already been working on, since it so inevitably falls out of developing the main Comments plugin. Hope also that Rory and I can touch base soon with Tim and Sebastien at Soton, and get their feedback on the Comments alpha.

There are too many other things I could report (all the usual themes, such as sustainability and interoperability, are well covered in the programme) but I’ll limit myself to some random things that caught my eye:

• Dave Millard’s refreshingly clear grasp of the issues around managing Learning Objects in his paper Towards an Open Repository of Teaching Resources.
• Licia Calvi’s expert usabilty evaluation of repository interfaces, that found them wanting in several critical areas.
• Julie Allinson’s poster for York’s Digital Library that sets a new standard for eye-catching OAIS advocacy.

OR09 will be in Atlanta.

To Westminster yesterday for the Gov 2.0 event organised by OII and POST, held at Portcullis House. (I’d love to have taken my own photos to include, but everywhere I turned there were “No photography” signs, and you know how I hate to break rules.)

The event was in two parts: on reflection I could happily have skipped the first session, in which several erudite professors raked over the well-trodden ashes of government IT disasters. All a bit superficial, really, and little, if anything, that wasn’t just as true 10 or 20 years ago, or that you won’t hear about on any decent project management course. But some of it may still have been news to the Whitehall and BCS elite present.

What I came for really was the second-half, chaired by the charming Earl of Erroll, and which seemed to be where the real meat of “Gov 2.0″ lay.

Jerry Fishenden from Microsoft UK kicked off, describing a typically Microsoft vision of Web 2.0 – that is to say, one that reinterprets the zeitgeist to ensure there’s a shrinkwrapped Microsoft solution to meet the needs of even the most befuddled IT director. In this version of the brave new world, Google doesn’t even get a mention, and we must unreeducate ourselves to say “Internet” not “Web”: presumably Tim O’Reilly’s seminal definition of Web 2.0 – “the Web as platform” – is literatura non grata in the MS empire.

Beginning with an inspirational quote from W. H. Gates III, the presentation went on to illustrate how a young offender might come to a bad end as a result of falling through the cracks between the disjoint systems of education, social, health and justice authorities. If only (Jerry insinuated) they’d had Microsoft’s vision of Internet joinedupness, that poor soul might have been saved. I winced when I saw this, and was glad, later, when one audience member, working in this sensitive area, rightly took him to task.

The best was kept for last: William Heath from IdealGovernment.com, and Tom Steinberg from mySociety.org each in their own way delivered compelling indictments of many of the approaches (laborious systems architecture and project management, inviolable specifications, long and expensive development cycles) venerated in the first session.

William Heath whirled away in a delightfully impassioned exposé of the problems with the traditional approach and the systems it has given us. Government IT systems (and their operators) seem to revel in “a culture of assertive non-listening and spin” leading us up the garden path to the point where “the computer says ‘no’” and officials can wash their hands of the problem. In essence, no one seems to care, listen or ask what we (customers, patients, taxpayers) actually want. Notwithstanding all that painstaking planning with PRINCE, OGC Gateway, and everything that went before, government has failed to lay the foundations of trust in its technological competence, not least in relation to data security.

Meanwhile real people with real needs struggle to get the right information at the right time, and feel as if they have to wring it out of government systems and literature. They have to become ad hoc, amateur experts in the regulations and systems they need to sort out their own lives and the lives of their families – which makes you wonder what we are paying officials for. Can there be anyone who’s wrestled with health, welfare, education, tax or any other government system – online or off – who doesn’t recognise the scenario?

Tom Steinberg went further still: “stop building big projects”, and on the evidence of his own successful projects (mysociety.org, theyworkforyou.com, writetothem.com, fixmystreet.com, planningalerts.com) – some independent, others sponsored by government departments for a tiny fraction of what, say, ID cards are going to cost us – it was hard to argue. Tom also developed the No 10 petition system.

You’ll clearly find Tom in the bazaar, developing rapidly and agilely, while others are laboriously designing their expensive cathedrals. Tom’s philosophy is “just go and do it”, and the best way to do it: see what’s already there, and harness existing functionality effectively (fixmystreet.com, for example, simply uses email to send reports to your local council). Small pieces loosely joined, and all that.

Does any of this have any bearing on us and our projects? I think it does. Tom is an advocate of intelligent reuse and repurposing of data, and his pièce de resistance was a demonstration of mySociety’s latest work on Travel-time maps, which merges house price information (from Land Registry, I assume) with transport information (from TfL’s Journey Planner) and overlays it on a map of London (from OpenStreetMap.org). Using a simple graphic tool, you can quickly see how long your journey to work’s going to take, depending on where you can afford to live, and vice versa.

As with Gapminder, this kind of visualisation is a really exciting, immediate, and meaningful application of statistical and geographical data. Wouldn’t it be great if we could see TNA promoting the reuse of NDAD datasets in a similar way? Why is it always Magna Carta and the Domesday Book that get the sexy expos? Mapping/GIS data and apps are getting cheaper, and NDAD’s many geographical datasets could take on a new life if we could at last give them the map-based interface they’ve always lacked.

So, if you’ve read this far: can you suggest any datasets in NDAD and/or UKDA that, separately or together, might reveal some interesting trends, or look good on a map or graph? Could info in Schools’ Census and Crime Statistics, for example, tell us something new when plotted together, or on a map? Suggestions in the Comments box below, please!

Ben, John, Chris, David, Joss, Ian, Sebastien, Les, Nina, Steve and Tim discussing the social side of repositories.

After lunch we had fascinating demos of, and discussions around, Southampton’s Rich Tags/mSpace, Nature’s Connotea, and our own progress to-date on SNEEP. Rory’s presentation was impressively clear and confident, given that we hadn’t actually prepared anything, and had to hijack Ian Mulvany’s laptop (thanks Ian). Having missed a chance to see it in action in Bristol, it was great to see a live demo of RichTags, which is a very impressive visualisation tool (the sort of thing that, like Gapminder, can totally change one’s ideas about data and metadata – check out the mSpace movies).

Trying to take in all permutations and implications of Web 2.0 is to risk an overload of total perspective vortex magnitude, so I’ll mostly restrict the rest of my thoughts to SNEEP’s and my own interests, and look forward to reading elsewhere how others interpreted the conversation. (There’s a section at the Eprints Wiki.)

It seems to me that one of the most important things we can achieve is to maximise the degree to which Eprints is susceptible to Social Networking. That may be in an “exo” way – (Rich Tags, Del.icio.us, CiteULike, Connotea, etc.) or from within the repository, a la SNEEP. But I’m reassured that, even within its modest scope, SNEEP has a part to play (and isn’t likely to be superseded by anything in EPrints 3.1). How it’s, in the end, used, remains to be seen. Web 2.0 Social Networking is what-you-make-of-it: some things fly, some flutter, others sink without trace (so long Eduspaces!), nevertheless it’s amazing what can be achieved with a few RSS or Atom feeds.

Apropos Bookmarking – as we’ve seen, it has many behaviours in common with Eprints search results. Good advice from Chris on reusing the Saved Search export functionality.

Commenting: Joss was right about the need to put on a little more style. Rory’s promised to embed plenty of CSS hooks, and it might be worth having a less skeletal default, based on standard Eprints livery.

Tagging implementation has some overlap with the Bookmarking, as Rory has already identified. We should also ensure if possible that SNEEP tags play nicely with Rich Tags (or at least are no more than an XSLT template away). Rory’s posted elsewhere his ideas on tagging: worth noting also that WordPress 2.3 seems to have a competent implementation of tagging (you’re looking at it!)

Of the other discussions, I’m particularly keen to follow up David Millard’s work on Learning Object repositories, with both my other projects and my E-learning studies in mind.

But I still haven’t got to the bottom of why some Web 2.0 sites grab me (Del.icio.us, LibraryThing, CiteULike…) and others don’t (BlueDot, Connotea). Not that Connotea’s not impressive: perhaps it is simply, as Ian Mulvany explained, that the strength of its “community” has a strong affinity with Nature’s readership – other SN sites are stronger in social sciences and humanities. Ian also confirmed that Connotea does not support the “save personal copy” feature, in the way CiteULike does: this is a deal-breaker for me, as I’ve found it invaluable to have quick and easy access to my MSc reading list from anywhere – work, home, laptop, desktop. In fact anything that didn’t make it to my CiteULike bookmarks probably didn’t get read!

This gives an idea of how the comments will work in conjunction with an EPrint abstract. It allows simple non threaded comments which can be displayed, added or updated within an abstract (or other page).

And now that’s out of the way I’ll give an idea of features that I would like to add. Some of these have arisen from the recent pow wow event and others were already stumbling around my mind. The existing interface is unlikely to change drastically most of the new features listed are tinkering under the hood.

• Comments as dataset – recode comments as an EPrint::Dataset and add the appropriate EPrint::DataObj::Comment
• Threading – implemented either by allowing comments on comments or by using some sort of stored threading data within each comment record.
• Better handling of markup – HTML Parsing is performed by default but failure is not well dealt with. Also would be good to offer a repo admins the opportunity to define an allowed subset using the sneep.xml config file.
• Comment on anything – Update the DB table structure so that the thing that is being commented on can be any other EPrint::DataObject. Also – with little or no work – allow for the possibility of commenting on external objects (might be useful where EPrints is running alongside other systems)
• Comment privacy (Notes) – Allow comments to be set as private so only the owner of the comment can view it (and re-badge as a note). Look into using EPrints security model for this as that will be neat and hopefully add some extra functionality.
• Object owner veto – Give the owners of various objects the ability to disallow comments being made on those objects (defaulting to permissive).

That’s the contents of my brains on this subject at the moment. If I have missed anything obvious or if anyone has any suggestions for the less obvious please feel free to add a comment.

The tagging feature has been recognised as a vital part of any self respecting set of web2.0 extensions. We’ve already had requests for them from the Linnean Society who would like to see them alongside the prototype comments and bookmarks already on their site. Consensus on the importance and usefulness of tags seemed to be one of the (many) themes to have arisen from the EPrints & Web2.0 pow wow.

It was also suggested that tags can be and are used in varying ways (eleven apparently!). For example bookmarks can be thought of as a tag under which an arbitrary set of references to things can be stored for the use of a single person. If bookmarks are implemented as a ‘view’ of the tagging system they immediately gain all the extra good things that tags offer (sharing, etc). Other uses mentioned were ‘rating’ (by tagging an item as ‘good’, ‘bad’, ‘terrible’) or even as a way of organising tasks (‘to read’, ‘to buy’, ‘to review’).

With this in mind I want to make sure that the SNEEP.tags database structure is as flexible, extensible and scalable as possible. A bit of effort on the DB side of things should hopefully mean that when it comes to producing views and interfaces for the tags the various uses can be easily accommodated and if appropriate (as in the case of bookmarks) re-branded as a new feature.

One resource on database design for tags is this webinar. I haven’t seen it in action as it appears to be significantly awkward to get hold of a Linux version of the Webex software needed to view it. I have attached the slides which seem to give a good gist. It is written from the point of view of MySQL so it is very much about how to get the most out of the DB no matter the amount of content.

Web 2.0 Tags and Folksonomy: Schema Design for Scalability and Performance

The examples in the slides correlate to a tagging system for a blog, so tags are related to posts. This approach, tagging elements by id in a closed system, is not difficult to map to EPrints where tags can be related to items (eprints) or indeed anything with a unique id (documents, comments, users).

One aspect of this approach that struck me is whether it might be difficult to do searches which combine the various item types. My approach so far has been to maintain an ‘all’ counter in the stas table. Any thoughts on the merits or otherwise of the approach outlined here would be welcome.

The agenda is flexible. This is an informal event, but it would help if anyone wishing to come could notify Richard Davis or Les Carr . A list of expected attendess is being maintained here: http://tinyurl.com/36sc9r

We’re extremely grateful to the London Knowledge Labs (Institute of Education/Birkbeck) for letting us use their excellent facilities in Holborn for this event.

EPrints Web 2.0 Pow-wow

December 13th 2007, 10:30 – 17:30
London

Agenda

Before lunch:

• Who we are, what projects we represent.
• What do we understand by “Web 2.0″?
• What do we want Web 2.0 to do for our repositories?
• What progress have we made so far?

Lunch arrangements TBC.

After lunch:

• A look at EPrints:
• What’s the easiest way to achieve our goals
• Overlaps between our projects & activities
• Co-ordinating activities and sharing outcomes

Followed by a chance to network further at a nearby hostelry.

Location

London Knowledge Labs
23-29 Emerald Street
London WC1N 3QS
http://www.lkl.ac.uk

LKL travel info: http://tinyurl.com/ywmsvc

Contact

Richard Davis, ULCC
r.davis@ulcc.ac.uk 020 7692 1350

Les Carr, Southampton University
lac@soton.ac.uk