Comments on: Working with Web Curator Tool (part 2): wikis

By: Richard M. Davis

Richard M. Davis — Fri, 03 Jun 2011 17:30:00 +0000

Thanks for the update about your tools, emijrp, we will look into this for future projects.

By: emijrp

emijrp — Mon, 30 May 2011 19:56:30 +0000

Hi;

There is an excellent tool for extracting all the info (text, histories and images) of a MediaWiki wiki, it is called WikiTeam tool.[1]

It has been tested with several MediaWiki versions. You can see a list of preserved wikis ad the downloads section.

The output format is a big XML with all the text and metadata (asy to import using MediaWiki import tools) and a directory with the wiki images.

Regards,
emijrp

[1] http://code.google.com/p/wikiteam/

By: JISC-PoWR » Blog Archive » Archiving a wiki

JISC-PoWR » Blog Archive » Archiving a wiki — Wed, 25 Mar 2009 14:05:24 +0000

[...] dablog recently I have put up a post with a few observations about archiving a MediaWiki site. The example is the UKOLN Repositories [...]

By: Gordon Paynter

Gordon Paynter — Sat, 14 Mar 2009 09:08:38 +0000

Hi Ed, Maureen:

We’ve encountered pretty the same Wiki issue as Ed describes at the National Library of New Zealand. The curators used exclusion filters much as Ed describes to limit the harvester (I will also be sending them this article in case they need to harvest mediawiki).

The biggest problem is not the edit history, but the differences between versions of a page. A page with 10 versions will have dozens of “diff” pages (Ed’s .*&diff.* pattern presumably takes care of this). So even if we want to capture the edit history, we don’t want the differences.

We have similar but more frequent issues with blogs. On some platforms each post appears on the homepage and on its own page, but also separately on the archive pages for the relevant year, month and day, and also on the page for any tags. So you can wind up harvesting the same content many times over. Again, crafting exclusion filters is the solution.

Gordon

By: Richard M. Davis

Richard M. Davis — Wed, 11 Mar 2009 17:11:14 +0000

Maureen – sort of… though I was thinking more of a view of the document, so that the timeline slider plays through the amendments as if they are being entered live – you know the sort of thing? Bit like this (but not too much

By: Maureen Pennock

Maureen Pennock — Wed, 11 Mar 2009 16:00:08 +0000

Ed – I agree, it comes back to the user community. I guess I’m just not clear on who’s ‘in’ the UKWAC user community, let alone how it may change in the future. Oh, for a crystal ball… !

Richard – hmm, a graphic, progressive change animator tool. Like a Dipity interface for the web archive? Neat idea

By: Richard M. Davis

Richard M. Davis — Wed, 11 Mar 2009 11:12:30 +0000

I don’t know if many people ever actually try to follow the authorship trail on (say) a Mediawiki page from start to finish. For example, the notorious Ronnie Hazlehurst/S Club 7 controversy on Wikipedia: there’s solid audit info there, no doubt, but there’s probably work for a whole new breed of auditor in actually working it out

Maybe a graphic, progressive change animator would be a useful tool in a wiki archive?

By: Ed Pinsent

Ed Pinsent — Wed, 11 Mar 2009 09:43:55 +0000

Hi Maureen
Many thanks for adding a comment. You make a good point. I agree there is an argument for capturing history pages from a wiki, and it’s likely “there will be people who want to know what was done by whom and when.” But I wonder if the UKWAC user community are those people? Perhaps a full record of this blog’s change history is more likely to be of value to UKOLN (primarily), and to the wider community of information specialists who are interested in digital preservation. So maybe this raises another possibility for web-archiving; different levels of capture, and capture of different types of content, depending on the requirements of your user communities.

By: Maureen Pennock

Maureen Pennock — Wed, 11 Mar 2009 08:52:07 +0000

Hi Ed,
this is really interesting, thanks for posting it. I’m particularly interested in your selection decision to capture the main pages and not the history/edit/discussion pages. I see where you’re coming from when you say that ‘scholars who browse the archived copy of DigiRep are not expecting to be able to edit pages, nor join in the discussions, nor browse the history of stored versions of pages.’ I agree insofar as the edit pages are concerned, but there’s an argument for capturing the history pages as an audit trail of who contributed what and when. This is interesting and valuable stuff – a particular (or should that be significant?) characteristic of wikis is that they are collaborative tools. The history pages are evidence of this and I imagine there will be people who want to know what was done by whom and when. You could make the same argument for the discussion pages, but in my experience the discussion pages are not widely used. I haven’t looked at the ones in the Digirep wiki so can’t say whether that’s the case there. But as I said, all interesting stuff, thank you!