You can read all about the project at the CORE Project Website. Without going into the clever system architecture and knowledge-management algorithms, there seem to me to be two especially important and interesting outcomes for Institutional Repositories:

• A CORE Plugin that can be easily dropped onto pretty much any repository web template, into the item’s abstract page. It will show a list of “Similar Items” selected from across the whole UK OA repository system (any repository registered with OpenDOAR, I think).
• A CORE Mobile app, providing keyword search across the UK OA repository corpus, also offering the “Similar Items” functionality.

Petr Knoth, Owen Stephens and the project team are working on improving both of these. You can see the Plugin working on EPrints at the Open University’s ORO repository (be patient, it takes a few extra seconds to load, something that’s being worked on right now).

The Mobile App is available from Google Play (formerly the Android Marketplace), and works fine on my HTC phone. An Iphone/Ipad version is expected in the ITunes store in the next few days.

This is a significant contribution to the growing landscape of enhanced search and discovery tools for repositories. On the one hand, we have the development of RDF-based linked-data/semantic metadata tools, notably the built-in support in recent versions of EPrints. Those of you with long memories may recall that using non-invasive JQuery scripts to exploit and expose embedded, rich, semantic metadata, was the basis for our winning entry in the Open Repositories 2010 Developer’s Challenge.

On the other hand we have search tools based on semantic textual analysis, such as the excellent MIMAS (formerly Intute) Repository Search, and the experimental work that ULCC and UCL undertook for the MERLIN project. When mobile computing really took off, after the launch of the Ipad in early 2010, we even discussed the possibility of a mobile interface for the MERLIN search tools, and submitted a demonstrator application for the Open Repositories 2011 Developer’s Challenge.

The CORE mobile app is extremely plausible and promising, and seems to me the best contender yet, to realise the dream of an ‘intelligent’, mobile OA VRE, and I’ll be watching it with interest.

There are some extra features I’d like to see it acquire.

• Distinguishing published, peer-reviewed papers from ‘other stuff’ in repositories
• Filtering by attributes such as institution, language, country-of-origin (assuming it might ultimately go global)
• Providing bookmarking within the mobile app (which inevitably leads us down the road export of citations, reading lists etc.)
• A rating system within the mobile app

Probably beyond CORE’s scope, but the icing-on-the-cake, would be any way to de-PDF the items in the majority of repositories. That probably requires a much bigger cultural shift, to steer authors and editors away from obsessing about the fussy, pseudo-print of PDFs, and make EPUB the preferred format for OA research articles.

We’re looking forward to hearing more about CORE in the near future…

A worker displays a page from the Le Description de l'Égypte, or Description of Egypt, that was salvaged from the ruins of the Institut d'Égypte.

Conflicts have shaped us all. From wars to civil unrest, in many ways, from the collective to the personal. Culture, often described as humankind’s greatest accomplishment is often a victim of humankind’s greatest failure, war. Memory, however has also shaped us all, from the personal to the collective. War frequently threatens the interpretation and reinterpretation of collective memory though the destruction of repositories of heritage, books, archives, the destruction of the record not to mention the repression of culture, language and the folk memory. These seeming benign repositories of heritage often transform overnight to being malevolent symbols of a previous regime and for many are ‘just’ targets for destruction, for the anihilation of the past. This is wrong and a tragedy among the tragedies that comprise war.

The House of Books conference on the risks and threats to cultural heritage in Europe and the Middle East was held in Florence last month. The NGO Un Ponte Per…’s House of Books project has worked in collaboration with the Iraq National Library and Archives (INLA) in relation to capacity building and the reconstruction of the INLA since the Iraq war in 2003 and I have participated in workshops in Jordan and Iraq for this project. It has been a fascinating project and you can resd more about it “>here.

Speakers included the main funders of the project including the EU and UNESCO as well as the Comune di Firenze. Dr Saad Eskander, the Director of the INLA gave us a progress review of their work and their work building a digital library for all types of materials related to Iraqi history about all Iraqi people. The British Library gave us a summary of their involvement in the reconstruction of the INLA through books and digital surrogates of documents and publicatiosn relevant to Iraqi history. The British Library also hosted and publicised the blog established by Dr Eskander and myself under the auspices of the Archives and Records association, UK and Ireland which brought the INLA to the world’s attention.

Destruction of the Institute D’Egypte, Cairo

William Kopycki of the Library of Congress in Cairo gave us an overview of the attack on the Egyptian Scientific Institute in Cairo in December 2012. You can read more about his experience during the initial uprising in Egypt here. The Institute, considered one of the most important and oldest Arab institutions, was set on fire during clashes between protesters and security forces on December 17th. A large collection of books, manuscripts and maps were damaged. Napoleon Bonaparte established the institute in 1798 to advance science and spread knowledge in Egypt, as well as to facilitate research and documentation of historical events and industrial facilities in Egypt. It was burned to a shell deliberately with the 20 volume work known as the Description de l’Egypte one of the casualties. This book was the collaborative work of about 160 civilian scholars and scientists, known popularly as the savants, who accompanied Napoleon’s expedition to Egypt in 1798 to 1801 as part of the French Revolutionary Wars, as well as about 2000 artists and technicians, including 400 engravers, who would later compile it into a full work. It is still avilable in its entirity in digital form here. Social media was the most powerful and quick method of spreading the news about the crisis. People came from all over Egypt to help restore and help in any way they could. William described how the Library of Congress assisted in the recovery and restoration of material. and noted the importance of taking photographs of the ownership stamps in the books. This was in an attempt to make sure that the books did not get displaced or sold and indeed some books were stopped from leaving Egypt at customs. William sent these stamps to various ministries so they could identify books which were leaving Egypt and some books were indeed stopped at customs.

Sound of Iraq

A very exciting project presented to us in Florence was the Sound of Iraq. It is an ambitious project spearheaded by the musician Khyam Allami (who also gave us a recital in the evening) dedicated to creating and supporting a new Iraqi National Sound Archive (INSA) as part of the Iraqi National Library and Archives (INLA) in Baghdad, Iraq.

‘During the mass looting following the 2003 war, the Iraqi National Radio and Television stations’ sound recordings disappeared into private hands and/or the black market (along with much of Iraq’s cultural heritage including books, artworks, and other critical cultural productions). This saw private collections of sound recordings scattered around the world, especially with the mass migrations of Iraqis to exile.

Post the 2003-war, culture and particularly music has been of little significance in the political agenda of development in Iraq. However, with the rise of online-media platforms such as web forums, and YouTube video channels, hundreds of Iraqi sound recordings have been shared with the public.’ Taken from Sound of Iraq website April, 2012.

The project has trained of staff from the INLA in sound archiving (our very own DPTP graduate Waleed Alnaqshabandi) at the British Library and is now looking towards the next stage. It is impressive to see how much thought and consideration is being given to good archival practise and the use of standards not just for access but also preservation. An example being the publication currently prepared is the Arabic/English dual language glossary of sound archiving terminology.

The ambitions of the INLA will culminate in the construction of a multi-media digital library. The INLA’s needs have changed a lot since the dark days of 2004. It is now a guiding force in terms of good practise and leadership in Iraq in relation to librarianship. Now we see a flagship library both intellectually and in terms of its infrastructure.

Threats to digital heritage

The extremely important part being played by digital media is exemplified in these projects, they enhance access and have played an important role in the regeneration of the INLA. Digital surrogates have also acted as preservation copies as is the case with the ‘Description of Egypt’. However the downside to the digital is ever present. While war poses a real and present threat to all heritage be it digital or analogue, a much greater threat to digital heritage is not in the shape of war but due to simply benign neglect. The value of the House of Books workshops was in providing a space for discussing these points and improving best practise in realtion to creation, management and sustainability of digital material. The many digitsiation projects taking place in the counries which took part in the workshops exemplified this. The future is digital and as much as we must mind our analogue past tomorrow’s historians will be using a largely digital historical record for their research, which will transform the way they research, and present the past. An example of this is social media which has been used to mobilise people to act in many ways in many countries. How shall we keep this as a record of the Arab Spring? History, be it personal or collective is now overwhelmingly digital and needs interventions and considerations sooner than later otherwise it will be lost and not due to a catastrophe such as war or natural disaster but due to simple benign neglect.

A cursory glance at how well the Hague Convention has protected cultural property during times of conlfict demonstrates that it doesn’t work very well, to put it mildly. That doesn’t mean that we shouldn’t have these laws, we must have them. But in the height of battle the Hague Convention is not going to protect the bridge/mosque/sysnagogue/library etc. Why are we not getting better at this? As convictions at the International War Crimes Tribunal add up so does case law and perhaps they could be a slight deterrant but the fact is that the bridge/mosque/synagogue or library are gone and irreplaceable. What is more powerful in my mind is intellectual protection allowing education about various perspectives with regard to history from a very young age, legislation which protects heritage both digital and analogue, at a national level and disaster planning.

If war is, as many consider an inevitable fact or muktab (an arabic word which means ‘it is written’) then we need to find realistic ways of protecting heritage. We need to transform people’s thinking from a young age about history, so that if their history is that of a dictatorship or occupation, develop understanding that it is their history: the evidence essential to rebuilding their state or a community. By destroying it they destroy themselves and their memory. Obviously this is not easy. However what impressed me throughout this weekend in Florence was that even if you have solidarity officially between sister institutions around the world, it always seems to depend very much on the individual at the helm. If they are engaged/committed then well and good, if they are not then it simply doesn’t work. So we should never underestimate the power of the individual to make a difference be they those working on the conservation of ancient texts in St Catherine’s Monastery in Sinai or those preserving digital libraries in Iraq. It is clear that irreplaceable cultural resources are the constant victims of unrest and this is a matter for concern of all of us who care about the world’s cultural heritage. Power to the people who make a difference.

It was a collaboration between myself and Kit Good, the records manager here at UoL. We’re quite pleased with the results. We wanted to see if we could create preservation copies of core business documents that require permanent preservation, but do it using a very simple intervention and with zero overheads. So we worked with a simple toolkit of services and software that can plug into a network drive; we used open source migration and validation tools. Our case study sought to demonstrate the viability of this approach. Along the way we learned a lot about how Xena digital preservation software operates, and how (combined with Open Office) it makes a very credible job of producing bare-bones Archival Information Packages, and putting information into formats with improved long-term prospects.

The project has worked on a small test corpus of common Institutional digital records, performed preservation transformations on them and conducted systematic evaluation to ensure that the conversions worked, that the finished documents render correctly, that sufficient metadata been generated for preservation purposes, and that it can feasibly be extracted and stored in a database; and that the results are satisfactory and fit for purpose.

The results show us that it is possible to build a low-cost, practical preservation solution that addresses immediate preservation problems, makes use of available open source tools, and requires minimal IT support. We think the results of the case study can feasibly be used by other Institutions facing similar difficulties, and scaled up to apply to the preservation of other and more complex digital objects. It will enable non-specialist information professionals to perform certain preservation and information management tasks with a minimum of preservation-specific theoretical knowledge.

Future-Proofing won’t solve your records management problems, but it stands a chance of empowering records managers by allowing them to create preservation-worthy digital objects out of their organisation’s records, without the need for an expensive bespoke solution.

Father Najeeb Michaeel examines a manuscript

Father Najeeb Michaeel is an Iraqi Christian priest who speaks Arabic, English, French, Aramaic and Syriac, not to mention being able to read Latin and Greek. In the garden of Zaytun library, Erbil I hear this gentle man tell me how his community of friars used to live in Mosul, a traditional centre for Christianity in Iraq, having the highest proportion of Assyrian Christians of all the Iraqi cities. Father Najeeb’s community has  had to leave Mosul due to persecution.  Later on during The House of Books workshop he gives us a presentation of the magnificent early Christian manuscripts they are digitising.  Over coffee he gives us a moving rendition of the ‘Our Father’ sung in Aramaic.  I wasn’t expecting to feel so moved by a  religion I have become increasingly frustrated by, and in Iraq.

Early Christian manuscript, Centre Numerique des Manuscrits Orientaux, Mosul, Iraq.

Iraq has often compared to a mosaic in terms of the diversity of its religious diversity.  Iraq is a Shia majority country and contains the sacred Shia cities of Najaf and Karbala. Most sources estimate that around 65% of Iraqis follow Shia Islam, and around 35% follow Sunni Islam. What is not so well known is that Christians have inhabited what is modern day Iraq for about 2,000 years. The person who is supposed to be respnsible for the transmission of Christianity in Iraq is St Thomas the Apostle. Assyrians (also called Syriacs and Chaldeans) most of whom are adherents of the Chaldean Catholic Church, Syriac Orthodox Church and the Assyrian Church of the East account for most of Iraq’s Christian population, along with Armenians.  Tariq Aziz was born to an Assyrian family and is a member of the Chaldean Catholic church. There are also small populations of Mandaeans, Shabaks, Yarsan and Yezidis. The Iraqi Jewish community, numbering around 150,000 in 1941, almost entirely left the country. There are also Gnostics in the form of Mandeans and sub sects thereof, Yazidis who believe in a god but have a blue peacock angel in their pantheon, and of course the Zoroastrians which the ancient Babylonians followed.

Early Christian manuscript, Centre Numerique des Manuscrits Orientaux, Mosul, Iraq.

Despite this diversity they share some things, one being religious persecution within Iraq.  Estimates for the numbers of Christians suggest a decline from 8–10% in the mid-20th century to 5% at the turn of the century, to 3% in 2008.  About 600,000 Iraqi Christians have fled to Syria, Jordan or other countries or relocated to Iraqi Kurdistan.which is also the traditional homeland of the Assyrian people. Those who remain are very aware of their minority status and the threat to their lives. Another shared thing they have in common is a respect for their heritage, both in terms of artefacts and records.  In the aftermath of the war in Iraq where countless manuscripts where destroyed, these communities are well aware of the importance of their documentary heritage. Each group have their own manuscript collection. In addition, an attempt by Saddam Hussein to centralise private collections of archives has also made them wary of any notion of centralisation/government control. This has resulted in many collections being hidden away and lost.

 It was in this context that I met Father Najeeb in Erbil. He was there to speak about his work at the Centre Numérique des Manuscrits Orientaux (CNMO) Mosul  and their work involving the digitisation of their  Iraqi Christian manuscripts. This is a small scale project conducted by the Dominican community in Kurdistan. Their move to Iraqi Kurdistan was due to their persecution in Mosul and the government in Kurdistan ensures as much they can the community’s safety.  Father Najeeb and his community are being helped by Father Columba Stewart, a Benedictine monk in St John’s monastery in Minnesota. Based at the the Hill Museum and Manuscript Library, the library began in the cold war, hoping to retain a record of Europe’s heritage in case the Soviets came.  Father Stewart’s  goal since 2003 has been to  digitise as many Eastern Christian manuscripts in the Middle East as possible, because  these manuscripts are endangered from a variety of causes. The main danger is the ethnic genocide which has afflicted Iraq but also neglect.

CD of a collection for donor

Father Najeeb’s aim is clear, to preserve and generate awareness and interest in these ancient Iraqi Christian manuscripts and protect their heritage from disaster and cultural genocide. The Centre actively collects and digitises collections from private donors, who in turn get a copy of their manuscripts on CD (see above). I met other members of the Christian community in Iraq who had found a safe haven in Ankawa a district of Erbil which we visited. They all speak several languages including Arabic and Syriac, an ancient language closely related to Aramaic.  While Erbil is known as a safe haven withing Iraq, driving around in Father Najeeb’s car  one realises that hanging a rosary bead from a rear view mirror is not a casual gesture a brave declaration of faith as is wearing the traditional clothes of a Catholic priest.

The manuscripts which Father Najeeb is digitising are extarordinarily beautiful, and all the more so considering their provenance, age and the number of destructive forces they have endured. They have of some similarity to other e arly Christian manuscripts of the age, as the religion spread as far east as Iraq and also to the wilds of western Ireland where scribes worked on  similar texts.  In some way digitisation and the digital age has brought people together again  to protect these texts and hopefully raise awareness of this and other vulnerable communities.

Young Iraqi Christian at Centre Numérique des Manuscrits Orientaux

We are very pleased to announce that the next Digital Preservation Training Programme will take place on the 14th -16th November 2011, in SOAS, London. The Digital Preservation Coalition is providing three scholarships to attend and applications are invited from DPC members.

Full details are available on the DPTP website.

Flying into Erbil at night

“What you destroy, we will rebuild, only better” – Slogan of Kurdish Peshmerga.

The garden I am standing in is so beautiful that I find it difficult to imagine that it was a former detention centre  operated by Saddam Hussain’s Ba’ath party, a place  of imprisonment and torture.  It is now a garden full of  flowers  and trees and in its centre rises the impressive Zaytun Library of Erbil.  This is no accident, the Kurdish Peshmerga vowed that all these sites would be rebuilt this way once Saddam’s regime ended and the people would reclaim such poisoned land for purposes such as libraries and gardens. Erbil or or Hawler as it is called by locals like much of Iraq has seen a lot of history pass its way, Alexander the Great sorted out the Persian King Darius near here and the citadel of Erbil is the oldest inhabited city in the world and a soon to be UNESCO heritage site.

Erbil citadel

But let’s take a step back. What is a London based Corkonian doing in the middle of former detention centre/ garden in Iraqi Kurdistan? This  region in the north is the ancestral homelands of the Kurds – the oft persecuted minority in Iraq.  The Kurds constitute the largest minority without a homeland. I was at the library as part of the third House of Books workshop funded by the EU and UNESCO and run by a Humanitarian NGO called Un Ponte Per…. You can read more about their involvement here. It is the last in a series of workshops which has been looking at digitisation of texts and their preservation and its main partner is the Iraq National Library and Archives (INLA). Many institutes from Iraq joined us including the National Museum of Iraq, Centre Numérique des Manuscrits Orientaux and other projects. From the Middle East the  National Library of Jordan and the  American University of Beirut also took part. My story with the INLA goes back to 2004 when I managed after some effort to persuade Dr Saad Eskander to write his  diary about his day to day life reconstructing the destroyed library in Baghdad.

Iraq National Library and Archives

The INLA was destroyed during and post war in 2003.  Of its 417,000 books, 2,618 periodicals dating from the late Ottoman era to modern times, and a collection of 4,412 rare books and manuscripts, an estimated 60 percent of its total archival materials, 25 percent of its books, newspapers, rare books, and most of its historical photographs and maps were destroyed in various ways. This was not just a loss for Iraq, it was a catastrophe for the world on many levels.

In 2011, Dr Eskander has built up a library few could ever have imagined possible. The INLA now leads the way in much best practise in librarianship for both traditional and digital material in Iraq.  There are also many digitisation projects being hatched in Iraq and around the Middle East, big and small and the drive to join up previously physically separated collections of journals, manuscripts, photographs is strong. The potential of the digital has long been recognised as a powerful means of disseminating information in the Middle East. The workshop  is trying to make projects understand that digitisation has a catch, and that is preservation or how to ensure that access is maintained over time. This idea was well introduced in Jordan during sessions such as ”Digitisation is not preservation’ and other catchy titles, and the event this time saw progression and developments since Jordan. In fact it seemed that projects had reassessed their approach to digitisation. Some would admit that where previously they were just scanning, (blogs passim) they revisited their projects in light of what they had learned in Jordan. Some  added technical metadata which they had not done at point of digitisation, others looked at having master copies as well as access copies of their digitised content and kept in different locations.  Policies were reconsidered. Storage solutions were considered. All steps in the right direction. The value of having 2 workshops in a year with a lot of the same people in both proved useful  as there seemed to be a lot of consolidation and desire to demonstrate improvements between one meeting and the next.

Workshop overview

The workshop was kicked off by UPP, and some short contributions by the EU’s representative in Kurdistan, Hala Al Sharifa , followed by the UNESCO programme officer for Kurdistan, Sami Al Khoji who is clearly dedicated to his role and the revitalisation and distribution by digital means of information about Iraq’s cultural assets. Kanan Mufti who is  director general for antiquities in the western Kurdish region in Erbil and who resided in the ancient citadel reflected on the importance of documentary heritage for Kurdistan and declared the protection of documents as a priority for them.  Presentations the INLA showed us they have been digitising and accessioning digital content since 2008 and we heard about their  plan to develop an Iraq digital library making its materials available to all Iraqis online.  The material will cover all aspects of Iraqi life and society and all forms of document. The INLA sees as vital its contribution to intellectual and scientific research in Iraq and also endeavours to support programmes which will end illiteracy in Iraq. The INLA has embarked on training programmes in many aspects of digital library management. They have even managed to send 2 people to attend the Digital Preservation Training Programme, thanks to the British Council and BISI.

The National Library of Jordan spoke about the need for standardisation in the region in relation to cataloguing and indexing. The  American University of Beirut ‘s presentation lead on nicely as Basma Chebani reflected  on Arabic ontologies and the need for authority files in the Arabic speaking world. My 2nd session on metadata fitted in well here and we did a nice hands on exercise, working with the group of 30 through Arabic and Kurdish, it seemed to hit the mark and the right level. The translator also did a great job helping me, he is now a metadata expert!

Father Najeeb of the Dominican Order in Iraq who spoke movingly about their ancient texts and manuscripts and their ongoing digitisation. A small project with great ambitions from a community constantly under threat. I plan to write more about this in another blog.

Early Christian manuscript, Centre Numerique des Manuscrits Orientaux, Mosul, Iraq.

Many copyright issues arose, in particular that of forgeries in the traditional manuscript environment. It seems that a lot of illegal copying takes place and it is difficult to contain. Issues such as translation were interesting. During my session on digitisation and preservation the Iraqi born but Aberystwyth-reared translator ran out of his booth proclaiming, ‘What is a plug in?!’ To which everyone loudly  had an opinion in return.

Translator in action

The translator explains metadata

I led the concluding session, working with the group about recommendations for next steps arising from the workshop. This proved interesting, considering I was again working with a translator with a group who spoke Kurdish, Arabic and Syriac, not to mention an Arabic English keyboard! Getting individuals to think by themselves and for themselves about what they would accomplish on their return to work is good in a big group. They then worked as teams of 4 to consider what the next steps would be in Iraq for libraries and archives.  The key issues were networking, the establishment of a national network or syndicate of librarians and archivists and information specialists involved in libraries and archives is deemed of great importance. The group also want to consolidate the information from the workshops and ensure that people are not reinventing the wheel in terms of developing best practise. The establishment of education and training programmes in all aspects of librarianship and archives is also vital.

Metadata matters but other things do too.

What was also key and hidden beneath the discussion of texts and metadata and the like was that this was a   moment or space away from the day to day. These people work in circumstances which we cannot begin to comprehend. Just to come to the workshop involved endless checkpoints and danger. Life is unsafe and violent. Civil society as we know it here in the UK is almost non existent in most of Iraq. Electricity cuts are regular, resulting in me being stuck in a lift fo 5 minutes. This is not the same for Erbil on the whole but most colleagues came from Baghdad, Mosul and other regions which are the news for tragic reasons.

The value of this little group of archivists and librarians from different ethnic and religious groups is more than just about metadata and file formats (as important as they are) but about bringing disparate groups of people together with a view to the flowering of a new nation where religious and ethnic difference no longer matter, where censorship doesn;t exist, where ideas flow freely once again. This is the vision of the INLA director Dr Saad Eskander.  It is not an easy vision in a divided society where sectarianism is rife. However it is not so unusual to consider the power of libraries as a social phenomenom and yet we seem to treat them purely as an informational phenomenon. The House of Books demonstrates that it works on at least 2 levels

Iraq as everyone knows has  a violent history of occupation and war, however during periods of serenity, the emergence of civilisations who have made numerous extraordinary contributions to the history of civilisation, these include  writing, and the concept of zero or  sifr to name but a few.  Original texts survive from the era of Babylonian mathematics. On day 1 of archives school baby archivists learn that the Babylonians wrote on tablets of unbaked clay, using cuneiform writing. The symbols were pressed into soft clay tablets with the slanted edge of a stylus and so had a wedge-shaped appearance (and hence the name cuneiform). Experts studying these learned that the Babylonians had developed the concept of sifr or zero.

Cunieform from Babylonian times: Top: 64 (1 sixty + 4 ones) bottom: 3604 (1 sixty2 + 0 sixty + 4 ones)

Sifr is also used in Arabic to denote a clean slate, a blank page. In Iraq hard work has begun of the rebuilding from scratch  of a rich cultural heritage of Iraq for the future.  I am glad that the preservation of digital heritage of Iraq is a part of this.  More later!

Map of Erbil

I have been invited to the 3rd House of Books workshop organised by Un Ponte Per…. It will take place in Erbil, Kurdistan and is under the auspices of the National Library and Archives of Iraq. The workshop is looking at best practise in relation to digitisation and preservation, follwing on on topics discussed at the last workshop. Our keen and alert followers will notice that this project has already been on DA blog. I will be posting when I get back about this workshop.

If you haven’t seen it, can I recommend Kristen Snawder’s recent post on the Library of Congress Digital Preservation blog, Digitization is different than digital preservation. Kristen reiterates familiar points about the long-term commitment necessary for serious digital preservation, contrasted with the quick hit of a scanning project. “In the hurry to meet user expectations, institutions may scan large quantities of materials without having a solid plan for preserving the digital images into the future.”

However another recent find on the Web compels me to make an additional point, namely that we might do equally well to differentiate between scanning and digitisation. Anyone can set to work with a scanner and create a bunch of digital images – but that barely scratches the surface of what I think we should be expecting of a digitisation project in 2011.

First and foremost, we need metadata: the more the merrier, but something at least. Even if we expect to come back later and polish it up (once the images can be browsed and examined on screen). In the absence of any established metadata profiles for a project, at least try to cover as many Dublin Core elements as possible – title, creator, date, subject/keywords… Images, in particular, may prove tricky or time-consuming to find again, especially once there are thousands of them on a disk. We should probably keep the metadata in a database, and perhaps additionally store metadata with the objects. This can be as XML or plain text files stored alongside the digital images, or embedded in the files we create (many common file formats – TIFF, JPEG, MPEG, PDF – support metadata embedding, and there are many free tools available to help).

There is yet more, though, that we should be doing, particularly when we are scanning text-based objects (articles, books, magazines, reports, etc). Most importantly, we really should try and extract the text from the image if possible. ^[1]

My recent web find was the teaching blog of Dr Toine Bogers at the Royal School of Library and Information Science (RSLIS) in Copenhagen, Denmark. One fascinating post describes a Lab Session exercise, From OCR To NER, a set of comparatively simple command-line processes to get the most out of a scanned-text project.

Toine’s post walks us through the process. Once the article is scanned, we should apply some OCR. The exercise goes further and also describes the use of tools to clean up and spell-check the resulting OCR’d text. This will, at the very least, result in a separate text file, hopefully containing a fairly accurate version of the article text. Finally, the cleaned-up text can be submited to a Named Entity Recognition service. Toine’s exercise uses NER tools at University of Illinois. (We’ve been using similar functionality provided by OpenCalais and GATE for our AIM25 Open Metadata project.)

Why do all this? The most important, instant, result of this is that we can now easily index our article for full-text searching – in a local repository system, such as EPrints or DSpace provide – and of course by Google. None of this is possible if we leave the scanned image as just that – an image.

Another  side-effect of any successful OCR outcome, is that the text is now free to be re-flowed. This means that we might consider sharing it with users in a variety of forms enhancing usability and accessibility.

It’s important not to confuse preservation formats with formats for access and dissemination. You probably will have your scanned image masters in TIFF, RAW, JPEG2000, PostScript, SVG. None of these are likely to be of much use to your users over the Web. Not only are the formats not widely supported by Web browsers, but most users probably don’t need or want your master image. If it’s a high-resolution scan of a 100 page book, they might be looking at 100Mb download, or worse – slow to load, and probably slow to render and navigate.

Time taken thinking what formats will give users the best experience is time well spent. What platforms might they want to use now and in the foreseeable future? It’s less than 18 months since Kindle3 made e-book readers affordable, and the Ipad made them sexy. E-books look and function very impressively on both platforms (albeit in different ways): for an overview of some of the benefits of the EPUB format, see Martin Fenner’s post Beyond The PDF… is EPUB. PDF outputs may yet have their uses, if users can at least search for text within them. The point is that only with properly digitised text, do these kinds of accessibility options become possible.

Even image collections can also be disseminated as E-books – nice offline items some users might care to flick through on their tablet computers, possibly even smartphones. I’ve demonstrated how we can create OJS XML from EPrints XML on-the-fly with XSLT: since EPUB and Mobi/Kindle are XML-based formats, we should be able to do something similar to create e-books using repository APIs. Also, by using appropriately sized images in dissemination formats (Ipad screen is 1024x768px; Iphone4 is 960x640px) we can not only ship our users a sensibly-sized download, we can protect any capital we may have in the master images, without having to resort to ugly tricks like watermarking. (Giving users full-size, high-res images with embedded watermarks seems to me the worst of all worlds.)

Therefore I’d suggest that, in order to get the best out of a digitisation project, consider what would you like to see at the end of the project – and, more importantly, what would give your users the best experience, or even win you new users? Ask around, do some tests, with users if possible, and get an idea how they want to use the materials and how they will get the best out of them. Maybe there are comparable projects and systems that you admire, with features you’d like to be available for your collection. What about in five or ten years’ time: will your current project outputs help or hinder longer term accessibility goals?

This kind of vision of is essential. Without some conception of the end result, how the materials will be used and managed most effectively, all the scanning in the world isn’t going to amount to a successful digitisation project.

This survey is part of BlogForever, an EU-funded collaborative project that ULCC collaborates through the Digital Archives department.

The results of the survey, available at the end of the summer, will help to develop digital preservation, management and dissemination facilities for weblogs. Hence, we are keen to gather information about the content, context and usage patterns of current weblogs, so we could identify blogs users’ views on their long-term preservation, management, analysis, access and use. If you would like to take part on the survey please use the following link:

For a naïve non-tech archivist like me, this felt like a new way of looking at and thinking about digital data. The principle seems to be that it’s possible to capture an image of an entire disk, and then perform stepped analysis on all the data contained within that image. Most models of digital preservation, management or curation that I’m familiar with tend to focus on the file as the unit which we must identify, migrate, catalogue or preserve; here the target is different, and we’re dealing with a whole wodge of related digital spew that includes files, technical metadata, automated logs, and lots of system elements from the registry.

Digital forensics is often applied to “personal archives”, such as the disk image of a single author / researcher; or is used by commercial organisations seeking to prove something in a court of law; or in extreme cases in tracking down evidence of a crime. As presented by Leighton John, a number of characteristics of the forensic approach struck me as being quite resonant with the aims of digital preservation and/or electronic records management:

• No change: you have to capture information without changing it; demonstrate you have not changed it; and analyse it without changing it.
• Audit trail: forensics requires working out what happened in the past from traces of information, then having to defend it in a public place.
• It looks at everything: context is provided by examination of all data, records, files and system components in a case.
• It tells a story: forensics is the creation of a narrative, with lots of details and evidence.

This struck me as overlapping with aspects of records management, and descriptive of many features of what we’d expect an EDRMS to deliver. The “No Change” dimension is a crucial part of records management when it comes to the authenticity of the record, and also aligns with the requirements of BIP 0008 on legal admissibility of digital documents. EDRMS systems offer audit trailing as standard, but it’s mainly the audit trail of what happens to file objects once in the EDRMS itself. I wondered if digital forensics could indeed build this detailed technical “narrative” of record creation activity through analysis of an individual user’s disk image, and I asked a bemused question along these lines.

Naturally, forensics alone wouldn’t satisfy all the MOREQ requirements for records management, and we’d still need our mechanisms for managing retention and disposal. Some of the more obvious barriers to success would include:

• Privacy and rights: what member of staff working anywhere would welcome having their entire profile analysed? At the very least they’d feel pretty sensitive about it! But maybe it need not be all or nothing; there’s plenty of “rich data” in any given system and a lot of it could be associated with individual files without capturing the entire data image.
• The costs of doing it: I’m not sure but I get the impression it’s neither cheap nor trivial to carry out a forensic analysis of a disc image; it requires specialist technical skills, a dedicated environment, there are numerous steps to the process, and it takes a long time to complete.
• Potential loss of context, a problem which may be partially resolved by a specialist form of search and analysis, which can be used as means of indexing one’s forensics findings.

This is all just speculation of course, but I do like the mental exercise of thinking about digital data in ways that don’t involve mapping to paper analogues, a place where I think we have been trapped for too long.

Question: What do alcohol and a VLE discussion forum have in common?

Clue: you’ll have to read to the bottom of this article to find out.

History Spot at Institute of Historial Research ( IHR) looks very good.  IHR is one of the many jewels in the crown of ten member Institutes of the School of Advanced Study, part of the University of London. It offers many training opportunities for historians and the like. History SPOT is a platform and an opportunity seized by IHR to launch and disseminate IHR activities.  It will significantly increase and enhance access to the materials by anyone who needs these skills/information/knowledge.  What we saw so far, looks intelligently designed with the user at the heart of the model.

On Tuesday we gathered to hear more about HistorySPOT and also from the Open University about their issues with VLEs, they knowing more than a little about such matters.  History SPOT was launched in March 2011 thanks to a grant from an anonymous donor to develop the IHR’s online presence (especially to widen participation of its research seminar programmes).  It was soon decided that they would use this donation to develop two core areas of the IHR’s remit.  The first was to widen knowledge of and participation in their internationally renowned research seminars through the use of podcasting, live streamed events, and vodcasting (video).  The second was to provide for the first time, free access to content from our research training modules and courses.

Dr Matt Phillpott gave us an overview of presently available online training courses relating to the Humanities and the concluded that there were not that many. The Open University is the closest we have and of course  iTunes is an option but this is not perfect as it neglects much contextual information about the courses on offer.  The OU also ran us through their various offerings, including the online tasters they provide (all in Moodle) which used to be called Learning Space but now is called Open Learn.

So all in all putting courses into a VLE is a good thing right? Well, of course! At IHR it increases access to the wealth of material being built up at IHR and hopefully adds to IHR’s income stream which will enable it to deliver more high quality and useful stuff to put into History Spot.

An interesting discussion arose about the age cohorts of people using our VLEs, many young people will be very happy with a purely online experience but the quite high drop out rate at OU reflects that not all can motivate themselves in this way. Agreement was reached that it is very difficult to teach about the meaning of something as well as it is to train via a VLE.

Of course putting our courses on a VLE is not as simple as it seems. Transferring the pedagogical experience to the environment of a VLE is not straightforward.  To recreate an experience of the  group dynamic which occurs in a training course or that magic of the learning moment, we can resort to forums, discussion groups and the like. However these will only go so far and the more is not the merrier. In fact the more forums the less likely people are to contribute and more problems can arise.

A positive side to online discussions is that people who normally wouldn’t contribute in a face to face course may be more inclined to do so in a VLE.  However, and here is the pitfall, the experience of many hardened and wise VLE providers will indicate that all these courses and discussion groups need  constant moderation. People can write things and express themselves in ways online which they wouldn’t dream of doing face to face. Like alcohol it seems to have an emboldening effect and people will say things which whether intentional or not can be misconstrued and cause great upset. This has to be decided on when planning and designing the course.  The moderator will need to be alert to this at all times while running their course, and so the lesson is again ‘Moderation in everything’.

CERN Computer Centre

CERN Computer Racks

Last week I was in a very welcoming Geneva, exactly in the European Organization for Nuclear Research (CERN) to meet other partners working on BlogForever and to have several Invenio workshops. I felt very lucky to be in the hub of such an organization and to see how many young international students are getting the opportunity to be in the forefront of high physics research.

The Globe at CERN

CERN is home of the world’s biggest and most powerful particle accelerator, the Large Hadron Collider (LHC). This machine is installed in a 27 km circumference tunnel. The LHC records around 15 petabytes per year. All the data is stored in their vast computer centre, where open access and sharing has been the driving principle since their foundation in 1954 and an inspirational environment for the Web to be born there.

Invenio’s workshops showed us that their electronic document management system is robust and versatile, targets the management of more than 1.2 million documents and it can be used in 19 different languages. Its content is clean and complete. In just their High Energy Physics domain, they have around 700 collections and approximately 20K queries a day. As well Invenio is used for special programs like the UNESCO funded digital repos in Africa and EU funded projects like D4Science and OpenAIRE.

Me at CERN

In the case of BlogForever and Invenio, plenty of work to be done by the Invenio Team at the User and Document Service Group. At the moment, they have more than 30 readily available Python modules that can be adapted to the case of preserving huge amount of blogs. From the point of view of my work with repositories as part of the Digital Archives and Repositories Team at ULCC, I was inspired by Invenio’s advance search engines; indexing and ranking methods.

In a more personal level, if you are ever crossing the border between France and Switzerland near Geneva, get Tram 18 and hop off at CERN to see their Microcosm and Globe exhibitions.

Chris Awre, William Nixon, Rory McNicholl at the Texas Longhorns stadium

The conference kicked off with a keynote from Jim Jagielski of the Apache Software Foundation, describing the history and organisation behind Apache and its projects. It was observed by some in the Twitter backchannel that the talk could as easily have been from 2001 as 2011, but for all that it was a worthwhile reminder that, in all our efforts, we stand on the shoulders of the giants who created and maintain the infrastructure of the Web and the Internet. And also that many our endeavours benefit from a little more dedication and commitment than you can usually squeeze between 9-to-5.

The closing keynote was by repositories stalwart Clifford Lynch, who managed to touch on so many perennial repository themes, I won’t attempt to summarise them. There is a handy anthology of tweets about his talk on Storify.

In between were plenty of presentations and opportunities to meet friends old and new from the United States of Repoland – some we have worked with, some we would like to work with, and many with challenging ideas and insights into the many facets of working with repositories.

The OR conference hops back and forth across the Atlantic (I’ve previously attended OR08 in Southampton, OR09 in Atlanta and OR10 in Madrid). Unfortunately when the conference is held Stateside, the representation of the EPrints community tends to be noticeably smaller. Not that there aren’t EPrints users in the USA (we were particularly pleased to meet the team from Cal Tech Library, very happy users and advocates of EPrints), but the distribution of software platforms is significantly different from Europe in general, and the UK in particular (if you are interested in such things, you can check out the statistics at OpenDOAR). And of course travel logistics (and costs) are non-trivial. Luckily Rory and I had been saving our prize money from last year’s Developer Challenge!

While it eluded me in previous years, I think at last I am starting to grasp at least some of the salient points of the thing they call Duraspace (launched, if I recall, in Atlanta)! I’m certainly hoping to find time to take my free Duracloud trial. However other aspects still remain opaque to me. At one panel discussion about the prospects for implementing DSpace over Fedora (or Fedora under DSpace, depending which way up you look at it), I was surprised to hear a description of ongoing DSpace-Fedora alignment efforts as “more about the journey than the destination”. An enviable luxury: for the time being we need tangible outcomes for our repositories and customers, and that’s one reason why we’ll be sticking with EPrints for the foreseeable future.

Personal highlights for me are described elsewhere: the Developer Challenge, which we enjoyed immensely, and Changing Platforms the talk that I presented with Imma Subirats, of the UN Food & Agricultural Organisation, where we discussed migrating between repository platforms. Rory also had a chance to meet developers from Yale, who had worked on the other end of the SOAS-Yale Islamic Manuscripts collaboration, and show off some of his work for the SOAS repository. We were also hugely appreciative of the generosity of the EPrints team, who kept us generally amused and amazed, and kindly included us in their group dinner on the last evening.

Adrian Stevenson's got them all-night late bar open repository blues...

The facilities at UT’s AT&T Conference Centre were outstanding, as was the surrounding campus generally, including the Longhorns football stadium (with its insanely massive west stand) where the conference dinner was held. Austin has far more attractions than we could see in such a short time, and it is an impressive and vibrant city, from the spectacular grandeur of the Texas state capitol, to the noisy entertainment on 6th Street, where virtually every bar has some kind of rock or blues band playing. We were most impressed by UKOLN’s Adrian Stevenson who jammed on a borrowed guitar with the blues band in the Blue Moon bar at 2am. As if that wasn’t enough, our visit also coincided with the massive Republic Of Texas biker rally – an insanely noisy procession of up to 50,000 bikers through the main streets of the city. Our ears won’t forget OR11 in a hurry.

In addition to her work at FAO, Imma is Chief Executive for the E-LIS repository, a major international and multi-lingual repository of articles about Library and Information Science. E-LIS has operated since 2003 on EPrints, but last year migrated to DSpace, because CILEA in Italy, who generously donate support and hosting, now focuses exclusively on working with DSpace. The E-LIS migration has been largely successful, however a number of EPrints features on which the E-LIS editors and users depended, have been difficult to replicate in DSpace, or had to be put on ice. This is no reflection on the specialists at CILEA, but perhaps indicative of more profound differences between EPrints and DSpace, that aren’t always reflected in the usual comparisons of repository platforms, such as the otherwise informative JISC RSP Repository Software survey.

ULCC of course has just completed a repository migration from DSpace to EPrints for the School of Advanced Study. Our motivation was in many respects the same as that of CILEA – our expertise lies firmly in the EPrints camp. But I think the outcomes for our end-user community are more demonstrably positive: in fact I don’t think there’s a single feature of the new SAS-Space-on-EPrints that isn’t a major improvement over its previous incarnation.

Migration of metadata and data (at least from DSpace to EPrints) presented few issues (that weren’t of my own making!) – export, transform, import. Here the similarities between the models of the two platforms was extremely valuable. But we did encounter other significant differences, some of which are set out in more detail below.

Richard presenting Changing Platforms at OR11

Issues in EPrints

Perhaps the most significant issue we encountered with re-implementing SAS-Space on EPrints was the absence of built-in support for Handle persistent identifiers. Handle support comes out-of-the-box with DSpace, but not with EPrints, so the choice we faced was between re-implementing Handle support, or dropping it. We chose the latter, since the benefits of Handles to a relatively small IR like SAS-Space were not obvious, and so it was hard to justify the extra cost and effort. By ensuring that items kept the same ID when migrated from DSpace to EPrints, and implementing a simple rewrite rule, we have ensured that Handle URIs created while DSpace was operational continue to point to the same item – but for items added since EPrints went live, no new Handle URIs are coined.

(Shortly after we returned from OR11, an extended discussion broke out on Twitter, amongst several well-respected gentlemen in our field, about the benefits of using Handles. A considerable amount of scepticism was expressed about their usefulness.)

Issues in DSpace

Imma described some workflow issues encountered with the new implementation of her repository. The E-LIS team is accustomed to a very flexible EPrints-based workflow that allows items to have their workflow status changed quite freely. DSpace, by contrast, has a unidirectional workflow model, so that items cannot (for example) be reverted from Live to Pending, if some kind of error is spotted, but effectively need to be deleted and resubmitted. This is obviously a significant divergence between the superficially similar repository platforms.

Another example Imma gave of a perplexing feature of the default DSpace UI is the button on each abstract page that says “View Full Item Record”. It leads to a rather intimidating web page displaying the item metadata as Qualified Dublin Core. It’s not a very attractive display, nor is it actually a “data” rendering of the metadata (as you would get by explicitly choosing to Export As XML, or from some new-fangled Linked Data features). It’s not clear why this view would be of interest to general users of the repository: why is it there?

At OR11 I talked to several people working with DSpace, and all agreed that there’s room for improvement in the default Web UI. In some cases they have completely reimplemented the web templates. It’s also worth noting that the page layout in the default JSP UI is entirely implemented using HTML tables, and doesn’t pass W3C validation. For a Web application that’s nearly 10 years old, this is disappointing. (The alternative Manakin XML UI implements an attractive vision of UI abstraction using XSLT, but reports suggest that configuring/maintaining it is not for the faint-hearted.)

Quite a few Web design infelicities are perpetrated in the default Community, Collection and Abstract page templates. (During the conference, many of us enjoyed and applauded Simeon Warner’s timely rant,”Don’t bold the field name”.) Of course we can change them – it’s Open Source, isn’t it? – but is it unreasonable to expect default Web templates that are at least potentially usable as is? Of course the natural and reasonable response of the DSpace community is to ask that we report the issue as a bug or feature-request to the development team. Or fix it ourselves and share the fix. But where an absent feature is really important to a user (by which I probably mean a repository manager), then the choice faced is between “getting by” until it’s implemented in the core distribution, or doing it themselves (which probably means hiring a specialist developer to implement it for them).

Out-of-the-box

At the DSpace User Group meeting in 2007, I described how we considered that, back in 2005, DSpace offered a better “out-of-the-box” experience than EPrints. I never thought it was anything to write home about – in fact I remember being disappointed by the very UI issues I’ve described above – but to my untrained eye it did seem better than EPrints, at the time. But, as I’ve mentioned elsewhere, EPrints has improved remarkably since.

Of course a lot of people we admire have proved that you can create impressive repository systems using DSpace. It performs and provides a lot of essential repository functionality. Its Lucene search engine is certainly better than anything EPrints currently offers. But I’m still surprised how much more work seems to be necessary to make a DSpace installation as readily useful and usable as EPrints, and this seems to represent considerable additional cost in setting up DSpace.

I’ve heard it sometimes argued – in both EPrints and DSpace camps – that Repository setup shouldn’t be too easy, lest repository managers get in a mess and endanger the integrity of their system. In my opinion, as developers and solution providers, our job is to provide as many features and tools as possible to enable Repository Managers to manage their collections effectively and easily – not act as as gatekeepers to their systems and data.

By way of contrast, we have recently supported the Institute of Education (IOE) in setting up an EPrints repository of UK government publications, and we were pleased to see the repository manager called on us very little, other than to answer some questions and apply a few small configuration changes. The experience with SAS-Space has also confirmed to me that EPrints now has strong out-of-the-box appeal, and a rich set of features available through the Web UI, that enable a reasonably confident repository manager to get to work without needing to initiate a major technical project.

In the current climate, of straitened library budgets, this could make a considerable difference to the viability of a repository startup project. For a growing number of libraries and information services – not least at smaller research institutions, or in developing countries – that could be the difference between having a repository, or not.

Excitement at the OR11 Developer Challenge Show-and-Tell (Photo by @sparrowbarley)

An event that asked developers to demonstrate the Future of Repositories can only be considered a great success when it receives entries that include:

• Multiple real-time examples of using “Repositories As A Service (RaaS)”, not only exchanging data but also sharing sophisticated functionality between EPrints and DSpace – and even including an Android application
• A tool for bundling and depositing a whole raft of research related outputs from the Web via RDF
• A tactile repository search interface with dynamic search suggestions, specifically designed for tablets and smartphones
• A complete gesture and voice-driven system for depositing and searching in repositories

All these – and other great entries too – were achieved in a couple of days’ work during the course of the conference, for the annual OR Developer Challenge, and presented at a packed Show-and-Tell session on Thursday afternoon (true, there was free beer).

Stuart Lewis’s team were worthy winners with their RaaS project, particularly as they showed a genuine commitment to a cross-platform approach – something which, sensibly, backgrounds the individual software platforms, that often receive too much attention, and focuses on the Repository as an application and entity in its own right.

We were also really pleased to see a prize go to Patrick McSweeney and Matt Taylor. And enjoyed seeing Dave Tarrant stealing the show (again) with his live demonstration of using a Microsoft Xbox Kinect to submit items to a repository.

Our own entry may not have won, but several people liked it, and you may see more of it in future. For the second year running, the Developer Challenge was a great opportunity for Rory and me to concentrate on an idea that we’ve been kicking around, without having found a home for it in existing work (yet). This was true for our Semantic Metadata popup tools that won the challenge with last year.

This year we set about achieving our  longstanding desire to take the very tactile and dynamic search interface that Rory created for the MERLIN project, and turn it into a touchscreen app, for smartphones and tablets.  The results were pretty effective.

The MERLIN Mobile interface ("TEXAS") as demonstrated at OR11

The MERLIN interface on LASSO is quite complex, but at the heart of it is the tag cloud of related terms that the Termine text-mining suggests. This always looked like it might be good on a touchscreen, so we stripped it down, rearranged and tweaked it to make it viable on an Ipad screen. If you’ve got an Ipad, you can try it out by pointing your Safari browser at http://is.gd/texasweb. (It will work on desktop browsers too, but it looks best on a portrait oriented screen.)

If you’ve got an Android device, you can even download an app-based version of it from http://is.gd/texasapp. (It’s a bit cramped in a smartphone display, but still essentially working.)

If you’ve got neither we created this page to give you a rough idea what it looks like.

It’s worth mentioning that this is a real, live, working application: enter a search term (“logical positivism”, “climate change”, “Jeremy Bentham”,  …) and it searches over the full-text corpus of all articles in University of London Open Access repositories (SHERPA-LEAP consortium members), and makes suggestions about additional or alternative search terms, based on the results of the text-mining analysis of the articles

The hackathon approach of working closely together to create something quickly worked well again: Rory hacks, and I test and review each time he hits ‘Save’! All very agile and iterative.

In honor of our hosts in Austin, we decided to call the new interface Touchscreen Enhanced Cross-Search with Augmented Serendipity – or TEXAS for short.

Kudos to Mahendra Mahey, Peter Sefton and the DevCSI project for putting (and keeping) it all together, and to the awesome panel of judges (even though they didn’t pick us)!

Fáilte go Sráid na Banrighinn

The whole purpose of education is to turn mirrors into windows.  ~Sydney J. Harris

Football and digital preservation seem an unlikely combination but on May 15th Ed and I arrived to a slightly damp Glasgow celebrating en masse the end of the football season, to deliver our DPTP north of the border.  There, I am afraid the comparison between football and digital curation ends but it is worth noting that on Monday our students did not consist, in the main of bleary eyed Scots.  As I am myself half Scottish I was looking forward to hearing about the digital state of play in my mother’s country but curiously we had a very international group, many had travelled far from places such as the European Central Bank,  European Commission, PRONI,  and near  such as from the National Records Scotland (who clearly cannot get enough of a good thing as we were with them 4 weeks previous).  The group was very creative who seemed to work well together. I know  I know, we say this a lot but this group seemed to shine for many reasons.  They were keen to work together and shout out about their ideas right from the start. And though being a disparate international group they managed to work together very neatly and develop some great case studies for us.

Internationally (and sometimes nationally!) language has always been a barrier  for communication, but  it is also an enabler  and this is again where we see something such as the OAIS being a useful way of empowering people to be able to communicate with each other, and other professionals.  The DPTP rarely (and rightly so) has students just from the traditional information management world (libraries/archives/etc).  Many people now being redeployed from different parts of their organisation and as such are often unfamilair with the idea/concept of an archive or indeed of the traditional notion of the lifecycle of a document/record. This is where OAIS is very valuable due to the way it which it expresses the way a digital object should be ideally kept and uses a language describe this.

In many of our case studies with OAIS, we often find that many organisations are actually mapping very well to the OAIS which indicates that it reflects  good and real practise when it comes to managing our digital repository. Our job in the DPTP is to take someone who has never heard of the OAIS (suprisingly more often than not) and by the end of the three days have them fluent in the OAIS concepts both through listening and through application to their own environments.  Our feedback from the course has proven that this is a job well done….DPTP abú!

ULCC Digital Archives & Repositories Team, May 2011

Please introduce yourself
We are ULCC’s Digital Archives & Repositories Team, and you can generally find us in the basement of Senate House along with the rest of ULCC. We are five:

• Ed Pinsent and Patricia Sleeman, our digital archivists, provide digital preservation training and consultancy.
• Rory McNicholl is our lead developer and repository systems manager.
• Silvia Arango-Docio is project officer for Web archiving projects and repository support.
• Richard Davis is team leader, and actively contributes to all of the team’s projects, as well as pursuing new and interesting opportunities.

What does your team do?
Our department was created in 1997 to develop the National Digital Archive of Datasets (NDAD), and operate it for The National Archives. NDAD was originally a joint project of ULCC and Senate House Libraries, providing a specialised cataloguing and preservation service for government databases.

Since then we’ve worked on many digital archive and library projects. The NDAD service ceased in 2010, but we continue working on innovative projects for born-digital and digitised records, developing repositories and related information systems for education and research. We provide specialist training and consultancy for the HE and cultural heritage sectors through the highly-acclaimed Digital Preservation Training Programme (DPTP).

Our current or recent partners and customers include most of the University’s colleges and institutes, the British Library, the Parliamentary Archives, the JISC, the UK Web Archiving Consortium, and the European Commission. We work closely with the Digital Preservation Coalition (DPC), and local groups such as the AIM25 archives network and theSHERPA-LEAP consortium of University of London repository managers.

Tell us one thing about your work that colleagues might not know.
Digital preservation issues might seem esoteric, but they are fundamental to every electronic and computer-based activity – in other words, pretty much everything in business, education and research. Understanding the best ways to manage, describe and preserve all our electronic information – from a single email, to complex Web sites, or collections of digital photos or videos – is an essential 21st century skill.

What aspect of your work gives you the most satisfaction?
We are very lucky to work closely with information professionals from around the UK and internationally. Patricia has recently delivered preservation workshops at the National Library of Jordan; Ed and Silvia are working on Web archiving projects for the JISC and the EU; Richard and Rory won the annual JISC Developer Challenge at last year’s Open Repositories conference in Madrid.

What is the most challenging part of your job?
Keeping up-to-date with constantly changing ICT landscape in education, libraries and archives. Electronic information systems evolve continually, as do the tools and methods for managing them. Luckily there is a wealth of current information on the Web, if you know where to look! We actively share our thoughts and experiences through our long-running Digital Archives Blog and on Twitter.

Name one thing that would make your working environment better.
Sometimes it seems we work with everyone except colleagues in Senate House! It would be great to work more with colleagues at the Central University, and bring more of our accumulated experience to bear on some of the University’s information and records management challenges.

If you could meet anyone (dead or alive) who would it be and why?
We’re not going to agree on this any time soon. Suggestions so far include Sofia Loren, Hypatia of Alexandria, Richard Brautigan and Pete Townshend.

Name three hobbies team members pursue – the more unusual the better.
Most of us have small children, so hobbies take a back seat. Ed is an accomplished artist, writer, broadcaster, musician and samizdat publisher. Patricia is about to learn how to stain glass before starting Arabic lessons for the third time.

If, as a group, you stormed the musical charts, what genre of music would that be in, what would be your stage name and who would be your lead vocalist?
We could be a psychedelic beat group called the Dates Of Creation. Our lead singer would have to be Patricia.

Recently the British Library team approached all the users of the shared WCT tool. It seems that the curators involved in UKWA have been using these metadata fields slightly differently and the BL team have initiated a project to move towards more consistency. The project will involve deciding on definitive interpretations of how to use these fields, followed by a process of cleaning up legacy data stored in the system. Some of it is potentially useful, some of it not so useful; some is legacy from the earlier PANDAS phase of the project, mostly not needed, or entered into the wrong field.

As noted, a lot of this metadata is mainly to do with selection and evaluation decisions, curation information such as changes in status of the site, and as such it’s never been exposed anywhere except within WCT. However, one descriptive field will eventually end up exposed on the UKWA live site, and provide us cataloguer types an opportunity to describe the resources in more detail. It will appear on the Title Entry Page (TEP) for each instance.

I welcome any move towards exposing more descriptive metadata on the UKWA public site. I have always taken the view that the phrase which currently appears alongside a Title “The live site may provide more information” is not really very helpful in the context of a web archive, for three reasons. (1) we don’t want our users clicking away from UKWA; (2) the link to the live site may be dead by now, and; (3) as archivists and curators, I feel strongly that we are the ones who should be providing that “more information” in the shape of a catalogue description of some kind.

The JISC project sites, as a collection, have high evidentiary value as stages in development of very specific tools, services and activities that benefit the UK Higher Education community. The sites by themselves don’t always explain their history or intentions; I would argue that a lot of rich contextual detail about the reasons these sites existed (the JISC programme under which they were developed, the dates, the staff involved, the themes, the outputs) would help interpret the collection to the users and make it more intelligible.

A glimpse of Petra, Jordan

‘Machine replication of human functions, like reading, is an ancient dream’ *

One of the many topics discussed in the House of Books project in Amman was the issue of OCR and Arabic texts. Optical character recognition or OCR has become one of the most successful applications of technology in the field of pattern recognition and artificial intelligence. It is now a necessary step in the transition from analogue text to the elctronic world, particularly due to the quantity of information now available in the electronic age as it enables rapdi searching and scanning. In the last five decades, machine reading of text has grown from a dream to reality.

Software for OCR is now almost 100% sucessful for Roman scripts. Middle Eastern library content however, particularly for Arabic and other non-Roman language materials, poses special challenges to the creation of digital repositories of arabic texts.  Arabic, being a diacritic language has many characters (letters) which have exactly the same form, and are distinguished only by the position of various dots over, above, or inside the main character block. This  poses special difficulty for OCR, as dots can be ignored by software as speckling or error, or even removed. Most institutions digitising Arabic manuscripts use Sakhr OCR software, but it does not seem to pick up the intricacies of Arabic script. What to do?

Some arabic fonts http://university.arabsbook.com/

It seems that if prepared well the Sakhr recognition software package has the capability to recognize generic Arabic fonts (called Naskh or Kūfī) with a fair degree of accuracy. However the software has to be taught to recognize any peculiarities or unusual characteristics in the font of the scanned volume in question. This is extremely time consuming and requires technical expertise. Also it is taken for granted in such a process that the font will be more or less consistent throughout any given volume; in many cases the hand can change in any manuscript so I imagine it would need to be reinstructed according to each section where the hand or the font changes. In addition the quality of any OCR depends on the quality of the original scanned file.  Also not everyone wants to use generic fonts, think of how much we like to personalise our own? Another headache for Sakhr.

Our group in Amman as a whole expressed frustration with  Sakhr and really hoped that it could in some way be generally instructed to recognise characters which it consistently fails to pick up. We felt sure that it will be solved soon and I personally cannot imagine that the military have not got a  solution up their sleeve  about this considering the politics of the world these days.

Interestingly in terms of resources discovery, Google Scholar does not allow searching in Arabic, while it allows for searching of both Japanese and Chinese scholarly texts. Surely as complex for an OCR piece of software to recgnise as Arabic?   This means that any texts written in Arabic cannot be accessed, which means that scholarship in Arabic is not being picked up by one of  the biggest and widest search engines for scholarly literature. Why such an oversight by Google scholar? I have contacted them and have yet to find out!

This of course brought home the real need for more collaboration between libraries and archives involved in digitsiation projects in the Middle East itself. There are many  projects based in North America such as Ameel and in the UK such as SOAS (which our own Repository folk in DART have been working on!) which unify and make available digital resources from the Middle East.  There was also an interesting JISC study with the University of Exeter about user requirements for digitised resources in Islamic studies. These are of course a western approach to arabic material, albeit in their own collections. It often also is concerned with transalations of arabic texts to greek or Latin as was the norm.

The issue of OCR and its sucess rate for non Roman fonts also raises questions about the power of the digital and askes the question that if OCR canot serve one of the great languages – Arabic,  how many minority languages which are also very diacritic are not being served well by the OCR sofwtare available.  The result of this must a tip in the balance of available reserach material in favour of texts in Roman script and sees an imbalance in what is being made available online.

Baghdad at night, 2011

There is a need for the countries which created this material to work together on such projects. Many very interesting and topical projects  to do with the emergence of which were being proposed in Amman relating to digitisation and working together to track missing journals as well as trying to avoid duplicating efforts.

So how to do this? Several libraries attending our workshop in Amman highlighted the necessity to coordinate the effort for  Arabic texts digitization in order to avoid duplication, share best practices and develop common standards, index and software. To enable this  a decision was made to work on  developing new cultural cooperation interventions for digitisation in the Middle East; to fund-raise for this and to set up groups in a social network (facebook, linkedin) including all the participants from the House of Books project. Importantly further workshops will be run to encourage this cooperation and hopefully see strides being made in cooperation and digitisation of arabic texts in the Middle East.

* http://www.nr.no/~eikvil/OCR.pdf

**Thanks to Qaiss Hatef  Saeed of the Iraq National Library and Archives for his help.

Newspapers in National Library of Jordan

Baghdad, 2003 -  when Domenico Chirico, Director of Un Ponte Per… first asked various organisations for support and resources for the reconstruction of the Iraq National Library and Archives (INLA) destroyed during the Iraq invasion and occupation, he was met with cries of bemusement and disbelief:

‘Why worry about books and archives when we have lives to save?!’.

Some if not many  struggled to understand that a library in Baghdad could be a priority during such horrific times.  But they had not yet met Dr Saad Eskander, Director of the INLA…but that is another story which has been told.

Cairo,  2011 and the ‘Spring revolution’ is happening in the Middle East.  In Egypt almost as soon as unrest broke out, two public libraries in Cairo were  burnt to the ground however cheeringly in Alexandria, news came of staff and citizens forming a human chain around the Biblioteca Alexandrina (much supported by Susan Mubarack) and another around the National Mususem of Egypt to protect it, though it wasn’t so lucky. These stories as told via Twitter cited what had happened in Iraq. Iraq, sadly is not alone in having suffered such destruction to its cultural property. These attacks,  fire, pillage, looting to order and just plain old theft  do more than just destroy a building or some documents. They are attacks on civil society, denying Iraqi people  access to engage in democracy, to access information as well as to their cultural memory.

Amman, 2011 – In light of all this, it was with great interest that I agreed to participate in the ‘The House of Books/ Dar El Kataub’ workshop run by the aforementioned Domenico Chirico’s NGO UPP and UNESCO.   Its purpose was to work on challenges facing the digitisation of Arabic texts.  I was there to look at the preservation aspect of digitisation, a shock to some as it is often thought that digitisation is itself a preservation strategy! So there was work to be done…

William Kopycki and I leading a session

We had a wide range of people from Iraq, Lebanon, Qatar/Australia, Jordan, Egypt/USA, Italy and the United Kingdom.  From an digital archivist’s point of view (and in my view the right one),  the projects represented presented us with the gamut of activities which are now present in a digital library or archive, from the development of impressives copyright legisalation in Jordan to a start to finish overview of a project digitsing  journals relating to the 19th centroy arab cultural renaissance Al-Nahda. We also heard about training and devloping infrastructures for digital object management in Iraq, as well as an overview of the current project in the INLA to reconstitute many of its collections through digitisation. We had a good deal of course about preservation from myself and Giovanni Bergamin from the University of Florence and Maurizio Messina.

My work consisted partly of leading the group in a consolidation of ideas discussed during day one.  A very important goal of this workshop is collaboration and as a result we wanted the group to think individually and in groups of four about why they saw the need to collaborate and then to tell us how they would approach collaboration.  We ended up with great points which reinforced the absolute need for collaboration in terms of standardisation, best practise, resource discovery, lobbying and important networking opportunities across the arab speaking world.

I also was there to speak and work with the group a lot about digital preservation and digitisation. This is an area which can be neglected often as many considered digitisation to be simply about simply delivering access to materials.  In addition these digital objects are considered digital surrogates and little consideration is given to their preservation as the analogue copies are available. It is important to consider preservation of these digital surrogates over time at the point of their creation.  Do we really want to invest the time and money again in their re-digitisation? I very much doubt it! However unless consideration is given to their long term sustainability this is what will happen, data loss or re-digitisation. This is time and money few of course can afford to spend.

Touching of some of the issues we covered in terms of preservation we looked at planning digitisation projects in light of preservation and their sustainability and explored the main points to consider for preservation of digitised content, drawing a lot from our very popular report on digitisation and preservation produced by DPC/ULCC and PORTICO. I reinforced the real need for honest sharing and for sharing failures as well as successes, we have all had them so we can only progress through acknowledgement of both.

Looking at the 1923 magazine on women’s issues called  Layla, we explored in a session what characteristics we would like to preserve over time.  Many things which we assume will be kept in the paper based world, have to planned for in detail  in the digital world, not much if anything can be left to chance.

Workshop over!

All speakers were extremely interesting individually and collectively as they gave an outsider like me the overview of the digital library/archives world in this part of the Middle East.  Qaiss Hatef  Saeed of the INLA spoke about the motivation of the INLA to establish a digital library  due to the loss of their holdings in the library during and post conflict.  Much of the library is being rebuilt literally from ashes by means of digital content,  and popular holdings need to be digitised to give as Quaiss said ‘ books a rest’ from handling.

Qaiss also proposed that in 100 years hard copy books will no longer exist and everything will be digital. Fighting talk! But  digital libraries however will only survive as long as we invest in their sustainability.  Digital resources have great power in terms of access but they are also very vulneable in terms of long term sustainabilty.  As such action needs to be taken an now to stop bit rot and technology obsolesence being the next threat to access to information which is now increasingly digital.

*Note: The House of Books workshop was picked up by 16  newspapers in Jordan and Iraq. These include  articles in the Baghdad press, The Jordan Times and even the Iraq National Congress.

Over the last year or so we’ve installed and configured (in some cases reconfigured) the IRStats package for several of the LEAP repositories, including those hosted by ULCC. It seemed a good moment to share a few thoughts about the process of getting “all statted up” with EPrints.

By default, and without any further action, IRStats provides a kind of smorgasbord control panel, demonstrating the many optional graphs, charts and list available. You can see an example on our own ULCC Publications repository.

More recently we’ve seen growing demand among repository managers to share data on downloads with both their depositors and users at large. It’s really important for repository managers to select carefully which statistics views they actually want or need to display – we can only suggest things we think might work. Once you’ve decided on the views you want, we can look at the most effective ways to display them: and this is why I’ve been having fun souping up some of the displays already offered by IRstats.

The first display we’ve been working on is the Statistics digest. These are common enough and we’ve used the example of UCL Discovery repository as the basis of work for both SAS-Space and SOAS institutional repository.

The second approach has been to re-style the IRstats “dashboard” view to lay the graphs on top of each other and then use some Javascript to handle the tabbed navigation. This seemed a more elegant approach than inserting lots of charts in the abstract page itself (as, for example, at ECS EPrints). I’ve used this display technique to display statistics for individual eprints for the School of Pharmacy, as well as SAS and SOAS.

The tabbed display of graphs and tables was also combined with a ‘modal box’ display that keeps the height of page the same (for example on this Abstract page at SOAS. At the bottom of the Abstract page I’ve added a statistics section showing the number full-text downloads, and a link that displays detailed stats in an overlaid box.

This method doesn’t just work for individual items, but can be used on other datasets in too. For example, on SAS-Space we have added it to the bottom of their Collection browse pages, so that at the bottom of each Collection view there is an opportunity to view download statistics for that collection as a whole.

Additionally in SAS-Space, since it is a repository for a number of discrete institutes, there was a requirement for institutional editors to have access to their own institute’s statistics. To achieve this, I allowed access to a constrained version of the IRStats control panel for editor-users who had the appropriate editorial permissions for the institute in question. (Unless you are a SAS-Space editor, you won’t be able to access this.)

Which statistics views to insert as tabs is the decision of the repository manager. Views we’ve used include:

• Monthly downloads
• Daily downloads
• Unique visitors
• Referrers
• Search Engines
• Top 10 items downloaded (only for a Collection, Repository or Division)
• Top 10 search terms

From a technical point-of-view, we will have to review these configurations when we upgrade to EPrints version 3.3, possibly later in the year (if it’s released!!), in conjunction with our VM infrastructure migration, and start doing things with EPStats rather than IRStats. But we now have an effective framework for adding statistics quickly to any EPrints installation.

During the ArchivePress project at ULCC, we briefly considered the data and metadata generally made available with blogs and blog posts. As ArchivePress focused on the representations of blogs in newsfeeds, we examined the metadata that is generated in common, and exposed in the newsfeeds of three of the most common blog platforms, WordPress, Blogger and TypePad. Blogger and Typepad prefer the Atom newsfeed format; WordPress (particularly WordPress.com) prefers RSS (though it can be made to publish Atom feeds too). This analysis was done, about a year ago, things may have changed, but here is a summary of what we found.

For each Blog, the following core information is available in the feeds:

For each Post, we established that the following core information is available in the newsfeeds:

One interesting point we noted was that neither Blogger nor Typepad published a link to a Comments Feed for each post. This made our work on ArchivePress more difficult since it was predicated on being able to easily identify the Comments feed for each post, and harvest new Comments as they were published. Obviously for blogs generated other than by WordPress, this was not going to be so easy. (Our ace developer Emanuele found some workarounds, but that’s another story.)

I think this offers us an interesting overview of the core of standard, structured blog data and metadata, in three of the leading blog platforms. This is the data structure and metadata profile that is maintained in blog databases, in one of its native forms, and I’d expect it to be present in all blog platforms, since it arguably represents the essence of blogs. I hope this will be useful background when considering the core models for data and metadata handling that will be developed for BlogForever.

Is there a particular blog or blogger you would like to see included in the BlogForever survey? We invite you to use this form to nominate them, and we will try to ensure that the blog is reviewed or the blogger contacted to participate in our survey.