Comments on: Marking and writing JISC proposals

By: Joanne Anthony

Joanne Anthony — Fri, 27 Mar 2009 16:30:29 +0000

Just to add some more food for thought on this. Yesterday I attended an excellent Executive Briefing titled “Costing and funding digitisation projects” [CILIP]. As one of the key speakers, Alastair Dunning (JISC Digitisation Programme) advised attendees on what funding bodies are looking for (which applies generally beyond digitisation projects). He highlighted that ‘Peer-reviewers who do the marking often interpret criteria differently’, that ‘you can be unlucky with a good bid; expect to fail before succeeding’; and that the ‘winning teams show evidence of broader strategic thinking’. Funding bodies are increasingly looking for things like: ‘evidence of impact and use’ ie testimonies of potential users (‘not just a build it and they will come approach’); ‘linking up with others’ eg economies of scale; ‘strong leadership’; ‘considered approach to IPR’; ‘workflows and standards’; ‘integrating web 2.0′; and dealing well with practical matters (eg proper preparation/answer specifics of the bid/proactive on partners and strategies/other sources of funding/don’t over-emphasize the tools/technology in your bid, focus more on users and outputs).

By: Kevin Ashley

Kevin Ashley — Sat, 21 Mar 2009 22:32:40 +0000

Joss, the stuff you’ve been doing at writetoreply is really great stuff and the community owes you thanks for doing it; we also stand to learn a good deal. I look forward to you getting a bid together. The hard part can be having a good idea, and you’ve done that. The other thing you need to do is to convince the evaluators that you can see the idea through to completion. You’ve already got some good evidence there as well.

By: Richard M. Davis

Richard M. Davis — Fri, 20 Mar 2009 15:43:09 +0000

Hi Joss

Love your JISCRI network, congrats on taking the initiative to knock this up. The whole JISC community as one big Buddypress network – why not? (It’s even defaultly Orange!)

Hope to check it out more next week, after an offline weekend’s cleared my head! (After all, real people don’t have time for social media

By: Joss Winn

Joss Winn — Fri, 20 Mar 2009 09:41:45 +0000

Tony Hirst and I are planning on making a bid to #jiscri to adapt what we’re doing on WriteToReply to assist and develop the process of JISC funding calls, bidding, marking and reporting. WriteToReply is pretty bare-boned right now, but with a little funding, has the potential to offer a place to announce and host Calls, invite comment and discussion (i.e. Town Meeting events), form interest groups and partnerships around bids, draft bids (even submit bids??) and finally mark bids. Each aspect of this process could be anywhere between open or private (I favour open at every stage).

We’ve not joined the two yet, but are thinking about how WriteToReply and http://learninglab.lincoln.ac.uk/jiscri social networking platform might be combined effectively to provide a community space focused around announcing, discussing, working on and marking calls and bids.

I think everything you’ve noted above could be accommodated by our proposal, although I would hope that we might also affect change in the Call process, too, during the requirements gathering and implementation of the service.

By: Richard M. Davis

Richard M. Davis — Fri, 13 Mar 2009 17:53:39 +0000

Another interesting approach to aspects of this process is being piloted by Joss Winn and Tony Hirst: http://writetoreply.org/jiscri. Using CommentPress - yet another exciting possibility out of the rich seam that is the Wordpress stable (see also P2 and BuddyPress - not to mention my imminent post for JISC-PoWR about using a Wordpress blog to archive Wordpress blogs!).

By: Kevin Ashley

Kevin Ashley — Wed, 11 Mar 2009 11:17:03 +0000

Interesting paper, Richard – perhaps the future is closer than I think. However, that paper looks at marking of material where a model answer exists, so the automated systems are effectively measuring how far the candidate document is from the ‘ideal’ document in some measure space. There isn’t a model response to a research funding call, although there are almost certainly some aspects of a bid (such as the budget) where models do exist and where automated assessment might well be practical, as well as fairer.

By: Richard M. Davis

Richard M. Davis — Wed, 11 Mar 2009 11:01:14 +0000

According to a paper by Robert Williams, describing automated essay marking using the Bayesian-based Text Categorization Technique:

When all the criteria for assessment were used the proportion of essays graded exactly the same as human graders was 0.60 and scores adjacent (a score one grade on either side) was 1.00. [...] The system performed remarkably well.

Couldn't the JISC employ this kind of technology (that it's no doubt funding research into elsewhere, directly or indirectly)? (Of course they might then need Turnitin too, to make sure we're not cheating...)