Assessing the impact of the JISCMRD programme

In this third post focusing on our recent RDM survey of UK HEIs, we consider the impact of the JISCMRD programme on the work UK HEIs are undertaking.

Jisc recently funded projects to undertake a range of initiatives investigating the management of research data. Specific programme strands were:

  1. Research Data Management Infrastructure (RDMI)
  2. Research Data Management Planning (RDMP)
  3. Support and Tools
  4. Citing, Linking, Integrating and Publishing Research Data (CLIP)
  5. Research Data Management Training Materials

Although the programme concluded in 2013 the influence these project on the broader UK HEI sector has yet to be established. While these projects were underway evidence of their research and development activities was recorded in a blog by Laura Molloy (JISC MRD: Evidence Gathering. Encouraging discussion across the JISC Managing Research Data programme, Laura Molloy, http://mrdevidence.jiscinvolve.org/wp/). Laura is the Curation and Preservation Officer at the Humanities Advanced Technology and Information Institute (HATII), University of Glasgow and Preservation Researcher with the Digital Curation Centre. Posts indicate the effectiveness of the programme in providing shared experience, expertise and materials for reuse. The last blog post (May 2013) describes the role of research funders in ensuring effective RDM (not least to allow for the implementation of their own policies), and in particular the need for them to consider the place and nature of peer review with regards to assessing the RDM component of proposals.

So, what has been happening since these projects ended and why is it important to consider the influence the work of these projects may be having on current RDM related activities in UK universities?

It is apparent from analysis of survey data from UK HEIs that JISCMRD funding does seem to put institutions in a better position to develop policy and services to support their research data management initiatives. For further information on this please see our previous discussion (Institutional readiness for managing research data). However, when considering the ‘ripple effect’ of these projects data is more difficult to locate. Personal communications across the sector for information about the work of a project or requests to use materials are not centrally recorded.

We believe that impact assessment is inherently valuable for evaluating the success of a programme and its impact over time. Information gathered can be used to inform and shape future work and to lobby for funding for such activities.

We therefore welcome responses from those involved in the JISCMRD programme and institutions grappling with RDM issues describing the benefits they derived from involvement in JISCMRD projects or having access to information from these projects. Comments can be added to this blog or by contacting Stéphane Goldstein <stephane.goldstein@researchinfonet.org>. We also hope that, in due course, Jisc itself will show an interest in following up on our findings.

Previous posts in this seriess:

Staffing research data management in UK HEIs

Following on from our earlier post about Institutional readiness for managing research data, Stéphane and I have been looking at staffing levels in UK HEIs for establishing and supporting RDM. The data comes from our recent survey of UK HEIs RDM practices and a mapping to JISCMRD programme institutions.

A further set of indicators can serve to substantiate the argument that HEIs which took part in the JISCMRD programme are more advanced in the development of institutional RDM practice. These are the figures that relate to staffing levels for all the different categories of RDM-related personnel identified in the Loughborough survey of research data management activities at UK HEIs. The following tables describe this:

  Number %
Number of institutions receiving Jisc funding who have any fixed-term staff in post 7 out of 15 47%
Number of institutions not receiving Jisc funding who have any fixed-term staff in post 6 out of 23 26%

 

  Number %
Number of institutions receiving Jisc funding who have any permanent staff in post 8 out of 15 53%
Number of institutions not receiving Jisc funding who have any permanent staff in post 5 out of 23 22%

 

  Mean Median
Average fixed-term staffing level (FTE) for institutions receiving Jisc funding 1.07 (*) 0
Average fixed-term staffing level (FTE) for institutions not receiving Jisc funding 0.54 0

(*) It is reasonable to say that the mean, in this instance, is somewhat skewed by one responding university having 5.5 FTE employed in RDM related roles.

 

  Mean Median
Average permanent staffing level (FTE) for institutions receiving Jisc funding 1.13 1
Average permanent staffing level (FTE) for institutions not receiving Jisc funding 0.24 0

These figures are perhaps hardly surprising, particularly for the fixed-term staff, who might have been recruited specifically for the duration of the Jisc projects; it’s noteworthy though that institutions receiving Jisc funding have also invested much more than others in permanent staff.

 Full survey results and discussion are available in a blog post by Martin Hamilton (http://blog.martinh.net/2013/10/metadata-is-love-note-to-future-uk.html).

Institutional readiness for managing research data

We have been working with Stéphane Goldstein, a Consultant at the Research Information Network (RIN), to see whether there is any link between the development of institutional research data management services and involvement in the JISCMRD programme.

Universities in the UK are preparing policies and services to better support research data management. This work is being undertaken to ensure that the maximum benefits are achieved for research outputs, particularly where these have been funded from the public purse.

For a three-year period between (2009-2011 and 2011-2013) Jisc funded 65 projects to consider a range of Research Data Management (RDM) issues (JISCMRD programme, http://www.jisc.ac.uk/whatwedo/programmes/mrd.aspx). Thirty-five UK Higher Education Institutions took part in the programme which involved considerable financial investment from Jisc and intensive work and effort within the projects concerned. A wealth of very valuable outputs arose from these projects which Loughborough University, along with others, have benefited from. Loughborough has also made good use of support available from the Digital Curation Centre via their Institutional Engagements and from the UK Data Archive online resources, and in discussions with other experts in this area. The University has also had discussions with the Research Information Network on the results of a national survey that Loughborough University undertook to uncover RDM activities in UK HEIs. The survey was advertised to a number of professional email lists whose readership would have a stake in UK HEI research data management. In addition, DCC colleagues actively promoted the survey to institutions it engages with. The survey was completed by 38 unique HEIs, 15 (39%) of these were involved in Jisc-funded MRD projects. The high proportion of respondents from the JISCMRD programme (15 out of 35 [43%]) may be attributed to our promotional activities and the community coalescing around the JISCMRD projects. Full UK HEI RDM survey results and discussion are available in a blog post by Martin Hamilton (http://blog.martinh.net/2013/10/metadata-is-love-note-to-future-uk.html).

An initial mapping of data from the survey against data on institutions receiving JISCMRD funding revealed that the latter were further advanced in their RDM policy making, infrastructure provision and support/training offerings. We discovered that 60% (9 out of 15) of the Jisc-funded institutions have a RDM policy compared with 26% (6 out 23) of those who did not receive such funding, and 39% (15 out of 38) of the whole sample.

Research data management policy

Research data management policy

Similarly, 33% (5 out of 15) of those receiving Jisc funding have a research data service in place, but this proportion falls to 4% (1 out 23) for those not receiving any such funding, and 16% (6 out of 38) for the sample as a whole.

Research data service

Research data service

This basic analysis appears to indicate a link between Jisc funding and presence of RDM policy and services. In our view this preliminary finding warrants further investigation, not least because it provides some evidence that universities involved in Jisc-funded MRD projects benefit from accelerated policy and service development.

The next step is to delve deeper into the data to look at project themes and to link this up with other sources of information.