HOST Policy Research: Projects and Consultations

Current Projects

 

Impact review of the Centres for Excellence in Teacher Training (CETTs) in England for Learning and Skills Improvement Service (LSIS, formerly Quality Improvement Agency (QIA)), and Department for Innovations, Universities and Skills (DIUS) led by Professor David Parsons with John Barry, Peter Foster, Dr Jacqui Hughes, Valerie Rowe, and Judy Staton

CETTs were set up in 2007 following a government (then Department for Education and Skills (DfES)) review of the quality of teacher training in the post-16 sector. Over 400 UK providers were involved in bidding for CETTs status which led to 11 partnerships being designated CETTs. In summer 2007, responsibility for the three-year funding of the start-up and capacity building of CETTs was passed to (then) QIA, who later commissioned HOST to conduct a national impact review. This has included an assessment of the quality of start-up, sector engagement partnership working and annual reviews of delivery against respective business plans.

The HOST evaluation is formative, and although independent of the programme, has worked closely with LSIS, and the cross-CETT network group now co-ordinated through the Association of CETTs (ACETT) to review early development messages. The evaluation team has also developed a cross-CETT guidebook on setting impact assessment strategies, and conducted a workshop (training) to help CETT managers embed the guide in their internal monitoring and evaluation activities.

The evaluation includes a series of staged reviews with CETT directors and co-ordinators, and selected CETT partners, together with a wider sector survey and (shortly) an impact survey across all CETT partner institutions and collaborators. Longitudinal analysis is conducted from baseline evidence to assess delivery, and the direct and indirect impact of CETT activities and collaborations. A baseline report (December 2007), and two annual reports (March 2008 and 2009) have been prepared. A final review report is expected in September 2009.

 

National Evaluation of Support for Excellence (SfE) for Learning and Skills Improvement Service (LSIS, formerly Quality Improvement Agency (QIA)), led by Professor David Parsons with Dorothy Berry-Lound, Jane Holland, John Austin, Dr Jacqui Hughes, Bob Hughes, Susan Lanz, Valerie Rowe, and Judy Staton

Following the Leitch and Tomlinson reports, government announced an overhaul of regulatory arrangements for the post-16 sector with a policy goal to prepare the sector for self regulation. Two parallel strands of early activity were expected with the LSC leading on the Framework for Excellence, and (then) QIA looking to cross-provider review through multi-provider Peer Review and Development (PRD) Groups. In autumn 2007, HOST was selected to design and conduct the national review of PRD capacity building and its effectiveness within this policy agenda. In July 2007, Deloitte were awarded the contract for PRD Group brokerage and support.

This two-year review of the support programme for PRD working (Support for Excellence) has involved close working with LSIS and Deloitte to provide an independent and formative evaluation. This has included a range of evaluation methods including some group observation, but with the main inputs through review of programme management information (MI) and reach, staged participant profiling, surveys of PRD partner, and effective practice case studies. The evaluation has also looked at awareness and attitudes among non-participant providers, and HOST has also provided impact guidance across the PRD network. A series of interim evaluation reports have been produced at key decision making points in the programme, and a final report is due in September 2009.

 

National Evaluation of Support for Excellence (SfE) – Peer Review and Development (PRD) Groups in Offender Learning in England for Learning and Skills Improvement Service (LSIS, formerly Quality Improvement Agency (QIA)), led by Professor David Parsons with Valerie Rowe, and Judy Staton

This project has been set up to parallel the mainstream evaluation of the Support for Excellence Programme. In summer 2008, a separate activity was commenced by LSIS to promote and support PRD Groups across Her Majesty’s Prison Service and the National Probation Service. HOST was asked to conduct an independent review of capacity building and its effectiveness across the 45 prisons and youth offending institutions taking part.

This one-year review includes a review of programme reach, e-surveys of all HMP/YOI participants, a non-engagement survey, and a small number of process case studies. Brokers and support consultants have also been engaged in the review. An interim evaluation report was produced in February 2009, and a final report is due in August 2009.

 

Evaluation of the Foundation Learning (FL) Capacity Building Programme in England for the Learning and Skills Improvement Service (LSIS, formerly Quality Improvement Agency (QIA)), led by John Austin and Professor David Parsons, with Alan Bloomfield, and Simon Bysshe

Following the Tomlinson and Leith reviews, government has been looking for substantial reforms to delivery and learner progression at Entry Level and Level 1 provision, and for feeder routes into Level 2 provision. To support this, LSIS has been providing phased implementation support to selected providers since 2007 to develop and test new delivery routes within new qualifications flexibilities. This has been substantially expanded since May 2008 to help an estimated 900 post-16 providers in England build provider capacity and responsiveness to the emerging FL reforms. In summer 2009, HOST was commissioned to evaluate this expanded programme, and to review implications for future provider support for 2009-2010 and beyond. The evaluation focuses on those areas of provider support centred on post-16 provision, and particularly for adults.

This seven-month study will combine both formative and summative evaluation methods. It will include a mixture of stakeholder review to establish policy intentions and goals, a series in-depth provider profiles looking at engagement and adjustment in FL, and an end-of-programme participant sample survey to also look at early impact and effective practices. An interim report is due in mid December to provide early evidence on participation and roll-out of the contracted support programme, and a final report is due for the end of March 2010.

 

An Evaluation of the Teaching and Learning Programme for the Learning and Skills Improvement Service (LSIS), and Department for Business Innovations and Skills (BIS) led by Professor David Parsons and Judy Staton, with John Barry, Peter Foster, Dr Jacqui Hughes, Valerie Rowe and Dick Willis

The Teaching and Learning Programme (TLP) is one of the longest-standing improvement programmes in the post-compulsory sector aimed at building teaching capabilities and innovation across the post-16 sector. Previously (with QIA) the National Teaching and Learning Change Programme (NTLCP), it was launched by the (then) DfES Standards Unit in 2003 as a critical part of the Success for All strategy. HOST were appointed (by then DfES) as the programme evaluators in 2003-2006, and subsequently also by QIA for 2006-2009 – for which a final evaluation report was presented to LSIS in September 2009. With a growing emphasis on the sustainability of the programme and embedding the practices and developments of the last six years, LSIS have now commissioned HOST to conduct the concluding stage of the programme evaluation to March 2010 and also to review evaluation options for any activity thereafter. TLP has developed as a large-scale and multi-faceted approach, and the evaluation will be widened significantly to include activities recently developed or not previously in scope, and in particular in the e-CPD programme. TLP now offers a series of sub-programmes designed to work in concert as a framework for quality improvement and ‘... supporting the delivery of learner success’, and including:

  • A widening series of teaching and learning resources – commonly accessed through the Excellence Gateway.
  • The development of the national Professional Training Programme (PTP) for Subject Learning Coaches (SLCs), and now also for Advanced Learning Coaches (ALCs).
  • A series of mainly regionally-based but interrelated Subject Coaching Networks (SCNs) were also developed, initially to support the roll-out and effective use of the teaching and learning resources.
  • A programme of activities aimed at supporting provider managers and Senior Management Teams (SMTs) in effectively integrating the TLP activities and making best use of the opportunities.
  • A programme of STEM-related activities run in conjunction with the mainstream activities but with some special cross-STEM activity.
  • Funding for the sector to take part in Excellence Improvement and Innovations Fund projects, and to share the resources and outcomes with others in the sector.
  • A programme of activities aimed at building e-CPD capacity across diverse providers.

This evaluation will report at the end of March 2010.

 

Evaluation of post-16 citizenship support programme for the Learning and Skills Improvement Service (LSIS, formerly Quality Improved Agency, QIA) led by Judy Staton with Dorothy Berry-Lound

Since September 2006 the Post-16 Citizenship Support Programme has been assisting staff and organisations from a wide range of post-16 settings to develop and enhance their citizenship provision. This built on an earlier development programme which was set up in 2001 following recommendations in the ‘Citizenship for 16-19 Year Olds in Education and Training (2000) report and the government’s decision to make citizenship education statutory at key stages 3 and 4. Over the last three and a half years the support programme has worked to promote the benefits of citizenship education to a wider audience, to develop good practice in this and to strengthen its links with cross-governmental policies such as the promotion of community cohesion and of Every Child Matters. In order to achieve these aims it has used a range of strategies and tools, for example, by making available teaching and learning and staff development materials, other promotional materials, a dedicated website and a range of national and regional network and training events.

HOST was commissioned by QIA in the autumn of 2006 to evaluate the first phase of the support programme and has undertaken evaluations of each subsequent phase, producing four reports in total. Each of the evaluations has been independent of the programme and formative: its recommendations have helped to influence the ongoing development of the programme. The current evaluation commenced in October 2009 and focuses on two specific aspects of the support offered which were viewed by LSN as key to achieving programme outcomes. These are:

  • Centre-based or customised training. Free sessions of ‘in-house training’, which were first offered in June 2008, are designed to help practitioners to improve the quality and range of their citizenship provision.
  • Development projects. These commenced in September 2009 and are intended to yield specific outputs, that is materials or resources and case-studies, to support the dissemination of innovative practice in citizenship education. It was also anticipated that they might help providers to develop their own provision as well as perhaps establishing arrangements for collaborative working which might continue beyond the life of the projects.

The evaluation aims to contribute to assessment of the impact of both of these programme strands on individual organisations’ delivery of citizenship. An e-questionnaire is being distributed to lead contacts for all providers participating in customised training between June 2008 and October 2009. This seeks to elicit their views on the way that the training has influenced their delivery of citizenship and on their capability and future plans for doing so. The main tool for evaluating the development projects will be a number of telephone interviews selected from the primary contacts of participating organisations supplemented by a review of documentation and attendance at the contractors’ evaluation meeting. HOST’s final report on the current phase of the support programme is due in March 2010.

 

Phase 2 of the Review of Management Qualifications and Training in the Learning and Skills sector for Learning and Skills Improvement Agency (LSIS) led by Valerie Rowe with Dorothy Berry-Lound and Simon Bysshe.

LSIS works in partnership with all parts of the learning and skills sector to provide support and enable continuous self-improvement and capacity building, delivered through a wide range of programmes and services. While LSIS and its predecessors Quality Improvement Agency (QIA) and the Centre for Excellence in Leadership (CEL) have been active in various areas of management development in the sector, these activities have tended to have a specific capacity building focus or to be embedded within distinct programme related needs. The new emphasis seeks to build on this track record, but to establish what may be a more directed and strategic approach. This, in turn, is recognised to need a firm foundation of understanding sector requirements, and the primary purpose of this review is to gather structured information from the sector about its distinctive needs and current requirements for management skills and qualifications.

Following HOST’s earlier review of available evidence and discussions with key stakeholders, LSIS has commissioned a Phase 2 research activity comprising a series of e-surveys aimed at managers within individual sub sectors of the learning and skills arena in order to fill research gaps identified from the earlier research, and to elicit more detail on junior and middle management skills, development and qualifications. The research is due to report at the end of June 2010.

 

Supporting the Department for Work and Pensions (DWP) in the development of a Delivery Balanced Scorecard led by Dorothy Berry-Lound with Professor David Parsons and Susan Sibley.

DWP is developing a Delivery Balanced Scorecard and has asked HOST to assist by conducting quarterly e-surveys of providers. This involves a first pilot in March and a second pilot in order to establish the structure of the questionnaires, style of wording and approach. Then the quarterly surveys will roll out from the autumn until March 2012 with a quarterly cycle of reporting.

 

An evaluation of the impact of a training intervention on behaviour towards carers and delivery of services to carers by workers (predominantly health and social care staff) who have undertaken the training. A project for Skills for Care working jointly with Skills for Health, led by Dorothy Berry-Lound, with Dr Jo Verrill, Stefan Burkey, Frances Chinemana, Dr Claire Harris and Susan Lanz.

The project will deliver a mix of quantitative and qualitative measures to provide a robust evaluation of the impact of the training programme. Activities will take place on a rolling basis to reflect training activity on the ground and involve research with carers and staff; the same methods will be applied across pilot and roll out sites.

A combination of self-complete surveys, depth interviews and focus groups will be used to thoroughly evaluate the impact of training on staff. A staff database will be created and maintained, containing contact information and a record of the individuals training plan including learning goals, mode of delivery, start date, completion date and final evaluation date. The database will act as the trigger for all research activities, ensuring follow up is timed appropriately, in a rolling programme of data collection and review. Baseline, completion of training and three months after completion of training are the trigger points for the three staff surveys.

A high quality colour survey leaflet will be produced for staff to hand out to all carers in touch with their service during the field period. Carers are the intended end beneficiaries of the training programme, with the aim being to improve their lives and experience by up-skilling workers they come into contact with. The leaflet will be self sealing with freepost response, enabling the carer to return their information by post or to a collection point at the service location. Data from returned leaflets will be captured to a database for analysis and used for recruitment to the extended research activities. This mechanism offers a flexible first step to engaging with carers which can accommodate a wide range of service scenarios, eg fleeting contact with signposting organisations that do not retain contact information for carers, through to more intensive services where longer term relationships are in place. The initial feedback form will be followed up with a phone call at a fixed interval/s to track the progress of the enquiry and impact on quality of life, etc. Call intervals will be determined by the nature of the enquiry, being responsive to the profile of carers coming into contact with services during the evaluation period.

A series of reports are proposed for the project, with the final report being due in June 2011.