{"id":764,"date":"2019-11-22T13:33:35","date_gmt":"2019-11-22T13:33:35","guid":{"rendered":"http:\/\/blogs.kcl.ac.uk\/behaviouralinsights\/?p=764"},"modified":"2020-02-10T13:03:26","modified_gmt":"2020-02-10T13:03:26","slug":"2019-may-have-been-the-year-of-null-results-but-its-moving-us-forward","status":"publish","type":"post","link":"https:\/\/blogs.kcl.ac.uk\/behaviouralinsights\/2019\/11\/22\/2019-may-have-been-the-year-of-null-results-but-its-moving-us-forward\/","title":{"rendered":"2019 may have been the year of null results, but it\u2019s moving us forward"},"content":{"rendered":"

Vanessa Todman, Head of Student Experience Research and Miriam Styrnol, Senior Research and Evaluation Advisor<\/em><\/strong>|<\/em><\/strong><\/p>\n

Null results are useful. In What Works we\u2019re always seeking to learn from the experiences of others, and then evaluate for the context of our students. For example, taking learning from organisations such as the Education Endowment Fund (EEF), we are testing if a broad approach might be helpful when constructing psychological interventions, with positive results. <\/em><\/p>\n

What Works has been going on a journey. When we started out, we focused mainly on nudges. We focused on creating and building our evidence base by nudging current students (see our KcLxBIT report<\/a>) and then in 2018\/19 started to add psychological interventions to our toolbox, focusing on belonging and social capital<\/a>. The results of these trials are starting to come through, and although inevitably not everything works, it\u2019s looking good.<\/p>\n

Positive psychological interventions use psychological tools to help people overcome mental blocks<\/strong><\/p>\n

Psychological interventions in education change how students think or feel about learning, or about themselves in a learning environment. For example, they can signal inclusion to students who are unsure if they fit in. A good example of this is Yeager and Walton\u2019s work exploring the use of value affirmation (giving students opportunities to reflect on their personal values in order to reduce stress and increase belonging[i]<\/a>). These kinds of interventions, also known as \u2018wise interventions\u2019<\/a> complement, but do not replace, traditional institutional reforms.<\/p>\n

Such interventions have shown to work for some students in some, mostly US, contexts. For example, Walton and Cohen have shown that students\u2019 doubts about belonging may take the form of a hypothesis, and \u2018wise interventions\u2019 can help certain students in certain contexts to identify experiences which prove or disprove that hypothesis[ii]<\/sup><\/a>.<\/p>\n

Recently, the EEF published findings of their UK-based trials encouraging the use of growth mindsets in schools, a\u00a0 psychological approach that has shown some success with children in America[iii]<\/a> . They mostly found little impact on attainment. and concluded that teachers should be cautious about using growth mindsets alone as a way of boosting pupil attainment.[iv]<\/a>.<\/p>\n

It\u2019s important to test When, for Whom and try to understand Why<\/strong><\/p>\n

When something works for one group, especially for something as important as attainment, it can be tempting to roll it out on a larger scale. This doesn\u2019t always work. Over the summer, Ben Castleman<\/a> came to speak at King\u2019s about the issues he\u2019s experienced trying to upscale local nudge interventions[v]<\/a>.\u00a0 Similarly, the evidence supporting Growth Mindsets interventions was strong, but mainly American, and so was tested in a very different environment to UK schools. The EEF sensibly ran well powered interventions to test if the findings were portable to the UK, and has found that- under the original experiment\u2019s circumstances- they aren\u2019t.<\/p>\n

Psychological interventions should be just one part of the toolkit<\/strong><\/p>\n

At What Works we\u2019re currently taking a \u201ceverything but the kitchen-sink\u201d type of approach to influencing attainment, with some positive results. Our reasoning is that no demographic groups of students are homogenous and therefore reasons for any behaviour will differ between individuals. In such a context, we feel it\u2019s not surprising that trying to influence one element had little effect (especially when taking into account Michael Sanders<\/a>\u2019 recent warning that effect sizes will often be small) .<\/p>\n

We are therefore currently testing a suite of interventions which aim to hit as many possible psychological causes of lower attainment as possible, as well as other possible non-psychological factors such as lack of key information needed to succeed. This project follows our ACES Framework<\/a>, we identified the following psycho-social factors which could be having an effect:<\/p>\n