Organising instruction & study: 7 recommendations to improve student learning

This blog is a summary of a Practice Guide by Pashler et al. from 2007, which sets out to provide teachers with specific strategies for instruction and study.

I came across it in a roundabout way via this paper by Dunlosky et al cited in the “What makes great teaching? Review of the underpinning research” by Rob Coe et al.

The central tenet of this particular Practice Guide is that learning depends on memory, which can in turn be strengthened by concrete strategies. These strategies help students to master new knowledge and skills, without forgetting what they have learned.

A note on Practice Guides

The Health Care professions have been using practice guides for some time now to communicate evidence-based advice to medical practitioners.

The recommendations contained within Practice Guides are intended to be:

• Actionable by practitioners
• Coherent in their approach
• Explicitly connected to the level of supporting evidence

Levels of evidence are determined by the types of studies used to draw conclusions, ranging from stronger levels of evidence that come from RCTs, with more moderate levels of evidence coming from non-randomised studies, down to lower levels of evidence that are drawn from the opinions of respected authorities.

Practice Guides are not systematic reviews or meta-analyses that have been subject to detailed literature surveys. Instead they rely more on their authors’ expertise to identify the most important research relative to the recommendations made, in order to characterise its meaning and provide specific, actionable steps. The recommendations contained in this Practice Guide have been agreed by the authors concerned and subjected to independent peer review.

Recommendations

1: Space learning over time – moderate level of evidence

Most of the research in this area has been focused on the acquisition of facts and remembering definitions of terms. To improve retention, students should be exposed to material at least twice, with a delay of weeks to months between exposures. Short delays of less than about 5% of the time between exposure and testing should be avoided. In other words, if you want students to remember material for a test in 6 months time, avoid re-exposure within less than a week or two. “Overshooting” the delay is better than reviewing too soon.

Teaching strategies:

• Regular, in-class review of previously covered material.
• Inclusion of previously covered material in homework assignments.
• Mid-term and final testing that includes cumulative material.

2: Interleave worked examples with problem solving exercises – moderate level of evidence

Experiments and some classroom studies have shown that students learn more when switching between studying examples of worked-out solutions to problems and solving similar problems independently. In the studies cited, alternating and interleaving was more successful than giving students only problems to solve, or a block of worked examples followed by a block of problems to solve. Increasing the amount of variability between successive examples and problems was also beneficial. The scaffolding provided by the worked examples should gradually be removed with time by “fading” more and more stages of the worked examples into problems.

Teaching strategies:

• Provide a worked example for every other problem, e.g. for 10 questions, make questions 1, 3, 5, 7 and 9 worked examples.
• Model the solution to a problem with the class, and then ask students to solve the next problem independently (just one!)
• Ask some students to present their solutions, while others explain the steps (a worked example in its own right) followed by another problem to solve independently.

3: Combine graphics with verbal descriptions – moderate level of evidence

Adding visual representations, e.g. graphs, diagrams or other graphic formats to text descriptions can lead to better learning than just using text. Any accompanying text should be positioned as close as possible to the relevant section of the diagram. This can be further improved with the use of verbal descriptions to accompany visual representations, which allow for both elements to be scrutinised simultaneously.

Teaching strategies:

• Provide visual representations to support the explanation of processes or concepts.
• Highlight the relevant parts of the visual representation while describing processes or concepts.
• Using simplified diagrams that show the relevant parts, rather than more complex representations is sometimes more beneficial.
• Share multiple visual representations, e.g. pictures, models, real objects etc. to illustrate how a single concept can be depicted in different ways.

4: Connect and integrate abstract and concrete representations of concepts – moderate level of evidence

Here, the research seems to suggest that teaching concepts using only concrete representations supports initial understanding but doesn’t support transfer to novel, but relevant contexts. Whereas, using only abstract representations initially can take longer to develop initial understanding, this greater initial difficulty is compensated for via improved application to different situations.

One proposal suggested to utilise the benefits of both is “concreteness fading” with initial concrete examples being gradually and systematically replaced by more abstract representations. Another is to explicitly identify and draw students’ attention to the relationship between the concrete and abstract components in representations of the same concept.

Teaching strategies:

• Show the same idea in multiple forms to show that deep structure is constant despite surface changes.
• Connect abstract ideas to relevant concrete representations and situations.
• Highlight relevant features across both abstract and concrete representations.
• Avoid using the same type of example repeatedly, i.e. examples all from one area, e.g. “sports”
• Avoid knowledge becoming “inert” by allowing time to draw connections between multiple, interleaved examples that vary in their concreteness or abstractness.
• Anchor new ideas in stories or scenarios that are familiar and interesting.

5a: Use pre-questions to introduce a topic – low level of evidence

Pre-questions are thought to activate prior knowledge and focus students’ attention on the material to be learned.

Despite recommending it as way to improve student learning, the panel deemed the level of evidence for quizzing to be low, as most of the research had been completed with college students, or based on laboratory experiments carried out on reading from written text, rather than tested as a component of regular classroom instruction.

The research does seem to suggest, however, that when pre-questions are used to preview the content of assigned material, there will likely be gains in learning of the pre-questioned material, providing students don’t read selectively based on the content of the pre-questions used.

Teaching strategies:

• Direct students’ attention to important facts and concepts by using pre-questions to introduce new topics.
• Prepare several pre-questions that students can attempt immediately on entering the lesson as part of the “do now”

5b: Use quizzes to re-expose students to information – strong level of evidence

The act of practising recalling information from memory enhances learning, reduces the rate of forgetting and cements information to memory.

Laboratory experiments across a wide range of materials and ages have repeatedly demonstrated that testing promotes remembering of material on a later test, and is almost always more powerful than spending additional time studying material.

Teaching strategies:

• Take every opportunity to prompt students to retrieve information.
• Use closed book quizzes after teaching material, prior to final testing.
• Ensure corrective feedback is provided following testing to ensure errors don’t remain.
• Use websites, e.g. http://www.quia.com to share or create quizzes.

6a: Teach students how to use delayed judgement of learning techniques to identify concepts that need further study – low level of evidence

The evidence in support of this recommendation comes mainly from experimental research in the laboratory, rather than in the classroom.

Without training, most learners cannot accurately assess what they know and what they don’t, and typically overestimate how well material has been mastered – “the illusion of knowing.” Knowing what you have and haven’t mastered accurately, is therefore essential in identifying what you still need to spend time studying, which in turn increases the likelihood of performing better when tested.

The “cue-only delayed judgement of learning procedure” is thought to be a key technique for breaking this illusion, which works as follows:

1. Students should test their mastery of material after a meaningful delay.
2. Students should only have access to “the cue” and not the answer when testing whether they know concepts or not, i.e. multiple choice questions should not be used for this purpose.
3. Students should judge how likely they are to get the answer right, as well as answering the question.

A similar technique, the “delayed keyword technique” supports students to judge how well they have retained material they have read after a delay, for example a section of a textbook or a chapter of a book, by asking them to generate keywords or sentences that summarise the main points.

Teaching strategies:

• Pre-prepare 10 questions (for example) that capture the core content to be learned.
• Give the students the questions one at a time, asking them to use a scale of 1 to 100 to judge how likely they feel they would be able to answer the question correctly tomorrow
• Ask students to review the material, use a text or ask the teacher to find out and record the answer to any question they did not score as 100.
• Use repeatedly over the course of the year, teaching students how to use this technique independently.
• Teach students to use the “delayed keyword technique” to generate four key terms and definitions following assigned reading out of class, followed by re-reading if they are unable to do this.

6b: Use tests and quizzes to identify content that needs to be learned – low level of evidence

The evidence in support of this recommendation comes mainly from experimental research of college students and laboratory tests, rather than in the classroom.

As previously stated: Inaccurate judgements by learners of what they have and haven’t learned well can mean subsequent study is focused on the wrong items.

Quizzing is thought to help students identify which items are not well learned, as does re-reading material when a test is interspersed between readings.

Teaching strategies:

• Closed book quizzes following presentation of material.
• Very short “spot check” quizzes covering material from the previous night or prior classwork.

7: Help students build explanations by asking and answering deep questions – strong level of evidence

The evidence base includes over a dozen experimental studies each, in both school and college settings, plus a large number of laboratory experiments.

Shallow knowledge is concerned with basic facts or skills, whereas deep knowledge is when learners are able argue with reason and logic, explore relationships between facts or concepts or answer “why?”

Interventions that specifically train students how to ask deep level questions while studying new material, e.g. classroom discussion, provision of exemplar materials and modelling how to ask and answer questions, have been shown to improve the rate and depth of student questions, as well as their comprehension of the material.

Teaching strategies:

• Identify and prepare deep level questions that require deep level responses.
• Ask questions that challenge students’ prior beliefs and assumptions.
• Model the process of asking and answering deep questions.
• Model and encourage students to “think aloud”.
• Encourage students to respond to explanations by their peers.
• Allow plenty of time to answer deep level questions.

---

The IES Practice Guide: Organizing Instruction and Study to Improve Student Learning by Pashler et al (2007) is available here.

 

10 Research Based Principles of Instruction for Teachers

I recently read an American Educator article from 2012 by Barak Rosenshine that set out 10 principles of instruction informed by research, with subsequent suggestions for implementing them in the classroom. It was also one of the articles cited in the “What makes great teaching? Review of the underpinning research” by Rob Coe et al and provided further elaboration on one of their six components of great teaching thought to have strong evidence of impact on student outcomes, i.e. quality of instruction.

Here’s my summary of the key messages from each of the 10 principles.

1: Begin with a short review of prior learning

Students in experimental classes where daily review was used had higher achievement scores. A 5-8 minute review of prior learning was said to strengthen connections between material learned and improve recall so that it became effortless and automatic, thus freeing up working memory.

Daily review could include, for example:

• Homework
• Previous material
• Key vocabulary
• Problems where there were errors
• Further practise of knowledge, concepts and skills

2: Present new material in small amounts or steps

Working memory is small and can only cope with small chunks at a time. Too much information presented at once overloads it and can confuse students, who won’t be able to process it. Sufficient time needs to be allocated to processes that will allow students to work with confidence independently. More effective teachers in the study dealt with the limitation of working memory by presenting only small amounts of new material at a time.

3: Ask a large number of questions and check the responses of all students

Questions allow students to practise new material and connect new material to prior learning. They also help teachers to determine how well material has been learned and whether additional teaching is required. The most effective teachers asked students to explain the process they used and how they answered the question, as well as answering the question posed.

Strategies suggested for checking the responses of all students included asking students to:

• Tell their answers to a partner
• Write a short summary and share it with a partner
• Write their answers on a mini-white board or similar, followed by “show me”
• Raise their hands if they know the answer or agree with someone else

4: Provide models

Students require cognitive support to reduce the cognitive load on their working memory and help them to solve problems faster. Examples include:

• Providing clearly laid out, step-by-step worked examples
• Identifying and explaining the underlying principles of each step
• Modelling the use of prompts
• Working together with students on tasks
• Providing partially completed problems

5: Guide student practice

New material will quickly be forgotten without sufficient rehearsal. Rehearsal helps students to access information quickly and easily when required. Additional time needs to be spent by students summarising, rephrasing or elaborating on new material so that it can become:

• Stored in long-term memory
• Easily retrieved
• Used for new learning and problem solving

The quality of storage relies on:

• Student engagement with the material
• Providing feedback to the students to correct errors and ensure misconceptions aren’t stored

The rehearsal process can be facilitated and enhanced by:

• Questioning students
• Asking students to summarise the main points
• Supervising students during practice

In one study, the more successful teachers spent more time guiding practice, for example by working through initial problems at the board whilst explaining the reasons for each step or asking students to work out problems at the board and discuss their procedures. This also served as a way of providing multiple models for students to allow them to be better prepared for independent work.

6: Check for student understanding

More effective teachers frequently checked for understanding. Checking for understanding identifies whether students are developing misconceptions as well as providing some of the processing required to move new learning into long-term memory.

The purpose of checking is twofold:

1. Answering questions might cause students to elaborate and strengthen connections to prior learning in their long term-memory
2. The answers provided by students alert the teacher to parts of the material that may need reteaching

A number of strategies can be used to check for understanding, e.g:

• Questioning
• Asking students to think aloud as they work
• Asking students to defend a position to others

7: Obtain a high success rate

When students learn new material, they construct meaning in their long-term memory. Errors can be made though, as they attempt to be logical in areas where their background knowledge may still be weak. It was suggested that the optimal success rate for fostering student achievement is approximately 80%. Furthermore, it was said that achieving a success rate of 80% showed that students were learning the material, whilst being suitably challenged. High success rates during guided practice led to higher success rates during independent work. If practice did not have a high success rate, there was a chance that errors were being practised and learned, which then become difficult to overcome. The development of misconceptions can be limited by breaking material down into small steps, providing guided practice and checking for understanding.

8: Provide scaffolds for difficult tasks

Scaffolds are temporary supports that help students to learn difficult tasks, which are gradually withdrawn with increasing competence. The use of scaffolds and models, aided by a master, helps students to serve their “cognitive apprenticeship” and learn strategies that allow them to become independent.

Scaffolds include:

• Thinking aloud by the teacher to reveal the thought processes of an expert and provide mental labels during problem solving
• Providing poor examples to correct, as well as expert models
• Tools such as cue cards or checklists
• Prompts such as “Who?” “Why?” and “How? that enable students to ask questions as they work
• Box prompts to categorise and elaborate on the main ideas
• A model of the completed task for students to compare their own work to

9: Require and monitor independent practice

Independent practice follows guided practice and involves students working alone and practising new material. Sufficient practice is necessary for students to become fluent and automatic. This avoids overcrowding working memory, and enables more attention to be devoted to comprehension and application.

Independent practice should involve the same material as guided practice, or with only slight variation. The research showed that optimal teacher-student contact time during supervision was 30 seconds or less, with longer explanations being required an indication that students were practising errors.

10: Engage students in weekly and monthly review

As students rehearse and review information, connections between ideas in long-term memory are strengthened. The more information is reviewed, the stronger these connections become. This also makes it easier to learn new information, as prior knowledge becomes more readily available for use. It also frees up space in working memory, as knowledge is organised into larger, better-connected patterns.

Practical suggestions for implementation include:

• Review the previous week’s work at the beginning of the following week
• Review the previous month’s work at the beginning of every fourth week
• Test following a review
• Weekly quizzes

---

The full report by Barak Rosenshine: Principles of Instruction – Research based strategies that all teachers should know is available here.

#NTENRED 2014

Deputy Head Veronica Waldie recently attended a conference jointly hosted by the National Teacher Enquiry Network and researchEd at Huntington School in York.  Here are her reflections on the day.

Why I went

I went to the ResearchED Conference hoping to be inspired, hoping to learn.

In school we are beginning conversations around “big issues “ such as how we can define, assure and improve the quality of teaching and learning.

I went to this conference to both deepen and clarify my own thoughts and to find ideas and practices which we could develop in school.

What I learnt

The conference undoubtedly helped me to do did this.

Part of this was because so much resonated with where we are as a school – with fires lit; a resurgence of interest in teaching and learning and the beginnings of curiosity about how research could help us to improve further.

However, there was also a buzz, an enthusiasm and a level of intellectual challenge which I have rarely encountered at CPD events. The speakers were all warm, engaging and inspiring. Above all, I felt the conference was a reflection of a positive and passionate movement for change, which we are starting to feel at the heart of our school.

What follows are my summaries of the sessions I attended. These deliberately concentrate on those aspects of most significance to us.

John Tomsett- Research Matters

This focussed on the question of how to get teachers interested in research and how this can make a difference.

Key points were:

• A key role of SLT is to create conditions for growth for teachers. More specifically, to develop a professional culture where teachers are working at the margins of their practice and are using research to help improve this. Part of how we will do this is by removing a culture of fear – for example, by taking away judgements from lesson observations.

We also need to:

• Keep a clear focus on impact – on “the Golden Thread “linking research to improving student outcomes.
• Think very carefully about processes. Implementation really matters.
• Put energy into evaluation.

 Mary Myatt- High challenge, low threat: Micro research in a macro world

This started by “de-mystifying” research, defining this as:

“actively thinking about practice and its impact on learning “

Other key points were:

• We all need challenge and stimulus – the converse is sterility.
• Innovation and research should not be high stakes – we need high challenge but low threat
• Mistakes can be a trigger for renewed insight – as long as what we do is legal, honest & decent we can afford to be wrong
• Don’t wait for everything to be in place to start. Go for brilliance not perfection
• Keep research manageable, very specific in focus and related to immediate practice
• Recognise that what we learn through a focus on one practice or on one child can impact much more widely
• Engaging in research can release huge amounts of positive energy and enthusiasm is infectious.

David Weston – What methods of “knowledge transfer“ and CPD will help teachers use research?

This started with a reminder that learning happens when:

• We are motivated to pay attention
• We are motivated to remain resilient; to put in time and effort; to reflect; to practise and to seek new knowledge
• We have the opportunity to connect new learning & skills to existing experience
• We are able to do this over time

For CPD to be effective it needs these characteristics.

Other specific points were:

• The value of NTEN lesson study is that it makes tacit, implicit, habitual knowledge explicit and draws on the power of the interplay between theory or expectation and reality
• In schools we need CPD toolkits with different strategies to inform, to influence and to embed
• Clarity of evaluation should start before we engage in any research or innovation – by asking ourselves: “which students will benefit; in which way and how will we know?”

Keven Bartle – Bottoms up

Keven focussed on teaching schools, with a theme of inverting hierarchies.

Key points were:

• The need to remember what it feels like to be at the bottom of a hierarchy
• The importance of “doing it well; doing it right; ensuring it lasts”
• The value of a “Trojan mouse” approach – empowering teachers to make small changes for big impact – rather than top-down “Trojan horse” initiatives
• Developing these ideas in practice-for example, teacher rounds – rather than (SLT) learning walks

Jill Berry – How to grow the next generation of school leaders

This focussed on behaviours and attributes needed to take organisations into the future. Jill identified that leaders need to:

• develop different professional persona and support networks
• deal sensitively with legacy – with “ ghosts of principles past”
• have hope, humanity, humility and humour (quoting from John Dunford)

Tom Bennett – Idiocracy – how did we get so stupid?

This presentation highlighted the harm muddled thinking and “bad science “ can and have caused in teaching.

More specifically, caution is needed in accepting “evidence” as:

• There is difference between RCTs conducted on buckets of water and on people
• Social science is useful as a commentary on human behaviour but does not provide normative laws
• We should always question the agenda of those who present “evidence”
• This doesn’t mean we should not engage with research. We actually need to bring research and practice closer – we need to ask the right questions to find what works for us, for our learners
• Suggestions for how to do this include “wise line management “ or “constructive tasking“ and creating research champions in schools

Stephen Tierney – The Babylon and Jerusalem of Professional Development

Stephen explored the tensions between the Jerusalem of professional development – where we gather people in one place at one time to “deliver what the new ritual will be“ and the Babylon – “wild”, more personal professional development , focussed on creating rather than passing on wisdom.

He suggested that Joint Practice Development may be a way of bringing these together and developing “Disciplined Innovation”

Further key points included the need to:

• Create opportunities for people to grow
• Give time and allow time as most innovations are abandoned before they reach maturity
• Be pragmatic– doing fewer things better leads to improved outcomes
• Evaluate – quantitative & qualitative evaluations are both valuable & valid – not evaluating is not

Specific ideas included creating or providing:

• Innovation Fellows
• System Redesign TLRs
• R & D communities
• Voluntary INSET
• Seed funding

Summary

Overall the speakers conveyed a passionate belief that when research and experience align we can create a growth culture for teachers and students alike. It also helped demystify research -research is already happening in schools and is profound in its ability to raise energy and passion. Clearly we need to recognise strengths and limitations – research evidence is complex; can be flawed and subject to conflicting agendas but can also be of real value. As John Tomsett pointed out, however, to have real impact, research must help us move from what we know to what we do.

What difference will it make and why it matters

At the point where we are planning for next year – and at time of massive curriculum innovation in school – I believe having these ideas as part of our dialogue from the start will make a difference.

As a first step, we want to make even more sure that the “good stuff” we already do is caught, shared, lifted and trialled. (Eduflections).

To develop further, joining NTEN and investing in technology, such as IRIS, could help.

We also want our professional development to be the best – to be “aspirational, collaborative, relevant, differentiated, sustained, underpinned by research and evaluated” (David Weston) Having this as our goal from the start, and planning meticulously for this, means it is more likely to happen.

More fundamentally, however, as a school I believe we are highly motivated by the need to make a difference, by a moral imperative to narrow gaps and improve outcomes for all. We know we will do this most effectively by improving teaching and learning, and everything we spend our time on should support this. Systematically reflecting and questioning all our practice is an essential part of this.

I will finish with an extract, quoted by John Tomsett, which I believe shows how doing this can help us create the school we all want:

“I would welcome the chance to work in a school characterised by a high level of collegiality, a place teeming with frequent, helpful personal and professional interactions……..where a climate of risk taking is deliberately fostered and where a safety net protects those who may risk and stumble……..where important differences…….were celebrated…….and which accorded a special place to philosophers who constantly examine and question……….”

Roland S Barth – A Personal Vision of an Idealised School Culture