Organising instruction & study: 7 recommendations to improve student learning

This blog is a summary of a Practice Guide by Pashler et al. from 2007, which sets out to provide teachers with specific strategies for instruction and study.

I came across it in a roundabout way via this paper by Dunlosky et al cited in the “What makes great teaching? Review of the underpinning research” by Rob Coe et al.

The central tenet of this particular Practice Guide is that learning depends on memory, which can in turn be strengthened by concrete strategies. These strategies help students to master new knowledge and skills, without forgetting what they have learned.

A note on Practice Guides

The Health Care professions have been using practice guides for some time now to communicate evidence-based advice to medical practitioners.

The recommendations contained within Practice Guides are intended to be:

• Actionable by practitioners
• Coherent in their approach
• Explicitly connected to the level of supporting evidence

Levels of evidence are determined by the types of studies used to draw conclusions, ranging from stronger levels of evidence that come from RCTs, with more moderate levels of evidence coming from non-randomised studies, down to lower levels of evidence that are drawn from the opinions of respected authorities.

Practice Guides are not systematic reviews or meta-analyses that have been subject to detailed literature surveys. Instead they rely more on their authors’ expertise to identify the most important research relative to the recommendations made, in order to characterise its meaning and provide specific, actionable steps. The recommendations contained in this Practice Guide have been agreed by the authors concerned and subjected to independent peer review.

Recommendations

1: Space learning over time – moderate level of evidence

Most of the research in this area has been focused on the acquisition of facts and remembering definitions of terms. To improve retention, students should be exposed to material at least twice, with a delay of weeks to months between exposures. Short delays of less than about 5% of the time between exposure and testing should be avoided. In other words, if you want students to remember material for a test in 6 months time, avoid re-exposure within less than a week or two. “Overshooting” the delay is better than reviewing too soon.

Teaching strategies:

• Regular, in-class review of previously covered material.
• Inclusion of previously covered material in homework assignments.
• Mid-term and final testing that includes cumulative material.

2: Interleave worked examples with problem solving exercises – moderate level of evidence

Experiments and some classroom studies have shown that students learn more when switching between studying examples of worked-out solutions to problems and solving similar problems independently. In the studies cited, alternating and interleaving was more successful than giving students only problems to solve, or a block of worked examples followed by a block of problems to solve. Increasing the amount of variability between successive examples and problems was also beneficial. The scaffolding provided by the worked examples should gradually be removed with time by “fading” more and more stages of the worked examples into problems.

Teaching strategies:

• Provide a worked example for every other problem, e.g. for 10 questions, make questions 1, 3, 5, 7 and 9 worked examples.
• Model the solution to a problem with the class, and then ask students to solve the next problem independently (just one!)
• Ask some students to present their solutions, while others explain the steps (a worked example in its own right) followed by another problem to solve independently.

3: Combine graphics with verbal descriptions – moderate level of evidence

Adding visual representations, e.g. graphs, diagrams or other graphic formats to text descriptions can lead to better learning than just using text. Any accompanying text should be positioned as close as possible to the relevant section of the diagram. This can be further improved with the use of verbal descriptions to accompany visual representations, which allow for both elements to be scrutinised simultaneously.

Teaching strategies:

• Provide visual representations to support the explanation of processes or concepts.
• Highlight the relevant parts of the visual representation while describing processes or concepts.
• Using simplified diagrams that show the relevant parts, rather than more complex representations is sometimes more beneficial.
• Share multiple visual representations, e.g. pictures, models, real objects etc. to illustrate how a single concept can be depicted in different ways.

4: Connect and integrate abstract and concrete representations of concepts – moderate level of evidence

Here, the research seems to suggest that teaching concepts using only concrete representations supports initial understanding but doesn’t support transfer to novel, but relevant contexts. Whereas, using only abstract representations initially can take longer to develop initial understanding, this greater initial difficulty is compensated for via improved application to different situations.

One proposal suggested to utilise the benefits of both is “concreteness fading” with initial concrete examples being gradually and systematically replaced by more abstract representations. Another is to explicitly identify and draw students’ attention to the relationship between the concrete and abstract components in representations of the same concept.

Teaching strategies:

• Show the same idea in multiple forms to show that deep structure is constant despite surface changes.
• Connect abstract ideas to relevant concrete representations and situations.
• Highlight relevant features across both abstract and concrete representations.
• Avoid using the same type of example repeatedly, i.e. examples all from one area, e.g. “sports”
• Avoid knowledge becoming “inert” by allowing time to draw connections between multiple, interleaved examples that vary in their concreteness or abstractness.
• Anchor new ideas in stories or scenarios that are familiar and interesting.

5a: Use pre-questions to introduce a topic – low level of evidence

Pre-questions are thought to activate prior knowledge and focus students’ attention on the material to be learned.

Despite recommending it as way to improve student learning, the panel deemed the level of evidence for quizzing to be low, as most of the research had been completed with college students, or based on laboratory experiments carried out on reading from written text, rather than tested as a component of regular classroom instruction.

The research does seem to suggest, however, that when pre-questions are used to preview the content of assigned material, there will likely be gains in learning of the pre-questioned material, providing students don’t read selectively based on the content of the pre-questions used.

Teaching strategies:

• Direct students’ attention to important facts and concepts by using pre-questions to introduce new topics.
• Prepare several pre-questions that students can attempt immediately on entering the lesson as part of the “do now”

5b: Use quizzes to re-expose students to information – strong level of evidence

The act of practising recalling information from memory enhances learning, reduces the rate of forgetting and cements information to memory.

Laboratory experiments across a wide range of materials and ages have repeatedly demonstrated that testing promotes remembering of material on a later test, and is almost always more powerful than spending additional time studying material.

Teaching strategies:

• Take every opportunity to prompt students to retrieve information.
• Use closed book quizzes after teaching material, prior to final testing.
• Ensure corrective feedback is provided following testing to ensure errors don’t remain.
• Use websites, e.g. http://www.quia.com to share or create quizzes.

6a: Teach students how to use delayed judgement of learning techniques to identify concepts that need further study – low level of evidence

The evidence in support of this recommendation comes mainly from experimental research in the laboratory, rather than in the classroom.

Without training, most learners cannot accurately assess what they know and what they don’t, and typically overestimate how well material has been mastered – “the illusion of knowing.” Knowing what you have and haven’t mastered accurately, is therefore essential in identifying what you still need to spend time studying, which in turn increases the likelihood of performing better when tested.

The “cue-only delayed judgement of learning procedure” is thought to be a key technique for breaking this illusion, which works as follows:

1. Students should test their mastery of material after a meaningful delay.
2. Students should only have access to “the cue” and not the answer when testing whether they know concepts or not, i.e. multiple choice questions should not be used for this purpose.
3. Students should judge how likely they are to get the answer right, as well as answering the question.

A similar technique, the “delayed keyword technique” supports students to judge how well they have retained material they have read after a delay, for example a section of a textbook or a chapter of a book, by asking them to generate keywords or sentences that summarise the main points.

Teaching strategies:

• Pre-prepare 10 questions (for example) that capture the core content to be learned.
• Give the students the questions one at a time, asking them to use a scale of 1 to 100 to judge how likely they feel they would be able to answer the question correctly tomorrow
• Ask students to review the material, use a text or ask the teacher to find out and record the answer to any question they did not score as 100.
• Use repeatedly over the course of the year, teaching students how to use this technique independently.
• Teach students to use the “delayed keyword technique” to generate four key terms and definitions following assigned reading out of class, followed by re-reading if they are unable to do this.

6b: Use tests and quizzes to identify content that needs to be learned – low level of evidence

The evidence in support of this recommendation comes mainly from experimental research of college students and laboratory tests, rather than in the classroom.

As previously stated: Inaccurate judgements by learners of what they have and haven’t learned well can mean subsequent study is focused on the wrong items.

Quizzing is thought to help students identify which items are not well learned, as does re-reading material when a test is interspersed between readings.

Teaching strategies:

• Closed book quizzes following presentation of material.
• Very short “spot check” quizzes covering material from the previous night or prior classwork.

7: Help students build explanations by asking and answering deep questions – strong level of evidence

The evidence base includes over a dozen experimental studies each, in both school and college settings, plus a large number of laboratory experiments.

Shallow knowledge is concerned with basic facts or skills, whereas deep knowledge is when learners are able argue with reason and logic, explore relationships between facts or concepts or answer “why?”

Interventions that specifically train students how to ask deep level questions while studying new material, e.g. classroom discussion, provision of exemplar materials and modelling how to ask and answer questions, have been shown to improve the rate and depth of student questions, as well as their comprehension of the material.

Teaching strategies:

• Identify and prepare deep level questions that require deep level responses.
• Ask questions that challenge students’ prior beliefs and assumptions.
• Model the process of asking and answering deep questions.
• Model and encourage students to “think aloud”.
• Encourage students to respond to explanations by their peers.
• Allow plenty of time to answer deep level questions.

---

The IES Practice Guide: Organizing Instruction and Study to Improve Student Learning by Pashler et al (2007) is available here.