Tag Archives: rubric

Designing a new post-levels curriculum and assessment model from scratch

This is the 5th post in a series about how we are designing our own post-levels curriculum and assessment model from scratch.

The story so far:

This latest update contains a miscellany of information and ideas that I’ve shared at our second curriculum conference and most recently at the Dare to imagine – Education for the 21st century conference and the Cramlington Festival of Learning TeachMeet.  It attempts to pull together more detail on:

  • context and why we are moving away from levels
  • the interplay between curriculum planning and assessment
  • tracking of progress

It also includes a number of curriculum planning tools that could be used to adopt a common planning framework.

A new taxonomy?

Most of us are familiar with Bloom’s taxonomy and the SOLO taxonomy, however, the end of statutory levelled assessment has brought with it a new kind of taxonomy that can be used to describe the various behaviours people often seem to exhibit in response:

a new taxonomy?

  • IGNORING – pretending it’s not happening
  • PANICKING when you realise it is happening
  • PROCRASTINATING – accepting it’s happening and deciding to deal with it later
  • WAITING for “something” to come along
  • SEARCHING what are others doing?
  • BUMBLING  trying to move forward without any real plan

Despite exhibiting a number of these behaviours ourselves this year, I’m pleased to say we are now at last well on the way to creating our own post-levels curriculum and assessment model.

Some thoughts on levels

  • Although originally intended to provide information on progress, there is a danger they have become a label that discourages a common intellectual mission and perpetuates a fixed mindset.  “Joe is a level 5” or worse still “I’m a level 5.”
  • The temptation to move up levels quickly in the name of “progress” is at odds with our desire to secure a deeper understanding of the big ideas, not just isolated content, and to allow more time for mastery of fundamental knowledge and skills.
  • The various models used to aggregate test scores, APP and the use of sub-levels by schools makes them unreliable.
  • High performing school systems don’t use levels

Can you re-think assessment in isolation without re-examining your existing curriculum?

Despite levels becoming non-statutory at Key Stage 3, the freedom to innovate and deliver the curriculum we want has always been there.  The limited amount of change in some cases between the old and new National Curriculum could offer little incentive to change, with some schools deciding to “stick” rather than “twist” or bolting on new assessment systems to their existing curriculum.

round-hole-square-peg

On the other hand, it also represents a golden opportunity to design curriculum and assessment systems that teach and assess what we value.

  • To make links and connections between big ideas explicit
  • To develop individuality and the ability to think
  • To specifically develop skills and habits of learning as well as knowledge.
  • To go “beyond” the traditional programme of study, to provide real stretch and challenge
  • To provide our students with formative feedback that means something
  • To allow for simple, meaningful reporting to parents and carers

Re-designing curriculum and assessment isn’t easy though.  We have to ask ourselves searching questions and think hard.  It takes time and we have to allow for that and ensure we provide ample opportunities within school.

Big ideas

Each of our subject areas have determined their own ‘organising concepts’ or ‘big ideas’ as well as the key knowledge and skills that weave through their curriculum – the golden threads.

For example, in science:

Slide04

Planning for excellence (and beyond)

Progression is then mapped out for each big idea by asking:

  • What does excellence look like?
  • What can my students do?
  • What do my students need to understand next?
  • What does this enquiry prepare students for next and how does it build on what they have already done?
  • How can we go beyond the boundaries of the existing Key Stage?
Slide09

Big Idea 1: All materials in the Universe are made of very small particles

Cognitive science and curriculum mapping

We’ve also looked at how a knowledge of cognitive science might support the way we construct programmes of study in each subject.  In particular how it could help us to:

  • encourage students to engage emotionally with content by ensuring appropriate degrees of challenge
  • avoid overloading working memory by linking to the big ideas / building on prior learning
  • build storage and retrieval strength by mapping our programme of study to incorporate spacing and interleaving

Slide1

Here’s an example of how we might space and interleave some of our big ideas to ensure progression of knowledge and skills across our science programme of study:

Slide07

A threshold assessment rubric is then developed for each unit that:

  • Sets the bar high
  • Focuses on assessing the key knowledge and skills for that particular unit
  • Scaffolds down from the beyond threshold
  • Supports the development of deeper understanding and skill development
  • Enables provision of formative feedback that supports progression to the next threshold

Slide08

It is only at this point that the lesson-by-lesson overview is then created, containing links to the last interleaved sequence; the learning intentions; specific, pre-planned probing questions that encourage thinking as well as the “products” we expect students to create.

Slide14

How often though, do we begin our planning at this point, rather than defining our…

  • Purpose
  • Big ideas
  • Key knowledge and skills
  • Progression
  • Mapping
  • Assessment criteria

…in advance?

Tracking

Establishing progress necessitates the need for a baseline, which can be a tricky business.  In the past, we have tended to lean heavily on KS2 test data,  however in our initial discussions we see this as an opportunity to use a wider range of data to include:

  • KS2 English + Maths test scores
  • KS2 Teacher Assessment and dialogue with feeder schools
  • MidYIS / CAT3 ability testing
  • FFT estimates
  • Internal tests on entry
  • Reading ages
  • etc.

Once a baseline has been established for each student, progress could then be measured relative to this using simple statements about progress relative to it, rather than targets that place ceilings on student achievement.

Slide16

Threshold performance could then be used to discuss “flight paths” to GCSE using the current grades A*-G or the new GCSE points system.

Slide17

e.g. if a student’s was currently working at the “securing” threshold they might usually be expected to progress to grade B/C (using current GCSE grades) or point 7/6

If they were working at the “developing” threshold then we might expect them to progress to grade C/D…etc.

What concerns me at the minute though, is how the use of some of this data fits with our thinking about a common intellectual mission and that all students are capable of excellence.

In fact, after months of reading, discussing, thinking and investing significant time (and cost) to allow joint discussion, planning and collaboration sometimes this feels as close as it gets to where my head is at right now.

Slide18

We’ve still got lots to work out and will need to evaluate the efficacy of all our work as we progress, however, in choosing to design a curriculum and assessment system that we value, it’s clear we share a real excitement, hope and optimism about the future.

Slide19

Here’s a link to an Excel version of our curriculum planning tools.  There are a number of planning sheets contained in the workbook, including some “Big Picture” questions by Pete Jones.  Feel free to use and adapt as you see fit.  I would love to hear from you if you decide to use any of them in your school.  Feedback is always welcome.

Dan

 

 

Advertisements

Better learning intentions

sailing-ship-sunset

“Imagine oneself on a ship sailing across an unknown sea, to an unknown destination……very quickly, the daily life on board ship becomes all important…..the daily chores, the demands, the inspections become the reality, not the voyage, nor the destination.”

(Mary Alice White’s view from the students’ desk from Dylan Wiliam’s: Embedded Formative Assessment)

Why focus on learning intentions?

A ‘no-brainer’ right?  Learning intentions when used well help both us and our students to see the destination and chart the voyage.  Great learning intentions can provide challenge, foster grit and determination, develop higher order thinking through good modelling and scaffolding and encourage us all to aim for excellence.

under the microscope

But how many of us have really put our learning intentions under the microscope lately to analyse their quality and effectiveness?  How much of our time and effort is spent framing them or developing success criteria that promotes opportunities for all students to achieve excellence?  As Wiliam says:

“this is why good teaching is so extraordinarily difficult….it is hard…..because it has to be designed backwards”

A brief history of learning intentions according to……me

I started teaching in 1995 nearly 20 years ago.  I have thought long and hard about this and can still remember vividly the first time I began sharing learning objectives.  In 2002 the Government launched the Key Stage 3 National Strategy for Science.  At the time I was Head of KS3 Science which inevitably resulted in my working with our Local Authority Consultant as part of this initiative.  It was she who first pointed out to me that our department should really be sharing objectives with our students.  There wasn’t any great discussion about the reasons as I recall – simply that our lessons would be better if we shared them.  So off we all went “bolting-on” objectives to our existing lessons – job done!

It wasn’t until not long after this, as a Head of Department, when working with another LA Consultant on “AfL unit 3 – Developing objective led lessons” that the penny began to finally drop and I began thinking more carefully about objectives, using them to plan lessons and series of lessons rather than creating them post-script.  Through this publication I was introduced to the work of Bloom and his taxonomy of learning and it was around this time that I first read Black and Wiliam’s groundbreaking work on formative assessment “Inside the Black Box” and subsequent publications by the King’s College London Assessment Reform Group.

From there, through my subsequent involvement with the National 8 Schools AfL project in 2005/6 via a brief stint as an LA consultant myself, I got a lot more opportunity to reflect on their importance and their use became central to my teaching.  By this time I had been teaching for 10 years.  Was I late to the party or had it yet to really start?

late to the party

Since then the way I’ve chosen to frame my learning intentions have seen more changes than Lady Gaga’s wardrobe and it has been interesting to look back as far as 2005, thanks to various memory sticks I have miraculously managed not to lose.  Unfortunately I can find no physical record of any I wrote prior to this on that revolutionary piece of kit that replaced the chalkboard – the rolling whiteboard!

Here are a few of my own examples from the last 9 years to give you a flavour of what I mean:

Reactivity of metals_lesson obs 2008 WALT 2007 WALT WILF 2007 learning intentions grid simpsons learning intentions SOLO learning intentions 2 SOLO learning intentions 1 WAGOLL1b WAGOLL 2

Opportunity knocks

This year, in order to further develop teaching & learning across our school we have chosen to focus our efforts on two areas:

1)      Better learning intentions

2)      Better feedback & response

The two main reasons for this were:

1)      Our evaluation of teaching and learning during the Summer Term identified both of these key features of better assessment as pivotal differences between ‘the best’ teaching & learning and ‘the rest’.

2)      Their known high influence on the achievement of students.

It also presented us with a great opportunity to look again at why learning intentions are important and how we can make them even better.  A simple act of deliberate practice for us all.

opportunity knocks

Where to start?

We began the year by developing a shared understanding of what we meant by objectives, outcomes and success criteria.  We did this by collecting and sharing exemplar learning intentions from different departments via an e-portfolio before following this up with discussions in departments during the course of the Autumn Term.  Towards the end of term departments shared their best examples of learning intentions and success criteria for inclusion in our e-portfolio.

What did this tell us?

  • That the routine sharing of quality learning intentions is becoming more consistent, especially the use of the terms objective, outcomes, success criteria & WAGOLL (What A Good One Looks Like)
  • That there is still some variation between the ‘best’ and the ‘rest’ in terms of

–          length and complexity

–          descriptions of tasks rather than learning as a result of them

–          degree of challenge

–          sharing of quality success criteria / WAGOLL

So what now?

For us it’s time to get out the microscope and to begin to examine in detail ‘what makes learning intentions tick’ in order to analyse how we can make ours even better.

watch ticks

Important features of learning intentions

In his book “Visible Learning for Teachers: Maximizing Impact on Learning” John Hattie identifies a number of important features of learning intentions from his extensive analysis of the research, i.e they should be:

  • Clear to students
  • Lead to a shared understanding of learning goals
  • Transparent – show how tasks relate to learning intentions
  • Inclusive of all students
  • Appropriately Challenging
  • Referred to – to help students chart their “journey”
  • Build in mechanisms for knowing that learning has been achieved

The 5 essential components

Hattie goes on to outline five essential components related to learning intentions and success criteria as follows:

  • Challenge

Related to prior knowledge and learning

Relative to a student’s current performance and understanding

Not unattainable – students must be able to see a pathway to challenging goals

Create positive tension –error is welcomed and encouraged

  • Commitment

Greater commitment from students = greater performance

More powerful when related to challenging tasks

  • Confidence

Supported by scaffolding along the learning journey

Helps build resilience amongst students to tackle challenging goals

  • High expectations

Support the development of students having high, appropriately challenging expectations of themselves

The most powerful influence on enhancing achievement (d=1.44)

  • Conceptual understanding

Move understanding from the surface level to deep and conceptual levels

The importance of language

As Dylan Wiliam points out, there is no shortage of advice for teachers on how to construct learning intentions and success criteria, although he does go on to encourage us to think about our use of language when framing them.  Using student-friendly language might help to engage students initially, however at some point students will still have to become familiar with key terms, phrases and language that define our subject disciplines.  “A sense of audience” can be simplified and broken down initially, however we still want our students to know what is meant by the term.

The role of context

Shirley Clarke advocates separating the learning intention from the context in order to achieve a greater “degree of transfer” of learning to other areas.  Too much detail can be seen as counterproductive – the clearer you are about what you want, the more likely you are to get it, but the less likely it is to mean anything!

She uses this example based on banana production:

“To understand the impact of banana production on banana producers”

-can assess the students on what they know.

“To understand the impact of production on producers in the developing world” (using banana production as the context for learning)

-allows us to assess students on their ability to transfer their learning to e.g. sugar production.

Further exemplification of this from Clarke can be found below.

Confused_clarified_context

Making the non-observable → observable

Bloom’s taxonomy will be familiar to many of us who have used it when planning our lessons.  Of his 3 domains of learning, it is the cognitive domain we usually use when framing our learning intentions.

As you will have seen from the examples I shared above, in the past I have used (and advocated the use of) Bloom’s taxonomy, however, I was interested to discover recently that the taxonomy was developed initially from a proposal by a committee of college examiners who collated curriculum objectives from their own institutions and sorted them, rather than being based on research of student learning.

Bloom & Krathwohl’s original work identified a number of cognitive levels we function at, from the most basic “knowledge” to the more sophisticated “evaluation”.

Bloom & Revised Bloom

In 2001, the taxonomy was revised and the nouns replaced with verbs, such as “remember”…”understand“…etc. and at first glance the sophistication appears to have been switched at the higher levels of cognitive function, with “evaluation” and “synthesis” changing order.

Despite often being represented as a pyramid or a set of increasing steps, it is possible for students to operate at a number of levels at the same time.  It might therefore be better to model the taxonomy as follows:

Modelling Blooms

The empirical evidence for the switch seems to be a mystery, although research has since shown that students remember more when they have learned to handle a topic at the higher levels of the taxonomy.

Returning to Hattie’s important features that learning intentions should be clear, transparent and provide us with a way of ascertaining whether learning has been achieved, how easy is it to observe whether a student “understands”?  More helpful verbs are needed to help students understand how to demonstrate their mastery at a particular level and to make the “non-observable → “observable”.  We usually frame learning intentions using these “observable verbs”.

Making non observable observable

One of the criticisms of Bloom’s however, is very evident in the table above – where you can see how the same verb can be used across multiple cognitive dimensions.  “Identify” highlighted in yellow is a good example of this.  Furthermore, when searching for the taxonomy you will find variations in the way each of the verbs are used, which can be off-putting for teachers and confusing for students.

Hattie also highlights the importance of getting the balance right between surface, deep and conceptual learning – either in the short term or across a series of lessons.  He advocates the SOLO taxonomy as a powerful model for understanding these three levels, which I started using myself last year.  Unlike Bloom’s SOLO is a theory of teaching and learning based on research of how students learn, rather than a theory of knowledge.  Many of us are already familiar with the SOLO (Structure of Observed Learning Outcomes) levels devised by Biggs & Collis to describe increasingly complex levels of thinking that can be observed.  If you aren’t already familiar with the SOLO taxonomy, our Head of Science Julie Ryder has written a great ‘Introduction to SOLO’ which is accessible here and has also blogged about her personal learning journey with SOLO available here.

SOLO levels

Through constructive alignment, a deliberate alliance is then made between the learning activities we plan and the learning outcomes we expect.  Constructive alignment is a conscious effort to provide students with clearly specified goals, well designed and appropriate learning activities and well designed assessment criteria to enable quality feedback.

It helps us to:

  • “Unpack” a particular objective
  • Align learning goals, activities and assessment criteria
  • Scaffold learning intentions & success criteria
  • Ensure that outcomes match objectives

constructive alignment 1

Put simply, constructive alignment requires our perceptions of assessment to match with those of our students.  Again, you can see below how verb use is central to this process.

constructive alignment 2

Here is an example of how verbs are used in the SOLO taxonomy to support constructive alignment.  Notice that unlike Bloom’s, there is consistent verb use at each cognitive level.

SOLO taxonomy & constructive alignment

The case for routine sharing of success criteria

Success criteria are crucial in helping to develop a shared understanding of what “excellence” looks like.  They enable us and our students to hold a shared concept of quality as well as enabling continuous ‘quality control’ of ‘products’ during production.

Ensuring that all students know what quality work looks like has a profound impact on achievement gaps.  Wiliam outlines the research of White & Frederiksen in illustrating the power of students understanding what they are meant to be doing as follows.

success criteria research

Here the reflective assessment group were introduced to the 9 assessment criteria that would be used to evaluate their work.  Students were asked to assess their own performance against the criteria as part of an ongoing process.  Peers also gave feedback using the criteria.

The control group on the other hand discussed their likes and dislikes of the topic they were studying once per week.   Both groups had similar prior achievement in literacy and numeracy.  Both groups studied the same science modules.

As you can see, the gains for the group exposed to success criteria are obvious.  Notice the benefits for lower achievers are even greater.

Developing quality success criteria

Developing quality success criteria means challenging students to develop quality rather than just completing tasks.

The two examples from Hattie below illustrate this difference well.

Hattie success criteria example

When thinking and subsequently writing about this I kept finding myself drawing comparisons to fellow blogger Andy Tharby’s ‘sentence escalator’ idea.  To paraphrase him “great success criteria are never finished”.  The more we focus on ‘escalating’ the success criteria, the closer we get to achieving excellence.

Choice of rubric

A rubric is basically an assessment tool that contains the assessment or marking criteria.

Task-specific rubrics include all the criteria for a particular task (like the example from Hattie shared previously) whereas generic rubrics contain non-specific guidance.  When we use task-specific rubrics with students one of the advantages is that they know exactly what to include, however, in doing so we could limit the transferability of learning to new areas referred to earlier.  For this reason task-specific rubrics work well for summative assessment.  The fact that a new rubric has to be constructed for each task also supports the case for the use of generic rubrics on a day to day basis.

SOLO rubrics

Unlike Bloom’s, SOLO verbs can be used to ask a question at one level of cognitive complexity while at the same time providing an opportunity for students to demonstrate their learning at multiple levels of the taxonomy.  For me it helps to think about SOLO verbs being able to be used in 3D as opposed to Bloom’s 2D use.

In SOLO Taxonomy: A Guide for Schools – Planning for differentiation, Hook & Mills show how SOLO assessment rubrics can be used to achieve this.  In the case shown below, you can see how when asked to “describe” – a verb used at the multistructural SOLO level – students can still demonstrate their learning at all levels of the SOLO taxonomy.

SOLO rubric

Co-construction

Co-construction is simply the process of ‘building together’.  By developing success criteria with students we increase the likelihood of them taking ownership of the criteria and applying them in the context of their own work.  By allowing students to help create rubrics, we help them to construct understanding for themselves.

Beyond rubrics – models of excellence

As it has done for countless others, the work of Ron Berger has transformed the way I think about and approach my own work, first and foremost as a teacher, but also as a school leader.  In his book “An Ethic of Excellence” Berger outlines “work of excellence” and argues that lists and rubrics are important, but aren’t enough in their own right, as they don’t leave a picture…a vision…an inspiration.  He urges us to admire models, to find inspiration in them and to figure out together what makes the work strong.  Rather than viewing this as simply copying what already exists, Berger refers to this as “tribute work” – borrowing from the best and building on it.  In doing so we create a vision of the goal measured against the best.

The use of student work has been shown to be a powerful mechanism for securing engagement with the success criteria and supporting student understanding of them.  Rather than talking in the abstract about what might make a piece of work good Berger encourages us to critique real examples.  Students tend to be better at spotting weakness in work produced by others and this could be as simple as asking students to rank and rate examples of work produced previously by other students in terms of quality, giving reasons for their choices.

In his book, Wiliam also shares a good example of how an MFL teacher supports her students to understand what a good French accent sounds like.  The teacher organised students into groups, in which each member of the group read the same passage of French aloud after which the group decided which member had the best accent.  The best French accent from each group was then heard followed by a teacher-led, whole class discussion to decide the strengths and weaknesses of each accent.

But why stop there?  As Berger points out, why not critique the work of real writers, scientists, artists…etc?

One final thought…

What better way to support the development of a culture of excellence within our own school than to “begin at the beginning” by focussing ourselves not just on producing “better learning intentions”…but producing learning intentions that are truly “excellent”.

Dan