Sunday, November 17, 2013

Chris Sarra's Stronger Smarter Schools: an Independent Evaluation

A Summative Evaluation of the Stronger Smarter Learning Communities Project
They [teachers] say you’re not going to be successful - that you can’t do this.

And just because you’re Indigenous or something, they expect nothing from you. That’s what they basically say. That’s what they want you doing; they want you to stay what they think you are.
—Indigenous Secondary School Student, 2012
Follow the link for the full report. Hopefully listing the findings and policy implications below will provide an initial incentive to some robust discussion of the full report.

ACRONYMS
Stronger Smarter Institute (SSI)
Stronger Smarter Leadership Program (SSLP)
Stronger Smarter Learning Communities Project (SSLC)
Index of Community Socio-Educational Advantage (ICSEA)

Abstract

This is a summative evaluation of the Stronger Smarter Learning Communities (SSLC) project that examines whether and how the SSLC project had an impact on Australian state schools which adopted its models and approaches. Drawing from qualitative and quantitative data sets, it also presents the largest scale and most comprehensive analysis of Indigenous education practices and outcomes to date.

It includes empirical findings on: success in changing school ethos and community engagement; challenges in progress at closure of the 'gap' in conventionally measured achievement and performance; schools' and principals' choices in curriculum and instruction; profiles of teachers' and principals' training and views on teacher education; and a strong emphasis on community and school Indigenoous voices and views on Indigenous education.

Key Findings

Introduction

Key Finding 1:
The transfer/mobility issue does not appear to be a major problem for continuity of school leadership: the average principal tenure in their current position is 5.74 years, but principals averaged 2.36 schools over the past 5 years.

Key Finding 2:
Remote/very remote schools are more likely to have less experienced staff with higher levels of transfer and turnover: respondents in remote/very remote schools were more likely to report having had 5 or less years of teaching experience compared to their colleagues in metropolitan or provincial schools; respondents in remote/very remote schools were more likely to report having spent 5 years or less in their current school compared to their colleagues in metropolitan or provincial schools.

Key Finding 3:
The teaching workforce is highly experienced with an average experience level of 14.63 years, but the large standard deviation (11.109) suggests that there is a wide variation in the age of teachers, with a significant proportion of highly experienced teachers and a significant proportion of beginning teachers.

Key Finding 4:
Overall credential levels of the administrative and teaching workforce are high, with over 80% of teachers and principals having at least a 4 year bachelor’s degree, and 9.7% of teachers and 19.6% of principals with masters or doctoral degrees.

Key Finding 5:
Overall levels of previous coursework on Indigenous education are low, with less than one third of the combined principal and teacher sample reporting any prior specialised pre- or in-service courses.

SSLC Operations and Processes

Key Finding 6:
SSLC encountered difficulties in staff retention and continuity.

Key Finding 7:
There were content and program transition issues in linking the SSLP leadership training model with SSLC’s focus on school reform.

Key Finding 8:
SSLC and SSI were not able to identify, document and circulate models and exemplars of successful practice for use by Hub and Affiliate schools.

Key Finding 9:
SSLC and SSI did not systematically provide advice on specific classroom- level reforms or innovations to schools.

Community Study

Key Finding 10:
The Indigenous community experience is that schools continue to work from a deficit perspective on Indigenous students, parents, communities and community members, and school staff.

Key Finding 11:
A significant proportion of teachers surveyed expressed deficit views of Indigenous students, families, communities and cultures.

Key Finding 12:
Many Indigenous education workers and teachers report the experience of marginalisation and disenfranchisement in schools, with reactive job roles and insecure working conditions.

Key Finding 13:
Community members interviewed consider many attempts at school consultation as token and superficial, with little real participation in school decision-making and governance.

Key Finding 14:
Indigenous students and staff interviewed report everyday experiences of labeling and mis-recognition of their actions, learning and social relations.

Key Finding 15:
Community members and parents interviewed acknowledge the importance of test score improvement, but are also concerned with other pathways, aspirations and goals, including cultural knowledge, awareness and relations, community participation, student safety and health.

Key Finding 16:
There is broad community support for the embedding of Indigenous knowledges in the curriculum, but Indigenous students and staff report significant problems with non-Indigenous teacher knowledge and intercultural sensitivity.

Teacher Knowledge and Community Engagement

Key Finding 17:
Teacher self-reported knowledge of Indigenous cultures, histories and communities is low.

Key Finding 18:
Teacher self-reported everyday engagement with Indigenous peoples and communities outside of the school is low.

Key Finding 19:
Teachers with higher self-reported levels of knowledge about and engagement with Indigenous communities and cultures are more likely to report that they are teaching Indigenous topics and knowledges in the classroom.

Key Finding 20:
Teachers reported that they were not satisfied that their pre-service teacher education adequately prepared them to support Indigenous learners.

Key Finding 21:
Teachers in SSLC schools report higher levels of engagement with Indigenous communities and cultures than teachers in non-SSLC schools.

School Cultural and Structural Reform

Key Finding 22:
There are no significant differences in SSLC and non-SSLC leaders’ reported foci on high expectations and Indigenous school climate.

Key Finding 23:
SSLC school leaders report stronger foci on Indigenous staffing and leadership, and community engagement and governance than non-SSLC school leaders.

Key Finding 24:
Teachers report 3 identifiable paths of reform in their schools: (1) from Indigenous school climate to high expectations promotion and enactment; (2) from Indigenous school climate to Indigenous community governance and Indigenous school leadership; (3) from Indigenous school climate to Indigenous community engagement and knowledge.

Pedagogy

Key Finding 25:
SSLC teachers report significantly more instructional time allocated to embedding of Indigenous content, knowledges and topics in the curriculum than teachers in non-SSLC schools.

Key Finding 26:
There are no significant differences in SSLC and non-SSLC teachers’ reports of their practices in other areas of pedagogy.

Key Finding 27:
The dominant approaches to pedagogy reported by SSLC and non-SSLC teachers are emphases on basic skills instruction and Vocational Education.

Key Finding 28:
Overall reported time allocated to the embedding of Indigenous content, topics, and knowledges is low.

Key Finding 29:
Reported time allocations for canonical pedagogy, progressive pedagogy and critical literacy pedagogy are low.

Key Finding 30:
Many teachers do not have the requisite background knowledge and cultural experience to teach topics and content on Indigenous knowledge and culture.

Key Finding 31:
When the overall school percentage of Indigenous students reaches key thresholds, it increases the likelihood of an emphasis on basic skills (>15%), Vocational Education (>11.5%) and embedding of Indigenous knowledge (>15.5%).

Key Finding 32:
Teachers in lower ICSEA value schools are more likely to report stronger emphasis on behaviour management (<933.5), basic skills (<922), Vocational Education (<952.5) and embedding of Indigenous knowledge (952.5).

Key Finding 33:
More experienced teachers (>10 years) report less time allocated to behaviour management and basic skills.

Key Finding 34:
SSLC Hub schools’ choices of curriculum programs, approaches and in- service programs are eclectic, with no discernible patterns of state, regional or school-type consistency.

Language

Key Finding 35:
Overall, teachers and school leaders reported low emphases on Indigenous languages and dialects in the classroom.

Key Finding 36: Overall levels of teacher awareness of Indigenous languages is low.

Key Finding 37:
Schools with higher percentage of Indigenous students are more likely to focus on Indigenous languages and dialects in the curriculum.

Key Finding 38:
The focus of current activity is in the teaching of Indigenous languages as part of LOTE and language revitalisation efforts, concentrated in a small number of schools surveyed.

Key Finding 39:
Schools working with LOTE programs are faced with complex local issues of language selection and the availability of linguistic corpus documentation, and with problems in securing qualified local speakers/teachers and curriculum resources.

Key Finding 40:
Teachers’ and school leaders’ understanding of, and engagement with, English as a Second Language and English as a Second Dialect issues facing Indigenous students is low.

Assessment and Certification

Key Finding 41:
Principals and teachers have limited expertise and training in the analysis and the use of test score and other performance data.

Key Finding 42:
The emphasis on improvement of NAPLAN test results is a dominant influence on school planning, policy and pedagogy.

Key Finding 43:
There is little evidence of innovation or the building of teacher expertise in classroom assessment (e.g., task-based assessment, high quality assessment, authentic assessment).

Key Finding 44:
Personal Learning Plans are a viable approach to authentic and negotiated assessment and planning, but these require training and systematic implementation.

Key Finding 45:
Streaming and ability grouping are common at all levels of primary and secondary education.

Systemic Data on Student Performance

Key Finding 46:
There are no statistically significant SSLC effects on improved school level attendance.

Key Finding 47:
There are no statistically significant SSLC effects on improved school level achievement on NAPLAN tests.

Key Finding 48:
In SSLC and non-SSLC schools, there are numerous individual instances of ‘closing the gap’ between Indigenous and non-Indigenous students in specific age/grade cohorts in specific curriculum areas – but there is no coherent pattern of school level, school type, jurisdiction or curriculum- area effects.

Sustainability

Key Finding 49:
SSLC has not reached sustainable levels of Hub-to-Hub communication and continues to rely on communication mediated by SSLC central administration.

Key Finding 50:
SSLC is not scalable and has not shown signs of increased or autonomous Hub-to-Hub communication as it has developed over time.

Key Finding 51:
Longstanding and durable regional clusters are the organisational units with the demonstrated capacity to sustain networked communications.

Key Finding 52:
School leaders do not report staff turnover as a major impediment to sustainable reform.

Key Finding 53:
School leaders report that the difficulty in hiring Indigenous staff and engaging with key community leaders is an impediment to sustainable reform.

Major Findings

Major Finding 1:
That the Stronger Smarter model’s recognition of the prevalence of ‘deficit thinking’ in schools is accurate – but the approach lacks an institutional analysis of how to reform and alter the effects of this phenomenon.

Major Finding 2:
That SSLC was successful at changing school foci on the need for Indigenous hiring, staffing and leadership in the school, on the need for improved community engagement and moves towards Indigenous participation in school decision-making and governance.

Major Finding 3:
That SSLC was successful at increasing teachers’ and leaders’ attention on the importance of knowledge of Indigenous cultures and communities, and on the need to embed these in teaching and learning.

Major Finding 4:
That despite these efforts, the general Indigenous community view and experience is that schools continue to work from deficit assumptions that preclude student enfranchisement, academic improvement and genuine community involvement and governance.

Major Finding 5:
That SSLC was not successful at generating the improvement of conventionally measured attendance and achievement.

Major Finding 6:
That the predominant, default modes of pedagogy for Indigenous students are basic skills instruction leading to vocational education pathways, part of a deficit model of testing/remediation/streaming and tracking.

Major Finding 7:
That there is an overall lack of school level curriculum program coherence in teaching/learning in SSLC and non-SSLC schools, with many principals and schools making eclectic and apparently idiosyncratic decisions about programs, curriculum materials and in- service approaches.

Major Finding 8:
That overall school leader and teacher knowledge of and engagement with Indigenous communities, cultures, languages and histories are a major impediment to community engagement, school reform and improved outcomes.

Policy Implications

Policy Implication 1:
That the current emphasis on NAPLAN without systematic state and regional-level curriculum assistance and advice has the effect of increasing principals’ tendencies to pursue ‘quick fix’ programs in a way that generates less coherent school programs and skewed test results.

Policy Implication 2:
That the push for increased principal autonomy without improved training in instructional/curricular leadership and data analysis risks exaggerating the skewed and idiosyncratic patterns of achievement described here.

Policy Implication 3:
That the Australian Curriculum mandate for the embedding of Indigenous knowledges raises major issues in terms of the requisite depth of teacher knowledge of Indigenous cultures, histories, issues and languages.

Policy Implication 4:
That - given the diversity of schools, communities and cohort demographics - the assumption that there is a single, ‘one size fits all’ curriculum or pedagogy solution for all Indigenous learners is not the solution to the problem of program incoherence, but risks exacerbating the problems identified here.

No comments: