Tag: data

The Evidence-based TL

noniharrison on

Evidenced-based practice should not only drive pedagogical initiatives of classroom teachers, it should also be at the core of the work of teacher librarians. Evidence can be used to identify a need or problem, plan and implement initiatives, and evaluate and reflect on those programs. It can also be used to justify library initiatives and programs, budgetary and staffing requirements, and a range of other resourcing needs. That being said, when evidence is gathered by the practitioner for advocacy purposes, the trustworthiness of the evidence and its interpretation comes into question. Therefore, it is vital evidence gathering practices are holistic and incorporate a range of methods.

Vertical and horizontal data sets can help schools better understand their students’ learning journey, while considering their specific context. Data is easily interpreted in a myriad of ways and can be manipulated to suit purpose. It becomes an issue when data is used to compare school performance, rather than focusing on individual student progression. The focus must be on student learning outcomes rather than outcomes. Therefore, vertical data gathered from high-stakes testing should be used in conjunction with horizontal data that provides a more holistic picture of students’ learning levels (Renshaw, Baroutsis, van Kraayenoord, Goos, & Dole, 2013).

Teacher librarians adopt a holistic approach to evidence-based practice by considering the whole school with which they serve and by drawing on a range of evidence in a range of ways (Gillespie, 2013). As seen in Figure 1, Todd (2015) proposes a holistic model for school libraries, which includes evidence for practice (Foundational: informational), evidence in practice (Process: transformational), and evidence of practice (Outcomes: formational). Furthermore, Robins (2015) highlights the usefulness of action research as part of a holistic approach to collect qualitative and quantitative data and connect educational research with improved practice. Oddone (2017) provides an overview of action research for school libraries as seen in Figure 2.

A table, which shows Todd's three categories of evidence-based practice. 1. Evidence FOR Practice - FOUNDATION Informational: Existing formal research provides the essential building blocks for professional practice. 2. Evidence IN Practice - Applications/Actions PROCESS Transformational: Locally produced evidence—data generated by practice (librarian-observed evidence)—is meshed with research-based evidence to provide a dynamic decision-making environment. 3. Evidence OF Practice - Results—impacts and outcomes; evidence of closing of gap OUTCOMES Formational User-reported evidence shows that the learner changes as a result of inputs, interventions, activities, and processes.
Figure 1: Holistic model of evidence-based practice for school librarians (Todd, 2015).

 

Infographic showing: Action Research for Teacher Librarians: A brief introduction and overview to Action Research as a tool for evidence based practice for teacher librarians.
Figure 2: Action Research for Teacher Librarians: A brief introduction and overview to Action Research as a tool for evidence based practice for teacher librarians. (Oddone, 2017).

Gillespie (2013) found that teacher librarians gather evidence through two key modes; by engaging and encountering. The nature of evidence-based practice is therefore not linear. Teacher librarians gather evidence through purposeful and accidental encounters, which involves research-based and/or practitioner-based evidence and application for improved practice through a combination of intuition and reflection (Gillespie, 2013). As with improvement in any field, particularly education, reflection is key to driving improvement as it provides the metacognitive prompts to interpret evidence and consider the practitioner’s professional experience and expertise that is needed to apply evidence meaningfully.

Teacher librarians can utilise evidence-gathering tools to gather evidence for, in and of practice. Gillespie (2013) recommends valuable evidence is drawn from and used within three areas; including, teaching and learning, library management, and professional practice. A tool such as School Library Impact Measure (which I spoke about in my final assessment for ETL504) equips teacher librarians with a framework to assess the impact of their instruction on student learning outcomes during a Guided Inquiry experience (Todd, Kuhlthau, & Heinstrom, 2005). To gather evidence of library management, including collection development, the environment, and services, teacher librarians can source benchmarks from peer institutions, questionnaires or surveys to evaluate performance, set new goals or standards and implement strategies. Benchmarks from peer institutions can be sourced through network meetings and through the Softlink survey.

As a school library that runs a college-wide academic reading and writing program, we utilise data to assess the extent to which the programs effectively improve student learning. Reading comprehension tests help the teacher librarians and teachers determine the effect of the strategies on student abilities. Teacher librarians should work with heads of department and classroom teachers to analyse and digest the data in meaningful ways. Collaboration is central to the perception of the data analysis process. Teachers do not want to fall victim to judgement due to the results within their class. While this is important to note, it is also important to hold honest and open conversations about opportunities the data may present. Viewing the results through an opportunities lens ensures all stakeholders feel safe and valued and student learning is central.

This year, I have been delving deeper into the world of data and evidence to assess the impact the Study Skills program has for our boarding students. The program is designed to assist boarding students with; organisation, prioritisation, study skills, assessment skills, and research skills, while growth mindset underpins the sessions. A teacher librarian (myself on Tuesday and Wednesday nights and our Head of Library on Monday nights) facilitate a half hour targeted session with the students in the senior library – one cohort per night on a weekly rotation. These sessions are timely and relevant to the assessment and class work to ensure the value is clear. After the half hour session, the students undertake their independent study for the remaining hour and a half, while the teacher librarian circulates and assist students in smaller groups and one-on-one. I am pleased to share that since the inception of the program last year, the boarders’ results are trending up – often at a greater rate than the day students. While many factors influence the boarders’ results (strategies from classroom teachers, tutoring, support from boarding supervisors and parents for example), the results are also indicative of the renewed impetus for study and a changing culture that has developed from the program. The program has been a feat of collaboration and has seen staff members from across the college come together for this common cause. To gather evidence, I was able to use data from both Learning Analytics and TASS to compare day and boarding groups, and boarding groups over time. This also provides me with opportunities to compare external high-stakes results, such as NAPLAN results, with curriculum results. This evidence was used to not only assess the effectiveness of the program but also to open conversations with staff and students about learning outcomes. From this, I created a report to document the findings and present to teaching staff and leadership team (Figure 3). I have had incredibly valuable conversations with students concerning their progress and strategies going forward. One-on-one, I talked to students about their GPAs, compared these against their report cards, then again against their individual assessment tasks to identify strengths and areas for improvement. The additional ownership students took of their results and the empowerment felt was palpable. The evidence is used to evaluate and reflect on the programs, to plan targeted sessions, to empower students, and to advocate for the work done by the teacher librarians.

Title page of Study Skills report. The pages states: "Study Skills, 6-monthly report 2018 Semester one.
Figure 3: Title page of Study Skills report.

Ultimately, learning must be at the centre of the analysis, discussion and subsequent initiatives or action. Evidence must be gathered on a local level to determine the effectiveness of library programs and through other means, such as empirical research, to ensure best practice. Evidence must be made explicit and concerted efforts must be made to “measure the relationship between inputs, outputs, actions and student outcomes” (Hughes, Bozorgian, & Allan, 2014, p. 15). These practices can help to ensure the longevity of the school library and assert the library and staff as invaluable to the school community.

 

 

References

Gillespie, A. (2013). Untangling the evidence: Teacher librarians and evidence based practice [Thesis]. Retrieved from https://eprints.qut.edu.au/61742/2/Ann_Gillespie_Thesis.pdf

Hughes, H., Bozorgian, H. & Allan, C. (2014). School libraries, teacher-librarians and student outcomes: Presenting and using the evidence. School Libraries Worldwide, 20(1), 29-50. doi: 10.14265.20.1.004

Oddone, K. (2017). Action research for teacher librarians [Infographic]. Retrieved from https://my.visme.co/projects/jwvj7ogk-action-research-for-teacher-librarians#s1

Renshaw, P., Baroutsis, A., van Kraayenoord, C., Goos, M., and Dole, S. (2013).  Teachers using classroom data well:  Identifying key features of effective practices. Final report. Retrieved from https://www.aitsl.edu.au/docs/default-source/default-document-library/teachers-using-classroom-data-well.pdf

Robins, J. (2015). Action research empowers school librarians. School library research, 18, 1-38. Retrieved from http://ezproxy.csu.edu.au

Todd, R. J. (2015). Evidence-based practice and school libraries: Interconnections of evidence, advocacy, and actions. Knowledge Quest, 43(3), 8-15. Retrieved from https://search-proquest-com.ezproxy.csu.edu.au

Todd, R., Kuhlthau, C. & Heinström, J. (2005). School library impact measure SLIM: A toolkit and handbook for tracking and assessing student learning outcomes of guided inquiry through the school library. Retrieved from https://cissl.rutgers.edu/sites/default/files/inline-files/slimtoolkit.pdf

Models and methods for collection evaluation

noniharrison on

Evaluation of collections assesses the extent to which library collections meet the goals of the library and the needs of the community it serves. There are various methods of evaluation that can be employed in school libraries. ALA identifies two categories of collection evaluation; collection-centered measures, and use-centered measures (Evans & Saponaro, 2014). A balanced way of evaluating the collection is to employ several of these methods and to ensure objective data is gathered in the mix. Grigg (2012) proposes methods of evaluating collections that are suitable for both print and digital resources. The methods and their usefulness in school library collection evaluation is outlined below:

Usage data: assessing circulation statistics.

  • Useful in determining the popularity of resources and the subject areas that are most borrowed and types of books used. This can be helpful in assessing how the collection meets learning needs and reading needs.

Overlap analysis: assessing duplications within journal collections.

  • Useful in minimising wasted dollars through unnecessary duplicate resources.

Survey instruments: quantitative method of collecting user feedback.

  • Useful in determining the users’ impression of the library and collection and the usefulness of the collection in meeting specific needs. This value of this method is increased when used in conjunction with another method of evaluation.

Benchmarking: comparing the collections of different libraries.

  • Useful in identifying potential gaps and strengths in comparison to other school libraries. This could be done during network meetings to provide suggestions and recommendations and collaborate with others in developing quality collections to meet the needs of the clientele.

Focus groups: qualitative method of collecting user feedback.

  • Useful in determining the strengths and weaknesses of a collection from the perspective of the library users. This method should be used in conjunction with other methods of evaluation, as the results may be bias or may only apply to a niche market.

Balanced scorecard methods: matching criteria or outcomes against circulation statistics.

  • Useful in assessing circulation statistics against particular goals; however, there is difficulty in developing realistic or accurate criteria and this does not account for the quality of the resource it only counts the use.

 

What are the practicalities of undertaking a collection evaluation within a school in terms of time, staffing, and priorities, as well as appropriateness of methodology?

Collection evaluation can be an expensive process in terms of staff time (Arizona State Library, n.d.); however, it is important to make time to ensure this process is undertaken. There are opportunities throughout the school year to allocate time to this process; for example, during staff days at the beginning of end of term, and during weeks when classes are limited (e.g. weeks 7-8) and TL time becomes available for other library management duties. This should be a priority when receiving assessment task sheets or unit outlines from teachers and when organising assignment help lessons. Methods must relate to all aspects of a school library; including, assessment and curriculum support, reading for pleasure, resources for teaching and other staff, resources to support the wider community. Additionally, the methodologies used should take advantage of the resources available to ensure the process is efficient; for example, using the OPAC to gather data and generate reports.

How does the need for, and possible benefits of an evaluation of the collection outweigh the difficulties of undertaking such an evaluation?

  • Despite the difficulties in undertaking collection evaluation it is vital in determining how and to what extent school libraries are meeting users’ needs and in cementing the library in the school context as a valuable commodity. As highlighted by Hernon, Dugan, and Matthews (2014), evaluation activities provide evidence of accountability and the opportunity to review and modify practices in a clear and formal manner. It also provides justification for changes in selection priorities and supports weeding or deselection decisions (Pattee, 2013). Collection evaluation is paramount in ensuring the collection’s usefulness and relevance to the school community.

Is it better to use a simple process with limited but useful outcomes, or to use the most appropriate methodology in terms of outcomes?

  • The answer to this question may change depending on the aspect of the collection needing to be evaluated – general or specific emphasis collections. Overall, the most appropriate methodology in terms of outcomes is the ideal method a TL would use when evaluating the collection; however, constraints including time and staffing may interfere with this. When evaluating the collection to meet curriculum needs, it is important to use the most appropriate methodology, as it is the role of the school library to meet these needs and assist teachers and students in understanding and navigating the new curriculum. If evaluation processes occur more regularly, they will become quicker and easier, as the process is streamlined and more regularly updated. Furthermore, the collection will more accurately reflect the library’s mission and changing needs of the community. Hernon, Dugan and Matthews (2014) even suggest that evaluation activities should become daily activities within libraries to review and improve services with the aim of meeting stated goals and objectives. The National Library of New Zealand recommends weeding monthly or quarterly, which could also be a useful time to undertake other evaluation processes (2012).

What are the current priority areas for evaluation in your school library collection?

  • Current priority areas for evaluation in my school library collection include, evaluation of the non-fiction collection in meeting curriculum needs of senior students studying the Arab Israeli Conflict, and Year 9 and 10 Science students undertaking new inquiry tasks requiring them to break down science claims and develop very specific research questions that require resources to support the question and their reading levels.

 

References

Arizona State Library. (n.d.). Collection assessment and mapping. Retrieved from https://www.azlibrary.gov/libdev/continuing-education/cdt/collection-assessment-mapping

Evans, G. E., & Saponaro, M. Z. (2014). Library and information science text: Collection management basics. Retrieved from https://ebookcentral-proquest-com.ezproxy.csu.edu.au

Griggs, K. (2012). Assessment and evaluation of e-book collections. In R. Kaplan (Ed.), Building and managing e-book collections: A how-to-do-it manual for librarians (pp. 127-137). Retrieved from https://ebookcentral-proquest-com.ezproxy.csu.edu.au

Hernon, P., Dugan, R. E., & Matthews, J. R. (2014). Getting started with evaluation. Retrieved from https://ebookcentral-proquest-com.ezproxy.csu.edu.au

National Library of New Zealand. (2012). Weeding your school library collection. Retrieved from https://natlib.govt.nz/schools/school-libraries/collections-and-resources/weeding-your-school-library-collection

Pattee, A. S. (2013). Developing library collections for today’s young adults. Retrieved from https://ebookcentral-proquest-com.ezproxy.csu.edu.au

 

[Reflection: Module 5.1]

Collection Analytics

noniharrison on

Formally analysing the collection using quantitative data is a powerful tool is assessing the value of the collection overall and identifying the areas in need of improvement. It also provides a means for better understanding the collection and its position in meeting library users’ needs. Collection analysis and mapping also provides a means of rectifying any bias that may be present in the collection to develop a more balanced collection overall. This also assists in collection development policies. Collection analytics provide a quantitative means with which to assess the collection, plan programs, assess budgetary requirements, compare libraries, evaluate the collection against curriculum requirements, develop library policies including collection development policies, and assist in weeding and deselection (Hart, 2003).

Hart (2003) suggests collecting data through LMS reports. Specifically, Hart mentions counting the number of resources in each Dewey class, calculating the average date of publication in each class, and collecting circulation statistics to determine “where supply was meeting demand” (Hart, 2003, p. 89). This data is then collated in a spreadsheet and analysed against a specific set of questions. Graphs are created to compare interrelated aspects of the collection; for example, age of the collection with the percentage of the collection by Dewey class, and number of titles with circulation. This last element of incorporating circulation statistics clearly allows libraries to identify how the needs of the users are being met and whether resources need to be increased and updated, and whether promotion is needed to increase readership related to particular curriculum areas.

TES recommends undertaking collection mapping that includes determining the average number of books per student in the general emphasis and specific emphasis collections, once the data is collected it can then be analysed in the following ways:

  • Examine strong sections to determine if the collection is evenly distributed,
  • Consider limiting selection in strong sections,
  • To improve weak sections, consider selecting from retrospective selection bibliographies and watch for new items,
  • Compare weak sections against the curriculum to be certain materials are needed,
  • Develop strategies to promote strong sections,
  • Compare the results with circulation statistics (Tangient LLC, 2018).

The Arizona State Library (n.d.) suggests first considering the library’s purpose and mission. This is used to provide a framework or focus for the collection analysis process. The data collected will be assessed against these standards. The types of quantitative data collected includes, number of titles, age and timeliness of materials, use, and per capita measures. These are also compared with qualitative data such as observations of use to better interpret the quantitative data.

 

References

Arizona State Library. (n.d.). Collection assessment and mapping. Retrieved from https://www.azlibrary.gov/libdev/continuing-education/cdt/collection-assessment-mapping

Hart, A. (2003). Collection analysis: powerful ways to collect, analyze, and present your data. In C. Andronik (Ed.), School Library Management (5th ed.) (pp. 88-91). Worthington, Ohio: Linworth.

Tangient LLC. (2018). Collection mapping. Retrieved from http://libraryadmin.wikispaces.com/Collection+Mapping

 

[Reflection: Module 5.1]