Strengthening Librarian Professional Development: An Assessment of Collection Development Training Evaluation Using Item Analysis

Abstract

Statement of the Problem / Objectives. As the Network of Academic Law Librarians, Inc. (NALL) prepares for subsequent iterations of the Academic Law Librarians Certificate Program (ALLCP), ensuring the pedagogical integrity of its assessment tools is paramount. This study aims to evaluate the diagnostic quality and reliability of the ALLCP post-training quiz on Collection Development. The objective is to identify specific test items requiring revision, specifically those with poor discrimination or inadequate cognitive demand, to ensure the instrument accurately measures professional competency.

Methods. The study utilized a quantitative descriptive design, applying Classical Test Theory (CTT) to evaluate a 10-item post-training assessment. The setting was a national training certificate program for Filipino law librarians, with a subject pool of 52 participants. Data analysis involved calculating the Difficulty Index (p), Discrimination Index (D), item variance, and internal consistency using Cronbach’s Alpha. Participants were categorized into top (27%), middle (46%), and bottom (27%) groups to assess the instrument’s ability to differentiate between performance levels.

Results. The assessment yielded a Cronbach’s Alpha of 0.6766, indicating moderate internal consistency. Item analysis revealed a diverse range of psychometric performance: Items 2 and 7 were identified as problematic, exhibiting high difficulty (p < 0.36) and low discrimination (D < 0.45), suggesting structural ambiguity. Conversely, Items 3 and 5 were categorized as "Very Easy" (p > 0.86), functioning as foundational recognition tasks rather than measures of analytical depth. While some items showed excellent discrimination (D = 0.80), the findings indicate that 40% of the instrument requires strategic revision to better align with higher-order cognitive objectives.

Conclusion. The study concludes that while the ALLCP assessment provides a functional measure of learning, its diagnostic rigor is currently hampered by items that are either structurally ambiguous or cognitively simplistic. To improve the tool, "Difficult" items must be refined to eliminate trick distractors, and "Very Easy" items must be transitioned from simple recall to scenario-based application. Implementing these psychometric improvements will ensure the ALLCP remains a robust benchmark for professional excellence in law librarianship.

Originality. This research provides a rare data-driven evaluation of professional development assessments within the specific context of Philippine law librarianship. It demonstrates a practical application of item analysis, often reserved for high-stakes academic testing, in specialized professional certificate programs, offering a replicable model for other library associations to institutionalize quality assurance in their training curricula.