Academic library services in Learning Management Systems (LMS): Methodologies for evaluating digital library content and system usability

Beskrivelse

Existing research estimates a rapid increase in the use of external sources of information, in particular Google, Google Scholar and Wikipedia, by library users (Carpenter, 2012; Tenopir, 2003; Tomlin, Tewell, Mullins, & Dent, 2017). Tomlin et al. (2017) estimates that 13% of the attending students has never used library resources. Carpenter (2012) finds that 30% of UK Doctoral students uses Google or Google scholar as their main source of information. In a literature review by Tenopir (2003) the same patterns are shown for high school students and college students. The study argues that the students perceive themselves as experienced users of the Internet. Biddix, Chung, and Park (2011) seek explanations to why students tend to use ressources other than the resources offered by their academic library. In a survey of 282 students show that students value efficiency over credibility. To address students, existing studies have explored how library visibility can be improved in learning management systems (LMS) as this is a digital environment already used intesively by students (Black & Blankenship, 2010; Schrecker, 2017). Schrecker (2017) finds that usage of the implemented library resources in fall semester outpaced spring semester use with 13%, after implementation of services in LMS-environments. This indicates that by implementing services in LMS-enivronments it has an impact on the electronic use of academic libraries. The abovementioned studies that have integrated electronic library resources successfully in LMS all use student perceptions as the key indicator for success. To depict a fuller picture of library services that students aren´t familiar with, further stakeholder views are needed. E.g. staff, domain-experts or management would be a valuable asset in such process since they possess specific knowledge of service utility and functionality. Hence, the aim of this study is to explore the effect of such multiple stakeholder-views when dealing with evaluation of electronic library services. Design and Approach This research has been conducted through a case study of students and librarians at the University of Southern Denmark. To attain multiple stakeholder-views two methods are introduced to evaluate respectively students’ and academic library staffs’ perceptions: the think aloud test and the heuristic expert evaluation. The think aloud test, was applied to observe behaviour from a user-centred view. This approach was used to explore potential users’ cognitive verbalisation through simulated use of the prototype. The verbal reports provided insights in both usability and user experience issues, bringing forth an important view from one stakeholder-group in particular: the users. The heuristic expert evaluation was initiated to assess functional aspects in the view of library staff. In general, the method is used in cooperation with domain and usability experts. In our case the domain experts are expected to possess knowledge about services represented in the prototype while the usability experts ensured to review general usability issues. Findings The think aloud tests shows usability issues regarding information-structure in relation to the content that causes confusion and lacking overview to the students. Moreover, all the participants indicate that the implementation of library services within LMS environments potentially will have a positive effect on students’ access to the academic library, mainly because of the decreased selection of services. The heuristic expert evaluation points out functional issues and the need of additional functions to support a more efficient use and experience of the services. Especially inconsistent use of language, information on how to pay overdue fees and additional functions for loan management was in interest of the staff. Moreover, the evaluation shows the experts perspective on which services that are relevant in LMS-environments e.g. the importance of access to offline dictionaries for exam use. Research or practical limitations or implications (as applicable) The study reveals a methodological framework when evaluating implemented digital resources in LMS. However, there is a theoretical concern on how to assess causality between the effects of implementing these resources in e-learning environments and improvement of student’s informational behaviour. Conclusions This study shows that by combining methods that aims to assess different stakeholder-groups, it is possible to provide insight into usability issues and the relevance of implemented services in LMS-environments. Furthermore, the study shows that the results from each methodology can act as complimentary elements. For instance, results attained from the think aloud test contributed with valuable aspects of what the users appreciated. In the conduct of the heuristic expert evaluation, this was applied as a valid argument for dealing with further decisions of picking, adding and removing services and elements from the LMS-environment. Moreover, the study has clarified essential aspects of stakeholder interest through the mixed conduct of evaluation methods. Originality and value of the proposal This study suggests an evaluation framework that ensures the practitioner to gain a full perspective when evaluating digital services implemented in LMS. Bibliography Biddix, J. P., Chung, C. J., & Park, H. W. (2011). Convenience or credibility? A study of college student online research behaviors. Internet and Higher Education, 14(3), 175-182. doi:10.1016/j.iheduc.2011.01.003 Black, E. L., & Blankenship, B. (2010). Linking Students to Library Resources through the Learning Management System. Journal of Library Administration, 458-467. Carpenter, J. (2012). Researchers of Tomorrow: The Research Behaviour of Generation Y Doctoral Students. Information Services and Use, Vol. 32, 3-17. Schrecker, D. L. (2017). Library orientation in blackboard: supporting online and distance learners. Library Hi Tech News, 6, 11-13. Tenopir, C. (2003). Use and Users of Electronic Library Resources: An Overview and Analysis of Recent Research Studies. Retrieved from Washington D.C.: Tomlin, N., Tewell, E., Mullins, K., & Dent, V. (2017). In Their Own Voices: An Ethnographic Perspective on Student Use of Library Information Sources. Journal of Library Administration, 631-650.

Periode	23. jul. 2019
Begivenhedstitel	Library Performance Measurement Conference: International Conference on Performance Measurement in Libraries
Begivenhedstype	Konference
Konferencenummer	13
Placering	Aberystwyth, StorbritannienVis på kort
Grad af anerkendelse	International