Effective Ways of Evaluating Productivity Tools and Software for Education
by Lorraine D. Ledger, M.S.
Successful technology integration into education depends largely on whether or not teachers promote technology use in their classrooms (Ottenbreit-Leftwich & Kimmons, 2020). It has become fairly common for teachers to use computers for administrative and classroom management chores, but more teachers should also consider introducing productivity tools into their classrooms for student use.
Productivity tools, such as those for graphic design or document creation and collaboration, afford opportunities for students to be creative and express themselves, while also providing a convenient means for peer collaboration and for demonstrating their learning through production of artifacts, which they can submit for grades, share, or store for future inspiration. When adopted for use by students, digital productivity tools should do more than merely replace passive tools; they should help amplify the teaching process and transform student learning (Gallagher, 2015; Ottenbreit-Leftwich & Kimmons, 2020). An evaluation process should be undertaken to assess whether an available productivity tool meets the essential educational criteria.
Will the tool transform student learning?
Deciding which productivity tools are appropriate and useful for enhancing learning in the classroom can be challenging, and it is important for schools and colleges to devise a sensible ongoing process for evaluating these tools before implementing them. Davies and West (2018) suggested three primary areas of concern that should be addressed when evaluating any technology for educational purposes: accessibility, usefulness, and effectiveness. Accessibility should be considered first. Once legal and ethical concerns have been investigated and the tool seems compliant, the next step is to determine what features are needed to enhance classroom materials and activities. EdSurge magazine offers a comprehensive guide for vetting and procuring edtech products (Gallagher, 2015). Their EdSurge Index offers reviews of more than 2,000 tools and apps, and their procurement guide also recommended the Digital Promise pilot framework (www.digitalpromise.org) and the educational product reviews and advice found at Common Sense (www.commonsense.org).
Several federal laws exist which govern things like student privacy, records management, and disability services which must be taken into consideration when evaluating technology, including digital tools, apps, and web-based software services. For instance, Titles II and III of Section 508 of the Americans with Disabilities Act of 1990 (ADA) require protection of “individuals with disabilities from discrimination based on disabling conditions” in online public spaces, including schools and businesses (Hidalgo & Watson, 2021). A good first step in any evaluation process, then, is to check for compliance with disability-related anti-discrimination laws. If a teacher or school is considering giving students access to web-based software as a service (SaaS) in class, especially if the tool is required for completing assignments, then it is important to find out if these services are following the latest WCAG 2.0 design standards for ADA-compliant accessibility (Hidalgo & Watson, 2021).
Federico et al. (2020) recommended, “The easiest way to start with your evaluation is to look for an accessibility statement or Voluntary Product Accessibility Template (VPAT) for the digital tool or app,” citing that Flip-grid’s statement says “it features closed captions, integration with Immersive Reader (a tool that makes reading more accessible), and text with video” (p. 6).
One handy tool for checking website accessibility is WAVE (http://wave.webaim.org/), a service of Utah State University, which can be used online or downloaded for desktop. It works by typing in a URL and clicking to submit the page for analysis. The analysis only takes a few seconds and produces a report that alerts users to accessibility problems. For instance, alerts may include lack of screen reader compatibility, fonts that are smaller than 12 pt, missing alt text for images, missing code needed by assistive technology, and in-page navigation barriers. Another evaluation method to check for accessibility is to apply the POUR Model (youtu.be/vVEvOGVKCdo), which stands for Perceivable, Operable, Understandable, and Robust (Federico et al., 2020). Students who rely on assistive technologies need to be able access the productivity tools and apps, so it would be a waste of time to adopt tools that are not universally accessible by all students in a class.
In terms of usefulness and effectiveness for learning, schools and colleges must evaluate new tools to match their unique organizational structures, policies, existing infrastructure, and specific student needs. For example apps and tools for use by fully online students must be checked for things like privacy and security, paywalls, browser compatibility, and WCAG 2.0 compliance. Apps and tools that will be used on non-networked computers, such as those for supervised use in a face-to-face classroom, lab, or library, may need less scrutiny.
Large-scale productivity tool adoptions should be evaluated by experts through a centralized and well-documented methodology, which may include multi-year evaluations and pilot programs (Schlosser, 2022). At the University of Denver, for example, the IT department maintains a calendar of ongoing technology evaluation projects and their status. The calendar reflects detailed processes that take several years to complete.
The IT department handles campus procurement and vendor agreements to ensure compliance with security protocols and student data privacy law (e.g., FERPA). IT evaluates software and tools that will be widely used throughout the campus, in order to ensure legal compliance as well as compatibility with existing computer infrastructure.
Denver’s Office of Teaching and Learning (OTL) has devised its own scheme for helping faculty with software adoptions, which it maintains separately from the campus-wide IT system. Both the IT and OTL departments accept requests for software evaluation. The OTL uses a stoplight evaluation method for faculty, categorizing popular productivity tools as either a red, yellow, green, or blue light product. Red meaning the university does not recommend using them due to privacy or other concerns, and blue meaning the tools “have cleared the rigorous DU technology review process, obtained a FERPA agreement, and DU has accepted the liability for the use” (Schlosser, 2022, para. 7).
University of Denver is an example of a current trend on college campuses to compartmentalize control of technology use and capital assets, while also allowing for stakeholder involvement and for user needs to guide the adoption process. Overall, the research literature seems to indicate agreement that, although tools and apps are readily available, there must be careful consideration before using them in classrooms, whether for adult students or children. Instructional goals, unique school infrastructure and organization, costs, laws, and individual student and faculty needs must all be included in criteria for software adoption.
The primary driver of productivity tool adoption should be whether the tool will enhance and, perhaps even, transform the experience of learning for students. If not, the tool or app that’s being considered—no matter how “cool” or trendy, inexpensive, available, or popular—should be rejected in favor of another resource that helps transform learning and achieve academic objectives.
References
Davies, R. S., & West, R. E. (2018). Technology integration in schools. In R. E. West (Ed.), Foundations of learning and instructional design technology (pp. 650-694). EdTech Books. https://edtechbooks.org/lidtfoundations
Federico, A., Shaikh, K., & Wang, M. (2020). Evaluating accessibility. In T. Trust (Ed.), Teaching with digital tools and apps. EdTech Books. https://edtechbooks.org/digitaltoolsapps/evaluatingaccessibility
Gallagher, K. (2015, June 24). Before choosing edtech products, ask yourself these three questions. EdSurge. https://www.edsurge.com/news/2015-06-24-before-choosing-edtech-products-ask-yourself-these-three-questions
Hidalgo, B., & Watson, N. (2021). A guide to designing accessible e-learning. In J. E. Stefaniak, S. Conklin, B. Oyarzun, & R. M. Reese (Eds.), A practitioner’s guide to instructional de- sign in higher education. EdTech Books. https://edtechbooks.org/id_highered/a_guide_to_designing
Ottenbreit-Leftwich, A., & Kimmons, R. (2020). The K-12 educational technology handbook (1st ed.). EdTech Books. https://edtechbooks.org/k12handbook
Schlosser, L. (2022, April 11). Picking the right educational technology tool. Office of Teaching and Learning, University of Denver. https://otl.du.edu/picking-the-right-educational-technology-tool/
