Choosing educational software can be confusing. There are so many products and they make many claims about how they help learners. So, what kinds of things should you consider to make sure that you choose the educational software that is best for your child? Here are a few questions to ask about educational software and the Educational Technology (EdTech) companies that produce them.
Dr Mike Timms, Director of EdTech Evaluation Consulting
Good EdTech companies put research about how people learn at the core of their product design. If a company says nothing about theories of learning that’s probably a bad sign. Also, some learning strategies are shown by research to have more impact. For example, going through worked examples that explain the rationale behind steps in problem-solving is more effective than just practising the same type of question over and over. Also, consider who was involved in the development of the product. Good products will have used teachers or education experts as part of the development process.
Is there evidence the product improves learning?
Above and beyond being based on learning theory, good products are able to show proof that they improve learning. Look for products that have been tried out with a large number (hundreds) of diverse learners. Also, good products can show evidence that their use is associated with learning gains, ideally through being included in studies done by independent researchers like universities. The gold standard for educational research is randomised control trials (RCTs) in which the product is compared to other types of product or teaching. Learners are randomised to one group or the other in order to reduce bias in the outcomes. Sophisticated companies are always researching their products by trying out different features in A/B testing where they compare the old feature to the new one to see which produces more learning. So, look for products from companies that seem committed to continuous improvement.
What feedback happens during learning?
Most EdTech products employ a ‘one-size-fits-all’ approach to feedback. Therefore providing the same kind of help to every learner who makes an error. But individual learners vary in their ability to process the feedback offered. Good products will provide nuanced feedback that goes beyond simple right/wrong statements or vague encouragement like “well done!” Good feedback explains what the learner did well and, if there are mistakes, it should point out what is wrong and why. Sophisticated products may vary the level of help for different learners. Higher ability learners do better when the learning system allows them more control, whereas lower ability students prefer more guidance.
How does the product report on progress?
Good products will show clear learning goals and report on how the learning is progressing toward those targets. Poor products may just show the number of questions answered correctly but with no interpretation of what this means for learning progress. Sophisticated products may report on what steps are recommended next for the learner. Also, consider if the product allows a parent or teacher to see the report so that they can monitor the learning progress too.
Is the product accessible to the broadest range of learners?
Good products will be designed to the latest accessibility standards such as the Web Content Accessibility Guidelines (WCAG). Those guidelines make sure that users who might have special needs, such as low vision, can access the content. Good companies will provide user features such as the ability to change text size, alternative text to describe visual elements, and even things like text-to-speech. Good products will also provide the ability to take a break from the learning task and come back to the same point in the process later.
How does the product handle Student Privacy?
Many countries now provide legal safeguards for citizens and their personal information and the protection of data on children is especially important. Good companies provide clear explanations of what types of student personal information that they collect. Also how the data is used. Make sure the company only uses the data for valid purposes. Also that it is not used for purposes unrelated to learning or shared with third parties. If the company is based in the US look to see if they have taken the Student Privacy Pledge.
While it is always good to review products yourself, there are so many that it can be overwhelming. Here are a couple of non-profit sites that have reviews of EdTech products. It might be worth checking there first to see if the product you are considering is already listed.
Common Sense a US non-profit organization. They are dedicated to improving the lives of kids and families by providing trustworthy information, education, and an independent voice.
EdSurge is owned by the International Society for Technology in Education (ISTE), an American non-profit membership organization for educators. They operate as an independent news and research initiative of ISTE.
Dr Mike Timms
Dr Mike Timms is Director of EdTech Evaluation, a consulting company dedicated to improving the effectiveness of education technology through applied research. He brings together decades of executive-level business experience plus expertise in education research to help education systems be informed about how education technology works and how to make informed decisions about buying products. Mike is a recognized leader in the development of innovative ways to assess learning in digital environments. Formerly, Mike was Director of Research Development at ACER and Associate Director of the STEM Program at WestEd, a preeminent US educational research organization.
If you want to learn more about the evaluation of EdTech products please contact the author,
Mike Timms at firstname.lastname@example.org