Professional athletes watch slow motion video of their performances to find areas for improvement. Armed with this information they can then work on these weaknesses and improve their performance. When studying for an exam how do you objectively measure your skills acquisition and areas of weakness that need to be worked on? Practice tests can help, especially if the questions are categorized into knowledge areas so we can tell which topics candidates understand and which they need more work on.

As a trainer I am also trying to get feedback from the group on whether people understand what I am talking about. I ask them of course, using questions like:  “Does this make sense?”, “Are there any questions on this?”, but I never really know. Cultural norms vary considerably, do polite nods and no questions mean am I preaching to the choir and they know all this stuff already, or they just don’t want to ask questions?

I recently started incorporating audience response systems (clickers) into my training courses, and while no silver bullet, they do provide useful objective feedback. I introduced them so that participants on my PMI-ACP Exam Prep course could answer end of module practice exam questions and get personal reports of how they did to help their study plan.

However the benefits go further, as a trainer I can poll the group with a quick question and if everyone gets it right move right along. Like Fist of Five voting a quick confirmation allows us to move efficiently, but if there is confusion or division of opinion then we can investigate and go deeper into topics. No longer do I have to decide if blank stares mean consent or incomprehension of my accent, now I have some hard data.

It allows for some fun games too, like prizes for most right answers, fastest responders, fastest correct responders, etc. Obviously leader boards just show the top 3 or so people, it is counter productive to show the lower part of ranked lists.

Using these tools we can provide detailed individual analysis of question responses that would otherwise require invasive supervision. Not only which categories did you score the highest and lowest on, but which questions you took the longest to answer, or changed you mind on the answer to select. This meta data helps target follow up studying for participants and also provides me with some useful feedback as I teach.

I used the system live for the first time last week in Bucharest, Romania and will be using them again for my Calgary course next week.