On Wednesday I got access to Google for Education’s new practice sets feature. I was interested to test out the final version of practice sets as I’d been involved in testing the alpha version of the software with my year 11 class about a year and a half ago. Whilst I was impressed with how polished the feature is, which you’d hope from a company with some of the best software engineering talent available to them, I ended up being underwhelmed by its lack of utility in the classroom. Disappointingly, after spending a bit of time exploring the features and trailing it with a few classes, I was left struggling to think of any situations in which it would be worth using.
However, let’s start with the positives as there are some. Usability of the software for students is excellent. I tried it with a year 8 class this week who are used to logging on to their google classroom and finding the starter task to do, but had never used practice sets before. Having seen for myself how intuitive the software is, I decided to see how they’d get on if (contrary to what is normally best practice) I didn’t model it at all and just told them to get on with it. Unlike experiences I’ve had with other programs which, even after multiple clear models, students struggle with using they managed practice sets without any instruction from me and I was able to focus on getting the class settled and taking the register which was excellent. This really shows how Google has the potential to leverage their software engineering and design capabilities to develop tools that reduce dead time in the classroom and increase the amount of time students spend doing “high value” tasks.
There are some positive aspects of the software for teachers too. The option to easily test out the practice set as if you’re a student helps you become more familiar with how it works. Additionally, the option to extract questions from a PDF is a useful tool although in my view it is not implemented brilliantly. As I fed back to the developers in the Alpha, the images of the PDF file it inserts into the practice set become almost unreadable if you resize them to fit the question space but they don’t seem to have done anything about this. I can’t help thinking that something could’ve been done here with optical character recognition especially given how good google lens can be. The omission of an “import from forms” feature seems to me a big oversight too since many teachers will have a large number of preexisting google forms they use with classroom.
Pedagogically speaking, I really do think it is a step backwards from the good old google form plus google classroom approach. All of the most useful things I’d thought it might be able to do for me after taking part in the Alpha and giving feedback are not in the final version. Furthermore some of the most important features for teaching and learning that are present in google forms are nowhere to be found in practice sets, narrowing their potential use cases (at least in my secondary computer science classroom) to almost nil.
There’s no obvious way to get data out of the system to conduct further analysis. This is a problem generally with classroom as even with regular assignments I need to use a third party google sheets extension to extract scores. Since that isn’t available for practice sets there exists a whole range of use cases for which I can’t use them, for example summative assessments that need to be entered into our MIS, because you can’t do anything useful with the data.
Practice sets give instant feedback on each question to the student if the question is auto marked. This can be great in some situations for example for a homework task where teacher support is not immediately available to the student. However there’s no flexibility with this which means I observed many of the weaker students in my year 8 class simply guessing each multiple choice question answer until they got it right rather than thinking deeply about the problem. For me, this is perhaps the most fundamental of the flaws I found with the software and is indicative of a fundamental lack of understanding of modern classroom practice in the development team. There’s no point in a student getting all the answers on a practice set right if they don’t understand why those are the right answers. It might make them feel good, but they haven’t learned anything.
Practice sets are being sold by Google as “a more personal path to learning”. As well as this transpiring to involve little more than checking multiple choice and short answer questions in a student facing sense, which google forms already does, insights for teachers are also supposed to be a key benefit but have offered no more insight for me than the equivalent google form would’ve done. Maybe it’s user error on my part, but I have “there are no insights for this assignment” which is not particularly useful at all.
Given that one of the few useful insights practice sets does give you is a breakdown of how many attempts it took for a student to get a question right, it’d be useful to be able to actually do something with that information. Namly, I’d like to be able to automatically create a new practice set based around the questions students in my class were weakest on. The only way to do this though is to manually duplicate the questions you want into a new practice set which is time consuming for us time poor teachers.
Another feature of practice sets is called skills hints which are, quite frankly, useless. I’ve tried a few times to get this to do something useful but to no avail. It’s supposed to provide students with useful scaffolding to support them in answering a question but it instead provides irrelevant options for resources to use. Below is a screengrab of me trying to get it to add help for a computer programming question. The only option it suggests is a resource on an A-level further mathematics topic and it must therefore have no ability to take into account the context of the question and the wider topic the practice set is based on. A simpler feature, allowing teachers to just attach their own resources from their google drive, would’ve been more useful. This has the potential to be very powerful in terms of saving teachers time but has unfortunately been executed poorly.
However, let’s start with the positives as there are some. Usability of the software for students is excellent. I tried it with a year 8 class this week who are used to logging on to their google classroom and finding the starter task to do, but had never used practice sets before. Having seen for myself how intuitive the software is, I decided to see how they’d get on if (contrary to what is normally best practice) I didn’t model it at all and just told them to get on with it. Unlike experiences I’ve had with other programs which, even after multiple clear models, students struggle with using they managed practice sets without any instruction from me and I was able to focus on getting the class settled and taking the register which was excellent. This really shows how Google has the potential to leverage their software engineering and design capabilities to develop tools that reduce dead time in the classroom and increase the amount of time students spend doing “high value” tasks.
There are some positive aspects of the software for teachers too. The option to easily test out the practice set as if you’re a student helps you become more familiar with how it works. Additionally, the option to extract questions from a PDF is a useful tool although in my view it is not implemented brilliantly. As I fed back to the developers in the Alpha, the images of the PDF file it inserts into the practice set become almost unreadable if you resize them to fit the question space but they don’t seem to have done anything about this. I can’t help thinking that something could’ve been done here with optical character recognition especially given how good google lens can be. The omission of an “import from forms” feature seems to me a big oversight too since many teachers will have a large number of preexisting google forms they use with classroom.
Pedagogically speaking, I really do think it is a step backwards from the good old google form plus google classroom approach. All of the most useful things I’d thought it might be able to do for me after taking part in the Alpha and giving feedback are not in the final version. Furthermore some of the most important features for teaching and learning that are present in google forms are nowhere to be found in practice sets, narrowing their potential use cases (at least in my secondary computer science classroom) to almost nil.
There’s no obvious way to get data out of the system to conduct further analysis. This is a problem generally with classroom as even with regular assignments I need to use a third party google sheets extension to extract scores. Since that isn’t available for practice sets there exists a whole range of use cases for which I can’t use them, for example summative assessments that need to be entered into our MIS, because you can’t do anything useful with the data.
Practice sets give instant feedback on each question to the student if the question is auto marked. This can be great in some situations for example for a homework task where teacher support is not immediately available to the student. However there’s no flexibility with this which means I observed many of the weaker students in my year 8 class simply guessing each multiple choice question answer until they got it right rather than thinking deeply about the problem. For me, this is perhaps the most fundamental of the flaws I found with the software and is indicative of a fundamental lack of understanding of modern classroom practice in the development team. There’s no point in a student getting all the answers on a practice set right if they don’t understand why those are the right answers. It might make them feel good, but they haven’t learned anything.
Practice sets are being sold by Google as “a more personal path to learning”. As well as this transpiring to involve little more than checking multiple choice and short answer questions in a student facing sense, which google forms already does, insights for teachers are also supposed to be a key benefit but have offered no more insight for me than the equivalent google form would’ve done. Maybe it’s user error on my part, but I have “there are no insights for this assignment” which is not particularly useful at all.
Given that one of the few useful insights practice sets does give you is a breakdown of how many attempts it took for a student to get a question right, it’d be useful to be able to actually do something with that information. Namly, I’d like to be able to automatically create a new practice set based around the questions students in my class were weakest on. The only way to do this though is to manually duplicate the questions you want into a new practice set which is time consuming for us time poor teachers.
It’s great to know that lots of students didn’t get question 6 right the first time, but what I really want is the option to create a new practice set with all questions like 6 for retrieval purposes.
Overall it’s disappointing and shows Google needs to better understand the needs of the profession they aim to serve with these products. Maybe hire some working classroom teachers to provide ongoing consultancy. I’d really hoped practice sets would represent a leap forward from google forms for digital assessment in GSuite but sadly the shiny new product has failed to cut the mustard when it comes to practicality for the working educator.
