“This is going to be really stats heavy and so I won’t be offended if you want to leave.” – Peter Whiting
Welcome back for part four in my review of FlipCon Adelaide. If you have missed the previous articles, you can find them by clicking here. For whatever reason, I had not registered for a session after Aimee Shattock’s and I decided to drop in on Peter Whiting’s (@mr_van_w) session where would be exploring the results from an action research project which was recently peer reviewed and published (you can find it here). Statistics and research is not a flavour that everyone enjoys and it was a small group in the room, however, it was, for me, an incredibly interesting session and I got a real kick out of hearing about the methodologies and the statistical results; it reignited a desire to engage in education research. It was a good session even before Peter spoke, however, as I saw this on the wall, encouraging a growth mindset and a persistent attitude to learning.
Peter spoke about his background, that he was a scientist before entering the teaching profession and so his research was driven by a science mindset, looking at the story told by the data. He also indicated that his working environment is hostile in many ways to flipped learning as a pedagogical strategy, but that the school has moved to action research as a basis for professional development, which sounds strategically sensible, depending on what guidelines are provided for topics of research and the structure. I had some conversations around this topic during the social event which I was intrigued by and will discuss further in a later article.
The action research was driven by two focus questions, what was the impact on student engagement and student learning outcomes when flipped content is made either by their own teacher, a team teacher or an external provider. It is an interesting question as the general feedback that highly experienced flipped educators give is that creation is better than curation for flipped content. Peter spoke about the relationship that he and his team teacher have which other as being very productive and safe vis-a-vis their ability to provide open and frank feedback to each other and that this was essential to the quality of their flipped content and also to the action research project.
This also provided the first departure point from standard flipped learning discourse as Peter noted that they do not necessarily have students engaging with video content in the individual learning space and therefore refer to the flipped content as learning objects or LOs.This allows for a discussion about the flipped content without limiting the discussion to video content.
The research was structured to allow for a number of data points. Peter explained that in a typical action research project, for each query, three data points are required. To this end, the research was structured to allow for a number of data collection points, with two sets of two parallel classes being utilised (an A and B class in each of Stage Four and Stage Five science) to allow for comparisons in different learning contexts. This enabled a comparison of the effect on engagement and outcome as a result of teacher-created, team-teach created or externally created LOs. The overall sample size was fifty-five students and Peter said that he would have liked to have had a larger sample size, however, that was what he had to work with. If you are not familiar with what team teaching is and why that is a topic of potential interest for research, you can find a good overview here.
Peter then did what he promised and went into statistics-mode. The first results that we were shown were the overall results around the engagement levels in the individual space (what would traditionally be referred to as homework). These showed markedly different results between teacher-created LOs and team teacher-created LOs; 91% completion in comparison to 85%. This trend continued when examined in the same way with the data clustered by the unit of study or topic.
The above photo is not the greatest, however, the darker column is Class A and the lighter column is Class B. The results demonstrated that students engaged with the LOs much more frequently and with greater interest when they were created by the class teacher, irrespective of the topic of study. The Class A teacher developed the LOs for the second unit, whilst the Class B teacher prepared the LOs for the first and third units and you can see the interaction patterns quite clearly in the results. It is interesting to note that the subject or topic of the unit (appears) not to have had any impact on the average results and I would be curious to hear about any inferences or conclusions that were made around that.
Following on from that, bookwork results were examined, and student effort was recorded using predetermined success criteria, with the results being clustered together by alternate and classroom teacher. It was reported that there was a significant different between the two sets of results; when students’ book-work marks were clustered together according to the book-work marks from their own and the alternate teacher. Peter reported that this indicated to them that students were taking detailed notes beyond the bare minimum when the learning object being used was created by their own teacher rather than the alternate teacher. Interestingly, it was also reported that as the end of the year drew closer the disparity between the two columns (book work marks for own vs alternate teacher) lessened. I am not sure what results you could infer from this other than potentially an impact of studying for impending exams or major in-class assessment tasks/tests. I do not recall what Peter said, if anything, about this, but he noted it as interesting.
Students were asked directly about whether they had a preference for the LOs that their teacher created in comparison to an alternate teacher and it is telling that although 70% of students thought the LOs were equivalent vis-a-vis quality, that 47% preferred the LOs developed by their own teacher. Peter did acknowledge that 49% of students were neutral on that question; that they did not mind either way. I found it very interesting that such a large proportion of students indicated they did not mind either way. A question along these lines was asked during the primary discussion panel (read the article here) and Matthew Burns (@burnsmatthew) responded that he asks his students about whether they prefer flipped pedagogies or traditional pedagogies. It is a slightly different question with a different focus, however, as far as I am aware, Matt creates the vast majority of his content and he indicated a roughly 70% / 10% / 20% split between preference of flipped/blended/traditional pedagogies. I do not know if Matt has done any similar research into the impacts of third-party created flipped content/LOs.
The above graph was shown to us next and it is a very intriguing set of results. It demonstrates that although there is a preference for teacher-created LOs, that the measured summative metrics revealed no statistically significant variance in the achievement of learning outcomes. This has significance for teachers interested in flipped learning as a pedagogical strategy. Engagement in the classroom or group learning time is an important factor in classroom management and the perception of whether you are a good teacher. John Hattie (@john_hattie) has written extensively around effect sizes, and engagement has an effect size of 0.45 which is not insignificant.
One potential reason for the preference for teacher-created LOs is that students are used to you; your vocal rhythms, patterns, tonal quality, and lilts, however, it is key that we remember that the LOs are not everything. Flipped learning is about videos, primarily, but that is not the goal of flipped learning. The goal of flipped learning to reclaim time for deeper learning and engagement with higher level thinking as envisioned on the reimagined Bloom’s Taxonomy.
Retrieved from tinyurl.com/zh44r2v on 30 November 2016
Peter related that Derek Muller (@Veritasium) completed a study for a PhD, which he (Peter) summarised as can we learn stuff from videos – the short answer from Derek is no. The learning happens in the class.” He pointed out that a video provides background and foundational information, but that it does not necessarily provide a context, an application or a synthesis of the skill or concept; that is what the classroom time needs to be used and as Jon Bergmann pointed out in his keynote address earlier that morning, the biggest mistake in implementing flipped pedagogies is not using the reclaimed group space time well.
The video does not teach students how to think critically around a topic or provide them with strategies for synthesising new information or evaluating the impacts of something, that is our role as teachers, to provide the opportunity for students to take that information and apply, analyse, evaluate and create with it. It provides the opportunity for teachers to build and strengthen the relationships with students which has a sizable effect size (0.52) on student learning outcomes according to Hattie.
We moved onto discussions around the human research ethics approval (HRECs), requirements around which varies depending on the jurisdiction. Essentially though, if the research is in-house for reflection and improvement of practice, ethical approval is not strictly necessary (unless otherwise indicated in your State or Territory), though it is still a good idea. If you intend to publish or share the results externally, then it becomes necessary. Even if it was not necessary, the process of completing a HRECs application is very useful. I found that it helped me to crystallise exactly what my guiding question was and how was going to go about researching that and understanding the results. Peter also said that there is money available via grants for research assistants and that we simply need to go through the processes. This was not something I was aware of, however, it would be very useful to have someone who can collect, collate and assist in data analysis.
We were told that the most basic interpretation of action research methodology is to ask a question, enact a plan to gather data, reflect and reiterate. The complication or the challenge comes from the need to continually ask so what and where to from here when the data is collected and conclusions have been drawn at each iterative step.
The question was asked how far away from your own institution do you go before content becomes external? Is it external content if it by anyone outside of your own Stage or Faculty? Your own school? your Local Learning Community or Dioecese? That, Peter indicated, is the next step for the research.
I personally found the session with Peter to be exciting and reinvigorating. My current long-term career goal is to end up in the education research space. I feel like this will be ongoing or multiple over a period of time, action research projects where specific questions are researched and iterations made to pedagogical practice and strategy with the end goal being to share results at each step for feedback and peer review (whether this is formalised for publication or merely social peer review through trusted colleagues I do not know). I am a teacher first and a researcher second, however, I genuinely enjoyed the process of reviewing the literature, synthesising it, researching, analysing the data and then writing the thesis. I would like to take it to the next step and be able to make iterative changes to my practice and to be able to share those results with peers. That is largely why I maintain this blog and also try to maintain the formal-ish academic style of writing, so that I do not lose the ability to write in that style when ( am determined it will be when not if) I get the opportunity to dig into some research again.
Thank you as always for reading this rather long article. I know that research and statistics is not everyone’s cup of tea, but I personally really enjoyed Peter’s session. We enjoyed a long conversation around it later on over dinner and drinks, and I daresay that when I read his article that I will have further questions for him. I would like to hear your thoughts on the research described and what direction you think it could go in next and what questions you feel would be valuable for research.
0 Comments
|
Categories
All
|
RSS Feed
