EER500 Educational Research: From research question to research design

From research question to research design

McMillan (2012) argues that a research project should reflect the curiosity and personal interests of the researcher.    I am currently working in one of many schools across NSW that is in the process of investing a significant amount of money in the refurbishment of classrooms and other learning spaces to incorporate more flexible, student-centred, technology-enabled spaces.   I am interested in the impact this will have on teaching practice and attitudes, so have developed this research question, included in Appendix A:

How does working in a flexible contemporary learning space in an urban co-educational secondary school influence teachers’ pedagogical approaches?”  (Quinlan, 2016)

I am interested in discovering if I will find evidence to support Dobbin, Diaz and Brown’s (2014) suggestion that instructors’ roles can be influenced by the space they work in, or if my investigation will support Blackmore, Bateman, Loughlin, O’Mara and Aranda’s consistent finding from their literature review that “…new built spaces will not move (teachers) to innovative pedagogies” (2011, p. 38).

This paper will justify conducting qualitative research to help investigate this question;  support the choice of a case study design for the research; discuss the strengths and limitations of the research methods of participant observation and semi-structured interviewing to collect data; describe dilemmas that may be faced when considering ethical issues and principles relating to the design and conduct of the research; and briefly describe how the chosen research design and methods align with an interpretivist paradigm.   The term ‘participant’ rather than ‘subject’ will be used throughout this paper, based on the premise presented in The National Statement on Ethical Conduct in Human Research (2007) that it is more respectful.

 

Justifying the research question

The development of my research question has been informed by the suggestion in research methods literature of asking “What exactly is it about this area of interest that I want to know?” to help focus down from a broader area of interest to a specific question that can be researched (Bryman, 2012; Punch, 2005).  The research question includes descriptors that act as boundaries to define and manage the specific focus of the study, as McMillan (2012) recommends.

Research has significance and integrity if it contributes to existing practice or knowledge within a broader field (Bryman, 2012; Creswell 2012; McMillan 2012).    In Appendix A, I outline the practical importance of my research to the participants involved, through the evidence it will provide to support informed reflection on existing practice.  The small scale of my study limits how the findings could be applied to a broader audience, although the research could have practical importance to people who see parallels between their context and the focus of my study, as it  “…illuminate(s) the general by looking at the particular” (Denscombe, 2010, p. 36).

Blackmore et. al.’s literature review was conducted within the context of an overarching question: “What are the benefits, challenges and unintended consequences of new learning space design in relation to pedagogy, school organisation and student learning outcomes?” (2011, p. 39).  This broad question recognises the interplay of a number of factors that need to be taken into consideration when evaluating learning spaces.  My specific, small scale research will contribute to this broader topic, by filling a gap in the literature identified by some authors when they write about the lack of empirical research into the design of learning spaces from a pedagogical perspective (Blackmore et. al., 2011, p. v; Fisher & Newton, 2014, p 916).

 

Research design

The research question, design and methodology should all be linked, to help maintain the integrity and credibility of the research.  My proposed research is qualitative in nature because it aims to understand how and why teachers use particular pedagogical approaches in different learning environments that occur naturally in their workplace, from their perspective.  Qualitative research is most often reported through case studies (McMillan and Wergin, 2010) because “…case study research is concerned with the complexity and particular nature of the case in question” (Bryman, 2012, p. 66).  Case study design is well suited to small scale research (Denscombe, 2010), which suits the scope of my proposed research.

There are different types of cases which can be studied.   My proposed research would be considered an example of a representative case by Bryman (2012) or a typical instance case by Denscombe (2010) because my aim is to record “…the circumstances and conditions of a…commonplace situation” (Bryman, 2012, p. 70).  The typical case which would provide a suitable context for the case study has been described in the research question – a co-educational, urban secondary school with both traditional and flexible contemporary learning spaces, which would need to be defined in the research report.

The strength of using a representative case study design to investigate my research question is that it provides the framework to holistically study the influence of learning spaces on teachers’ pedagogical approaches in the natural setting of a classroom.  Denscombe (2010, p. 36) argues that focusing on one particular case allows a researcher to focus on the case as a whole, providing the opportunity to “unravel the complexities of a given situation” to gain a better understanding of how the different parts of that situation affect one another.  A case study design supports the use of a range of methods and multiple sources of data, which is useful when wanting to gain an in-depth perspective of a particular situation.

Bryman (2012) writes about ecological validity as an indicator of the credibility of a piece of research.  Ecological validity is concerned with how the findings from a piece of research can be applied to people’s every-day, natural settings.  Denscombe (2010) argues that case study is a credible research design because it takes place in a natural setting that already exists, which suggests that it would have strong ecological validity to the participants involved.

The limitations of using a case study design to investigate my research question relate to practical considerations and concerns regarding the credibility of generalising from one case to a broader context.   Small scale research usually has a relatively short timeframe.  Using a case study design for this research requires access to an urban, co-educational secondary school with both traditional and flexible contemporary learning spaces. I am reliant on permission being granted to access the site, the teachers and students, then gaining their informed consent to participate in research in order to collect data to help answer my research question.  Allowances may need to be made if this takes longer than expected.

A case study design provides the framework for a detailed and intensive analysis of a typical case that may be representative of other similar contexts, and readers of the research may find parallels between the case study and their own contexts, but it would not be ethical or credible to suggest that findings from this research are generalisable.  Bryman (2012, p. 69) writes about the impossibility of a single case being representative enough to yield findings that can be applied more generally to other cases.  McMillan and Wergin (2010) also advise not to be tempted to generalise findings beyond the context, but to consider what insight the results of this study might offer to other settings.

 

Research methods

Qualitative methods are often regarded as the best way of generating detailed data to help depth case studies (Bryman, 2012).  The precise focus of my research question suggests a small-scale study, which Bryman identifies as a further characteristic of qualitative research (2012, p.408).

Punch (2009) writes that the language used in a question should point to the type of data that needs to be collected to answer it.    Using “How” to begin my research question points to the need to collect qualitative data to build an understanding of the behaviour, values and beliefs of teachers that inform their pedagogical approaches.   I will need to closely observe and speak with teachers to collect and analyse data about their pedagogical approaches; I will need to use teachers’ perspectives as the point of orientation for my research; and I expect concepts and themes about pedagogical approaches in different learning spaces to emerge from the data I collect.  These are further features associated with qualitative research as described by Bryman (2012, p.408).

I propose using the methods of observation and interview to address my research question, which McMillan and Wergin (2010) recognise as the main methods of qualitative research.    I will use a method of participant observation to collect data, where my identity as a researcher will be recognised through informed consent of the participants, but I will not participate in the lessons I observe.  I will position myself as unobtrusively as possible within a learning space and observe behaviour that points to the pedagogical approaches that two or three teachers use with the same group of students, in the same subject, in a traditional classroom and in a flexible learning space over at least two teaching cycles (approximately four weeks).  Wilson and Randall’s recommendation from their study that more than one observation of teachers’ use of space is needed to draw conclusions about their approach to teaching (2012, p. 15) has informed this choice of timeframe.   My overt observation will involve indirect contact with students, however the school can arrange for standing parental consent in these cases (SERAP Guidelines, 2014).   Ideally, I would also like permission to use a video recorder to film the lessons, or photograph particular behaviours during the lessons, to check and depth my field notes, and for use later in an interview.  I will give a copy of the video footage to the teachers for their reference, which they may wish to use for personal or professional reflection, as the AARE (1993) suggests that one of the aims of educational research is that participants should benefit from participation in research.

One of the strengths of this method is that it draws on direct evidence rather than what people say they do (Denscombe, 2010).  The participant observation method helps to support the ecological validity of research (Bryman, 2012) because data is collected in the context of the teachers’ natural environment in a working classroom.  However, my presence as a researcher immediately changes the naturalness of that environment, which is one of the limitations of this method.  Denscombe (2010, p. 69) writes about the potential for “the observer effect” to influence participants to behave differently because of their knowledge that they are being observed in some way.  The presence of myself and the video recording equipment over a four week period should hopefully become an accepted part of the natural environment, so that it will not excessively influence behaviours.

Another limitation of this method is the issue of perception (Denscombe, 2010) or subjectivity (Bryman, 2012).  For example, my understanding of different pedagogical approaches and the behaviours that are linked to them will influence what I record.  McMillan and Wergin (2010) suggest that this issue can be addressed by taking extensive field notes, which my proposed use of video footage would support, and having field notes reviewed by participants for accuracy.

I will meet with teachers individually after the observations are completed for a semi-structured interview, which I would seek permission to record.  I will encourage the teachers to talk about the values and beliefs that underpin the pedagogical approaches captured in the images I filmed or photographed during the lessons.  Bryman (2012, p. 480) suggests that the use of photos in qualitative interviews helps to ground the researcher’s interview questions and acts as a stimulus to help the participant understand what the researcher means.   I will transcribe these interviews and return a copy of the transcript to participants as a check for accuracy.

A semi-structured interview will help me gain a deeper insight into the participants’ perspectives than observation allowed, which is one of the strengths of this method (Bryman, 2012, Denscombe, 2010; McMillan and Wergin, 2010).  It will allow teachers to give rich, detailed responses about their pedagogical beliefs, and their feelings about working in traditional classroom settings and flexible learning spaces.  I will have a guideline of questions to address specific issues that I want to investigate (eg. Can you talk me through what is happening in this photo?  I noticed that you were doing X here.  Can you tell me why you did this?  Why did you arrange the flexible learning space like this?  These are photos of you working in classroom A, and these are photos of you working in classroom B.  Do you have a strong preference for one of these classrooms?  Why?), but the flexibility to prompt, probe and check to follow up leads and clear up inconsistencies in responses, rather than stick to a standard script.  The aim of the interview is to discover rather than check (Denscombe, 2010).

Bryman (2012, p.494) recognises one of the limitations of using a one-off semi-structured interview to collect data is that it does not adequately allow for the development of rapport between the researcher and the participant, which has implications for the quality of responses.  I will have spent approximately four weeks with teachers prior to the interview during the participant observation process, so hope to have developed some rapport with them during this time.  I am a naturally open person who finds it easy to talk with people, and I possess some of the characteristics that Bryman (2012, p.473) associates with a good interviewer such as good listening skills and trying to be non-judgemental in manner and responses.  However, Denscombe’s warning that conducting a semi-structured interview is not the same as having a conversation with someone (2011, p. 173) is worth noting.

Data from field notes and interview transcripts will be coded throughout the data collection phase, then triangulated and analysed to help develop concepts and theories about how working in a flexible contemporary learning space influences teachers’ pedagogical approaches in an urban co-educational secondary school.  Bryman (2012, p. 392) and Denscombe (2011, p.89) recommend triangulating data from more than one source as a process to cross check information, which can generate greater confidence in the credibility of the concepts and theories presented in findings.

 

Ethical issues

Bryman (2012) writes about the need for researchers to be aware of ethical principles and issues relating to the design and conduct of research, so that they can make informed decisions about the implications of choices that they make throughout the research process.   The ethical issues that will need to be taken into consideration in the design and conduct of my research include ethical practice as a researcher, the ethical integrity of the research itself, and respectful treatment of participants.

Bryman (2012) writes that it is impossible to conduct completely neutral qualitative research because values intrude in all phases of the research process.  I must consciously check if and how my personal values may be influencing the research process (eg. choice of participants, labelling behaviours I observe, coding data), because the credibility of the research can be called into question if personal bias is not addressed.   It would be preferable to conduct this study in a secondary school that is not my workplace, as this would raise further ethical dilemmas in terms of objectivity and collegiality.  My existing relationship with teaching colleagues could impact on the objectivity of observations and interviews, or observing and interviewing colleagues could affect our working relationship.

Bryman (2012) recognises the link between ethical integrity and research quality.    I have an ethical responsibility to maintain high standards of responsible research, such as intellectual honesty, integrity and scholarly rigour, to defend the importance of educational research in general (AARE, 1993).   I do not foresee any dilemmas in relation to the ethics of the design and rigour of my proposed research, other than the fact that as a novice researcher I do not yet have the experience and expertise that the National Statement on Ethical Conduct in Human Research (NHRMC, 2007b) identifies as one aspect of research merit and integrity.  I have endeavoured to meet the other aspects described in the Statement by arguing the potential benefit of my proposed research in relation to its connection to existing literature and how it could be of practical importance to the field of learning spaces and teacher pedagogy; I have linked research question, design and methodology so that I can collect and analyse data that will help answer the question I have created; and my consideration of the issues in the following paragraphs will help to ensure that respect for the participants in this research are not compromised.

I have an ethical responsibility to respect and protect the rights of participants and myself throughout the research process, although the AARE Code of Ethics (1993) maintains that respect for the dignity, worth and welfare of participants takes precedence over the self-interest of researchers.  I will need to respect and uphold  the rights of participants outlined in various publications such as the right to informed, voluntary consent to participate in or withdraw from a research project, the right to be safe from physical, psychological and social harm, and the right to anonymity and confidentiality by making use of pseudonyms or demographic descriptors (Bryman, 2012; AARE Code of Ethics, 2016; Australian Code for the Responsible Conduct of Research, 2007; SERAP Guidelines, 2014; National Statement on Ethical Conduct in Human Research, 2007).

My research will take place within a school setting, so I am bound by the Department of Education’s requirement that active, written consent be granted by all participants in proposed research (SERAP Guidelines, 2014).  Bryman (2012) highlights the dilemma this requirement can cause when he writes that requiring informed written consent can prompt concern in some participants who may then decline to be involved in the research project.  He also argues that it is rarely desirable to inform participants completely about what a piece of research is about because it may influence their behaviour and tarnish the data collected.    Adding to this dilemma is that the direction of some of the qualitative research strategies that I will be using such as semi-structured interviews and participant observation may be difficult to predict and include in information given prior to written consent, which could result in some participants withdrawing their consent later.

 

Paradigms in research

A paradigm is the set of beliefs or world-view that frames how a researcher sees, studies and interprets knowledge (McKenzie and Knipe, 2006), which influences what they will research and how they will conduct their research.   Punch (2009) suggests that research can begin with an articulated paradigm within which research questions and methods are developed, however the proposed research that is the subject of this paper has been developed using a pragmatic approach, or problem-based approach as O’Donoghue (2007) terms it.  The design and methods chosen for the proposed research align with an interpretivist paradigm, however the paradigm did not explicitly drive the choice of design and methods; the question did.

Hughes (2010) describes an interpretivist paradigm as the belief that our interpretation of the world influences our behaviour on it; and Bryman (2012) and O’Donoghue (2007) both write about researchers within the interpretivist paradigm having an interest in understanding the meaning behind something.  The proposed research question “How does working in a flexible contemporary learning space in an urban co-educational secondary school influence teachers’ pedagogical approaches?” (Quinlan, 2016) sits within an interpretivist paradigm because it involves interpretation of the interplay of flexible learning spaces and teachers’ pedagogical approaches from the researcher’s and teachers’ perspectives.  The case study design of the proposed research, and the predominance of qualitative data collection and analysis methods which aim to capture the participants’ perspectives within a natural setting are also consistent with the interpretivist paradigm as described by McKenzie and Knipes (2006).

 

 

References

Australian Association for Research in Education (1993). AARE Code of Ethics.  In Research Ethics.  Retrieved from http://www.aare.edu.au/pages/aare-code-of-ethics.html

Australian National Data Service (ANDS) (n.d.). Ethics, consent and data sharing.  Retrieved from http://www.ands.org.au/guides/ethics-consent-and-data-sharing

Blackmore, J., Bateman, D., Loughlin, J., O’Mara, J. & Aranda, G. (2011, June).  Research into the connection between built learning spaces and student outcomes: Literature review.   Melbourne, Victoria: Department of Education and Early Childhood Development.  Retrieved from http://www.eduweb.vic.gov.au/edulibrary/public/publ/research/publ/blackmore_learning_spaces.pdf)

Brooks, D. C. (2011). Space matters: The impact of formal learning environments on student learning. British Journal Of Educational Technology, 42(5), 719-726. doi:10.1111/j.1467-8535.2010.01098.x

Bryman, A. (2012). Social research methods (4th ed.). Oxford, UK: OUP.

Byers, T. & Imms, W. (2014, October).  Making the space for space:  The effect of the classroom layout on teacher and student usage and perception of one-to-one technology.  ACEC2014.  DOI: 10.13140/2.1.1849.7288

Creswell, J. W. (2012).   Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston, MA: Pearson.

Denscombe, M. (2010).  The Good Research Guide: For small-scale social research projects (4th ed.). Maidenhead: McGraw-Hill Education.

Dobbin, G., Diaz, V. & Brown, M. (2014, November).  Re-imagining learning spaces: Design, technology, and assessment: A report on the ELI fall focus session. (EDUCAUSE Learning Initiative Paper No. 3). EDUCAUSE. Retrieved from http://www.educause.edu/library/resources/reimagining-learning-spacesdesign-technology-and-assessment

Fisher, K. & Newton, C. (2014).  Transforming the twenty-first-century campus to enhance the net-generation student learning experience: using evidence-based design to determine what works and why in virtual/physical teaching spaces, Higher Education Research & Development, 33(5), 903-920. DOI: 10.1080/07294360.2014.890566

Hughes, P. (2010). Paradigms, methods and knowledge. In G. MacNaughton, S. A. Rolfe, I. Siraj-Blatchford, Doing early childhood research: International perspectives on theory and practice (pp. 35-61). Crows Nest, NSW: Allen & Unwin.

MacKenzie, N. & Knipe, S. (2006). Research dilemmas: Paradigms, methods and methodology. Issues in Educational Research, 16(2), 193-205.

McMillan, J. H. (2012). Educational research : Fundamentals for the consumer (6th ed.). Boston: Pearson.

McMillan, J. H. & Wergin, J. F. (2010). Understanding and evaluating educational research (4th ed.). Upper Saddle River, NJ: Pearson/Merrill.

NHMRC (2007a).  Australian Code for the Responsible Conduct of Research. Canberra: Commonwealth of Australia.  Retrieved from http://www.nhmrc.gov.au/guidelines-publications/r39

NHMRC (2007b).  National Statement on Ethical Conduct in Human Research. Canberra: Commonwealth of Australia.  Retrieved from http://www.nhmrc.gov.au/book/national-statement-ethical-conduct-human-research

NSW Department of Education and Communities. (2014). State Education Research Applications Process (SERAP) Guidelines. In SERAP Online. Retrieved from http://www.serap.det.nsw.edu.au/

NSW Department of Education and Training. (2015, November 6).  First look at NSW classrooms of the future.  NSW Department of Education and Training Media Release.  Retrieved from http://www.dec.nsw.gov.au/about-us/news-at-det/media-releases1/first-look-at-nsw-classrooms-of-the-future

NSW Government Centre for Education Statistics and Evaluation (2015, October).  School assets and student outcomes.  NSW Department of Education.

O’Donoghue, T. (2007).  Planning Your Qualitative Research Project: An introduction to interpretivist research in education.  London: Routledge.

Punch, K. (2009).  Introduction to research methods in education methods in education.  London : Sage Publications.

Punch, K. F. (2005).  Introduction to social research: Quantitative and qualitative approaches.  (2nd ed.). London: Sage.

Wilson, G. & Randall, M. (2012).  The implementation of a new learning space: a pilot study.  Research in Learning Technology, 20(0), 1-17.  DOI 10.3402/rlt.v20i0/14431

 

 

Appendix A:  Proposed research question

Topic:  Learning space design

The NSW Department of Education and Training’s media release “First look at NSW classrooms of the future” (Nov 2015) suggests that flexible interactive learning spaces are “…being used by teachers to test new ways of engaging students” in ways that reflect 21st century learning, allowing education to adapt to the digital age.   Many schools designed using an industrial model of education are currently investing a significant amount of money to reconfigure their existing classrooms and libraries as flexible, interactive learning spaces.  How is this investment in new spaces, furniture, lighting and technology impacting on teachers’ pedagogical approaches?

Proposed research question

An appropriate small-scale research question would be:

How does working in a flexible contemporary learning space in an urban co-educational secondary school influence teachers’ pedagogical approaches?” 

Connecting research question to literature and practice

Fisher & Newton (2014) argue in the conclusion of their study for the need for more evidence-based research methodologies to be leveraged into learning environment evaluation, beyond technical issues such as lighting and acoustics. They suggest that while it is difficult to argue that the physical learning environment by itself can enhance teaching and learning, it “…can inhibit the practice of some forms of progressive and effective pedagogy” (2014, p. 916).   This research would provide evidence to contribute to this conversation.

Byers & Imms’ mixed method research (2014) concluded that the arrangement of physical learning spaces in a secondary school setting can assist teachers to better integrate the affordances of technology into their pedagogical practice.  This would be one aspect that I would expect to see evidence of in my study, however my research question will consider additional ways that teachers’ pedagogical approaches might be influenced by flexible learning spaces in a secondary school setting.

Dobbin, Diaz & Brown (2014) maintain that the design of a space sets expectations and establishes the roles of instructor and student in that space, yet Wilson and Randall (2012, p.13) found that more than half the students surveyed in their study reported no difference when comparing teacher behaviour in a traditional classroom compared to a refurbished flexible learning space in a university setting.   Blackmore, Bateman, Loughlin, O’Mara, and Aranda (2011,  p. 38) also maintain that a consistent finding from their literature review was that “…new built spaces will not move (teachers) to innovative pedagogies.”  My proposed study will be able to investigate which of these findings align with teachers’ pedagogical approaches in flexible learning spaces in a secondary school setting.

Wilson and Randall (2012, p. 15) found that the theme of teachers working in new learning spaces requiring initial and ongoing support in relation to pedagogy was found regularly in the literature they reviewed.  My proposed research will provide evidence to support informed reflection on pedagogical practice for the participants involved, and have practical importance to a broader audience who see parallels between their context and focus of my study.

References

Blackmore, J., Bateman, D., Loughlin, J., O’Mara, J. & Aranda, G. (2011, June).  Research into the connection between built learning spaces and student outcomes: Literature review.  Paper No. 22.  Melbourne, Victoria: Department of Education and Early Childhood Development.  Retrieved from http://www.eduweb.vic.gov.au/edulibrary/public/publ/research/publ/blackmore_learning_spaces.pdf)

Byers, T. & Imms, W. (2014, October).  Making the space for space:  The effect of the classroom layout on teacher and student usage and perception of one-to-one technology.  ACEC2014.  DOI: 10.13140/2.1.1849.7288

Dobbin, G., Diaz, V., & Brown, M. (2014, November).  Re-imagining learning spaces: Design, technology, and assessment: A report on the ELI fall focus session. (EDUCAUSE Learning Initiative Paper No. 3). EDUCAUSE. Retrieved from http://www.educause.edu/library/resources/reimagining-learning-spacesdesign-technology-and-assessment

Fisher, K. & Newton, C. (2014).  Transforming the twenty-first-century campus to enhance the net-generation student learning experience: Using evidence-based design to determine what works and why in virtual/physical teaching spaces. Higher Education Research & Development, 33(5), 903-920. DOI: 10.1080/07294360.2014.890566

NSW Department of Education and Training. (2015, November 6).  First look at NSW classrooms of the future.  NSW Department of Education and Training Media Release.  Retrieved from http://www.dec.nsw.gov.au/about-us/news-at-det/media-releases1/first-look-at-nsw-classrooms-of-the-future

Wilson, G. & Randall, M. (2012).  The implementation of a new learning space: a pilot study.  Research in Learning Technology, 20(0), 1-17.  DOI 10.3402/rlt.v20i0/14431