Although your actual data collection takes place months after you decide on your research topic and methodology, it is a vital consideration from the very first steps in developing the basic plan for your study.
When you get to the point of completing your statistical or qualitative analysis, the data you’ve collected need to align with your topic and methodology. So, planning for a data collection strategy that aligns with your study and is also practical and doable is quite important. Here are some tips to help you through this planning for your dissertation or thesis.
Consider Your Population
You need to pick participants based on how they do or do not mesh with your target population. If your population of interest is high school math teachers working in urban schools, you will need to plan ahead to ensure that your sample is selected accordingly. If you sample from a group of middle school teachers working in rural Alabama, then you’re not going to end up with data that speak to your research questions related to high school math teachers working in urban areas. This is a problem that will cause your results chapter to be rejected due to misalignment, and it can be very stressful and time-consuming (possibly even expensive) to recruit your sample all over again because you didn’t plan ahead mindfully.
Think About Challenges for Vulnerable Populations
Gaining access to your participants requires going through the institutional review board (IRB) application process first, and so it is wise to consider any possible barriers to obtaining IRB approval when you first plan for data collection. Without IRB approval, you will not be permitted to even post flyers or send emails to solicit people to participate in your study. Planning ahead accordingly therefore makes great sense, as there are certain groups of people who are considered vulnerable and in need of greater levels of protection when included in research.
It is possible to gain IRB permission to recruit from vulnerable populations for research, but it is often discouraged during dissertation or thesis research because of how difficult it is to demonstrate direct benefit to the population. Dissertation consulting with experts in research ethics is definitely worthwhile if considering recruiting participants who might be considered vulnerable. Let’s go over some of these vulnerable populations:
Incarcerated individuals. Although the stereotyped portrait of the prisoner is anything but vulnerable, in truth, individuals who are incarcerated are incredibly vulnerable in the research context. Vulnerability in research participants derives from any obstacles to providing informed consent for truly voluntary research participation. Living in incarceration may limit these individuals’ experience of voluntariness of participation, making it that much more difficult for researchers to gain access to them as participants.
There are additional requirements researchers must meet to demonstrate ethicality of research using prisoners as participants, including the presence of at least one prisoner on the actual IRB. Research including prisoners is clearly very important, but it is typically recommended that you wait until you are a bit more seasoned as a researcher before moving into this field of research.
Individuals with intellectual or developmental disabilities (IDD). Individuals with IDD also experience vulnerabilities as research participants because voluntariness of participation can be compromised due to deficits in cognitive abilities and/or communication skills (see Bigby, Frawley, & Ramcharan, 2014). These deficits might adversely impact the ability of a person with IDD to fully comprehend the informed consent process, and/or to communicate a decision to participate or not in a clear manner.
Individuals with IDD also often live within social contexts that deprive them of autonomy on a routine basis, and which emphasize compliance and obedience over self-assertion and choice. This is not exclusively so, especially as self-determination becomes more prevalent in this population, but the social histories and interpersonal “training” these individuals have experienced might incline them to cooperate even when they are uncomfortable.
Because of these social and disability related factors, researchers must demonstrate to IRBs that they have taken extra measures to safeguard individuals with IDD around informed consent and voluntariness of participation (see McDonald, Conroy, Olick, & Panel, 2017). You should not necessarily be deterred from planning dissertation or thesis research involving persons with IDD, and there are many researchers who call for increased participation in research by this population as a means of empowerment and inclusion in the activities of adulthood to which all are entitled. However, if you choose to work with this population, it will be wise to develop informed consent materials that are accessible (i.e., using pictures, easily understandable language) and that emphasize the right of the individual to decline to participate at any time.
Minors. Participants who are under 18 years of age at the time of data collection can also be a challenge to recruit for purposes of research, because their young age makes them vulnerable in terms of comprehension of informed consent materials and lack of true capacity for consent. In general, IRBs will be tougher with approval with children of younger ages, and even when applications are approved for research involving minors, parent or guardian consent is also required.
As minors are not considered legally capable of providing true informed consent, these participants are required to furnish what is referred to as assent, which communicates their agreement to participate. Again, however, parent or guardian consent must also be obtained, and at the thesis or dissertation level, research that involves minors is often discouraged or even denied.
Consider IRB Challenges Related to Sensitive Topics
The potential for IRB challenges that hold up or even block your approval to recruit participants and collect data can also extend to participants who are at heightened risk of harm due to participation. The “Beneficence” section of the Belmont Report requires that researchers avoid harm to participants to the greatest extent possible. This might not be related to any inherent vulnerability in a specific population, but rather to the nature of the inquiry as experienced by participants. For example, psychological harm is a big one to avoid or minimize to the extent possible, and if you are not able to establish safeguards that adequately protect your participants from such harm, then you might end up with a research plan (and full three-chapter proposal!) that has to be revised due to lack of aligned data.
As one example, imagine that you are proposing to conduct qualitative analysis of interviews with an overall aim of understanding how the experience of sexual assault influenced survivors’ subsequent feelings of trust for others in their social networks. You might have great plans for your findings, and propose that the implications of your study will benefit survivors broadly, but you need to also consider the potential psychological effects of participating in interviews on sexual assault survivors. Of course, understanding the perspectives of sexual assault survivors is extremely important, and yet researchers must ensure that they can offer appropriate referrals to supportive resources (i.e., counseling), and adequate protections around anonymity and confidentiality to fully safeguard participants who disclose painful or sensitive information.
As with vulnerable populations discussed previously, these barriers do not necessarily prevent research on sensitive topics, but gaining IRB approval can be extremely difficult and at some universities impossible for graduate level researchers. Working with a dissertation consultant is often very helpful in avoiding any such “dead ends” related to data collection.
Another sensitive issue to consider early on when planning ahead for data collection is conflict of interest or power differential between you as the researcher and your future participants. For example, you might work as a director of a social services agency and have an interest in factors that impact job satisfaction and organizational commitment in such agencies. Given your ready access to employees in your own agency, you might think that recruiting from your own employees would be a great idea. It would certainly be easier, but your power in the organization would create vulnerability for your participants.
Although you might imagine that you are a great leader who is fair and impartial, your employees might feel pressured to participate out of fear of retaliation. Similarly, those who participate might feel pressured to respond to your interviews or surveys in ways that are less than honest, out of concern for job security or even favor in the organization. This would clearly create issues of validity in your findings. Understanding these possible—and likely—barriers to IRB approval for such potential participants is definitely advised at the earliest stages of planning your study and data collection methods. It will truly save you time and heartache after putting in all that hard work to research and write your proposal.
Consider Your Recruitment Options
Once your proposal is approved and the IRB gives you the green light to collect your data, you’ll want to be sure that you can actually recruit the right people as participants for your study. How are you going to gain access to these participants for your study? Because obtaining data from your target population is essential to moving your study forward successfully through the analysis and results stage, planning around recruitment logistics very early on is definitely important.
If you are planning a qualitative study, you will need comparatively few participants, often between 10 and 20 at the graduate level, but they will need to meet highly specific criteria to qualify for inclusion. Qualitative research is conducted to gain extremely rich and in-depth information about very specific processes or experiences, gained by exploring the points of view of individuals whose particular shared backgrounds or characteristics uniquely position them to shed light on these. In other words, only a few people need to participate in qualitative interviews because of the depth of the data that result, but the people who are included in the sample must be able to speak meaningfully and in depth about the issues addressed in your research questions.
This is a central issue to consider when planning qualitative research: can you locate a sufficient number of people with those specific attributes and experiences to fulfill the requirements of purposive sampling? For example, let’s say that you are really interested in how astronauts perceive the Earth after returning from space. This would be quite interesting, I’m sure, but are you confident that you will be able to access 10 or so astronauts who have been to space and back, and convince them to engage in interviews with you? This might be an obstacle, and it would be a shame to write up a full introduction, literature review, and methods chapter only to find that you cannot locate enough participants to achieve data saturation.
For a study following a quantitative method, to ensure adequacy of your statistical analysis you will of course need many more participants in your sample to meet sample size requirements determined through power analysis. If your study has a local or regional significance, you’ll need to think about whether your local recruitment options will be sufficient.
For example, let’s say that you are imagining a quantitative study in which you’ll investigate the relationship between salary, years of education, years of professional experience, and retention in cardiac nurses working in rural Oregon. Because of the specificity in nursing specialty and the lower numbers of nurses overall who work in rural areas, you might have good reason to wonder whether the population of cardiac nurses in rural Oregon would be sufficient to support a quantitative study.
Another recruitment issue to consider is whether you feel comfortable with and capable of recruiting participants via online venues such as social media. Gathering a large sample together, especially if you are attempting to recruit across a broad range of localities or settings, often requires use of online media to distribute your recruitment materials and collect your data (i.e., SurveyMonkey). Understanding how these logistics come into play down the road in your study will help you to select a data collection plan most mindfully in your early stages.
Evaluate Possibilities for Site Permission
Another consideration that might not leap to mind when planning out the basic topic and method of your study is the process of site permission. This is often a requirement when recruiting from specific organizational settings, such as schools, medical facilities, specific businesses, etc., and it can have implications for how you set up your problem, purpose, and research questions early on. Again, the importance of planning ahead for feasible data collection cannot be overstated, as anyone who has had to rewrite a full dissertation proposal due to lack of data access well knows. This is painful and stressful, and yet it can certainly be avoided.
Let’s imagine that you plan to conduct a quantitative study in management and leadership, and that you are really interested in investigating the impact of a participatory leadership intervention on subordinate employee job satisfaction and intentions to remain in their positions. In setting up this study, a central requirement is that you will have access to participants who work in subordinate employee positions in organizations that have implemented a participatory leadership intervention. This type of study will likely involve recruiting participants from specific organizations that have (or have not) implemented this intervention.
If you are recruiting individuals through a specific organization, you will need to obtain written permission to conduct research at that site. Site permission can often be obtained quite easily but there are definitely those cases in which permission is denied, leaving you to either find another site or change your study focus. This type of site permission denial might be more likely to occur if you are investigating something sensitive (i.e., allegations of abuse and neglect in a skilled nursing facility), and it is good practice to plan ahead to make sure you have suitable site options before planning your whole study around it.
Think About Your Sample Size
This consideration was mentioned previously in the context of recruitment, but it is very important and deserves careful consideration and planning. Sample sizes for quantitative studies are based on power analysis, while sample sizes for qualitative research studies are based on the expectation that the sample will be sufficient to arrive at data saturation. Having adequate power in a quantitative study is necessary for assessing differences or relationships between variables through your statistical analysis. In qualitative research, data saturation refers to the state at which no new findings are contributed to the dataset with the addition of more participants. It is necessary for ensuring that the findings you arrive at through your qualitative analysis fully cover the extent of the phenomena and do not leave out any important aspects. Recognizing how many individuals you will need to meet sample size requirements established through power analysis or estimations of data saturation will help you to plan ahead for recruitment that is actually doable and possible for you.
Check Into Availability and Applicability of Existing Data
In some studies, instead of recruiting participants to support data collection, you find that you can develop a gap and research questions that can be suitably addressed using an existing dataset. A wealth of data exists in areas such as student performance and other outcomes, local or global health and medical statistics, online communications in public venues, etc. Or, you might be working on a study that uses other publications as sources of data, which you might pull together for purposes of systematic review or content analysis.
Although use of existing datasets can definitely be useful and often more convenient than collecting new data, there are also important questions to think about when planning for this type of data collection. For one, will you be able to access the data? If it is not publicly available, will you be able to gain permission to access it?
Another very important consideration when planning for use of an existing dataset is whether it can answer the research questions you are developing. If you set up a whole research problem and rationale around a proposed correlational analysis of the quality of data-based decision making and student math achievement, but your existing dataset provides only a yes/no indicator of data-based decision making as present or not, then your dataset does not adequately answer your question about the quality of such decision making. Knowing if you can access your dataset, and also knowing what types of items and data it holds, are essential early steps in planning data collection for a quantitative study that uses an existing dataset.
Consider the Time You Have Available for Data Collection
Another crucial aspect of your data collection plan to think about is how much time you have to collect your data. Doctoral and master’s candidates often juggle multiple demands such as work and family along with their studies, which can definitely limit the amount of time available for data collection. In general, data collection for quantitative studies requires a lower time investment from the researcher in terms of administering surveys, but the need to obtain a certain number of responses to surveys can drag out the data collection period in many cases.
On the other hand, qualitative studies require fewer participants, making sample size requirements often easier and quicker to reach compared with quantitative studies. However, conducting the actual data collection in the form of interviews or focus groups with participants is more time-intensive compared with distributing quantitative surveys. Qualitative interviews usually last anywhere from 30 to 90 minutes, and when you multiply this by 10 to 20 participants, this clearly creates a much bigger personal investment of time on the part of the researcher.
Consider Your Skills for Data Collection and Analysis
Another major question to think honestly about when planning your data collection strategy is what kind of skills you have in areas of data collection and analysis. If you are thinking about a quantitative study, do you have the know-how to navigate online recruitment and data collection platforms? Most importantly, do you feel confident in your abilities to use statistical analysis software like SPSS to run your analysis? Once your statistical analysis has been completed, you will also need to be able to interpret the meanings of any significant or nonsignificant results. Statistical consulting is definitely a useful option if your heart is set on a quantitative study but you lack the novice or intermediate statistician competencies typically required for dissertation or thesis research.
On the other hand, conducting qualitative research will not require such adeptness with statistics, but it does require that you know how to conduct an interview such that it yields quality data. In most cases, graduate researchers use what is referred to as a semi-structured interview process, which is great for making sure that you hit all key questions consistently in each interview. But, this approach also allows for use of probe questions to encourage participants to share more detail about interesting points they bring up during the interview.
Knowing how to phrase questions in an open-ended manner, knowing how to avoid leading questions, knowing how to avoid discouraging or encouraging certain responses, and knowing how and when to probe for more information are all key skills for conducting qualitative data collection. Qualitative data analysis also frees you from the duty of conducting statistical analysis, but it does require that you have the time and capacity to wade through a mountain of text-based data to identify commonalities and differences through processes of coding and analysis. Recognizing your strengths and capacities for data collection and analysis will definitely be important for setting the course of your study from its earliest stages.
References
- Bigby, C., Frawley, P., & Ramcharan, P. (2014). Conceptualizing inclusive research with people with intellectual disability. Journal of Applied Research in Intellectual Disabilities, 27(1), 3-12. doi:10.1111/jar.12083
- McDonald, K. E., Conroy, N. E., Olick, R. S., & Panel, T. P. E. E. (2017). What’s the harm? Harms in research with adults with intellectual disability. American Journal on Intellectual and Developmental Disabilities, 122(1), 78-92. doi:10.1352/1944-7558-122.1.78
Leave a Reply