Assignment: Test of validity

Assignment: Test of validity

Assignment: Test of validity

Permalink:

Developing quantitative research studies that can stand the test of validity is a complicated and nuanced practice. Burkholder (2016) touched upon the experimental design validity framework of Shadish and colleagues that focuses on validity in the context of experimental designs. The framework outlines four components of validity in experimental design research; internal, external, statistical conclusion, and construct validity (Ch 2, Validity Considerations). Focusing upon internal and external validity it is important to note the threats and disparities of each in developing and conducting quantitative research.

Internal validity refers to the ability to test selected variables to an extent that allows the researcher to develop a causal inference. The ability to show that a selected variable had a change directly impacted by or caused by an independent variable absent of another plausible rival explanation (Burkholder, Ch 2, Internal Validity). Internal validity means that a study design enabled the researcher to account for all possible intervening variables related to the focus of the research. Internal validity in a way is the elimination of competing variables to explain a given phenomenon. For example, a study examining student retention and tuition affordability would have to account academic variables (among others) in order to isolate the focus upon economic driven negative retention rates. A researcher would have to control for students (or eliminate) from a study those where negative retention was the result of poor academic performance in order to show a valid relationship between retention (dependent) and affordability (independent). Failure to do so would result in the study failing to meet the standard of internal validity. Diminishing any correlation based upon non-controlled variables that could be used as plausible explanations for the dependent variable.

The plausibility of the implantation of a research study across contexts while still holding true is external validity (Burkholder, Ch.2, External Validity). This is another research design practice that requires an astute attention to detail and a great deal of perspective to eliminate. The greatest threat to external validity in my opinion is a failure to account for environmental context. Or the inability to account for environmental context when conducting a study or examining research results. For Instance, in my field of higher education, a study examining the effectiveness of retention policies in New Jersey would fail an external validity test if it did not account for the variations of state policies from state to state. This would be an example of generalization of a research study findings. If a researcher accounted for plausible factors that enabled a study to replicate across other states by utilizing qualitative data could then account for external validity of the study.

In addition to validity concerns in developing quantitative research studies, ethical concerns are also a pressing trepidation for researchers. For instance, when conducting a study that is localized to a specific geographic area, whereas the publication of the data could potentially be traced to individuals, is a paramount ethical and legal concern for researchers (Protecting Human Research Participants, NIH Office of Extramural Research). A study design must account for the anonymity and protection of any research subjects. When designing a research study it is important to measure the necessity of ethical responsibility, with the selection of study variables and testing tools. A failure to do so can leave a researcher ethically responsible for failing to meet the obligation of participant privacy or a study whose measurements could not balance with those obligations and fails to meet validity standards.

Finally, in developing a research study, one must derive ones methodology from the basis of the research question. Thus after developing a research question one can determine the most appropriate methodology to answer the research question. Not every research question is amenable to quantitative methodology. This means that in order to employ a quantitative methodology a researcher must be able to identify variables that can be numerically quantified as well as a tool that can accurately test the quantified data. Some research questions cannot be answered without a qualitative approach. Thus, the importance in allowing the research question to delineate the scientific methodology to employ for a given study and to analyze the potential limitations or flaws that may arise if choosing quantitative methods for a study that is not amenable to quantitative approaches.