Article Critique – Task Stream Assignment

Article Critique – Task Stream Assignment

Introduction Summary

The article I critiqued is called “Interactions Among Online Learners: A Quantitative Interdisciplinary Study” by Pawan Jain, Sachin Jain, and Smita Jain, 2011. The study focuses on

the amount of interaction students have with each other and with their professors to identify if

interaction levels differ among disciplines.

Research Problem

The major research problem identified in the study is that there is not enough research

concerning the matter of online interaction. The majority of the completed research is very

discipline specific and cannot be generalized. The author’s justification for conducting the study

was to remedy the lack of prior research on this increasingly important topic. The purpose is “to

fill the gap and try to understand the relationship between the interaction and differences in

discipline;” however, the authors also noted that this was “one” of the purposes of the study and

failed to mention any other purpose.

While the authors did not specifically discuss their decision to utilize a quantitative

approach in this study, it was clearly justified by their need to examine the relationship between

discipline and study interaction. The theoretical basis that is used as the basis for this study is

that increased interaction within an online course will ultimately lead to a better designed course

and better outcomes for students grades. Further, the authors attest that much of the literature is

inadequate for their study due to the specificity of the studies, disallowing them to be universally

applied.

ARTICLE CRITIQUE 3

The research question is “do the differences in the discipline area impact the overall

interaction among learners as defined by the number of learner-learner interactions?” The

hypothesis that was tested is “there is no significant relationship between the number of learner-

learner interactions and discipline area.” The only noted relationship that could be inferred

between the theory and the research question/hypotheses is that there are not any studies that

have been conducted that measure said information in a way that can be used for their purposes.

They note the importance of learner-learner interaction as a major pedagogical design; however,

they continued to note the lack of available research.

METHODS SUMMARY

Measurement

The primary concepts/variables in this study were the observations of the amount of

discussion posts by students in 39 different courses across 4 disciplines (College of Education,

College of Business, College of Arts and Sciences and the College of Health Sciences). The

identified independent variable was the 4 categories of discipline; the dependent variable was the

overall interactions per student per week. The conceptual or operational definition of these

variables was not mentioned beyond the above listed information. The authors did not address

whether the measures were valid or reliable.

Research Design

Although the authors in this study did not explicitly state the specific research design, one

can infer the researchers used a Randomized-to Groups Pretest-Posttest Design. Maturation

could be a threat to the internal validity because of the time elapsed between the pretest and

between posttest 1 and 2. Experimenter effects could also be a threat to internal validity because

ARTICLE CRITIQUE 4

the study does not assure the reader that experimenter bias has not influenced the results. Subject

effects could also be a threat to internal validity. Students may have responded differently

knowing they were part of a study. The authors did not address whether or not the children knew

they were being videotaped or used for a research study, which may have affected their behavior

and responses. Because this study only included a sample from the majority one race/ethnic

group (White), and was not specific to the SES of the participants, there is a clear threat to

external validity. The study may not be generalized appropriately to a larger population.

Sampling

The population that the authors wanted to study was interdisciplinary students enrolled in

online courses. The population from which the sample was taken does not represent all of the

appropriate people. The sample was taken from Graduate online courses at a major university in

the Rocky mountain region from four different disciplines. The sample is limited in that it

excludes all students besides graduate students. There was no mention of a specific sampling

technique that was used. I presume that it was a convenience sample. Additionally, there was

not any address of the implications of the sampling technique for the findings.

There may be issues with external validity based on the limited population of the sample.

Since they only sampled graduate courses, it cannot be generalized that the results would be

similar for other class levels. The difference in intensity for graduate courses compared to

undergraduate courses is not noted as it should be.

Data Collection

The researchers collected data based on observations of the number of discussion posts

per student per week in each course across the four disciplines. The advantage of this data

collection is that the information from the courses was archived and researchers needed only to

ARTICLE CRITIQUE 5

count the amount of discussion posts. The disadvantages to this use of data collection include

human error in counting, as well as the lack of distinction between quality and quantity of the

posts.

Data Analysis Critique

The data was organized using SPSS 15.0 statistical software. Descriptive statistics were

utilized to summarize, organize and simplify the data collected for the study Additionally, a one-

way analysis of variance was used to find the relationship between the dependent variable and

the nominal independent variable, discipline. Follow-up tests were conducted to analyze the

pairwise differences among the mean and Scheffe’s post hoc comparison test was employed for

this purpose.

Results Critique

The major findings of the study were that “the mean number of interactions per student

per week was 4.76; the standard deviation for this variable was 3.89” (Jain et al, 2011, p. 541).

The use of a one way analysis of variance was conducted and concluded that “differences in

discipline accounted for 22% of the variance of the dependent variable” (Jain et al, 2011, p.

542). The results showed that the overall interaction in Arts & Science courses was significantly

different than the overall interaction in Health Science courses. The interaction in Business

courses was significantly different than both, the interaction in Health Science courses and the

interaction in education courses. But no significant differences were found between the

interactions in Arts & Sciences courses and interaction in Business courses and interaction in

health Science courses and between interaction in education courses and the interaction in Arts &

Sciences courses. Overall, the health sciences courses had a higher interaction rate than the rest,

leading the authors to conclude that the “interactivity in an online class depends on the discipline

ARTICLE CRITIQUE 6

it belongs to” (Jain et al, 2011, p. 543). I am not highly confident in the results because of the

nature of the data collection as well as the limited sample used. Additionally, they do not

provide for a practical use.

Implications of the Findings Critique

The conclusions that the authors reached were that interactivity was different based on

the discipline of the course; these conclusions are appropriate but the author did not provide

practical implications of the study. The only implications the author noted was that additional

research would need to be conducted. The authors did not provide any practical information on

how the conclusions of this study could be used to further develop courses. In my opinion, the

significance of findings was minimal for my area of interest. Online education design and

implementation is my area of interest and I did not find this study to be particularly helpful or

beneficial.

My Contribution

Overall the study did not provide any practical information. I would suggest adjusting

the data collection method to include additional information about the particular students from

whom data was being collected (Are they full time or part time? Are they married? Do they have

children? Do they work? If yes, do they work part time or full time?). The information gathered

as to which students provided higher levels of interaction would be far more beneficial in

designing online courses. I would keep the data collection of the graduate students, but also

expand it to include undergraduate as well. I feel it is important to note the difference in

interaction between graduate level business courses and undergraduate level business courses.

ARTICLE CRITIQUE 7

Simply knowing that there is a difference among interactions does not provide any beneficial

information in developing courses and activities designed to increase interaction.

The authors noted on more than one occasion the lack of research already in this field;

however, I felt that their contribution should have been more significant. They conducted this

study, which in my opinion, still leads to a lack of necessary research.

Additionally, the actual layout of the information in their paper is not easy to navigate

through. There was key information that was left out such as the sampling technique used, as

well as any external factors that were not controllable. I would suggest changing the layout of

the article and address key information in a clear and easy to follow way. The independent and

dependent variables were not clearly addressed until the Results section of the article, when they

should have been listed at the beginning of the Methods section.

The Theoretical Framework section was well laid out and the authors did a good job of

defining any new terms that they used; however, they lacked a clear connection between the

question and the theoretical framework. They were not clear as to specifically what the

theoretical framework had to do with the questions at hand. Further, the purpose of the study

was mentioned in the theoretical framework section as opposed to the opening portion of the

study. The paper also noted in the theoretical framework that said purpose was “one of the

purposes” (2011, p. 541), but another purpose was never clearly identified. I think it is important

to lay out a clear purpose toward the beginning of the article so that the reader can easily identify

what they will get out of the article.

As mentioned above, there was a significant lack of practical implementations. The

results failed to provide information and suggestions on how to move forward. They simply

suggest that interaction is good based on their theoretical framework and that in fact there is a

ARTICLE CRITIQUE 8

difference in interaction amongst disciplines. I would change the procedures of the study to be

able to provide practical uses, such as what can be done to increase interaction for students who

have additional responsibilities such as full time jobs, spouses, and children? Information on

when the discussion posts were completed as well as the timeframe as to how long the students

were given to complete each post, would also be beneficial. I also feel that information should

be provided on the length of posts, to determine if the students in each discipline are contributing

quality comments and discussion or are they simply replying with “yes, I agree”.

In conclusion, I thought that the study lacked focus, and implications. I felt as though the

authors took the easy way out in stating that “more research was necessary”, which is generally

true of most studies. I felt as though the authors could have completed additional data collection

in relation to the particular students in each course as well as expanded the study across

undergraduate courses as well. There was no mention in the article about specifically studying

graduate students. I was disappointed in the article; I did not feel as though I gained any

information from reading it, other than a conclusion that I could have inferred on my own.

9 ARTICLE CRITIQUE

References

Jain, P., Jain, S., Jain, S., (2011). Interactions among online learners: a quantitative

interdisciplinary study. Education, 131, 3, p. 538-544.

References

Walker, D. D., Stephens, R., Roffman, R., DeMarce, J., Lozano, B., Towe, S., & Berg, B.

(2011). Randomized controlled trial of motivational enhancement therapy with

nontreatment-seeking adolescent cannabis users: A further test of the teen marijuana check-

up. Psychology of Addictive Behaviors, 25(3), 474-484. doi:10.1037/a0024076

10 ARTICLE CRITIQUE