Group+Two

Group Two (Clarice, Sarah, Pam, & Nicole)

Discussion of Wolfe & Goldman article:

I. RATIONALE FOR THE STUDY Wolfe and Goldman (2005) stated that "prior research has shown that, when learning from multiple texts, students rarely integrate information across texts" (p. 468). They further acknowledged that "integration across texts is particularly relevant for developing historical accounts" (p. 468) and asserted that their study "has the potential to inform history instruction" (p. 468). They listed two goals of this study: 1) they wanted to critically examine adolescents' processing strategies when they read conflicting accounts about a particular historical event - the Fall of Rome, and 2) they wanted to relate these processing strategies to students' efforts to construct their own writing piece after reading the conflicting accounts.

The previous research is adequately rationalized because Wolfe and Goldman (2005) examined research that addressed the text processing skills of college, high school, and adolescent (fifth grade) students, using historical text, which according to the authors "is ideal as a domain in which to examine students' processing and reasoning from multiple texts" (p. 470). The reasons are because learning about history involves gathering information from various texts and resources that relate to the same event, "it is an authentic educational task whose cognitive underpinnings can be subjected to the sort of scrutiny afforded by text-processing methods such as think-aloud protocol analysis" (p. 470) (the researchers could study their participants in a natural setting and in a natural context), and historians and adolescents work differently when processing historical texts. In the research studies, Wolfe and Goldman (2005) found: 1) that "readers who are not historians rarely construct documents models" (p. 471) as in the case of the college students, 2) that adolescents process multiple texts at a low level but are developmentally capable of being "critical readers of historical documents" (p. 472), and that 3) the results of their research findings should not be surprising given that history and social studies curricula rarely prepare students for higher levels of thought than memorization of historical facts - "other people's facts" (p. 472).

The review is organized around studies that pertain to the authors' goal of critically examining adolescents' processing strategies when they read multiple texts. The authors stated that their research "builds on VanSledright's (2002a, 2002b) recent work by using think-aloud methodology" (p. 472) as a framework for their study. Additionally, they framed their study around Palincsar and Magnusson's (2001) work with students, measuring their content learning and reasoning with science documents. Though not explicitly stated, I would argue that the researchers also drew on their knowledge of schema theory for this study. Wolfe and Goldman's (2005) research questions are: "1) How do students read and process multiple accounts of an historical event? 2) Having processed such accounts, how do they reason about them? and 3) What is the relation between processing and subsequent explanations of the historical event?" (p. 473). The review leads directly to the study because Wolfe and Goldman's (2005) questions are directly related to the research which they cited in this study.

II. CRITIQUE OF RESEARCH METHODS
 * What research method(s) is used? The main method is quantitative analysis of students' processing and reasoning from multiple texts. However, there also was a sort of mixed methods approach in that participants were directly interviewed by the researchers. The mixed methods approach can also be seen in the way the researchers codified the think aloud responses. Their methodology was vary similar to that which occurs in ethnography. Should this just say mixed methods? I don't think it's 'sort of' mixed...it just is.

||  || The researchers eliminated one student who was an outlier regarding degree of prior knowledge. ||  || Materials included two specially created "historical" documents written by fictitious researchers, a map with a timeline, and a list of facts. The two texts contained information that agreed, as well as information that disagreed.
 * Who are the participants? How were they selected? What are the demographics? Participants were 44 sixth-grade students (25 girls) who attended five different public schools in an urban school district in the southeastern U.S. The students were members of classrooms in which the teachers were involved in ongoing professional development about designing inquiry projects. All participants had completed a 6-8 week unit on the Roman Empire about 2 months before the study.
 * What are the independent variables?

The independent variables were: 1)Think-Aloud Events, which were coded into several categories: paraphrases, evaluations, comprehension problems, comprehension successes, elaborations. 2) Six sources of information: prior knowledge; information from earlier in the text (same text); information from the previous text read, the map, or the time line (previous text); or information from a previous think-aloud comment. 3) Four types of elaborative processing: self-explanations, surface text connections, irrelevant associations, and predictions. 4) Explanations for the fall of Rome: number of causes generated, complexity of reasoning, integration of causes ||  ||
 * What are the dependent measures? Mean frequency of students' answers; categories of questions generated by students; reasoning about why Rome fell

--Are the measures operationalized? For the most part, although the processes used to evaluate the questions generated by students (and the subsequent categories these were put into) seemed a little fuzzy. LET'S DISCUSS THIS IN CLASS. (HILLER)

--Are they reliable? Yes? p = less than 1. Isn't that good??

--Is there treatment fidelity? Not sure... the historical texts were generated by the researchers themselves; no picture provided of the map or timeline used; also the list of facts about the Roman Empire that the students received was not included. Yes, in that all participants received the same treatment - the same texts written specifically for their age group and reading-skills range. ||  ||
 * How are internal and external validity addressed?

Internal Though the research was conducted in five different public schools and the content was taught by five different teachers, "the teachers were covering the same district-mandated content on similar schedules" (p. 474). The researchers used a specific method of showing the kids the text one line at a time with a piece of paper that had a window cut out. The students were then asked to "think aloud" after reading the sentence. The kids were taught how to do read-aloud protocols through a sample/pre-test before they were given the fake historical documents to read. Also, all students had undergone a similar curriculum focus on the Roman Empire for 6-8 weeks prior to the study, so prior knowledge was the same. In addition, one subject was rejected because of overly extensive prior knowledge. The texts were intentionally constructed to be structurally isomorphic. The students' comments were coded using Chi's idea unit. The researchers also had other people codify the data to ensure that there was agreement regarding the categorization of student statements from the think aloud activity. There were four discrepancies, which the researchers corrected through discussion to reach a decision regarding which category the utterance belonged in.

External The authors recognized that their study had limited generalizability due to the historical accounts the students read being tailor-made for their age and reading-skill range. They also admitted that they sacrificed authenticity in order to study this specific population. (p. 494)

--Random assignment? The researchers did not elaborate on how they chose the participants. Their study simply said "Participants were 44 sixth grade students (25 girls) who were attending five different public schools" (p. 474).

--Control group intervention? No control group was specified.

--Counterbalancing of instructors? The study stated that "the five teachers were participating in professional development designed to encourage an inquiry orientation to social studies" (p. 474).

--Treatment conditions clearly described?

--Equivalent instructional time and mortality rates? "the teachers were covering the same district-mandated content on similar schedules" (p. 474).

--Task transfer and effect maintenance? ||  || Yes. This was a mixed methods study and the researchers clearly addressed the qualitative parts of their methodology, data collection, and results. HOW DID THE RESEARCHERS USE CORRELATION AND MULTIPLE REGRESSION IN THEIR ANALYSIS? (From Hiller) ||  ||
 * If qualitative methods are used, are they clearly explained?

III. CRITIQUE OF DATA ANALYSIS

How is the data treated & What analytical approaches were used: The researchers identified seven phases of data collection and used elaborate coding and scoring schemes to analyze this data. Data is comprised of audio recordings of student interviews from think-aloud sessions and phase seven interviews. The "think-alouds" were transcribed by their separate comments; some comments contained multiple "events" and were analyzed as such. Two coders were used to analyze the think-alouds in order to allow discussion for disagreements in coding; these coders were also utilized for the coding of the phase seven responses. The phase seven interviews resulted in three codes: the number of causes cited by students, the complexity of the students' reasoning process, and the integration of causes. These responses were scored by means of a points system. While the points system is clearly explained on pg. 484, a sample rubric would enhance the report.

Were they appropriate to answer the research question: Yes; upon examining the data analysis and returning to the three research questions clearly stated, they appear to be clearly linked.

If data is analyzed qualitatively, are the analytical approaches clearly described: Yes, all approaches to analyzing qualitative data are clearly described. An explanation of the coding process is offered explicitly, along with an explanation of how certain comments could be coded as more than one event. Furthermore, the mention of multiple coders enhances reliability. The incorporation of tables enhances the readers understanding of the process.

Is the data triangulated?: I believe that the data is triangulated because multiple phases (7) were observed and data collection was spread throughout. Finally, because the phases range from prior knowledge to post-interviews, it would reflect that the data is analyzed upon multiple levels and continuously examined. //I am not sure as to whether or not the teacher perspective could be used to further triangulate the data in this case??//

IV. CRITIQUE OF DISCUSSION
 * Are the results clearly rationalized? They appear to be clearly rationalized. The authors continue to connect their rationale for completing the study, reminding the reader of their initial purposes, while also elaborating on their research findings. They give specific examples of utterances from the students in their explanations. They also clearly relate their findings to the implications in classroom teaching - these explanations add to the relevance of the study.
 * Were unexpected outcomes explained? I did not observe any unexpected outcomes.
 * Were results tied to previous research? Yes, Wolfe and Goldman relate their findings to many previous research studies. For example, the say that the activities in multi-text comprehension are similar to activities in single-text processing as suggested by Chi and Cote (P. 493). They also claim to have given further support to Van Sleright's notions that some students are able to comprehend and reason with multiple history texts (p. 495).
 * Was theory generated or extended? For the most part, it seems that Wolfe and Goldman provided additional support for theory that had already been generated. For example, they discuss how prior research indicated that students read historical texts just to complete scholastic tasks - usually just reading to memorize facts. They reiterated that finding with their own study. They did add to the literature that cross-text connections become easier for students when the structure of the pieces are similar. This finding does solve a problem presented by Van Sledright's research. They also suggest the importance of explicit instruction for students in how to be historians.
 * Were limitations clearly addressed? Wolfe and Goldman mention that some people may say that the findings were skewed because the historical texts were not complex. They explain that this lack of complexity was specifically designed so that all students could access the texts.
 * Was future research suggested based on findings of the study? Yes, Wolfe and Goldman suggest that future studies "involve more systematic exploration of the processing and reasoning that can be promoted by systematic design of both tasks and texts in ways that scaffold and promote not only corroboration but sourcing and contextualization, albeit in developmentally appropriate ways" (p. 497). They also suggest that these studies examine students' thought processes in conjunction with their epistemological notions of history.
 * How would you assess the significance of the research contribution? The article is interesting and provides further support for many other theories. It also provides pedagogical applications for the work. Even though they didn't discover any groundbreaking theories, I do think the article was significant for the two aforementioned reasons.

V. ADDITIONAL OBSERVATIONS (QUESTIONS/CONFUSIONS??) This article connects well to the Coiro article for the week. It seems that the ability to read critically and synthesize would not only behoove students in the history classroom but would also provide a path for the improvement of students' online literacy. The skills seem inherently connected. My question for Coiro centered around classroom strategies for online literacy in schools where there may not be a plethora of technology, perhaps the appropriate strategies could start with this research.