Smagorinsky, Peter. “The Method Section as Conceptual Epicenter in Constructing Social Science Reports.” Written Communication 25.3 (2008): 389 – 411. Print.
In this article, the author argues that Method sections in social science research reports, particularly those that employ qualitative methods, often lack sufficient detail to make any results that follow from the analytic method trustworthy. The author provides a brief review of the evolution of the Method section from the 1960s to the present, makes a case for a more robust reporting of research method, and then outlines one way to achieve the end of providing a detailed, specific account of research methods that enable readers to understand unambiguously the means by which data are rendered into results. This consideration includes attention to the reporting of data collection, data reduction, data analysis, and the context of the investigation to make it clear why an illustrative presentation of data supports the claim that it offers.
- In this article S. is arguing that a careful attention to the methods section of any article submitted for publication would go a long way in increasing article acceptance rate.
- S. notes that in the era before the “pluralism of methods” that exploded in the 1980s the methods section of most articles was very straightforward. Aside from explaining the differences between the different groups under consideration, the kinds of data, the control variables, and the statistical tests, there was little explaining to do (391-2). In other words, because the article looked a lot like other hypothesized experiment-based science articles the method passed. This all changed when folks in comp/rhet began to borrow methods from outside the “experimental” paradigm. So, researchers had to make some new moves: first, they needed to clearly dexplain their new method; second, they needed to absolve themselves of any notions of objectivity. . . researcher data was as constructed as language itself. Third, the contextual nature of research came to the fore as the rhetorical elements of any particular investigation had to be accounted for. All these methodological disclosures still had to occur in the space of a 20-30 page journal article. This necessitated some choices by authors on what to include and what not to include (392-3).
- As S. notes, “The Method section, then, has evolved to the point where, in order for results to be credible, the methods of collection, reduction, and analysis need to be highly explicit” (392). As such, he’ll discuss what constitutes a good Method section and highlight why it matters.
- First, S. recommends explicit detail when writing the Method section. Don’t merely state that categories were used, name them and detail how they changed/moved across the research study. Be EXPLICIT.
- S. notes that he isn’t necessarily seeking a Method section that has replicability as its goal; rather, he sees the value – especially considering the “social turn” in literacy studies – of particularity as a research goal; however, a detailed explanation of the method (RAD or not) is essential in understanding how the data moved the author to particular claims (394).
- DATA COLLECTION: S. notes that the description for data collection is usually the most straightforward part of the Method section. Yet, it is important to be specific, provide linkages to previous interview work, and be explicit about the social pressures of data collection. S. provides an example where recounts the long process of coming to a particular research method by trying out, dismissing, and integrating various forms of protocol analysis. He provides this example as a way to describe data collection in more nuanced and robust ways. As he notes, “My approach is to err on the side of abundant detail and allow the editor to decide where to draw the line” (397).
- DATA REDUCTION: Data reduction is the process of moving through raw data to identify trends, specific lines of further inquiry, codes, and themes. As S. notes, most researchers recognize this in their research; however, they don’t detail the process of how they managed their data reduction. By providing a detailed account of data reduction, the researcher can avoid making the mistake of finding in the data they conclusions they seek rather than the conclusions that the data yields (397). In addition to recognizing and detailing how data is reduced to themes and trends, it is important for researchers to recognize the discrepancies in their data. . . no research project is airtight and these anomalies might provide new lines of flight or new research trajectories for the researcher or researchers that follow. In other words, those discrepancies might well be the serendipitous research event that makes something particularly worth-while.
- CODING DATA: S. sees coding as the manifestation of theory. Said differently, “coding makes evident the theoretical approach used to analyze the data by applying code names to segments of text” (399). To elaborate, S. notes that coding makes a researchers theoretical orientation explicit; further, by establishing particular principles within a theory as codes for division and categorization, a researcher demonstrates the relationships between different principles in a theory through data. . . coding is data made theoretical or theory made dataretical. Codes establish the authors hermeneutic and theoretical stance. . . they aren’t static and shouldn’t be.
- S. himself codes using a Vygotskian A-T approach because he buys the codes/principles of human action/cognition that are articulated by that theoretical paradigm.
- Collaborative coding and reliability: typically the way to ensure reliability is to train a second coder and have him/her code 15% of the material to an agreement level of at least 80% (401). This isn’t the way that S. works anymore though. . . now he collaboratively develops codes with another researcher in order to reach collaborative coding principles (not independent corroboration of codes developed by a singular researcher) (401). S. recognizes that it is important to disclose the relationship between collaborative coders in order to ensure full disclosure and mutual benefit.
- THE CONTEXT: S. writes a context section that is differentiated from the Method section. S. recognizes that working in A-T approaches is infinitely complex because every researched subject/scene has a whole lot of culture and a whole lot of history (404). Ultimately, context is too large to be reigned in and S. recommends that researchers negotiate the length and extent of the context section in relation to the journal wherein it is being submitted (404-5).
- A very important element in the success of a researched piece is what S. calls the “epicentric” role of the Method section. In other words, the theory and method that frame a study must find their way throughout the article – through the theory, framework, method, context, results, and discussuion (405). As he notes, “In my experience, studies work best when an author poses a limited set of answerable questions and then aligns the paper around them, making sure that they are theorized, that the method produces the data that serve as evidence for the claims, that the results are presented in clear relation to the questions, and that the discussion follows from the analysis” (405). To achieve this, S. recommends writing the methods section first (or at least very early on in the piece). Specifically, this usually takes the form of outlining the coding system as this is where the theoretical precepts instantiate themselves in data-form (406-7). As he notes, “If my codes reflect a sociocultural orientation to the data, then I need to frame the study from this theoretical perspective, and the same goes for information processing theorists, postcolonialists, phenomenologists, and everyone else” (406). This means that when a researcher borrows a coding system they might set themselves up for some troubles as the system wasn’t developed for the data it is being used to classify and categorize.
- S. notes that a constant referral to the coding is essential: “Referring back to the analytic method while reporting results ought to be, I believe, a standard move in published research” (407).
- S. sees the Method section as critically important for two reasons: first, the researcher can use the Method section as an organizational heuristic that integrates theoretic perspective, framework building, results reporting, and suggestions/interpretations/finding. Second, the Method section can create an environment where in the reader develops a trust of the findings of a study. Sound Method = Sound Arguments. (408)