Book Review: “Measuring Culture”

Mohr, John W., Bail, Christopher, Frye, Margaret, Lena, Jennifer, Lizardo, Omar, McDonnell, Terence, Mische, Ann, Tavory, Iddo, Wherry, Frederick. 2020. Measuring Culture. New York: Columbia University Press.

Review by Alex van Venrooij (University of Amsterdam)

Picture: Measuring Culture Book Cover (Source: Amazon.com)

In 2003, John Mohr wrote a short essay for this newsletter on what he thought other sociologists should know about cultural sociology. For Mohr, cultural sociology could potentially play a leading, paradigm-shifting role within our discipline if cultural sociologists could convincingly show mainstream sociologists how to incorporate the study of cultural meaning-making. But this would require vigilance against, in his eyes, false stereotypes about the study of cultural meaning as a purely interpretative endeavor. As his own work had shown, the hermeneutic study of meaning could also be done using advanced quantitative modeling techniques. There was no fundamental division between meaning and measurement, between positivism and interpretivism, between the two cultures of the sciences and the humanities. Yet at the same time rigorous and systematic study of culture should also not lead to crude simplifications and insensitivity to the subtleties, the ephemeral qualities, and ambiguities of the cultural dimension.
To achieve this, we needed more dialogue across the methodological and epistemological divides. “Can’t we all just get along?”, he asked. 

Measuring Culture is the result of John Mohr’s efforts at bringing people together. Emerging out of several meetings (co-)organized by Mohr, it is written by a collective of some of the leading scholars in our field, with expertise on a diverse range of methods (ethnography, interviewing, network analysis, computational social science) as well as the most sophisticated theoretical developments. Their aim was to develop a consensus on an analytical approach to the study of culture that could indeed transcend the opposition between formal and interpretative analysis. The result is an extraordinary piece of work. The authors managed to incorporate a wealth of insights into a short and highly readable book, which students can use to navigate the current state of the art in cultural sociology, and which accomplished researchers will admire for the seamless integration of various theoretical and methodological discussions. 

Although the title perhaps suggests otherwise, the book is not a methodological handbook on how to measure culture. There are no detailed discussions on scale construction, similarity measures, or any other specific measurement techniques. The discussion is pitched at a more general, more theoretical level. The consensus that the group achieved seems first and foremost a theoretical consensus on culture as a levels of analysis phenomenon. Culture is considered to be public and private, existing inside and outside of individual minds, contextualized in micro-situations as well as in larger discursive and social structures. This allows the authors to organize their discussion in separate chapters on culture as located in persons, in objects and in social relations. At each of these levels, the authors discuss the history of measurement, the current state of the art and potential issues that remain to be solved. 

Chapter 1 starts by discussing the measurement of culture in people. This involves analyzing how people think, talk and act. Even though the study of thinking had been considered as too subjective during the early days of the cultural turn and had been replaced in favor of studying “external” culture, the chapter shows how the study of thinking is currently one of the most exciting areas in cultural sociology. It highlights the work on (the relation between) declarative and nondeclarative thinking, the measurement of cognitive schemas, and other ways in which especially social and cognitive psychology has influenced cultural sociology in recent years. To circumvent the “Talk is Cheap” debate, the authors reserve the study of talk for analyzing how people imagine their futures or tell stories (irrespective of whether those lead to action or not). For the study of action, the authors look to ethnographic methods and digital trace data for more distanced but detailed analysis of action.

Chapter 2 discusses cultural meaning in objects. The problem, of course, is that meaning is not literally “in” objects, but objects can be carriers of potential meanings, whose material qualities more likely afford some rather than other interpretations. It therefore makes the case for measuring the observable qualities of objects, and interestingly points to the sociology of the senses as a potential source of inspiration. The chapter also reminds us that although measurement might stabilize or “fix” the qualities of objects, we need to be aware that material qualities, and meanings, can change over time (as when statues lose their original colors). Objects also have a life cycle of production, distribution, and reception, and we need to follow the object to understand their potential and actualized meanings.

Chapter 3 then brings us to measuring meaning in social relations, again at different levels: in face-to-face interactions, social networks, and larger fields. Although the local production of meaning in interaction was often considered to be the domain of purely qualitative methods, the chapter shows how the emergence of patterned interaction opened the door to (basic) forms of measurement, as in “ethnographic counts”, and more recently, more complex forms of analysis that analyze patterns in online social interactions. Similarly, the tools of network analysis emerged to a large extent out of the attempts of anthropologists to formalize the patterns of social interaction observed during field work. Network analysis can therefore be seen as the abstract version of ethnographic field work. Moreover, even though network analysis was oftentimes conceived as a purely social structural and anti-cultural perspective, the authors discuss the literature on how culture and networks interact, showing that we do not have to hold on to this culture-structure opposition. Finally, field analysis is described as a way to get at the more abstract relations and positions in which both networks and interactions can be embedded. Also, here, where others might hold on to the opposition between “substantialist” and “relational” thinking, and for example regression analysis and correspondence analysis, the authors again show that what was once thought of as a clear demarcation line is now a fuzzy boundary.

As is clear from the above descriptions, these chapters cover a lot of ground. The idea of Measuring Culture is no longer exclusively focused on the measurement of meaning as originally described by John Mohr, i.e., the use of formal, relational techniques to measure meaning structures (in textual material and at the meso-level of fields and institutions). This is now only one of many options that students of culture have at their disposal. But by expanding the range of methods and including qualitative methods such as ethnography or interviewing as a form of “measuring culture” (and not just as providing the raw material for quantification), it reiterates the general point that since culture is not one monolithic thing, we need different methods for different elements of culture. It would be counterproductive to privilege one style and level of analysis since this would limit our ability to understand meaning-making processes. Ethnography might be good at studying the local production of meaning, interviews are needed for the imagined futures, and topic modeling is useful for the larger discursive spaces that might limit the range of possible meanings that people can mobilize. And we need this methodological pluralism to do justice to our subject matter. 

The fourth chapter discusses three “classic” research projects that used multiple measurement strategies to “pivot” across different levels of analysis: DiMaggio’s work on the culture wars, Mische’s ethnographic and network analysis of political youth movements in Brazil and Mohr, Wagner-Pacifici and Breiger’s computational hermeneutics of national security documents. This chapter differs in style from the other chapters as it relies on interviews with the researchers and pays attention to the “context of discovery” of these projects: the false starts, the serendipitous discovery of new methods over lunch with a friend-of-friend from another department, the discussions that lead to new moves across the methodological landscape. The anecdotes are entertaining and strangely reassuring. To read that DiMaggio had, for example, also forgotten about kurtosis is a good antidote for anyone’s lingering imposter syndrome. But we can also draw more serious lessons from this chapter, as it nicely shows the gains that can come from pivoting across measurement strategies: that DiMaggio, for example, found no strong evidence for polarization when measuring culture in persons through surveys, while a topic modeling of newspaper discourses could clearly show evidence of polarization in public culture.

Given the centrality of the idea of “pivoting” to the argument of the book I expected perhaps a more systematic discussion of how to proceed in this kind of analytical strategy. The exemplary studies of chapter 4 show how this has been done in the past. But what about the future? Or in the language of the book, where do the potentialities lie and where are the current absences? An ideal typical overview of possible designs could possibly suggest ideas on how to proceed. Can we perhaps make a distinction between the more classic combination of different methods for different types of data (as in the work of Mische where ethnography and network analysis amplified each other) or the pivoting across methods for analyzing the same data (as when Mohr, Wagner-Pacifici and Breiger alternate between computational methods and traditional hermeneutic analysis to analyze the same documents). And along which kinds of dimensions can we pivot and turn? Even with only a few dimensions, the number of permutations would probably be innumerable, and quite daunting to analyze, but some concrete guidelines would have been welcome — also because currently the advice to readers remains somewhat general: read widely and outside of your own discipline, work together with others, and keep up with the latest in machine learning algorithms.

That being said, if asked by a colleague in my department who wonders about what cultural sociology is all about, and whether there is a logic to the methodological madness that indeed also characterizes my own cultural sociology group, Measuring Culture would definitely be the book I would recommend.