Hello, my name is Stephen Paff. I am a data scientist and an ethnographer. The goal of this blog is to explore the integration of data science and ethnography as an exciting and innovative way to understand people, whether consumers, users, fellow employees, or anyone else.
I want to think publicly. Ideas worth having develop in
conversation, and through this blog, I hope to present my integrative vision so
that others can potentially use it to develop their own visions and in turn
help shape mine.
Please Note: Because my blog straddles two technical areas, I will split my posts based on how in-depth they go into each technical expertise. Many posts I will write for a general audience. I will write some posts, though, for data scientists discussing technical matters within that field, and other posts will focus on technical topics withn ethnography for anthropologists and other ethnographers. At the top of each post, I will provide the following disclaimers:
As a data scientist and ethnographer, I have worked on many types of research projects. In professional and business settings, I am excited by the enormous growth in both data science and ethnography but have been frustrated by how, despite recent developments that make them more similar, their respective teams seem to be growing apart and competitively against each other.
Within academia, quantitative and qualitative research methods have developed historically as distinct and competing approaches as if one has to choose which direction to take when doing research: departments or individual researchers specialize in one or the other and fight over scarce research funding. One major justification for this division has been the perception that quantitative approaches tend to be prescriptive and top-down compared with qualitative approaches which tend to be to descriptive and bottom-up. That many professional research contexts have inherited this division is unfortunate.
Recent developments in data science draw parallels with qualitative research and if anything, could be a starting point for collaborative intermingling. What has developed as “traditional” statistics taught in introductory statistics courses is generally top-down, assuming that data follows a prescribed, ideal model and asking regimented questions based on that ideal model. Within the development of machine learning been a shift towards models uniquely tailored to the data and context in question, developed and refined iteratively.[i] These trends may show signs of breaking down the top-down nature of traditional statistics work.
If there was ever a time to integrate quantitative data science and qualitative ethnographic research, it is now. In the increasingly important “data economy,” understanding users/consumers is vital to developing strategic business practices. In the business world, both socially-oriented data scientists and ethnographers are experts in understanding users/consumers, but separating them into competing groups only prevents true synthesis of their insights. Integrating the two should not just include combining the respective research teams and their projects but also encouraging researchers to develop expertise in both instead of simply specializing in one or the other. New creative energy could burst forth when we no longer treat these as distinct methodologies or specialties.
[i] Nafus, D., & Knox, H. (2018). Ethnography for a Data-Saturated World. Manchester: Manchester University Press, 11-12.
This is a follow-up on my previous article about the difference between anthropology and ethnography. In this article, I discuss recent trends within anthropology to either revitalize ethnography and/or rethink its status as the primary research methodology within the discipline.
We anthropologists should consider expanding beyond the ethnographic toolkit. That could involve redefining what it means to conduct ethnography in such a way that includes other types of practices outside of the traditional ethnographic toolkit and/or rethinking the role of ethnography as our primary methodology.
For context, ethnography has been the primary tool within the discipline for the last several decades. I would define ethnography as a methodological approach that seeks to holistically understand and express the lived experiences of those in a particular sociocultural context(s) (see this article and this paper). Ethnography conventionally entails a specific set of qualitative methodologies that help to understand and analyze these lived experiences, including participant observation, interviews, qualitative coding, and so on. Anthropologists and other ethnographers have built this set of practices because they are excellent at capturing people’s lived experiences, and I agree that they are powerful for that.
I do not, however, believe that these are the only potential ways to do that. For me, ethnography is an orientation, an approach that seeks to make sense of the social world by focusing on the lived experiences of others, not necessarily some collection of qualitative methods. Seeing ethnography as an orientation, for example, would enable ethnographers to use data science and machine learning tools within ethnographies (see this and this).
My perspective here exemplifies the first way some anthropologists have sought to expand beyond the traditional ethnographic toolkit: by redefining ethnography. For us, viewing ethnography as a specific set of qualitative research techniques pigeonholes what ethnography can be. Although these techniques are powerful and useful, their exclusive deployment within anthropology stifles what ethnography can become.
Other anthropologists will seek to expand beyond this toolkit by advocating for non-ethnographic anthropological research. For them, anthropologists should cultivate other research practices in addition to or sometimes instead of ethnography. I am passionate about applying this specifically to data science and machine learning, and Morten Axel Pedersen is a counterpart to me who in this specific area. He thinks anthropologists should move beyond ethnographic research, which could include incorporating data science and machine learning research (see his talk as an example). Similar to me, he wants to see more utilization of data science and machine learning within anthropology, but he presents this as an alternative to doing ethnographic research not as a potential part of ethnographic research like I do.
The difference between the two approaches is subtle: the first advocates for reimaging ethnography and the second for reimagining anthropology and anthropological research while potentially keeping ethnography the same. On a practical level, though, they are not that different. Not only are they not mutually exclusive: one can seek to redefine ethnography and ethnography’s hold within anthropology. But they each also have their place in seeking when encouraging the expansion of the anthropological toolkit. In some situations, the promotion of redefining ethnography beyond its traditional qualitative practices is most beneficial, and other times, advocating for non-ethnographic forms of research would be.
As an anthropologist and data scientist, I often feel caught in the middle two distinct warring factions. Anthropologists and data scientists inherited a historic debate between quantitative and qualitative methodologies in social research within modern Western societies. At its core, this debate has centered on the difference between objective, prescriptive, top-downtechniques and subjective, sitautional, flexible, descritpive bottom-up approaches.[i] In this ensuing conflict, quantative research has been demarcated into the top-down faction and qualitative research within the bottom-up faction to the detriment of understanding both properly.
In my experience on both “sides,” I have seen a tendency among anthropologists to lump all quantitative social research as proscriptive and top-down and thus miss the important subtleties within data science and other quantitative techniques. Machine learning techniques within the field are a partial shift towards bottom-up, situational and iterative quantitative analysis, and business anthropologists should explore what data scientists do as a chance to redevelop their relationship with quantitative analysis.
Shifts in Machine Learning
Shifts within machine learning algorithm development give impetus for incorporating quantitative techniques that are local and interpretive. The debate between top-down vs. bottom-up knowledge production does not need – or at least may no longer need– to divide quantitative and qualitative techniques. Machine learning algorithms “leave open the possibility of situated knowledge production, entangled with narrative,” a clear parallel to qualitative ethnographic techniques.[ii]
At the same time, this shift towards iterative and flexible machine learning techniques is not total within data science: aspects of top-down frameworks remain, in terms of personnel, objectives, habits, strategies, and evaluation criteria. But, seeds of bottom-up thinking definitely exist prominently within data science, with the potential to significantly reshape data science and possibly quantitative analysis in general.
As a discipline, data science is in a uniquely formative and adolescent period, developing into its “standard” practices. This leads to significant fluctuations as the data scientist community defines its methodology. The set of standard practices that we now typically call “traditional” or “standard” statistics, generally taught in introductory statistics courses, developed over a several decade period in the late nineteenth and early twentieth century, especially in Britain.[iii] Connected with recent computer technology, data science is in a similarly formative period right now – developing its standard techniques and ways of thinking. This formative period is a strategic time for anthropologists to encourage bottom-up quantative techniques.
Conclusion
Business
anthropologists could and should be instrumental in helping to develop and
innovatively utilize these situational and iterative machine learning
techniques. This is a strategic time for business anthropologists to do the
following:
Immerse themselves into data science and encourage and cultivate bottom-up quantative machine learning techniques within data science
Cultivate and incorporate (when applicable) situational and iterative machine learning approaches in its ethnographies
For both, anthropologists should use the strengths of ethnographic and anthropological thinking to help develop bottom-up machine learning that is grounded in flexible to specific local contexts. Each requires business anthropologists to reexplore their relationship with data science and machine learning instead of treating it as part of an opposing “methodological clan.” [iv]
[i] Nafus, D.,
& Knox, H. (2018). Ethnography for a Data-Saturated World. Manchester:
Manchester University Press, 11-12
For my first interview in the Interview Series, I interviewed Astrid Countee. She is a business anthropologist and technologist with a background in anthropology, software engineering, and data science. She currently works as a user researcher at the peer-to-peer distributed company Holo, as a research associate at The Plenary, as an arts and education nonprofit, and as a co-founder of Missing Link Studios which distributes the This Anthro Life podcast.
If the audio does not play on your computer, you can download it here:
For my second interview in the Interview Series, I interviewed Schaun Wheeler. Schaun is co-founder of Aampe, a startup that embeds an active learning system into mobile apps to turn push notifications into part of the app’s user interface. Before he co-founded Aampe, Schaun was the data science lead for the award-winning Consumer Graph intelligence product at Valassis, a U.S. ad-tech firm. And before that he founded and directed the data science team at Success Academy Charter Schools in New York City. Then before that, Schaun was one of the first people to champion the use of statistical inference to understand massive unstructured data at the United States Department of the Army. Schaun has a Ph.D. in Cultural Anthropology from the University of Connecticut.
If the audio does not play on your computer, you can download it here:
For my third interview in the Interview Series, I interviewed Priyanka Dass Saharia. Priyanka Dass Saharia is an anthropologist working in tech startups, mostly seed to early venture stage in their product development. These companies broadly fall under the rubric of tech-based initiatives that aim to accelerate positive social, environmental and governance change. To do this, she routinely employs a variety of qualitative and quantitative techniques. In addition to anthropology, she has also studied economics and sociology in both India, where she grew up, and the United Kingdom, where she currently resides.
Over our conversation, we discussed the following:
Priyanka’s experiences as an anthropologist in tech, including her current work in fintech
Strategies for socially and environmentally equitable entrepreneurial investment
Skills anthropologists might need when working in tech and suggestions for how to develop them
Recommendations for how anthropologists can cultivate their quantitative thinking
To learn more about Priyanka Dass Saharia, check these out:
This is a quick and dirty summary of my master’s practicum research project with Indicia Consulting over the summer of 2018. For anyone interested in more detail, here is a more detailed report, and here is the final report with Indicia.
Background
My practicum was the sixth stage of a several year-long research project. The California Energy Commission commissioned this larger project to understand the potential relationship between individual energy consumption and technology usage. In stages one through five, we isolated certain clusters of behavior and attitudes around new technology adoption – which Indicia called cybersensitivity – and demonstrated that cybersensitivity tended to associate with a willingness to adopt energy-saving technology like smart meters.
This led to a key question: How can one identify cybersensivity among a broader population such as a community, county, or state? Answering this question was the main goal of my practicum project.
In the past stages of the research project, the team used ethnographic research to establish criteria for whether someone was a cybersensitive based on several hours of interviews and observations about their technology usage. These interviews and observations certainly helped the research team analyze behavioral and attitudinal patterns, determine what patterns were significant, and develop those into the concept of cybersensitivity, but they are too time- and resource-intensive to perform with an entire population. One generally does not have the ability to interview everyone in a community, county, or state. I sought to address this directly in my project.
Task
Timeline
Task Name
Research Technique
Description
Task 1
June 2015-Sept 2018
General Project Tasks
Administrative (N/A)
Developed project scope and timeline, adjusting as the project unfolds
Task 2
July 2015 – July 2016
Documenting and analyzing emerging attitudes, emotions, experiences, habits, and practices around technology adoption
Survey
Conducted survey research to observe patterns of attitudes and behaviors among cybersensitives/awares.
Task 3
Sept 2016 – Dec 2016
Identifying the attributes and characteristics and psychological drivers of cybersensitives
Interviews and Participant-Observation
Conducted in-depth interviews and observations coding for psych factor, energy consumption attitudes and behaviors, and technological device purchasing/usage.
Task 4*
Sept 2016 – July 2017
Assessing cybersensitives’ valence with technology
Statistical Analysis
Tested for statistically significant differences in demographics, behaviors, and beliefs/attitudes between cyber status groups
Task 5
Aug 2017 – Dec 2018
Developing critical insights for supporting residential engagement in energy efficient behaviors
Statistical Analysis
Analyzed utility data patterns of study participants, comparing it with the general population.
Task 6
March 2018 – Aug 2018
Recommending an alternative energy efficiency potential model
Decision Tree Modeling
Constructed decision tree models to classify an individual’s cyber status
Project Goal
The overall goal for the project was to produce a scalable method to assess whether someone exhibits cybersensitivity based on data measurable across an entire population. In doing this, the project also helped address the following research needs:
Created a method to further to scale across a larger population, assessing whether cybersensitives were more willing to adopt energy saving technologies across a community, county, or state
Provided the infrastructure to determine how much promoting energy-saving campaigns targeting cybersensitives specifically would reduce energy consumption in California
Helped the California Energy Commission determine the best means to reach cybersensitives for specific energy-saving campaigns
The Project
I used machine learning modeling to create a decision-making flow to isolate cybersensitives in a population. Random forests and decision trees produced the best models for Indicia’s needs: random forests in accuracy and robustness and decision trees in human decipherability. Through them, I created a programmable yet human-comprehensible framework to determine whether an individual is cybersensitive based on behaviors and other characteristics that an organization could be easily assess within a whole population. Thus, any energy organization could easily understand, replicate, and further develop the model since it was both easy for humans to read and encodable computationally. This way organizations could both use and refine it for their purposes.
Conclusion
This is a quick overview of my master’s practicum project. For more details on what modeling I did, how I did it, what results it produced, and how it fit within the wider needs of the multi-year research project, please see my full report.
I really appreciated the opportunity it posed to get my hands dirty integrating ethnography and data science to help address a real-world problem. This summary only scratches the surface of what Indicia did with the Californian Energy Commission to encourage sustainable energy usage societally. Hopefully, though, it will inspire you to integrate ethnography and data science to address whatever complex questions you face. It certainly did for me.
Thank you to Susan Mazur-Stommen and Haley Gilbert for your help in organizing and completing the project. I would like to thank my professorial committee at the University of Memphis – Dr. Keri Brondo, Dr. Ted Maclin, Dr. Deepak Venugopal, and Dr. Katherine Hicks – for their academic support as well.
