By Humintell Director Dr. David Matsumoto
We’ve heard a lot in the past year and a half about “listening to science.” Certainly, I believe that science is important to us today in our society. But when I heard what was said about science, the messages I heard were not so much about the science as it was to listening to those who said to listen to the science.
I thought we were losing a valuable teaching opportunity to educate the public about science. Because the message that has been given during this time is basically that “science” is something that’s beyond everyone’s grasp and people just need to listen to those who kind of get it.
I don’t think so.
So, I decided to put my thoughts about science in this brief video, as a scientist but also as an educator. I believe what I have to say about science could have and should have been covered through the education of a society. Yet it’s apparent to me that that kind of education has not occurred for many people, in this society and probably a lot of other societies as well.
To me, understanding science is all related to critical thinking because understanding science is about exercising critical thinking.
What is Science?
To me, science is an enterprise to generate knowledge.
So how does science generate knowledge? I believe there are three parts or layers or levels to the enterprise of science.
-
Individual studies
Probably the most basic level is the individual study. Science involves scientists conducting studies.
There are many, many different types of studies:
- Controlled experiments
- Correlational studies
- Surveys
- Case studies
- Longitudinal studies
- Double-blind controlled study
On the individual level of studies, “science” includes what may be considered specialized methods to test ideas that are called hypotheses. But these specialized methods, called methodologies, are ways to collect data- to test the hypotheses, and get a finding that can say that the hypotheses are either right or wrong.
Those ways of collecting data, those methodologies, essentially evolve around an idea of reducing uncertainties and any ambiguities that are inherent in any findings based upon those same methodologies.
Thus, understanding individual studies requires some degree of critical thinking about how one can generate findings. Again, that’s not the purpose of today’s thoughts, but that’s essentially what methodology is.
Conducting a single study well requires scientists to employ strict procedures to control as many factors as possible in order to isolate effects and eliminate rival, unintended hypotheses from explaining their findings.
Research methods, these ways of collecting data, are really exercises in critical thinking with big important labels.
Most people can understand methodology if it’s explained to them as critical thinking in terms of how we can reduce uncertainty and ambiguity in findings based upon a certain way of collecting data.
Whenever we conduct a study, we do everything we can to reduce that uncertainty or ambiguity in the findings- quality control. Increasing the quality of control of an individual study, is called internal validity.
Yet increasing that quality control comes at a cost of decreasing applicability to other samples with other methodologies in the real world, which is called external validity.
Whenever we increase internal validity to reduce ambiguity, we generally decrease external validity or how well those findings can situate with other samples.
This is just a general characteristic of any individual study.
There are scientists like me, and many, many other people who conduct studies, and generate individual findings.
-
Studies that study studies and analyze data from previous studies
The second level of the enterprise of science, involve studies that study studies, or studies that analyze data from previous studies.
If you can imagine the first level involving many individual studies, this next level takes those individual studies and studies those studies.
Oftentimes, this second layer of study involves the statistical findings from the individual studies and puts them all together in a grand kind of analysis. Those are called meta-analyses.
But there are other ways of doing this as well.
Another way would be to examine the body of evidence just on the basis of logic. This is done by examining the findings, understanding the methodologies by which those findings were generated, and then drawing some conclusions about them.
This second level involves drawing conclusions about groups of studies, and conglomerations of studies or bodies of evidence.
-
Conceptual work to extract knowledge from studies or studies of studies
The third layer of the enterprise of science is more conceptual.
The third layer involves extracting knowledge. Scientists examine the body of literature that could involve individual studies, as well as the second level study of studies and extract knowledge from those bodies of evidence.
At that point, we can get to some generalizable greater knowledge. No matter how important an individual study is, it generally cannot tell us about extracted knowledge that’s applicable to a lot of people in general.
When we can extract that knowledge from a body or bodies of evidence, from studies and or studies from studies, we can generate theories and models. These theories and models should then generate more hypotheses which should be tested in more individual studies, which goes back down to the beginning.
Thus, the enterprise of science actually is a circle.
This cycle is where theories and knowledge that’s extracted at the top feed and produce more hypotheses. These hypotheses feed back to more individual studies and that creates a body of evidence or bodies of evidence. Those bodies of evidence rise up again through studies of studies, where we scientists extract knowledge, etc, etc, etc.
This entire process can be referred to in many different ways.
This chart below is known as a knowledge pyramid. You can see the bottom of the chart refers to data. To me, the facts, these individual findings that are created by individual studies are represented by data at the bottom of this pyramid. That’s the lowest rung on the pyramid. It’s a bunch of facts.
Any one individual study can generate many facts and thus you’ve got many studies generating many facts.
The next level of this knowledge pyramid involves the conglomeration of those facts from study. It really corresponds to my level two: studies of studies. This level takes those facts on the bottom (even disparate, contradictory facts) and find ways to make sense of them. This produces some level of information.
The third level is where scientists can extract knowledge from those studies of studies. In this pyramid that is referred to as knowledge.
The Messiness of Science
Now one thing to remember about all of this, is what I call the messiness of science.
The findings from individual studies are rarely clean in any one study. Then across studies, there’s always contradictory and confusing findings about any one single domain of research.
This is especially true when those studies regard humans and human behaviors.
I’ve been studying humans and human behaviors for over 40 years and I know how messy it can be in one study, and then especially across studies. It’s likely true in all other fields of study, but I’m not an expert in those other areas. So I can’t, I won’t comment on that. But at least in psychology, with regard to human behavior, there’s a lot of messiness in any one study.
Thus, when people try to convince me about something, I’m not convinced about it by hearing about a study or two that supports that claim. I’m more interested in hearing about the conglomeration of evidence; that body or bodies of evidence both pro and con, both for and against that claim.
On the level of individual studies, I believe all findings, all findings from all studies are true given whatever methodology given whatever ways of collecting data, were used to generate those findings.
However, in order to believe larger claims, you need to understand the findings from many studies, both for and against the claim. And understand the methods or the ways in which those findings were generated. Because that’s how we can understand that information. That’s how we have knowledge.
Also, when people report about studies, or studies of studies, anything can be cherry picked.
Thus, for me, one better way to think about a claim or knowledge is understanding all of the studies or all of the findings, for and against something, and then making a decision about that. Not just hearing one side and not the other.
Where is the Wisdom?
I want to turn to this topic now about wisdom and ask “where is the wisdom in all of this?” because in all of this discussion about science, we’re missing what the top of the chart refers to.
A shared understanding of why it all matters, and what we should do about it, which we can generally call wisdom.
Although studies are of course important, and I spent a lifetime conducting studies, it seems like we may need less facts about some things and more shared understanding. This is especially true about important topics that are facing us today.
I hope that we can all search for that wisdom and get that wisdom for all of us.
Finally, one thing to consider is that “science” is a method that some or many societies have decided is the primary method of knowledge creation that they will rely on. Nothing wrong with that, I believe in science. I have done science for as I said, over 40 years and I will continue to do science.
But at the same time, I recognize that science is not the only method to generate knowledge and especially not the only method to generate wisdom, which is a shared understanding of the knowledge and knowing what to do about it.
Knowledge is different than that shared understanding and wisdom.
I know a lot of people who have had a lot of wisdom to impart in my life who were not scientists. They didn’t have advanced academic degrees, didn’t have a job in a university, didn’t publish a paper, but they had a lot of wisdom.
I think there’s a place for that in society, in addition to the science. Not to substitute or replace science or vice versa. But there’s a place for that. We can also consider wisdom generated from other sources as well, not just the science as we consider the advantages and disadvantages of the enterprise of science.
Let me close with some words from TS Eliot’s poem, Choruses from the Rock. These are two lines extracted from that.
I hope this has been interesting. For many of you, it’s a topic that I feel passionate about as I proceed with my own attempts at doing science and generating knowledge and not forgetting about wisdom.