On Science 4: The Scientific Method

If you are like most people once upon a time you heard/learned that the scientific method includes a series of predefined, more or less rigid steps that if followed, like a recipe as shown below, will result in "science."  This view of how science is done is just way too simple.

All the images in this posting are from a fantastic web site on the process of science put together by the good folks at UC Berkeley.  You can access it at: http://undsci.berkeley.edu/article/howscienceworks_01.

To get back to the process of science...it is true that the general steps outlined above do matter and are part of what happens, but there are MANY ways that scientists work.  A more accurate representation of how science is really done is shown below:

Rather than a static set of steps, the scientific approach to asking questions, obtaining empirical evidence, analyzing those data, and developing conclusions is a dynamic process.  Most scientists start the process in the top sphere "Exploration and Discovery" where they either ask questions about new observations or see things differently through new technology, or identify a meaningful problem through curiosity and inspiration, but ideas and motivation for carrying out research can happen at any stage of the process and in any of the spheres in the figure above.

Regardless of where a scientist starts, the process of inquiry almost always includes the generation of a scientific argument.   This step is shown in the upper half of the center sphere, "Testing Ideas."  A scientific argument includes three parts - hypotheses (Null and Research Hypotheses - see the earlier posting on the Power and Strength of Science), expected observations, and actual observations.  The Research Hypothesis is the researcher's best explanation of what they thing the eventual answer will be, and it is used to generate a set of expected or predicted results.  The researcher then uses one of any number of approaches to collect empirical observations. This can be done through experimentation, observation, etc.  Once the empirical observations are in hand the researcher moves to the lower part of the central sphere and carries out an analysis of the results...that is, they almost always usually use statistical methods or other mathematical tools to see how well the observed data and the predicted data match up.  This can lead to a variety of outcomes.

The observed observations may not support the expected observations, so the expected set and the hypothesis that led to them has to be upgraded, improved, changed, or even tossed out in favor of a better hypothesis and set of expected explanations.  This lead the researcher back to the top sphere or over to the sphere labeled "Community Analysis and Feedback."  No matter what path a project follows through this overall process, the research must pass through the "Community Analysis" sphere at least once.  This is where the observations and explanations are subjected to critical peer-review.  Peer-review is a rather ruthless process where other scientists who are experts in the field look over the hypotheses, expected data, observed data, methods of analysis used to examine the data, the conclusions reached by the researcher, etc.  The basic outline of what happens during peer review is shown in the figure below:

Major benefits of subjecting the outcome of a study to peer review include: minimizing or eliminating researcher bias in the results of the paper, rejecting papers that had poor research designs, inappropriate or internally inconsistent conclusions, helping the researchers address weaknesses in writing, analysis, etc.

If the results and conclusions of a study make it through peer-review (usually with a significant amount of additional review and revision work) the study can be accepted for publication in a professional journal.  Once this happens, the conclusions can inform new research in the top sphere or inform policy or be recognized as new knowledge, etc., in the sphere labeled "Benefits and Outcomes."  This is also when the larger scientific community can access the work, assess its validity and value and, if desired, replicate the study or carry out studies similar to it that will help determine whether the conclusions of the paper hold up under additional scrutiny.

Remember that the Null Hypothesis is always the preliminary explanation that is tested.  By so doing the scientific community eliminates false explanations, and thereby moves progressively closer to the true explanation about the question of interest.

Keep in mind that the outcome of any sphere can lead to any other sphere in the chart, and this dynamic, cyclic process is a much better representation of how science is actually done than the over-simplified "recipe" shown at the top of this posting.

Once a preliminary explanation has been tested using as many available approaches and sets of data as possible, and it still holds up, the hypothesis can become a theory - the most powerful kind of explanatory statement in science.

I heartily refer you to the website indicated above for the full treatment of how science works.  It's well worth the time, and I have all of my general education students review this site as part of the course.

On Science 3: The Power and Strength of Science

In earlier postings I shared some thoughts on the assumptions and limitations of science.  This posting focuses on the power and strength of science as a way of learning and knowing about the natural world.

In my earlier postings you read that one limitation of science is that science we can address only questions that are objective AND empirical, and another limitation is that there is no way we can be absolutely sure that a scientific explanation is correct.  These are limitations, but they are also the basis for some of the great power and strength of scientific inquiry.

Strength #1: Independent confirmation or refutation via empirical evidence.


The scientific approach to answering objective questions about the natural world always includes the collection and analysis of empirical evidence.  (Objective questions are those that have a definitive answer, and something that is empirical can be investigated through observations via our physical senses or technology that extends those senses.)

Because scientific explanations are based on empirical observations, anyone with access to the right kind of equipment (if needed) can replicate an experiment or collect their own observations independently, and independently test their own evidence to see whether a scientific conclusion is confirmed or refuted.  If claimed evidence cannot be replicated, then the conclusion is put into significant doubt and others will carry out their own studies to add to the body of observations which eventually becomes so compelling that the original explanation is either accepted, modified to explain all available evidence, or rejected in favor of a different, but stronger explanation.  Plus, this process of discovery and dissemination of scientific explanations includes independent, critical peer-review before it can be accepted for publication in a professional journal.  Then, once published, the information is read and assessed by the larger scientific community that in most cases carries out independent tests that allow for independent confirmation or refutation before it is accepted by the larger scientific community as a viable explanation.

This approach minimizes researcher bias and the chances that poor methodology or faulty or poorly supported conclusions will make their way into the accepted body of scientific knowledge.

Point #2: The self-correcting nature of science.


Statistical analysis of empirical data, and consideration of new evidence as it becomes available are a routine part of most scientific studies.  Result of these tests provide a statistical level of confidence we have in relation to hypotheses we test.  These levels of confidence are based on mutually accepted levels of confidence that are based on statistical critical values that have been calculated by statisticians for data sest of definitive sizes and for each kind of statistical test that exists.

These critical values allow us to identify the likelihood or level of confidence we have in accepting a scientific explanation as valid.  The scientific community typically requires a researcher to be at least 95% confident that a particular explanation (hypothesis) should not be rejected before it can be considered a viable possibility.

When a scientist carries out a research project they most often employ two preliminary explanations - hypotheses.  One represents the researcher's best prediction of what the outcome or eventual explanation will be.  This is called the Alternative or Research Hypothesis.  The other hypothesis is called the Null Hypothesis.  This hypothesis is a statement that says that the correct explanation is anything other than the Research Hypothesis.  The Null Hypothesis is the one that a researcher tests and must decide whether to reject or fail to reject based on the analysis of empirical evidence.  The decision about what to do about the Null is determined by the amount of possible error that is associated with the outcome of the statistical tests. What this means in practice is that a researcher must be more than 95% confident that the Null Hypothesis is NOT correct before they reject it.  The other 5% represents the amount of error that exists in relation to that decision.

Actually, there are two types of error associated with this kind of decision-making.  One type is the possibility of accepting an explanation when it should have been rejected, and the other type is the chance of wrongly rejecting an explanation when it should have been accepted.

So if the outcome of a statistical test shows a p-value (probability value of the null hypothesis being correct, or level of error in decision-making) is greater than 0.05 or 5% the researcher is compelled to fail to reject (i.e., accept) the null hypothesis.  If the p-value is smaller than 0.05 or >5% the researcher is compelled to reject the null hypothesis.  Only if this happens can the researcher consider the research hypothesis as a viable possible explanation.  it does not, however, mean that the research hypothesis is correct.  It means only that it has not been rejected as a possible explanation.

Since this process of eliminating possible explanations has been going on systematically for around 300 years now, many, many weak or incorrect scientific explanations have been corrected or rejected in favor of better ones.  What this also means is that whenever this process is applied, there is ALWAYS a margin of error, slim though it may be, associated with every decision.  What this also means is that as hypotheses are tested and rejected or not, we get progressively closer to describing truths about the natural world and how it works.

About now I hope you are asking yourself  "Is it possible to discover absolute truth through science?"

I strongly contend that not only can science discover absolute truth, but that it does so on a regular basis.  The problem though is that while absolute truth can be discovered, it is impossible to be absolutely confident that what science has been discovered is the absolute truth.

Because science cannot be absolutely confident in its discovery of truth, scientists continually test explanations with new sets of data collected in new ways or with new types of technology.  Hypotheses that that bear up under repeated testing become theories.  Theories that have withstood the test of time and many repeated tests for validity are considered strong theories.  Those that do not hold up under this type of scrutiny are either modified to explain all previously existing pertinent data and new data, or they are rejected in favor of new explanations that are able to explain all pertinent data.


This approach to testing, re-testing, and improving, or revising and replacing explanations is referred to as "The Self Correcting Nature of Science" which is one of the greatest strengths of the scientific approach to discovery of truth.  Dr. John Moore expressed the power of this aspect of science when he wrote, "Great art is eternal; great science tends to be replaced by greater science" in his book, Science as a Way of Knowing: The Foundations of Modern Biology."

In conclusion, science relies on independent review in order to minimize the effects of personal bias and to maximize the quality of scientific explanations.  It also includes a systematic process for eliminating weak or incorrect scientific explanations in favor of more complete or better-supported explanations.  These two strengths make the scientific approach an extremely powerful way to discover truth about the natural world and how it works.

Approaching Arctic Ocean Annual Sea Ice Maximum Extent For 2012

One of my favorite climate-related web sites is the http://nsidc.org/. This is the site of the National Snow and Ice Data Center, housed at the University of Colorado, Boulder.  This is a great place to check on the state of snow and ice around the globe: the Arctic, the Antarctic, Greenland, mountain glaciers, etc.

The NSIDC posts a near-real time map and graph showing the current sea ice extent and recent trend of sea ice compared with a baseline average of the years 1979 through 2000.

Anyway, it's late February 2012, and according to the baseline data, this is the time of year that we are approaching maximum sea ice extent for the year.  The NSIDC defines an area to be "covered" by sea ice if a location has at least 15% of its surface area covered by sea ice.

Here is the most recent map showing sea ice cover, compared to the 1979-2000 baseline average:

The orange line on the map shows the baseline average extent of sea ice cover from the years 1979-2000. The white area shows the reported current extent of at least 15% sea ice cover based on satellite data provided by NASA.gov.  NSIDC scientists note that there is more ice than usual in the Bering Sea north of the Aleutian Islands - remember the challenge faced by residents of Nome, Alaska, earlier this winter when sea ice prevented shipping from reaching them?  At same time, just about everyplace else in the Arctic shows a lower sea ice extent than the historical baseline.  This is especially true in the Kara Sea and Arctic Ocean North of Scandinavia.

The figure below shows that we are fast approaching the annual sea ice maximum for 2012.  The maximum extent is usually reached sometime between mid-February and mid-March, so we are in the window.  Current Arctic sea ice extent (blue line) shows that the current sea ice extend is about 1.2 million square kilometers less than the historic baseline (dark gray line).  The current extent is also well below the + 2 standard deviation range (light gray zone) around the average baseline.  This means that yet again, the current sea ice extent is statistically significantly lower than the baseline.  And sea ice extent in the Arctic is comparable to the sea ice extent observed in 2006-2007 which produced the lowest summer sea ice extent on record.

Is this the record lowest extent for this date?  No.  The record for the lowest extent for this date goes to February 2011: last year - when we also saw the second lowest summer sea ice extent on record.

Does this mean that we will have a record low sea ice extent in Summer 2012?  No one knows.  The lowest sea ice extent is a product of not only a warming climate, but of prevailing short-term wind patterns and other weather conditions between now and then.  All we can really do is sit back and see what does happen.

So, until next time, keep an eye on the sky, the thermometer, and the ice.  Cheers!

On Science 2: The Limitations of Science

The scientific approach is a powerful way to discover natural laws and describe how the universe works, but it is not all-powerful.  Science has limitations that any student of the sciences needs to be aware of.

Limitation #1: Science is objective and empirical

The scientific approach can be used to address only questions that are objective and empirical.  An objective question is one for which a definitive answer actually exists.  And a question is empirical only if the answer can be discovered through the collection and analysis of empirical evidence = observations made via the physical senses or technology that extends the senses.

An example of an objective, empirical question is: "What is the fastest way to get to work?"  This question has an actual answer, and it can be discovered by collecting empirical observations on the amount of time it takes to walk, bike, drive, take a train, bus, helicopter, plane, etc., from point A to point B under a wide range of conditions.

An example of a question that is neither objective nor empirical is "What is the best way to get to work?"  This question can be answered any number of different ways - fastest, most economical, most comfortable, most prestigious, most environmentally-minded, etc., depending on the opinion or perspective of the question asker.  So the answer is based completely on the subjective measure a person may want to apply to it, and is therefore not scientific, because someone else is likely to apply a different subjective measure, etc.

An example of a question that is objective but not empirical is "Does God exist?"  This question is objective because there are only two possible answers to the question, "Yes" and "No".  But it is a non-empirical question since we have no way to collect empirical, demonstrable, repeatable observations that can be shown to anyone who wants to see them.  This means that science simply cannot answer questions about things outside of our ability to observe and measure the physical/natural world.  Consequently, science is God-neutral and religion-neutral.  So if anyone ever tries to content one way or another about the existence of God using only scientific evidence, you should realize that they don't understand the limitations of science, and a little warning flag should pop up in your head.

Limitation #2: Scientific explanations always contain an unavoidable element of uncertainty

One of the first things a scientist does when s/he prepares to do some research is to state what they think they will observe and explain what those observations might mean. This preliminary explanation is called a hypothesis, a Research Hypothesis to be more precise.  It would be bad process, though for a scientist to set out to prove their Research Hypothesis correct, so in order to reduce personal bias the scientist develops a second option or explanation that is the opposite or negation of their Research Hypothesis.  This second explanation is called the Null Hypothesis.

After the researcher has collected as many empirical observations as possible, given time and other constraints, s/he analyzes those data.  This usually involves the application of statistical methods.  The interesting thing about this overall process is that the Research Hypothesis should never be the one that is tested.  Rather, the Null Hypothesis is the possible explanation that the researcher has to decide to reject or fail to reject (i.e., accept).  The really interesting thing about this process is that the researcher is forced to accept the Null Hypothesis unless the data and analysis of the data provide overwhelming evidence that the Null Hypothesis is not correct.  How overwhelming?  The scientific standard for most disciplines is that the research has to be at least 95% confident that the Null Hypothesis is not correct before it can be rejected.  Of course, some data sets allow the researcher to be more than 95% confident.  Sometimes they can be 99% confident, 99.9% confident, or even more, but no matter how confident a scientist is in their decision, they can never be 100% confident.  The 5%, 1%, or 0.1% represents the chance of making a wrong decision regarding the Null Hypothesis - that it should have been rejected when it was accepted, or that it should have been accepted when it was rejected.  This unavoidable element of uncertainly means that the researcher can never be absolutely 100% confident that their decision about the Null Hypothesis is the right one, though levels of confidence typically exceed reasonable doubt.  I mean, I would LOVE it if I could be at least 95% confident that was I making the right choice whenever I make any kind of decision, wouldn't you?

You should be aware that there are people out there that use this unavoidable, but usually minuscule amount of uncertainty to say that because there is some uncertainty in the data that we should feel free to reject any scientific explanation or conclusion that we don't happen to like.  This is faulty logic, so don't be misled by the kind of misguided decision-making promoted by this faulty logic.

Limit #3: It is possible to make the wrong decision regarding the Null Hypothesis, even if the scientist applies good process


As mentioned above in Limitation #2, it is possible that the decision to reject the Null rather than accept it is not the right one, even though the data suggest that this is the correct thing to do.  It is possible, for instance, that the set of observations you collected are not really representative of a population or a real cause and effect relationship, and this can result in faulty decision-making.

Does this happen?  Yes, occasionally, and more often when a scientist has only a small set of data than when they have a large one.

Is there a way to safeguard against or reduce the rick of making the wrong decision?  Yes.  The first line of defense is called the peer-review process.  Before a scientist's work can be accepted for publication in a professional journal, their work is sent out to a hand full of experts in the field.  These experts provide a no-holds-barred critical review of the research that was done.  They critique the overall research design, amount and type of data collected, the statistical and other analytical methods employed, and the conclusions reached by the researcher.  This peer-review process catches most errors that exist in research papers before they are published.  Then, once some research is published, other researchers will read it and some of them will carry out independent tests of their own to see if the obtain similar data and conclusions.  If they do, then the conclusions of the original research are supported.  If not, they are refuted, and the other researchers publish their work after peer-review.  This process of independent replication catches most of the other errors, but even so we will never be 100% confident of our decisions, though our confidence in the decision typically exceeds any reasonable person's criteria for surpassing reasonable doubt.

Conclusion

So to sum up, there are three primary limitations of science:

1. Science can address only objective, empirical questions.
2. Scientific conclusions and explanations always contain at least a tiny amount of uncertainly
3. Scientists can make the wrong decision about what to accept when they do their research

I hope that this overview has been helpful and will help you understand what the main limitations of science are, and how risks of reaching faulty decisions and explanations can be minimized.

Looking For A Great Place To Go Cross Country Skiing? Try Warm River, Idaho

Last Saturday we went cross country skiing at one of my favorite places in the world: the Rails to Trails Route at Warm River, Idaho.  This trail is readily accessible, beautiful, and doable for just about any skier.

This image (courtesy of GoogleEarth) shows the general location of Warm River.

To get to Warm River, drive north on Hwy 20 toward Yellowstone National Park.  When you reach Ashton, Idaho, turn right on Hwy 47, which is also Main Street in Ashton, and follow that road.  You will drive past many farm fields and then down into the river valley of Henry's Fork of the Snake River.  As you continue on Hwy 47 you will see the Warm River Campground to your right as you drive up out of the valley.  This campground is on Warm River. 

If you want to, you can access the bottom of the Warm River to West Yellowstone Rails to Trails route at the north end of the campground.  It is well marked, and is located on the west bank of Warm River.  During the wintertime the entire trail is open to snowmobiles, so there is sometimes quite a bit of activity on the trail from the campground northward.  Personally, I choose not to ski this part of the trail because the snow mobiles tend to chew up the course.  My preference is to access the Rails to Trail route at Bear Gulch.

Bear Gulch is a turnout at the end of the stretch of Hwy 47 that is maintained and plowed by the state.  This is also a popular jump-off spot for snowmobilers, most of the snow machine riders who start here do not go over to Warm River, they continue up Hwy 47 where they can really open up their machines.  We, however, pop up onto the road from the parking lot and drop down to Warm River via a well traveled trail.  Granted, this route is also accessible to snow machines, and can also get chewed up, but we were able to find pretty good snow along the edges of the main track.

I enjoy going to Warm River not only in the winter, but it is also my favorite fishing spot.  You don't tend to catch a lot of big fish there, but it's highly unusual to get blanked.  

Anyway, when we went on Saturday we got there around 11:30a or noon, and we skied up the trail for about 3 miles, as marked on the image above.  This is a beautiful place.  It was a nice day, though overcast.  The temp was not too cold, and it snowed on us on and off, with huge flakes.  A perfect winter day outing.

Here are some things you will see if you visit this area in the winter time.

This is a shot looking south, up the hill from the Bear Gulch turnout - note the trailers and snow machines.  We drive to the north end of the lot and park there.  This puts us out of the way of most traffic.  This is a nice turnout, and it has restrooms (though without running water).  It is, after all, on National Forest land.  If you look closely you will see where people have hiked up or were driven up this hill on snow machines and then skied or snow boarded down.  It beats paying for a lift ticket - at least according to some.

This is a panorama collage I put together using 6 shots taken from just down the trail from the RR tunnel.  The tunnel is visible as the dark object at the end of the trail at the left-hand end of the image.  The Warm River is visible at the bottom of the valley.  It is surrounded by pine and snow-covered hills. It's beautiful.

This is a shot of just the river valley.  In the spring, summer, and fall, there are also stands of aspen trees that you can see in the valley as well.

This is the southern end of the old RR Tunnel.   A few years ago here was a partial cave-in midway through the tunnel, and the USFS blocked both ends, and built an extension of the trail around the outer edge of the tunnel so you can still easily access the trail above the tunnel.

This is a shot of my wife and daughter on the bypass trail around the tunnel.

Once you finish the bypass trail you can look back and see the north end of the tunnel.  Well, you can actually see all the way through it...the tunnel isn't all that long.

While the section of the trail below the tunnel is scenic, my favorite stretch is above the tunnel.  Here are some shots from the trail as it travels along on the west bank of the Warm River.

This is a shot I took through a natural frame of snow-covered pines.  I love the way the snow piled up on boulders in the middle of the river.  This next set of shots were taken as we continued north on the trail.  There are places where there are volcanic outcroppings above the river, as shown below.  

The shot below shows ice cycles that form where the river water splashes onto logs that have fallen across the river.  I also really like the layering in the snow along the riverbank.

This shot shows a favorite fishing hole of mine that I visit several times each summer.  This trip though, all I'm taking are photos.  Well, actually, I am mostly a catch-an-release fisherman, so I don't tend to take that much even when I do come here fishing.

Note the large snowflakes falling with the dark water in the background.

And you will be treated to many scenes like this one all along this trail...and did I mention that you don't even have to pay any access fees!  Yeah!

Eventually it was time to turn around and head back.  Because the trail is an old railroad bed the grade is quite gentle, and when you are outbound it's not all that noticeable that you have been climbing in altitude.  But when you turn around you can tell the difference, and while you don't ski all the way down, you will find that the going is quite a bit easier.  And, you will notice that the difference in perspective of heading south gives you a different view of things you passed when you were outbound. 

And here we are, back below the RR tunnel, with Warm River in the background.  Until next time!

Oh, and when you're done and you're on your way home, don't forget to stop at Dave's Jubilee in Ashton for ice cream or a bite to eat at the deli!  Yum!

On Science 1: The Foundation of Science

What is the foundation of science?

If you asked 100 people on the street this question they might say, "Experiments", "Observations", "Hypotheses", "Theories", or even "Scientific Laws." All these things are important to doing science and developing meaningful scientific explanations, but these are not the foundation of science.

The foundation of science, scientific investigations, and scientific explanations is assumption, or rather a set of assumptions.

About now you might be saying, "Whoa! Assumptions? What are you talking about!?"

Before anyone can ask a scientific question, carry out a scientific investigation, or develop a scientific explanation it is imperative that they know the bounds and framework for doing that work. The bounds and framework for science are included in a set of assumptions that are used by the scientific community worldwide.

Before I lay out a list of assumptions of science it is important to realize that many such lists exist, and no single list is universally accepted, but the list I present here is representative of such lists and includes the main ideas found in them.

Assumptions of Science:

1. The natural world is understandable.  This assumption states that we can systematically observe the natural world and develop meaningful explanations of the natural world based on observations. If we do not accept this assumption there is no point in trying to study or understand the natural world.
2. Events in the natural world are the result of cause and effect relationships. This assumption states that when we see something happen that event is the consequence of some natural force, process, or factor.  These cause and effect relationships are governed by natural laws.
3. Cause and effect relationships can be studied by collecting empirical observations.  Empirical observations are made via our physical senses or by using technology that extends those senses, and these observations can be repeated and demonstrated. 
4. There is consistency in the natural world (i.e., the universe). Cause and effect  relationships and the natural laws that control them that function one way in one location or under one set of conditions will function the same way everywhere in the universe where the same conditions exist. In other words, natural laws are universal and function the same way everywhere in the universe, past, present, and future.

Collectively this set of assumptions support the conclusion that there is a set of natural laws in effect in the universe.  The work of scientists and the goal of science is to discover those laws and explain how they work.  In other words, scientists to not develop natural laws, they discover natural laws that already exist.

An idea...maybe even a plan!

As you may know (if you read my short bio) I teach a general education course on climate change. As you probably also know, there is a lot of information and a lot of misinformation out there on this topic.  What I would like to do is to start a series of postings that deal with the practice, power, strengths, and limitations of the scientific approach to learning about the natural world, and then build on that foundation by sharing information about the science of climate change.

My goal in posting this information is to provide you with the tools and data you need to reach your own conclusions regarding this somewhat heated (pardon the pun) topic.  In other words, I am not going to be posting any passionate, emotionally charged arguments or persuasion pieces designed to proselytize anyone into either camp that is engaged in the current "climate change debate."  Once you have an understanding of what science is and how it works, and you have the data about global climate in hand it will be up to you do decide what you think about the question of climate change.

I will post entries as time allows, but I've been pretty busy lately so I hope you won't expect to see too many postings right away...still I'll do what I can.

BTW, the order of these postings will roughly follow the same order that I present ideas and data that I use in my course on climate change.  I hope you find it to be helpful and informative.

Best wishes!