If you have not already done so, please see the article, "What Is Good Science" first.
There are tons of bad science reports out there these days. It really isn't science, but many people consider it science, so we will cal it "Bad Science".
Bad science generally has a bad start.
What is a bad start?
- Basic assumptions that cannot be verified, and/or are simply unreasonable.
- Conclusion written before doing any studying. (Often this is fraud-science)
- "I thought up this idea so it must be true." (Ego-centric-science)
- Not bothering to consider what else has been done on the subject. (Lazy-science)
Let's look at some faulty basic assumptions.
- Looking at minutia will answer the big questions.
- If the math is correct, then the theory must be correct.
- Using initial conditions which cannot be realistically determined.
- This is so simple that a very small data set is sufficient.
- The impossible is possible given enough time.
- There may be alternate universes with alternate rules, that make this possible.
- A celebrity said it, so it must be true.
- A computer can be programmed to simulate this.
Looking at minutia and missing the big picture, lets consider an example.
I recently read a discussion about DNA proving or disproving that the Book of Mormon is a true history. It was a fascinating discussion, in which both sides claimed victory, but that's not what got me. They were so involved discussing the details of DNA and heredity science, and how to analyze paternity using DNA, that they totally missed looking at the big picture which totally invalidated all of their arguments on both sides. You see, both of them were trying to compare the current DNA of native Americans with current DNA of the Jews. This is because the Book of Mormon says that people who came from Jerusalem to America about 600 AD are the ancestors of native Americans.
But because they jumped so quickly into the debate about DNA, they missed that they couldn't get the answer to the question from DNA. If they would have read the Book of Mormon more carefully they would have realized that Lehi (the leader of the group from Jerusalem) did not claim to be Jewish. In fact he very plainly said that he is a descendant of Joseph (Judah's brother).
Brothers can have very similar DNA, but Joseph and Judah were brothers from the same father, but different mothers, and the DNA being used for the discussion was mitochondrial DNA passed down through the mothers. So Joseph and Judah had different mothers, hence different DNA. Then Joseph and Judah married women from different parts of the world.
The ancestry of Lehi went back to Joseph through the paternal line, but we have no idea at all of the maternal line divergences since Joseph and Judah. Hence, the whole discussion was worthless for either side. They were trying to compare things that had no relevance.
Looking at minutia very often has this problem. The big picture is missed, and so much time and effort is wasted. But the saddest part is that neither side ever backed up to see the big picture and realize that the time was wasted. They each think that they proved their point.
This is actually a very major problem for today's scientists in general, because of all the specialization. Of course specialization is necessary in our technological world today, because nobody can know everything. But we need to keep administrators who oversee specialists and keep them on track by looking at the big picture. Some people need to be trained as big picture people, and not specialists.
It appears that our best science researchers today are those who have a decent understanding of basic principles in a wide range of subjects, and know how to think critically about the data created by specialists, and combine the various ideas into a reasonable whole picture.
Organizations looking to further scientific discovery should hire 'integrators' who can analyze and integrate knowledge from many sources into accurate big pictures of reality.
Correct math does not prove that a theory is correct.
There are many mathematical formulas which can be written and solved that don't have anything to do with the reality of the world around us. Mathematics can project multiple universes, but can not make them real. Math proved that the Wright brothers could not create a flying machine.
Math proved relativity, but still nobody can actually explain it or demonstrate it, and some scientists are now proposing that it is totally wrong. Still others are saying that the formulas simply need to be redone and refined a bit.
The problem actually lies in the fact that modern math is a figment of imagination. There are so many imaginary constructs in math, it is a bit surprising that people expect math to represent reality. Remember when it used to never work to divide by zero?
Now, don't get me wrong. Of course math is a necessary tool to be used in analyzing data. But math must remain a tool and not become the master. It is a measuring and comparing tool. It is not a building block.
I can take 3 apples and add them to 5 apples, and math will tell me that I have 8 apples in my bag. But I can't multiply 8 by 3 and suddenly have 24 apples in my bag. Even though I did the math correctly, it did not change the reality of my bag containing 8 apples.
While it may be possible to put 24 apples into my bag, doing the math does not make it so. Possibility does not equal probability.
Using initial conditions that cannot be known. (Falsifying assumptions)
A common mistake is making what we call falsifying assumptions that are not based in reality.
Consider, for instance, carbon dating. To make carbon dating work we must assume that an object had a certain amount of carbon in it at the time of it's creation. But since we have no way of knowing how much carbon it had at the time of it's creation, we assume that it has the same amount of carbon in it that a similar object created today would have. We then look at the rate of decay of carbon (assuming that carbon always decays at the rate that it is decaying at now) and calculate how old the object must be by how much carbon it still contains.
So two major assumptions here; every similar object is created under the same circumstances, and rate of decay is always constant. If either one of these assumptions is incorrect, then carbon dating doesn't work.
I think that it is pretty obvious to any thinking individual that the first assumption is erroneous. We are currently involved in a very well know discussion about changes in carbon in our atmosphere. If honest, we have to admit that we have no idea what carbon levels were in the past, nor what they will be in the future.
And for those following science, it is also obvious that the second assumption also fails, decay rates are not always constant, but also can vary depending on various environmental factors. In fact multiple samples from the same object have different levels of carbon.
The interesting thing is that our society, despite knowing these things, continues to use carbon dating. The excuse is that we need some kind of measurement to work with. But, we still pretend like it is accurate, ignoring the reality of it's failure. Then we also ignore that every theory that relies on carbon dating is also a failure, because it is based on faulty assumptions. A house of cards, if you will.
Some have become so reliant on carbon dating, that they even try to defend the accuracy, claiming it to be very accurate. They have even built complex mathematical models to magically math away the possible errors.
All radiometric dating has the same problems, but they use "Concordia curves" to magic away the discrepancies. Much like Ptolemy's epicycles, they work mathematically, but have no relation to reality.
Keep it simple with a small data set.
Many studies get cited these days that have such a small data set, that they are totally unreliable. Some are designed that way on purpose. A small data set is easy to control and analyze, but due to statistical randomization it can be totally useless for drawing meaningful conclusions. Six people can not realistically represent 6 billion, or even 6 thousand. For instance if I grab 6 people from the edge of a crowd of 6000, it is likely they will have more things in common that the rest of the crowd (that is why they were together). So I can't poll them and assume that the rest of the 6000 are the same. But the bigger chunk of people that I grab, the more reliable my generalizations about the crowd can become.
The worst case of small data sets, which is actually quite common, goes something like this. "My mother's cousin's boyfriend had cancer, and he sat on the toilet for 14 days and his cancer was cured."
Now, while that may be a true statement, it does not prove the sometimes implied conclusion that if we all sat on toilets for 2 weeks at a time we would all be cancer free.
Now, I made that a bit nonsensical to make the point, but I hope that you recognize that no matter what you replace the toilet-sitting with it doesn't change the fact that this is very bad science.
Because something happened with one person, it does not mean that it will happen with all. There are way too many variables and differences between people to make such claims.
OK, what if it happened with 2 people? OK, that is interesting, and it does make one wonder how the two might be connected, but it doesn't show causation. What about 3 people, or 4 people, or 5 people? Well, the more people that it happens with the more interesting it becomes. But, because there are so many variables that could contribute to the outcome that we haven't discussed at all, we can make no conclusion. However as the number of people that this is happening with continues to grow, and as we begin to ask questions about other factors involved we may start to come to some useful ideas that may help to fight cancer.
To continue with this (fictional) model: As we look into this toilet-sitting more, we begin to get more details.
- Well he didn't actually sit on the toilet non-stop all the time, but he did have a bowel movement three times a day.
- He and the lady next door both were drinking about a gallon of water everyday.
- The first 100 people found doing this, that had success, were also using a teaspoon or two of real salt in their water every day.
- Most of the first 1000 people that went into remission were also taking about 20 grams of ascorbate with their water and salt each day.
- 80% of the first successful 10,000 people were also eating lots of fresh whole foods.
- Out of 100,000 successful toilet sitters 75 percent had stopped the use of all vegetable oils and started using coconut oil.
So as we learn more and get more people involved in a study, it becomes more meaningful and perhaps predictive of what could happen for many other people.
But if you have just a very small sample, it could just all be coincidence. The bigger the sample the more probable the connection.
The impossible is possible given enough time.
This one isn't actually science at all. It is just speculation that something might be possible given enough time. If something proves impossible, just say that it takes much longer. After all, maybe laws of physics change over time. Again not scientific, just fantasy.
This is the pit that evolution of species has fallen into. As it has become more and more obvious that biologically one species can not turn into another (requiring more and more miracles) those wishing to stick with the theory keep adding longer and longer time periods. At first evolutionists said that evolution of species took thousands of years, then tens of thousands, then hundreds of thousands, then millions, and so forth. The idea that you can take an impossibility and throw it in a time machine and it will become a probability is just wishful thinking.
The alternate universe theories are just more of the same thing. Fanciful imagination, not science.
Some celebrity said so.
Are you kidding me?!!! It still shocks me that people give credence to something just because some celebrity said it.
Computer simulation science.
This is one of our big problems these days. For some reason people got this weird idea that if you can simulate something on a computer then you must be able to do it in real life. Didn't they ever watch cartoons on television?
Computers are great tools for crunching data and analyzing it, but just like with mathematics, it must remain a tool, and be based in reality.
Because computers can make computations so much faster than people, that doesn't mean they are smart. Computer's have this major flaw called GIGO. It stands for 'Garbage In, Garbage Out'. A computer's output can only be as accurate as it's input. In fact computers actually compound errors very rapidly. One little mistake, or one little thing left out, can create chaos very quickly.
Since all computer input comes from people originally, there are nearly always mistakes or omissions. People enter their assumptions and expectations into computers. A computer therefore cannot create truth. It can be used to analyze and help to graph data, but the validity of that data is not assured or improved simply because it was passed through a computer.
An obvious case in point; all of the predictions of global warming disasters have been backed up by computer simulations. All the predicted disasters (and even the symptoms associated with those disasters) are failing to materialize in the real world. We don't even understand weather enough to predict the next 12 hours, but it was assumed that computer models (with incomplete information) could predict it over the next 50 years? Now that is a new kind of idiocy.
If you don't even know what variables are involved, or how many, can you write a mathematical equation to solve what the weather will be tomorrow? It's silly. It might be a fun exercise, but anyone knows that it would be totally worthless in regard to reality.
Now, if you want to look at past trends, a computer is great for analyzing data. But as far as projecting those trends into the future you have to make assumptions and predictions. Everyone knows that the further into the future you look, the hazier it becomes, and it becomes so very quickly. So compare the output of any forward looking program to the next minute, hour, or day. If it isn't accurate in the short term , it will only get worse in the long term.
There is a lot of fraud-science these days. Companies use studies to make money and politicians use them to gain power. Now don't assume that just because a study comes from a company or a government entity that it must be fraud. But do beware of the possibility. Be more diligent about checking the facts and making sure that they match the conclusions. They often do not match.
Often data is falsified, or shifted, or thrown out, because the original data didn't say what the authors wanted it to say. For example a well know film pretended to put one graph over another to show how well they matched up. But they cheated. They shifted the time scale on one graph so that they could make it appear to match the other. It was a lie, but they got the reaction from the audience that they wanted.
Unfortunately this kind of stuff goes on more than you would hope.
It is also not uncommon to read through a study and look at the data and then be surprised by the conclusion. The data often does not match the conclusion, or was not gathered in such a way as to allow the stated conclusion to be verified. These are the studies where they wrote the conclusion first.
This is another form of fraud actually, but it is self-deception. The author is so sure of his basic premise that he interprets all data to mean what he wants it to mean, even though an outside observer might come to a totally different conclusion. We all have to be careful of this one, because it is human nature to be a bit egotistical. We think that the ideas we think up must have some validity simply because they are ours. We talk ourselves into being right before looking at the observations, and so naturally interpret what we see as what we expected to see. This is why peer review is a good idea.
However, peer review problems also arise due to the same ego problem. Peer review would ideally be people with the same basic understandings of science that we have looking over our research to make sure that it was done well and makes sense. As well as perhaps giving us some constructive criticism and catching mistakes before the research is published. Peer review used to actually include repeating the experiments to make sure that the outcomes were not just flukes or anomalies.
Today however most peer review is simply a journal editor reading over your paper and deciding if he agrees with it or not. This leads to the editors ego getting in the way sometimes. He gets lots of papers to look over, so he does it as quickly as he can. Many editors fall into the trap of thinking that man has already discovered most of what he needs to know, so any paper that is out of line with what the editor thinks to be true is not a well done paper. While this saves the editor time, it often overlooks and discards new and innovative ideas that may be based on very solid observation and analysis. This has a tendency to solidify old ideas and discard new ideas. Not a great paradigm for science and it's future.
Lazy science comes from the guy who just says, "I'm curious about this so I'll do this little experiment." A great start, but if he does no research to see what else has been discovered about the subject, it can be like the experimenting blind man. While coming at an idea from a fresh prospective is always a good idea, if one remains blind to the other realities around him it gives a very incomplete picture.
This narrowness of vision can cause major problems. For instance a man who has had no exposure to smoking, suddenly comes across another who is smoking outside on a cold fall day. He decides to experiment and finds that indeed smoking can make him feel warmer on cold days. He soon begins to tell everyone that they need to smoke on cold days to stay comfortable, and that constant smoking is the best, because then you are always warm.
Has he discovered truth? Yes. So, what is the problem? There is a huge part of the picture that he hasn't been exposed to, and so he is heading downhill fast towards a miserable death.
This is a constant problem for science, and why we always need to be looking around, and asking more questions. Science can be a great boon to us, but also a huge stumbling block if just used casually.
Bad science can cause bad problems, but a little diligence can turn it into a great blessing.
If you are not willing to do lots of studying yourself, then find those who like to do it, and who you feel you can trust, and communicate with them on a regular basis to stay apprised of what they are learning. You will be glad that you did. It can save you lots of heart ache in the future.