- Joined
- Mar 2, 2016
- Messages
- 372
- Reaction score
- 490
The simple, honest answer is this: If you are not at least semi-competent in statistical analysis (300+ level coursework), you don't really understand what you are looking at.
What is worse is this: We know all sorts of problems exist and why they exist, but we still can't really do anything about it.
For example, scientific journals prefer to publish new material. New findings are sexy. New findings sell, even to academics. It is almost impossible to get them to publish studies that result in outcomes rejecting previous findings. (Particularly because there will almost ALWAYS be these rejection findings ... even if the original data is good.)
We also know that a certain percentage of "positive" findings will be in error, even when everything is done correctly.
These two things together mean bad data gets published by legitimate scientific journals, without the aid of corporate conspiracy, and will by sheer inertia of the publication system pose a difficult target to displace.
The best, simple method to identify reliability in data is reproduction of the findings. Look for multiple studies, from different sources, that give essentially the same conclusion. This will still have flaws, but is really the best the average person is going to do.
What is worse is this: We know all sorts of problems exist and why they exist, but we still can't really do anything about it.
For example, scientific journals prefer to publish new material. New findings are sexy. New findings sell, even to academics. It is almost impossible to get them to publish studies that result in outcomes rejecting previous findings. (Particularly because there will almost ALWAYS be these rejection findings ... even if the original data is good.)
We also know that a certain percentage of "positive" findings will be in error, even when everything is done correctly.
These two things together mean bad data gets published by legitimate scientific journals, without the aid of corporate conspiracy, and will by sheer inertia of the publication system pose a difficult target to displace.
The best, simple method to identify reliability in data is reproduction of the findings. Look for multiple studies, from different sources, that give essentially the same conclusion. This will still have flaws, but is really the best the average person is going to do.