Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.
These days, it is extremely easy to produce and disseminate information, which has resulted in a proliferation of information, and some pieces of data are guaranteed to contradict others. Search engines and systems have evolved to the point that almost any query will yield some sort of result. The challenge is not to find information but to find the right amount of quality, believable information. This guide introduces some things to consider when evaluating information in general, and chemical data in particular and introduces four credible sources of physical and chemical properties of substances.
Accuracy and Bias in Scientific Information
It is tempting to think that all scientific information is accurate, particularly in fields like chemistry that rely on quantitative data to support conclusions; however, even the best journals issue retractions (pulling papers from circulations) when articles are discovered, after publication, to be fraudulent, inaccurate, or based on erroneous premises.
It is also tempting to think that chemical information is unbiased. After all, reactions either work or they don't, and the "data don't lie." In point of fact every piece of scientific information is biased, for two reasons.
- Bias of experience: Scientists use the word "interpret" to describe their analysis of data. This means that, while there is no surety that their premises are correct, they are attempting, to the best of their abilities, to explain the data in terms of things that they know or believe to be true. Experience comes into play here greatly; a scientist with more experience in the field or who has read more widely on the subject will have more information at her fingertips and will be able to bring this experience into the interpretation of the data. A scientist with less experience will have a much more narrow perspective and will not be able to bring as many diverse findings to the interpretation. Unfortunately, the only cure for the bias of experience is age.
- Bias of expectation: This is also known as "confirmation bias," and it begins at the time of project selection. A scientist will only elect to do a project if it has a good chance of success... and if he can secure the funding for it! Once he begins the work, he develops a working hypothesis, which he alters as the project progresses. It is very tempting to "see what you want to see," and he must be careful not to let his hypotheses or his expectations color the interpretation of his results.
General Criteria for Evaluating Information
One can apply the same questions to evaluating a piece of research that one can apply to dissecting a story or a piece of literature: who, what, where, when, why, and how. When evaluating research, however, these questions take on a slightly different meaning. You should ask yourself all six questions whenever looking at scientific information in both familiar and unfamiliar areas.
- Who did the research?
- What are the authors' credentials in the field of the research?
- What kind of reputations do they have?
- With which institution(s) are the authors affiliated?
- What is the reputation of the authors' institution(s) in the field of the research?
- Is the research relevant to your current information need?
- Is the information presented clearly so that it is easy to understand the purpose of the research?
- Are the data appropriately recorded and documented?
- Do the authors appear to have done a full and complete background literature search, documenting all relevant prior research?
- Where is the research published?
- Is the source peer reviewed?
- What is the reputation of the publisher and/or the source?
- What is the publication date of the research?
- Is it current for its field?
- Have techniques or thinking greatly evolved appreciably in this field since the research was done?
- What makes this piece of research interesting or important in its field?
- What were the authors' motivations in publishing the research (if you can determine them)?
- Who funded the research?
- Do the authors declare any interests in the document?
- Do the authors' institutions have any easily-recognizable interests in the area of the research?
- Do the authors have any relationships or biases that would throw the integrity of the research into question?
- Does the information presented appear to be accurate?
- Are the methods recorded in such a way as to make it possible to reproduce the results?
- Do the methods agree with other published methods?
The National Institute of Standards and Technology asks the following three questions of all pieces of data.
- How well is the data generation described?
- How do the data follow the known physical laws?
- How do the data compare to other measurements or calculations of the same phenomena?
Source: Reproduced from https://srdata.nist.gov/CeramicDataPortal/pds (Accessed June 4, 2017).
In addition, NIST offers some definitions of various types of data that can be applied to pieces of data located on a substance or phenomenon. They are presented in the order of their "authority."
- Certified: Standard reference values, specific to known production batches
- Validated: Confirmed via correlations and models
- Qualified: Basic acceptance criteria satisfied
- Commercial: Manufacturer's data for specific commercial materials
- Typical: Derived from surveys of nominally similar materials
- Research: Preliminary values from work in progress
- Unevaluated: All other data
Source: Reproduced from Munro, Ronald G. Data Evaluation Theory and Practice for Materials Properties, NIST Special Publication 960-11. Washington, DC: National Institute for Standards and Technology, 2003.