Methods that New Zealand information managers use to evaluate the degree to which items of information reduce uncertainty or otherwise contribute to knowledge

Abstract

This project seeks to identify, describe and compare information evaluation methods used by selected New Zealand information managers with reference to appropriate decision analysis techniques. Four methods from decision analysis theory for dealing with uncertainty when evaluating information are covered: • Bayes' Rule which uses likelihood Ratios to represent learning when new information is found and combined with old information. • Dempster Shafer theorem • Fuzzy sets and • Rule Based Systems. Heuristic errors and other biases that can arise in decision analysis / information analysis situations are also discussed, as are the benefits of utilizing hypotheses, levels of inference and the elements of informal argument structure to help evaluate information. Taped interviews in the form of "think aloud" and "retrospective" protocol analyses were conducted with six information managers. The interviews of two information managers, a librarian and an intelligence analyst, were examined in detail to find differences in their treatments of uncertainty and information. Evaluation was found to be the focus of the analyst's thinking. Search strategy and execution was more important to the Librarian. No evidence was found that information managers use formal decision analysis techniques when evaluating information. Recommendations are made for follow up research.