Metrics are a form of information and are ideally developed to answer specific sorts of questions. Which measures a society collects and analyzes about the effects of a technology depends largely upon what it wants to know, and we would not expect that all societies necessarily want to know the same things. The metrics and analyses presented here are based on four central questions:
Available metrics exhibit considerable weaknesses in their ability to answer the above questions. The single most important obstacle to effective data collection is the lack of standardized definitions of IT, and the exclusion of important costs associated with IT use. For example, in some economic studies, IT reflects only computers, while in others it captures computers and telecommunications hardware. Research shows that IT support personnel and training expenses are significant elements of the cost and effective use of IT, but that these expenses are not always included in research studies or data collection. To fully capture the extent of the technology, IT should be defined as computers and telecommunications equipment. In addition, IT-associated costs should be included when collecting expenditure data on IT. Key associated costs include software, personnel expenses for IT support staff (e.g., network administrators), and training expenses for individuals who use the technology. One major obstacle to more effective data collection is the lack of appropriate budgeting and accounting reporting systems at the organizational level. Another is that IT itself continues to change rapidly.
A further weakness is the relative absence of systematic information on how IT is actually being used. IT is a means to an endprincipally information processing. A real appreciation for the impacts and consequences of IT requires understanding what information it allows us to collect, access, and process. The presence of the hardware itself does not tell us to what ends it is put, and it is the actual use of the technology that determines its effects. Systematic surveys of IT applications are thus in order. For example, time-on-task audits would reveal how individuals actually use computers and networks at their office, school, or home; analytical questions about the impacts of specific IT activities would develop from patterns of real use. Similarly, diffusion estimates for specific types of applications (such as CAD-CAM, electronic data interchange, inventory management systems, and business management systems) could narrow down and help identify impact-related questions. Systematic knowledge about the degree of importance of different uses and applications of IT is missing.
Diffusion indicators for IT are relatively abundant and analytically useful. Several good data series exist that could be compiled into an ongoing set of diffusion metrics. These indicators are presented in text table 8-7, and include IT investments and stocks by industry and IT in K-12 schools. A lack of diffusion/IT intensity metrics is notable for both the education and economic sectors with respect to IT-associated costs-personnel, software, and training. These IT expenses are emerging in the research as significant determinants of IT effectiveness, and need to be tracked on a systematic basis. The most striking lack of data relates to distance education: by definition, this is learning that takes place through the use of information technologies, and there are simply no reliable metrics on the scope and growth of this unique educational practice.
Impact measures for the economy are problematic, primarily because of the difficulty in measuring economic output for many of the service sectors. One alternative to this dilemma is to select a representative set of sectorsor those that are the most economically significant-and develop a set of impact metrics unique to each. The research evidence suggests that IT impacts are highly firm- and industry-specific; it is unlikely that a single measure could capture the economic benefits of IT for all types of enterprises. Three potential measures-as illustrated by the banking industry-are those that reflect the volume of transactions processing, human versus electronic transaction costs for key types of transactions, and processing times for key transactions.
Impact assessments for IT and learning are complicated by a more severe measurement issue, which is the need to collect data through observational studies or controlled experiments. Meta-analysis suggests that computer-based instruction generates real learning impacts, but more rigorous and comprehensive studies need to be conducted. A large-scale controlled study would be one way to avoid the statistical dilemmas of small classroom experiments.
Finally, IT clearly raises quality-of-life issues for the individual citizen. Occupational injury, psychological stress, and violations of privacy are clearly potential dangers of the widespread use of information technologies in the workplace and in information-intensive industries. Consistent tracking of the hazards to the individual represented by extensive use of IT is in order. Because IT also can clearly enhance quality of life, inequity in IT access could create more social stratification in the United States. Ongoing monitoring of equity indicators is critical as the significance of IT to employment, health, and well-being grows.