Information Technology Innovation Survey:  Fall 2001

Technical Notes


Survey Universe top

The universe of the main study was divided into two sectors: an IT sector comprised of three strata that included those companies closely associated with IT-based innovation activities, and a non-IT sector comprised of four strata that included companies most likely associated with the use of IT-based innovation. These strata included companies whose primary business is classified within the manufacturing; transportation, communication, and public utilities (TCPU); finance, insurance, and real estate (FIRE); and services industries based on primary four-digit SIC codes reported in Dun & Bradstreet's Marketplace© database.

To provide robust independent estimates at both the stratum and total levels of aggregation, a determination was made to establish an equal allocation of 400 completes for each of the seven identified strata. Based on pretest results, it was estimated that approximately 4,000 companies would need to be selected to account for attrition due to ineligibility and nonresponse. A disproportional random sampling schema was used to divide companies within frame into replicates of 1,000. Four sample replicates were initially selected at random for use in the study. During the survey, a decision was reached to add 344 companies to the IT communication sample in order to account for unexpected sample attrition in this stratum. These additional IT communication companies were randomly drawn from the unused replicates. Table A-1 gives the final frame and sample sizes for the survey sectors.

  Table A-1 Source Data File: Excel File

Four primary industrial sectors used in the SIC coding scheme were identified as the target population of interest. Each contained businesses thought to be most likely associated with IT-based innovation, had significant dependence on the use of IT, or otherwise appeared to be within the scope of the research objectives. The identified sectors were manufacturing; transportation, communications, and public utilities (TCPU); finance, insurance, and real estate (FIRE); and services.

Three strata were identified as the IT universe. They included companies most likely associated with IT-based innovation activities. The IT manufacturing stratum included companies within SIC codes 3571–3579, 3651, 3661–3679, 3695, and 3823–3826. The IT communication stratum included companies with SIC codes of 4812–4899. Ultimately, radio and television broadcasting stations (SIC codes 4832 and 4833) were eliminated from this universe because questions did not appear to elicit responses that were aligned with the survey objectives. Adjustments were made to the final weights to account for their elimination from the frame. The IT computer service stratum included companies within SIC codes 7371–7379. No IT stratum was identified in the FIRE sector.

The four strata identified as representing the non-IT universe included companies most likely associated with the use of IT-based innovation. The non-IT manufacturing stratum included companies with four-digit SIC codes of 2011–3569, 3581–3648, 3652, 3691–3694, 3699–3822, and 3827–3999. The non-IT transportation and public utilities stratum included companies with SIC codes of 4011–4231, 4412–4789, and 4911–4971. During pretest verification, it was found that most credit unions (SIC codes 6061 and 6062) fell under public ownership. A decision was reached to exclude these two SIC codes from the universe. Therefore, the non-IT FIRE stratum included companies within SIC codes 6011–6036, 6081–6153, and 6162–6799. Ultimately, miscellaneous business credit unions (SIC code 6159) were eliminated from this universe because most companies within this group were publicly owned. Adjustments were made to the final weights to account for their elimination from the frame. The large non-IT service stratum identified in the pretest was reduced to include only companies within SIC codes 8711 (engineering services) and 8721 (accounting, auditing, and bookkeeping services) for the main survey.

Eligible companies had to have a combination of a minimum of 25 employees and minimum annual revenues of $2.5 million, as reported by Dun & Bradstreet. They also had to be a public for-profit corporate headquarters or a single location business.

Data Collection Techniques top

Survey implementation was divided into two stages. In the first stage (designated as NSF-1) 2,000 sampled companies were divided into batches of 1,000 companies each. Companies in the first batch were sent introductory letters on July 6, 2001, informing them of the study and directing them to the Website to complete the questionnaire on the Web. Companies in the second batch of 1,000 were sent contact materials approximately 1 month after the first batch.

Computer Assisted Telephone Interviewing (CATI) follow-up began 2 weeks after the launch of batch-one letters and continued throughout the data collection period, which ended April 1, 2002. Follow-up also included sending several reminder packets. Each of the follow-up mail packets included a letter from NSF, a letter from the data collection agent, a paper version of the questionnaire, and a postage-paid business reply envelope.

Response to the first-stage launch was relatively constant, but the pace of receipt was unexpectedly slow. The timing of the launch during the summer vacation season may have played a part in the slow pace of receipt. In October 2001, NSF requested Dr. Don A. Dillman [3] to review the implementation protocol and offer suggestions for improving response. As a result of his review, several changes were made to the data collection protocol for the second stage (designated as NSF-2) launch. These changes were:

In the second stage (NSF-2) 2,344 sampled companies were divided into two batches of 1,000 companies and one batch of 344 companies. Companies in the first batch were sent the new introductory letter on October 30, 2001, followed by an initial mailing to batch 2 approximately 2 weeks later. Approximately 45 days after the launch of the second stage, a supplemental batch of 344 companies were mailed initial invitations. This final batch was necessary to adjust for an unexpectedly high number of companies identified as out-of-frame or out-of-scope within the IT communications sample stratum.

Stage 2 telephone interviewing began 3 days after the launch of the initial letter and continued throughout data collection. Two weeks later sampled companies were sent a reminder postcard asking them to respond to the survey when called. Approximately 1 month after launch, nonresponding companies were sent letters introducing the Website and encouraging them to respond via the Web. Two weeks later a postcard reminder was sent that included the Website information. Approximately 15 days after the postcard was sent, nonresponding companies were sent mail packets that included letters from NSF and the data collection agent, a paper version of the questionnaire, and a postage-paid business reply envelope. This protocol was followed for each of the three batches in stage 2. Finally, all nonresponding companies from each batch were sent a priority mail packet that included a letter from NSF, a letter from the data collection agent, a paper version of the questionnaire, and a postage-paid business reply envelope.

To help mitigate the impact of nonresponse, an aggressive follow-up campaign was planned for the main survey. The plan called for the use of three personalized mail reminders, up to 30 telephone (CATI) reminder calls, self-administered paper facsimile, and e-mail versions sent upon request; forwarding information was to be requested for all USPS postmaster returns, and a personalized last mail request would be sent by priority mail.

Response Rate and Calculations top

A total of 2,005 companies responded to the survey, representing an estimated population of 72,406 companies. Of the 2,005 respondents, 66 percent responded by CATI, 22 percent by Web, and 12 percent by paper. After adjusting the sample for companies determined to be out-of-frame and out-of-scope, an overall 57.2 percent response rate was achieved. The highest response rate (65.4 percent) was for the non-IT services stratum, and the lowest response rate (50.3 percent) was for the non-IT manufacturing stratum. Table A-2 provides a summary of response by sector and stratum.

  Table A-2 Source Data File: Excel File

The following steps were employed in the pretest and main surveys to minimize the impact of nonresponse:

Item nonresponse was minimal due in part to the fact that 88 percent of response was received either by CATI or Web. Interviewers administering the CATI version of the survey were professionally trained to counter objections raised by respondents to individual questions. The Web administration of the survey required respondents to answer each question before proceeding.

See Survey Methodology Report (forthcoming on SRS Website, http://www.nsf.gov/statistics/).

Confidentiality and Disclosure Analysis top

An extensive tabulation process of the raw data was undertaken as part of the confidentiality and disclosure analysis. Based on these tabulations several steps were taken to minimize the possibility of inadvertent disclosure of individual company responses. These steps include:


Footnote

[3] Dr. Dillman is a consultant to NSF and recognized expert in the field of survey research.


Previous Section Top of page Next Section Table of Contents Help SRS Homepage