All mailout materials are produced by the contractor, Quantum Research Corporation (QRC). Institution facsimiles and mailing labels are generated from the previous year's survey database. QRC staffers assemble the survey packets, checking them individually to ensure accuracy before mailing. All survey materials are returned to QRC (except during new contract years when materials are returned to NSF and forwarded to QRC). Institutions that requested an Automated Survey Questionnaire (ASQ) in a previous year also receive an ASQ diskette and instructions in their mailout packets. The most recent questionnaire form and other mailout materials are available.
As with the postcards, completed questionnaires are returned to NSF, stamped with the date received, and forwarded to QRC, where the receipts are recorded in the institution response control logs.
At the end of each week a report is generated from the survey database. This weekly report provides the following information on each institution: Top 100 or other standing, type of academic institution, highest degree granted, type of control, state, and stratum code. The report also provides, for each institution, date of postcard receipt, expected and actual receipt date of survey questionnaire, whether the ASQ has been requested and receipt date, and a flag marking a change in any of these conditions during the latest week. Institution counts, aggregated by week, provide similar information on the receipt of postcards, ASQ, and survey questionnaires for academic institutions, Top 100 institutions, and FFRDCs. A count of institutions by response status code is also displayed. This report is sent to the NSF project officer each Monday during the survey processing cycle.
A list of nonrespondents is generated from the control logs, and in early December, follow-up postcards are sent to all institutions that have not acknowledged receipt of the survey packet.
The date of mailing a follow-up postcard is recorded in the institution response control screen. The receipt of follow-up postcards is handled in the same manner as the receipt of acknowledgment postcards.
Follow-up telephone calls begin 2 weeks after the follow-up postcards are mailed to the institutions. A control log listing is again generated to identify all institutions that have not acknowledged receipt of the survey packet (either through an acknowledgment postcard, telephone call, or a follow-up postcard). The data collection coordinator telephones nonrespondents while referring either to the log listing or the database. The survey packet is described to the respondent in sufficient detail to determine whether or not it has been received. In cases where the packet has not been received, the address is updated and a new packet is mailed. In cases where the previous year's respondent is no longer available, a new respondent is identified and a new packet is mailed. Institutions are telephoned until acknowledgment of the survey packet has been obtained. All contacts and mailings are recorded in the institution response control log.
A response control log is generated of all institutions that have not returned a completed survey questionnaire. Beginning 2 weeks after the survey due date, a second round of follow-up telephone calls begins that emphasizes responding to the survey rather than acknowledging its receipt. These calls are handled in the same manner as the first round, except that additional emphasis is placed on obtaining responses from the Top 100 institutions. All contacts are recorded in the institution response control log.
Respondents who indicate that they lack sufficient time to properly complete the survey are offered a deadline extension. Respondents who show reluctance to participate are reminded of the national scope of this effort and its consideration by various decision making bodies, including Congress. During a sample survey year, respondents are informed that the survey is limited to a sample, thus making each institution's participation even more crucial. Respondents who continue to have difficulty in finding the time to complete the survey or are relunctant to participate are asked to provide only data for item 1--current fund expenditures for separately budgeted R&D in the sciences and engineering, by source of funds. Respondents who still refuse to participate are thanked for their time and informed that NSF personnel may contact them at a future date.
Response control logs are generated periodically to help identify those institutions requiring additional contact. The data collection coordinator maintains periodic telephone contact with nonrespondent institutions until completed survey questionnaires are received.
Beginning in March the NSF project officer is advised of the status of nonrespondent institutions through monthly memoranda. As mentioned earlier, the project officer receives weekly reports on the status of data collection. At the end of each month, the project officer receives an up-to-date log report, a listing of respondents and their address information, and a database status reports showing institution response status.
The receipt of a completed survey questionnaire and any accompanying comments are recorded in the institution response control screen. All survey forms are reviewed by the data collection coordinator in preparation for data entry. When questionnaires are returned with incomplete data, the coordinator determines whether imputation of any missing data cells would be required. Cells requiring imputation are marked in red with a "B" status code. When respondents report in dollars rather than thousands of dollars, the data collection coordinator revises these figures appropriately. Respondents occasionally report negative numbers (i.e., credits to accounts), which are acceptable only for survey item 3. If negative numbers are reported for other items, the data are adjusted proportionately to correct the error without altering the subtotal/total figures reported. The revised figures are recorded in red on the questionnaire and assigned an "E" status code to indicate that they are estimates. Each original survey form is filed in survey folders that are maintained for each institution.
The final step in preparing for data entry is to change the status of institution records from "awaiting response" to "awaiting processing." For the actual data entry, copies of institution records awaiting processing are sometimes exported (copied) to a diskette or a network drive. This allows personnel to perform data entry at a remote computer while the data collection coordinator continues data processing on the main survey database. Upon completion of data entry the institutions' records are imported (copied) back to the database.
The institution response status after data entry becomes one of the following:
The monthly database status report provides information on institution response status and compares data on source of funds for each institution for the current and the previous year.
"Edit letters" are generated for all institutions that have arithmetic errors and/or trend warnings. These letters thank the respondent for participating in the survey, explain the facsimile, provide a brief explanation of discrepancies in the data, and request revision of data for current and/or previous years. Edit letters and facsimiles are assembled into "edit packets" and mailed to the respondents. Mailing dates are recorded in the response control logs, and copies of the edit packet materials are stored in the survey folders maintained for the institutions.
Prior to mailout, all edit packets are reviewed by the NSF project officer and the data collection coordinator. Error messages and trend warning messages are reviewed. Facsimile messages that represent the most significant reporting problems receive individual notes in red explaining the discrepancy in detail. On occasion the data collection coordinator corrects or verifies the data for institutions that have minor errors (usually rounding errors) or minor trend warnings. Altered data cells receive an "E" status code in the database to indicate that they are estimates, and the response control logs are updated to reflect that the data were corrected or verified. In these cases the institutional respondents are usually contacted by telephone, informed of the changes, and provided an opportunity to adjust the data.
Respondents who do not return their edit packets within 2 weeks of mailing are identified through periodically generated logs reports. These respondents are contacted by telephone to resolve data errors and explain trend warnings.
The data collection coordinator records the receipt of edit packets in the response control screens, along with any comments. A new facsimile is generated after all data revisions are made to the institution's screens in the database. If the institution's status code is awaiting imputation or clean, a facsimile is mailed to the institution and a copy of the data revision is filed. When a returned edit packet does not change the response status to awaiting imputation or clean, the respondent is telephoned for additional data revision or explanation.
To ensure quality control, preliminary ranking tables are produced several weeks prior to close-out and reviewed by the data collection coordinator and the NSF project officer. These tables provide several years of data by source of funds and by the major fields of science and engineering. The ranking tables provide an added check for irregularities or unusual trend changes in reported data.
Approximately two weeks before survey close-out, the NSF project officer visits QRC to review any major trend fluctuations in institutional data. All institutions that are still are reviewed. The NSF project officer and the data collection coordinator develop estimates--marked with an "E" status code--correcting or verifying these institutions' data. If an edit packet is received for one of these institutions prior to close-out, the estimates are replaced by the new data.
The survey questionnaires received from the institutions are stored in each institution's file folder. The questionnaires are retained for 2 additional survey years, after which time they are discarded.