APPENDIX A

RESEARCH METHODOLOGY

APPENDIX A: RESEARCH METHODOLOGY

The sections that follow describe the research methodology used for the National Science Foundation Principal Investigator FY 2001 Grant Award Survey and for the National Science Foundation Institutional FY 2001 Grant Award Survey.

A. QUESTIONNAIRE DEVELOPMENT

The initial phase of questionnaire development included two focus groups with NSF representatives who could identify key issues to be included in the two questionnaires. A third focus group with institutional representatives was scheduled for September 2001, however the events of September 11 resulted in a cancellation. Instead institutional representatives were contacted by telephone to discuss key issues to be included in the survey. After draft questionnaires were developed, they were cognitively pretested with PIs and institutional representative, and revisions were made based on the findings from the pretests. The following provides details about the steps that were followed:

Date Type of Group Number of Participants

August 8, 2001 NSF Focus Group 12

August 9, 2001 NSF Focus Group 11

October 2001 Institutional Representatives 4

(Telephone interviews)*

December 4, 2001 Principal Investigators

Cognitive pretest/group discussion 8

January/February Institutional Representatives 4

2002 Cognitive pretest/individual

interviews

*Re-scheduled from the Federal Demonstration Project Group discussion because of September 11,2001.

B. PROCEDURES FOR PRETEST WITH PRINCIPAL INVESTIGATORS

Eight PIs of a sample of 30 potential respondents participated in the pretest for the Principal Investigator FY 2001 Grant Award Survey. The sample was randomly selected from a total of 156 PIs throughout New Jersey representing a variety of grant types and award sizes. We decided to limit the sample selection to New Jersey because we assumed that MPR’s Princeton office in New Jersey would make it easier for the respondents to participate.

Respondents were asked to complete the draft questionnaire and comment on the questions. When respondents had difficulty understanding a question, MPR reworded the question or divided it into parts to make it more understandable. MPR also added some probes to better focus respondents on questions. Because participants voiced concerns about the amount of time it took to complete the questionnaire, the length of the questionnaire was reduced. Also, feedback about the focus of questions was implemented into a revised questionnaire. In particular, the concept “fully enabled” was discussed and rejected by the group. A preferred concept to describe the goals was “ongoing research and educational activities.”

The final questionnaire was programmed into a Web format to be conducted as a Computerized Self-Administered Questionnaire (CSAQ). Extensive testing was conducted on the Web questionnaire to insure compatibility with a wide range of different computers and servers that would be accessing the questionnaire.

C. SAMPLE APPROACH

1. Principal Investigator Survey

The universe for the PI survey comprises all 6,180 FY 2001 NSF award grantees. NSF decided to collect data from the universe of PIs instead of a sample to ensure that the most robust information. Since the primary mode of data collection is the World Wide Web, the additional costs associated with using the universe, instead of a sample, were minimal. In addition, examining the universe eliminates both the additional costs needed to develop a sampling plan and the potential sampling bias associated with sampling plans.

2. Institutional Survey

The universe for the institutional survey comprises all 582 institutions where at least one PI received an NSF award in FY 2001. Each institution in the universe was mailed a questionnaire and afforded the opportunity to participate. However, a sample of 100 institutions was drawn from the universe, based on institutional size and type (for example, private research institution, academic institution), the number of grants received, the type of grants received, and the institution’s geographic region.

The sampling design is based on the purpose and analytical objectives of the study. The purpose of this study is to determine the burden of the grant awards on institutions receiving grants from NSF. The analytic objective is to investigate the burden of the grant awards using both institution-level and grant-level measures. Therefore, there is an interest in both the estimate of the proportion of institutions that have a level of burden and the estimate of the average burden per grant for specific types of grants or type of institutions. The sampling design accounts for these two analytical objectives, which indicate somewhat different designs. A stratified random sample of institutions was selected that included an over sampling of institutions with a larger number of grants.

The number grant awards per institution is highly skewed with 40 percent of institutions (233) receiving one award and 16 institutions receiving in aggregate more than 1,500 awards. To account for both analytical objectives, sampling strata were developed that permit an over sample of the institutions with the greatest number of awards, and allocate a sufficient number of sampled institutions to the strata of the institutions with one or only a few awards. Within each stratum, a sample of institutions with equal probability and without replacement were selected. A larger initial sample was selected and then partitioned into random sub samples called waves. Some waves were released for data collection at the start of the fielding period and others were held in reserve. Three reserve waves were released because of institutions on the original data base that NSF determined to be ineligible. At the end of the data collection, sampling weights were applied to the final data file based on the inverse of the selection probabilities and computed adjustment to compensate for non-response among sampled institutions.

The following provides a description of the universe and the sampling frame, the sampling design, sample allocation, and expected precision from the sample.

a. Description of the Universe

The target population and the universe for this study is a listing of current recipients of grant awards by NSF. The population includes 582 institutions receiving a total of 6,180 grants, an average of 10.6 grants per institution. In total, 440 institutions (75 percent) received 9 or fewer grants with 233 (40 percent) institutions receiving one award and 85 (15 percent) institutions receiving two awards. On the other hand, 16 institutions (2.7 percent) accounted for 1,523 (25 percent) of the grant awards.

3. Sampling Design and Allocation

The analytical objectives indicate two variations on a stratified sampling design. For institution-level survey estimates, the sampling design that can offer smallest sampling variance is an equal probability sample of all institutions. For grant-level measures of the burden of the grant awards, the sampling design offering smallest sampling variance has the institutions selected with probability proportional to the number of grant awards. The sampling approach that offered a reasonable comprise between these two designs.

A classical process to develop sampling strata that account for the “size” (in this case, the number of awards at the institution) of a sampling unit is to use the square root of the size factor and partition a list of sampling units into strata so that the aggregate value of the square root of the size factor for institutions in each strata is equal (see Cochran 1997 for the “cumulative square root of f rule”).[1] Using the cumulative square root of f rule, estimates of totals (in this situation grant awards) is improved over an equal probability sample of institutions. For example, if 5 sampling strata are desired, the cumulative square root is summed over all units and then divided by 5. This value is used to identify the units that are assigned to each stratum. In developing the strata, there was a slight modification of this procedure to achieve better precision for institution-level estimates.

The proposed sample size is 100 institutions. The precision available from a sample of 100 units is assessed by using an estimate of an institution-level proportion around 0.50. The estimated half-width of a 95 percent confidence interval is 0.098, that is an interval of .402 to .598 (see Table B.1). Using the cumulative square root of the frequency (f) rule, we looked not only at the square root but also the cube root. When the finite population correction is accounted for, using the cumulative square root of f rule, resulted in a half-width of a 95 percent confidence interval of 0.115, whereas using the cumulative cube root of the frequency, resulted in a half-width of a 95 percent confidence interval of 0.100. That is, the use of the cube root can achieve nearly the precision of a simple random sampling of all institutions, but includes over sampling of the institutions with the largest number of grants. Increasing the number of strata beyond 3 had only a slight effect on the precision, and the plan was to use 5 strata for operational ease. For grant-level estimates, the level of precision is based on the correlation between the number of grant awards at an institution and the outcome measures. The anticipated precision will be as good and most likely better than will be available for the institution-level estimates.

In summary, for the institution survey there was a stratified random sample of institutions using 5 strata for respondent sample of 100 institutions. The sampling strata were developed to achieve good precision for both institution-level estimates and grant-level estimates.

TABLE B.1

SAMPLE ALLOCATION AND STRATA FOR INSTITUTION SAMPLE

	Number of Institutions
Strata	Sample Size	Equal Size Strata	Square Root Algorithm	Cube Root Algorithm
1	20	116	269	197
2	20	116	154	159
3	20	116	79	106
4	20	117	47	70
5	20	117	33	50
Half-Width of 95% Confidence Interval		0.098	0.115	0.100

Source: Mathematica computations.

Note: Half-width of 95% confidence interval = 1.96 * %variance for a stratified random sample where the variance within a stratum is computed from p * (1 – p) with p =0.50.

D. DATA COLLECTION

The PI survey was conducted using a mixed-mode format of Web and mail methods and the institution survey was a mail survey. A database containing contact information (telephone numbers and e-mail addresses) for potential respondents was provided to MPR by NSF.

The following provides additional detail of the data collection steps that were taken:

January 2001	NSF Director Dr. Rita R. Colwell sends PIs e-mail message announcing the survey.
January 30, 2002	MPR begins sending PI e-mail invitations with Web site access username and password on a rolling schedule.
February 4-19, 2002	MPR sends e-mail remiders to non-responders on a 3 day schedule.
February 15, 2002	MPR sends questionnaire mail packets to 778 PIs who have responded to the Web questionnaire.
March 8, 2002	Deadline for data collection

Original PI grants in NSF data file	6,180
PIs with multiple grants randomly selected a single grant for the survey (375) or questionable grant information (12)	5,793
Total completes and partials	5,221
Cases screened out during quality assurance process for criteria such as inconsistent grant award or duration information	232
Total cases used for analysis	4,989

A tracking system was developed to monitor participation. Figure A-1 illustrates the PI participation in the Web mode of the questionnaire. A total of 778 mail packets were sent to insure participation from PIs who may not have had Web access or would prefer to complete the questionnaire on paper.

The institutional survey was a mail only survey that used an e-mail approach to identify the most appropriate institutional participant. The data collection process was as follows:

January 2001	NSF Director Dr. Rita R. Colwell sends institution presidents an e-mail message announcing the two surveys.
January 24, 2002	MPR sends e-mail messages to institution contact people identified on the NSF data file to identify the appropriate person to participate in the survey.
February 15-March 6, 2002	Questionnaire mail packets are sent as institutional representatives contact information is identified.
March 8-30, 2002	MPR contacts non-responders in the institution sample by phone and e-mail.
March 30, 2002	All data collection is completed.

Total institutions with 2001 NSF grant recipients	582
No contact information	60
Total number with contact information	471(total); 105 (sample)
Total questionnaires returned	369 (total); 95 (sample)
Questionnaires not acceptable after quality assurance	359 (total); 95 (sample)

E. INSTITUTIONAL SURVEY ESTIMATES OF STANDARD ERROR

As described in Section D, the results from the institution survey are based on a sample, not a census of all institutions. Therefore, the results discussed in the report have standard errors. The estimates of the standard error for the key items included in the analysis are on Table A-1.

F. PRINCIPAL INVESTIGATOR SURVEY MEAN CALCULATIONS

The report includes information about means that are calculated in two different ways. There are means that are calculated for a single question in the PI questionnaire or for a single item of information from the NSF FY 2001 grant data files. In addition, there are means that have been calculated using measures constructed from either two items in the survey data or using a combination of questionnaire items and items from the NSF FY 2001 grant data file. The means for these constructed variables are calculated by taking the individual PI information for the included items, doing the calculation for each individual PI, and then getting an average. The following describes the information that is based on means calculated from multiple items. Appendix G has the central tendency distributions for these constructed variables.

CONSTRUCTED VARIABLES	CALCULATION AND DATA SOURCE
Option 1: Award Efficiency and Effectiveness Deviation from Requested Award Amount	(FY 2001 Award Request – FY 2001 Award Amount)/Number of FY2001 Grant Award Years (Information from NSF data file)
Option 2: Award Efficient and Effectiveness Percent of Research Being Funded	(FY 2001 Award Amount/(Q3.2¸100)-FY2001 Award Amount) Divided by 5 Years to annualize (NSF information and survey question)
Option 4: Award Efficient and Effectiveness NFS’s Contribution	Q3.3 X Q3.4 Divided by 5 Years to Annualize (Survey questions)
Difference in FY 2001 Award Amount Request and Amount Awarded	FY 2001 Amount Request-FY 2001 Amount Award (NSF data file)
Difference in FY 2001 Duration Request and Duration Award	FY 2001 Duration Request-FY 2001 Duration Award (NSF data file)
Additional Duration Needed	FY 2001 Duration Award + Q3.1 (NSF data file and survey question)

G. SURVEY MEASUREMENT ERROR

It should be noted that in any survey there are sources of both sampling and non-sampling error. Some examples of sources of survey measurement error are non-response to the survey, skipped questions, context effects, data collection methodology, and question wording. In conducting this study, all efforts possible were taken to minimize survey measurement error.

APPENDIX B

ANNOTATED QUESTIONNAIRES

APPENDIX B CONTENTS

A. NATIONAL SCIENCE FOUNDATION PRINCIPAL INVESTIGATOR 2001 GRANT AWARD SURVEY

B. NATIONAL SCIENCE FOUNDATION INSTITUTIONAL SURVEY

OMB Approval Number: 3145-0185

Welcome to the

National Science Foundation

Principal Investigator

2001 Grant Award Survey

BY MAIL	Questions or Comments?
TO: Matt Mishkind Project Director Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543	Contact Matt Mishkind at 877-236-4185 or E-mail: nsfgrantsweb@mathematica-mpr.com

2001 NSF GRANT INFORMATION

#1	Grant Title	__________________________________
#2	Grant Effective Date	__________________________________
#3	Requested Amount	__________________________________
#4	Awarded Amount	__________________________________
#5	Amount Change 5% or Greater	__________________________________
#6	Requested Duration	__________________________________
#7	Awarded Duration	__________________________________
#8	Duration Change 1 Year or Greater	__________________________________

· You will be asked to reference the information listed above throughout this questionnaire. This information is from our database and is specific to the NSF grant you were awarded funding in 2001.

· When a question asks you to think about any of the above information, a notation will be made in the questionnaire. Therefore, it is important to keep this information attached to the rest of the questionnaire.

· If this is your grant, please check the box and begin the questionnaire.

· If any of this grant information is incorrect, please contact Matt Mishkind at 877-236-4185 or nsfgrantsweb@mathematica-mpr.com before you complete the questionnaire.

· You may also complete this questionnaire on the Web:

Log onto

http://nsfgrants.mathematica-mpr.com

and enter the following

USERNAME: xxxxxx

PASSWORD: xxxxxx

PRINCIPAL INVESTIGATOR

2001 GRANT AWARD SURVEY

SECTION 1

REMINDER: Please check grant information provided on back of cover page.

1.1 Was your 2001 NSF grant [#1 GRANT TITLE] awarded on [#2 GRANT EFFECTIVE DATE] a first‑time submission or a revision of a previously declined NSF proposal?

· A revised proposal does not refer to changes made in your 2001 NSF grant proposal after the initial review

mark one

71% a first time submission

29% a revision of a previously declined NSF proposal

1.2 NSF research grants can be classified along a number of different dimensions. Which ONE of the following definitions best describes the research that is funded by this grant?

· If your work involves several of these categories please choose the one that is most appropriate

THEORETICAL research can be accomplished with minimal physical resources beyond the investigator’s institutional research library, computing capability and office space.

LABORATORY research requires an equipped laboratory, for example, research often found in chemistry, biology or engineering university laboratories requiring research and/or testing equipment, plumbing.

FIELD research requires fieldwork, specimen collection, sample survey, location of sensors, etc. away from the principal investigator’s institution, for example, some science activities in geosciences, biology, social sciences.

mark one

37% Theoretical Research

44% Laboratory Research

18% Field Research

1.3 Does your 2001 NSF project require the use of a national or international research facility such as access to an accelerator, a light source, a ship, major telescope or supercomputer center?

16% Yes

83% No

1.4 In general, would you say that this 2001 NSF grant is funding:

mark one

7% A specific product or deliverable

89% A project that is part of your ongoing body of research and educational activities

4% Other (Please Describe)

1.5 For each of the following, how much advice did you get from NSF staff when you were preparing your grant proposal:

		MARK ONE FOR EACH
		A Great Deal	Some	Not Much	None At All
a.	The amount of funding.............	12%	27%	17%	43%
b.	The duration of the grant proposal.................................	11%	21%	16%	51%
c.	The substance or focus of the grant......................................	7%	25%	19%	49%

1.6 Based on the advice provided by NSF staff, did you increase, not change, or decrease:

		Increase		Not Change	Decrease	Not Asked
a.	The amount of the award you proposed.......................................		4%	36%	27%	31%
b.	The award duration you proposed....		3%	58%	6%	31%

SECTION 2

As part of the review process, NSF may ask principal investigators to revise their proposal before they are awarded funding. The following questions are about your revised budget and award duration.

ONLY ANSWER Q2.1 IF #5 AMOUNT CHANGE >5% IS LABELED “YES.” See inside cover.

2.1 In your proposal, you requested [#3 REQUESTED AMOUNT] and in your award you received [#4 AWARDED AMOUNT].

Overall, how much will this change in the award amount impact your ability to do what you expected to accomplish with this 2001 NSF grant?

mark one

1% Can do a great deal more than expected

2% Can do somewhat more than expected

7% Can do about the same as expected

28% Can do somewhat less than expected

15% Can do a great deal less than expected

1% Don’t know

47% Not asked

ONLY ANSWER Q2.2 IF #8 DURATION CHANGE >1 YEAR IS LABELED “YES.” See inside cover.

(IF BOTH #5 AMOUNT CHANGE >5% AND #8 DURATION CHANGE >1 YEAR ARE LABELED “NO,” PLEASE SKIP TO SECTION 3).

2.2 In your proposal, you requested [#6 REQUESTED DURATION] and in your award you received [#7 AWARDED DURATION].

Overall, how much will this change in award duration impact your ability to do what you expected to accomplish with this 2001 NSF grant?

mark one

1% Can do a great deal more than expected

1% Can do somewhat more than expected

1% Can do about the same as expected

4% Can do somewhat less than expected

5% Can do a great deal less than expected

88% Not asked

IF YOU RESPONDED AS 6 “CAN’T ANSWER” OR -1 “DON’T KNOW” TO BOTH Q2.1 AND Q2.2, PLEASE SKIP TO SECTION 3.

ONLY ANSWER Q2.3 IF YOU PROVIDED A RESPONSE OF 1, 2, 3, 4, OR 5 TO EITHER Q2.1 OR Q2.2.

2.3 The following are some possible consequences of the changes in your NSF award funding and/or duration. Will this change have a positive impact, no impact, or negative impact on your ability to ...

		Positive Impact	No Impact	Negative Impact	Not Applicable	Not Asked
A. Goals and Objectives
a.	Pursue innovative ideas.............................................................	4%	25%	23%	1%	47%
b.	Pursue high-risk ideas..............................................................	3%	20%	26%	4%	47%
c.	Obtain other funding.................................................................	6%	35%	6%	5%	47%
B. Applications and Outcomes
d.	Disseminate research findings...................................................	4%	32%	17%	1%	47%
e.	Develop instrumentation or other enhancements for the research and education infrastructure......................................................	2%	17%	20%	13%	47%
f.	Develop partnerships with industry, other educational institutions, or national laboratories..............................................................	3%	26%	13%	11%	47%
g.	Integrate research activity into your teaching and training.............	4%	27%	18%	4%	47%
h.	Nurture connections between research activity and its potential for: health benefits, economic benefits, and national security benefits...................................................................................	2%	24%	10%	16%	47%
i.	Develop programs with K‑12 teachers and/or students.................	1%	23%	7%	22%	47%
j.	Improve public understanding of the project.................................	3%	31%	11%	9%	47%
C. Process and Team Building
k.	Collaborate with researchers in your area of research..................	5%	21%	26%	1%	47%
l.	Broaden participation of under-represented groups in the research activity....................................................................................	3%	27%	18%	6%	47%
m.	Collaborate with researchers in different areas of research............	4%	25%	21%	3%	47%
n.	Achieve the research objectives within the specified time.............	4%	14%	34%	1%	47%
o.	Obtain quality personnel...........................................................	3%	17%	28%	4%	47%
p.	Establish mentoring or other research-based education activities..	3%	23%	21%	5%	47%
D. Research Tools							4%	47%
q.	Access state-of-the-art equipment.............................................	2%	28%	17%	6%	47%
r.	Access facilities.......................................................................	2%	34%	10%	6%	47%

SKIP Q2.4a IF NO POSITIVE ITEMS IN Q2.3

2.4a Among the items you marked “Positive Impact,” please rank order (write in the number(s)), up to three, those that had the most positive impact.

#1

#2

#3

SKIP Q2.4b IF NO NEGATIVE ITEMS IN Q2.3

2.4b Among the items you marked “Negative Impact,” please rank order (write in the number(s)), up to three, those that had the most negative impact.

#1

#2

#3

2.5 Please describe any other impact(s) that resulted from the change in your 2001 NSF award or give more details on any in the list that need further explanation.

SECTION 3

The next group of questions is your assessment of how this grant fits into your ongoing body of research and educational activities.

· Our records indicate that your 2001 NSF grant is for $[#4 AWARDED AMOUNT] over a period of [#7 AWARDED DURATION] Years. See inside cover.

3.1 Thinking about the timeframe for your ongoing body of research and educational activities, about how many additional years do you think you would need to accomplish your key goals?

· DO NOT include the years for the 2001 NSF grant

· Enter “0” for “Do not need any additional years”

Median=2

Mean=3

Mode=2

Range: 0 to 40

3.2 If you think about your ongoing body of research and educational activities as 100 percent of what you’d like to accomplish in the next five years, about what percent of what you’d like to do will be achieved with your 2001 NSF research grant?

Median=30

Mean=37

Mode=20

Range: 0 to 100

Now, speculate on what changes, if any, you would need to accomplish all you would like to in the next five years.

3.3 In the next five years, how much additional funding from all sources, if any, would you need to achieve what you would like to with your ongoing body of research and educational activities?

· Exclude funding you currently have for this NSF grant and from any other funding sources

· Enter “0” for “Do not need any additional funding”

Median=$500,000

Mean=$1,149,000

Mode=$500,000

Range: $0 to $300,000,000

IF YOU DO NOT NEED ADDITIONAL FUNDING OR DON’T KNOW, SKIP TO Q3.6.

3.4 What percent of this additional amount do you think is appropriate for NSF to fund?

Median=70%

Mean=67%

Mode=100%

Range: 0% to 100%

3.5 About how many additional grants do you think you would need to get this funding?

Median=2

Mean=2.39

Mode=2

Range: 0 to 32

ONLY ANSWER Q3.6 IF YOU NEED ADDITIONAL YEARS (Q3.1) AND/OR ADDITIONAL FUNDING (Q3.3).

IF YOU RESPONDED “0” OR “DON’T KNOW” TO Q3.1 AND Q3.3, SKIP TO SECTION 4.

3.6 If NSF provided this additional funding and/or duration to support your ongoing research and educational activities, would there be a positive impact, no impact, or a negative impact on each of the following:

		Positive Impact	No Impact	Negative Impact	Not Applicable	Not Asked
A. Goals and Objectives
1.	Pursue innovative ideas.............................................................	87%	2%	<1%	<1%	10%
2.	Pursue high-risk ideas..............................................................	76%	9%	<1%	4%	10%
3.	Obtain other funding.................................................................	54%	26%	5%	3%	10%
B. Applications and Outcomes
4.	Disseminate research findings...................................................	74%	14%	<1%	1%	10%
5.	Develop instrumentation or other enhancements for the research and education infrastructure......................................................	61%	16%	<1%	13%	10%
6.	Develop partnerships with industry, other educational institutions, or national laboratories..............................................................	62%	19%	<1%	8%	10%
7.	Integrate research activity into your teaching and training.............	73%	13%	<1%	2%	10%
8.	Nurture connections between research activity and its potential for: health benefits, economic benefits, and national security benefits...................................................................................	48%	24%	<1%	16%	10%
9.	Develop programs with K‑12 teachers and/or students.................	32%	36%	<1%	20%	10%
10.	Improve public understanding of the project.................................	58%	25%	<1%	5%	10%
C. Process and Team Building
11.	Collaborate with researchers in your area of research..................	83%	6%	<1%	<1%	10%
12.	Broaden participation of under-represented groups in the research activity....................................................................................	62%	23%	<1%	3%	10%
13.	Collaborate with researchers in different areas of research............	76%	12%	<1%	2%	10%
14.	Achieve the research objectives within the specified time.............	83%	6%	<1%	1%	10%
15.	Obtain quality personnel...........................................................	76%	9%	<1%	3%	10%
16.	Establish mentoring or other research-based education activities..	71%	14%	<1%	3%	10%
D. Research Tools
17.	Access state-of-the-art equipment.............................................	60%	22%	<1%	7%	10%
18.	Access facilities.......................................................................	49%	32%	<1%	8%	10%

SKIP Q3.7a IF NO POSITIVE ITEMS IN Q3.6

3.7a Among the items you marked “Positive Impact,” please rank order (write in the number(s)), up to three, those that had the most positive impact.

#1

#2

#3

SKIP Q3.7b IF NO NEGATIVE ITEMS IN Q3.6

3.7b Among the items you marked “Negative Impact,” please rank order (write in the number(s)), up to three, those that had the most negative impact.

#1

#2

#3

3.8 Please describe any other impact(s) that would result if NSF provided you what you need for what you want to accomplish, or give more details on any in the list that needs further explanation:

3.9 If you received this additional funding and/or duration from NSF that you need for your ongoing body of research and educational activities, how likely would you increase each of the following?

		MARK ONE FOR EACH
		Very Likely	Somewhat Likely	Neither Likely Nor Unlikely	Somewhat Unlikely	Very Unlikely	Not Applicable	Not Asked
Personnel
1.	The number and/or months of senior personnel.....................................	29%	21%	13%	7%	13%	4%	10%
2.	The number and/or months of post doctoral associates........................	43%	22%	7%	4%	5%	7%	10%
3.	The number and/or months of technicians..............................................	15%	14%	14%	7%	16%	19%	10%
4.	The number and/or months of programmers..........................................	7%	11%	14%	6%	19%	24%	10%
5.	The number and/or months of graduate students...................................	70%	11%	2%	1%	1%	3%	10%
6.	The number and/or months of undergraduate students..........................	45%	26%	7%	3%	3%	4%	10%
Equipments
7.	The number of equipment purchases.....................................................	33%	31%	11%	4%	5%	5%	10%
8.	The quality of equipment purchases........................................................	28%	22%	20%	5%	7%	7%	10%
Travel
9.	The number of trips..................................................................................	29%	32%	18%	5%	4%	1%	10%
10.	The cost per trip.......................................................................................	3%	8%	36%	11%	29%	2%	10%
Experiments
11.	The number of experiments, tests, subjects...........................................	49%	14%	5%	1%	1%	20%	10%
12.	The size of the experiments or tests.......................................................	30%	17%	14%	3%	3%	22%	10%
13.	The quality of the experiments or tests....................................................	36%	13%	13%	2%	4%	20%	10%
Other Direct Costs
14.	Participant support...................................................................................	21%	18%	19%	4%	10%	16%	10%
15.	Consultant services.................................................................................	6%	10%	19%	7%	23%	22%	10%
16.	Computer/Publication costs....................................................................	16%	28%	21%	7%	11%	5%	10%
17.	Other (Please Specify)............................................................................
27% gave a response

3.10 Among the items you are “Very Likely” to increase, rank order (write in the numbers), up to three, those that would have the most impact on what you want to accomplish.

#1

#2

#3

3.11 And, if you received this additional funding and/or duration from NSF that you need for your ongoing research and educational activities, would your ability to do each of the following be:

		MARK ONE FOR EACH
		Increased A Great Deal	Increased Somewhat	About the Same	Decreased Somewhat	Decreased A Great Deal	Not Applicable	Not Asked
a.	Recruit post doctoral associates.......................................	47%	23%	8%	<1%	<1%	9%	10%
b.	Recruit graduate students.................................................	56%	23%	6%	<1%	<1%	4%	10%
c.	Recruit undergraduate students........................................	27%	31%	24%	<1%	<1%	6%	10%
d.	Provide adequate support for a graduate student to shorten time to degree.................................................................	29%	26%	25%	<1%	<1%	8%	10%
e.	Provide stability for technicians........................................	17%	15%	13%	<1%	<1%	42%	10%
f.	Provide stability for programmers......................................	8%	9%	14%	<1%	<1%	57%	10%
g.	Conduct more experiments, tests or subjects.....................	42%	22%	5%	<1%	<1%	19%	10%
h.	Have higher-quality experiments or tests............................	31%	22%	16%	<1%	<1%	20%	10%
i.	Duration of experiments...................................................	17%	19%	24%	1%	1%	26%	10%
j.	Other (Please Specify)....................................................

29% gave a response

3.12 Thinking about all the different aspects of what you would like to accomplish, which of the following would have the greatest impact on your ongoing body of research and educational activities:

mark one

54% More funding

35% Longer duration

10% Not asked

SECTION 4

The following are questions about NSF funding and your general field of research.

4.1 In your opinion, if NSF increased the funding and the duration of the awards in your field of research, how likely would these changes …?

		MARK ONE FOR EACH
		Very Likely	Somewhat Likely	Neither Likely Nor Unlikely	Somewhat Unlikely	Very Unlikely	Not Applicable
a.	Widen the focus of the research in your field......	63%	28%	7%	1%	1%	<1%
b.	Increase the number of proposals to NSF with innovative ideas.................................................	46%	35%	13%	3%	1%	1%
c.	Increase the number of proposals to NSF with high-risk ideas..................................................	37%	38%	17%	4%	2%	2%
d.	Attract more established researchers to apply for NSF funding......................................................	37%	31%	23%	4%	2%	2%
e.	Decrease the amount of time to answer research questions..........................................................	31%	32%	20%	6%	6%	4%
f.	Attract more graduate students...........................	65%	26%	6%	<1%	<1%	2%
g.	Attract better graduate students..........................	62%	25%	8%	1%	<1%	2%
h.	Improve access to facilities and databases..........	36%	34%	20%	1%	1%	7%
i.	Decrease interruptions in funding.......................	70%	23%	4%	<1%	<1%	1%

4.2 If NSF had more money to award each year, please rank in descending order of importance from (1) most important to (3) least important, the following possible actions for awards in your area of research:

rank order 1^st 2^nd 3^rd

Increase only the amount of funding per award................................... 40% 36% 23%

Increase only the length of time per award.......................................... 24% 37% 38%

Increase only the total number of awards per year.............................. 36% 26% 37%

SECTION 5

This section asks about your experience preparing this NSF grant and about some other research experiences.

5.1 Thinking about the proposal you submitted to NSF for this grant, what is your best estimate of the total hours of preparation for submitting this proposal?

In determining your estimate, please make sure you:

· consider all of your own time for writing the proposal, preparing the budget, completing forms, and consulting with others about your proposal

· consider the time other people such as graduate assistants, secretaries, and budget administrators put into the preparation of this proposal

· DO NOT include any institutional personnel who might review or internally process your proposal such as staff from the sponsored research office

Median=100

Mean=157

Mode=100

Range: 1 to 9,000

5.2 What’s your best estimate of the percent of hours that were devoted to the intellectual content of the proposal and the percent devoted to the mechanics of proposal preparation?

· Your total must equal 100%

Preparation of intellectual content..........................

Median=75% Mean=68% Mode=80% Range: 5% - 100%

Mechanics of proposal preparation........................

Median=25% Mean=32% Mode=20% Range: 0% - 100%

5.3 How helpful is having an NSF research grant in obtaining funding from other sources?

mark one

39% Very helpful

33% Somewhat helpful

25% Neither helpful nor unhelpful

2% Somewhat unhelpful

<1% Very unhelpful

Now, think about any other funding you may be getting for your ongoing body of research and educational activities.

5.4 Right now, are you getting NSF funding for any other projects for your ongoing body of research and educational activities?

· This includes funding from grants on which you are a collaborator or subcontractor

· DO NOT include the 2001 NSF grant identified for this survey

44% Yes

55% No

1% Don’t know

5.5 Not including the 2001 NSF grant identified for this survey, what is the total number of current NSF grants funding your ongoing body of research and educational activities?

Median=1

Mean=2

Mode=1

Range: 0 to 236

5.6 What is the total amount of annual funding you currently have from these other NSF grants?

· DO NOT include the 2001 NSF grant identified for this survey

Median=$100,000

Mean=$207,000

Mode=$100,000

Range: $0 to $30,000,000

5.7 Did you divide your ongoing body of research and educational activities into several proposals and submit them to NSF?

38% Yes

62% No

Now, think about any non-NSF funding you are getting for your ongoing body of research and educational activities.

5.8 In addition to your NSF funding, do you currently have other funding for your ongoing body of research and educational activities?

· This may be funding from sources such as your institution, another federal agency, a state agency, a non-profit foundation, or a for-profit company or organization

72% Yes

27% No

<1% Don’t know

5.9 What is the total number of current non-NSF funding sources for your ongoing body of research and educational activities?

Median=2

Mean=2

Mode=1

Range: 0 to 420

5.10 And, what is the total amount of annual funding you have from non-NSF sources?

Median=$100,000

Mean=$199,000

Mode=$100,000

Range: $0 to $10,000,000

The next set of questions are about your research activities and professional duties.

5.11 What’s your best estimate of the percent of your time spent conducting research in each of the following ways:

· Your total must equal 100%

Work as part of a team with researchers from other disciplines..........

Median=10% Mean=14% Mode=0% Range: 0% - 100%

Work as part of a team including other senior investigators in the

same discipline...............................................................................

Median=20% Mean=25% Mode=20% Range: 0% - 100%

Work individually with students and post doctoral assistants................

Median=55% Mean=54% Mode=50% Range: 0% - 100%

Other (Please Specify)....................................................................

Median=0% Mean=6% Mode=0% Range: 0% - 100%

5.12 How many peer-reviewed articles have you published during the past 5 years where you have been the primary author?

Median=9

Mean=13

Mode=5

Range: 0 to 500

For the following question, please think about your current experience.

5.13 How many people in the following categories work with you on your current research projects?

	Type of Institution
Undergraduate Students Median=2 Mean=2 Mode=1 Range: 0 – 50	4% Non-Academic 5% Non-PhD 18% Other PhD 25% NSF Funding Top 20 26% NSF Funding Top 21-50 22% NSF Funding Top 51-100
Graduate Students Median=3 Mean=4 Mode=2 Range: 0 – 300
Post-doctoral fellows Median=1 Mean=1 Mode=0 Range: 0 – 100

5.14 Questionnaires by their nature are limited. Please write in any other comments you have about your experiences with the NSF grant process that you think are important.

Thank you very much for completing this questionnaire! Please return in the postage-paid envelope.

Sample Institution

OMB Approval Number: 3145-0185

National Science Foundation

Institutional Survey

Please Complete and Submit Questionnaire

By March 8, 2002

BY MAIL:	Questions or Comments?
TO: Matt Mishkind Project Director Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543	Contact Matt Mishkind at 877-236-4185 or E-mail: nsfgrants@mathematica-mpr.com

Thank you for participating in this study of institutional representatives who are responsible for applying for and administering National Science Foundation (NSF) grants. We know that your time is valuable and we greatly appreciate your assistance.

Dr. Colwell, Director of the National Science Foundation, sent a letter informing your institution about this study. Mathematica Policy Research, Inc. (MPR) is conducting this study for the National Science Foundation (NSF). To assist in their future planning, NSF is very interested in learning more about NSF grants from the perspective of the institutional representatives responsible for NSF grants.

Your participation is critical to the success of the study and to the quality of the information we get about NSF grants. If you have any questions about the background of the study you can contact Bob Abel at NSF (nsf-survey@nsf.gov). If you have any questions or require any assistance while you are completing the questionnaire, you may contact Matt Mishkind at MPR (877‑236‑4185/nsfgrants@mathematica-mpr.com).

CONFIDENTIALITY

All of your responses to the questionnaire are strictly confidential. We will not use your name or email for any other purposes than this study. All information from the study will be kept in a secure place. Only the MPR researchers directly working on the study will have access to this information. Any reports of the results of this study will be presented in the aggregate.

INSTRUCTIONS

As you answer some of these questions, you will focus on the NSF FY 2001 grant(s) awarded to your institution. It will include questions about the NSF proposals submitted by your institution and the NSF grants administered by your institution. For your convenience, a list of these grants is enclosed.

The process of applying for and administering NSF grants may vary from institution to institution. The purpose of this questionnaire is to get a general assessment of the resources your institution uses for this process. Please make sure the most informed person at your institution completes each section of the questionnaire. For some institutions, multiple people may need to respond.

Number of 2001 NSF grant awards Median=3 Mean=12 Mode=1

Number of 2001 NSF grant declines Median=7 Mean=32 Mode=1

The following questions focus on the proposal process at your institution.

1.1 Does your institution have a formal, standardized process that is followed to submit grant proposals?

· This is only your institution’s process for grant proposals, it does not refer to others such as NSF FastLane

98% Yes

2% No

1.2 Are there specific individuals or administrative offices assigned to work with grant proposals?

· Do not include principal investigators

99% Yes

1% No SKIP TO Q1.4a

1.3 What is the total number of each of the following assigned to grant proposals:

| | | individuals Median=4 Mean=6 Mode=3

| | | administrative offices Median=1 Mean=2 Mode=1

1.4a In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal process for grants.

· Do not include individual academic departments or research centers

1.4b For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal.

Administrative Office	Average Number of Hours Per NSF Grant Proposal
	Median=4 Mean=6 Mode=1
	Median=2 Mean=4 Mode=1
	Median=1 Mean=2 Mode=1
	Median=3 Mean=10 Mode=1
	Median=1 Mean=2 Mode=1

The following questions are about the process of negotiating grant proposal revisions.

2.1 Does your institution have a formal, standardized process that is followed to negotiate grant proposal revisions?

72% Yes

28% No

2.2 Are there specific individuals or administrative offices assigned to work with grant proposal revisions?

· Do not include principal investigators

87% Yes

14% No SKIP TO Q2.4a

2.3 What is the total number of each of the following assigned to grant proposal revisions:

| | | individuals Median=3 Mean=5 Mode=3

| | | administrative offices Median=1 Mean=2 Mode=1

2.4a In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal revision process for grants.

· Do not include individual academic departments or research centers

2.4b For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal revision.

Administrative Office	Average Number of Hours Per NSF Grant Proposal Revision
	Median=2 Mean=3         Mode=1
	Median=1 Mean=2 Mode=1
	Median=1 Mean=1 Mode=1
	Median=1 Mean=3 Mode=1
	Median=0 Mean=<1 Mode=1

2.5 For a typical NSF grant that your institution is awarded, approximately how many hours are spent communicating directly with NSF on revisions to the original proposal?

· Do not include principal investigator hours

| | | | average number of hours per nsf grant

Median=1 Mean=2 Mode=1

After a grant is awarded, institutions are responsible for administering the grant and providing additional oversight. For the following questions, please think about grant administration.

3.1 Are there specific individuals or administrative offices assigned to administer grant awards?

97% Yes

3% No SKIP TO Q3.3a

3.2 What is the total number of each of the following assigned to administer grants:

· Do not include principal investigators

| | | individuals Median=4 Mean=8 Mode=3

| | | administrative offices Median=2 Mean=2 Mode=2

3.3a In the grid below, please identify, up to five, the key administrative offices at your institution involved in administering grant awards.

· Do not include individual academic departments or research centers

3.3b For each office, please give your best estimate of the average number of hours individuals in that office spent to administer a typical FY 2001 NSF grant award.

Administrative Office	Average Number of Hours Per NSF Grant Administration
	Median=8 Mean=21 Mode=5
	Median=5 Mean=10 Mode=1
	Median=4 Mean=11 Mode=1
	Median=2 Mean=5 Mode=1
	Median=10 Mean=7 Mode=<1

3.4 Approximately how many hours are spent to complete and submit NSF required reports for a typical FY 2001 NSF grant?

| | | | average number of hours per nsf grant

Median=3 Mean=6 Mode=2

NSF is considering increasing the amount and duration available for grants. Think about how these potential changes would impact how your institution applies for and administers NSF grants.

4.1 If NSF had more money to award each year, please rank from most important (1) to least important (3), the following possible actions for awards to your institution.

mark one
Ranking
1	2	3
44%	38%	13%	Increase only the amount of funding per award
9%	23%	62%	Increase only the duration per award
50%	30%	17%	Increase only the total number of awards per year

4.2 Overall, if NSF made each of the following changes, would it increase, decrease, or not make any difference in the administrative time your institution uses to mange all aspects of NSF awards?

		Increase Time Needed		Decrease Time Needed	No Difference in Time Needed
a.	Increasing the amount of funding for NSF awards................................................................		12%	7%	81%
b.	Increasing the duration of NSF awards...........		42%	24%	33%
c.	Increasing the total number of NSF awards....		86%	--%	14%

4.3 In your opinion, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average dollar amount for each grant award?

4.4 Now, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average duration for each grant award?

4.5 Please outline any suggestions you have for NSF changes that would result in a reduction of the amount of time and resources used by your institution to manage NSF grants.

The following questions will provide an overview of the grants managed by your institution.

5.1 Thinking about all the grant awards managed by your office in FY 2001, approximately what percent is for NSF grants?

| | | | percent Median=10% Mean=16% Mode=10%

5.2 And, approximately what percent of the total dollar amount of all grant awards managed by your office in FY 2001, is for NSF grants?

| | | | percent Median=12% Mean=18% Mode=1%

Questionnaires by their nature are sometimes limited. Please write in any other comments you have about your institution’s experiences with the NSF grant process.

Thank you very much for completing this questionnaire. Please use the enclosed postage-paid envelope to return it to: Matt Mishkind, Mathematica Policy Research, Inc., P.O. Box 2393, Princeton, NJ 08543.

Type of Institution

11% Non-Academic

28% Non-PhD

44% Other PhD

5% NSF Funding Top 20

3% NSF Funding Top 21-50

9% NSF Funding Top 51-100

APPENDIX C

NONSAMPLE INSTITUTIONAL SURVEY INFORMATION

APPENDIX C

NONSAMPLE INSTITUTION SURVEY RESULTS

This section of the appendix provides a general overview of the results on the completed questionnaires from the 264 institutional representatives who were not selected as part of the scientific sample of institutions described in Appendix A. These results can be categorized as a convenience sample rather than a scientific sample. The results of the scientific institution sample discussed in the report text can be projected on the population of all FY 2001 institutions who had PIs that received NSF grants; the results from this convenience sample describes the responses from these 264 institutional representatives.

The results from these nonsample institutions follows in two forms: (1) an annotated questionnaire with the responses and (2) tables that have the percentages of responses from the open-ended questions. It should be noted that in Appendix G there is a table with the central tendency distributions for the nonsample institutions.

	Nonsample Institution

OMB Approval Number: 3145-0185

National Science Foundation

Institutional Survey

BY MAIL:	Questions or Comments?
TO: Matt Mishkind Project Director Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543	Contact Matt Mishkind at 877-236-4185 or E-mail: nsfgrants@mathematica-mpr.com

Thank you for participating in this study of institutional representatives who are responsible for applying for and administering National Science Foundation (NSF) grants. We know that your time is valuable and we greatly appreciate your assistance.

Dr. Colwell, Director of the National Science Foundation, sent a letter informing your institution about this study. Mathematica Policy Research, Inc. (MPR) is conducting this study for the National Science Foundation (NSF). To assist in their future planning, NSF is very interested in learning more about NSF grants from the perspective of the institutional representatives responsible for NSF grants.

Your participation is critical to the success of the study and to the quality of the information we get about NSF grants. If you have any questions about the background of the study you can contact Bob Abel at NSF (nsf-survey@nsf.gov). If you have any questions or require any assistance while you are completing the questionnaire, you may contact Matt Mishkind at MPR (877‑236‑4185/nsfgrants@mathematica-mpr.com).

CONFIDENTIALITY

All of your responses to the questionnaire are strictly confidential. We will not use your name or email for any other purposes than this study. All information from the study will be kept in a secure place. Only the MPR researchers directly working on the study will have access to this information. Any reports of the results of this study will be presented in the aggregate.

INSTRUCTIONS

As you answer some of these questions, you will focus on the NSF FY 2001 grant(s) awarded to your institution. It will include questions about the NSF proposals submitted by your institution and the NSF grants administered by your institution. For your convenience, a list of these grants is enclosed.

The process of applying for and administering NSF grants may vary from institution to institution. The purpose of this questionnaire is to get a general assessment of the resources your institution uses for this process. Please make sure the most informed person at your institution completes each section of the questionnaire. For some institutions, multiple people may need to respond.

Number of 2001 NSF grant awards Median=2 Mean=10 Mode=1

Number of 2001 NSF grant declines Median=9 Mean=27 Mode=1

The following questions focus on the proposal process at your institution.

1.1 Does your institution have a formal, standardized process that is followed to submit grant proposals?

· This is only your institution’s process for grant proposals, it does not refer to others such as NSF FastLane

94% Yes

5% No

1.2 Are there specific individuals or administrative offices assigned to work with grant proposals?

· Do not include principal investigators

96% Yes

3% No SKIP TO Q1.4a

1.3 What is the total number of each of the following assigned to grant proposals:

| | | individuals Median=3 Mean=5 Mode=2

| | | administrative offices Median=1 Mean=2 Mode=1

1.4a In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal process for grants.

· Do not include individual academic departments or research centers

1.4b For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal.

Administrative Office	Average Number of Hours Per NSF Grant Proposal
	Median=4 Mean=8 Mode=2
	Median=2 Mean=4 Mode=1
	Median=1 Mean=3 Mode=1
	Median=1 Mean=4 Mode=1
	Median=2 Mean=7 Mode=1

The following questions are about the process of negotiating grant proposal revisions.

2.1 Does your institution have a formal, standardized process that is followed to negotiate grant proposal revisions?

65% Yes

34% No

2.2 Are there specific individuals or administrative offices assigned to work with grant proposal revisions?

· Do not include principal investigators

85% Yes

14% No SKIP TO Q2.4a

2.3 What is the total number of each of the following assigned to grant proposal revisions:

| | | individuals Median=3 Mean=4 Mode=2

| | | administrative offices Median=1 Mean=2 Mode=1

2.4a In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal revision process for grants.

· Do not include individual academic departments or research centers

2.4b For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal revision.

Administrative Office	Average Number of Hours Per NSF Grant Proposal Revision
	Median=1 Mean=3 Mode=1
	Median=1 Mean=2 Mode=1
	Median=1 Mean=2 Mode=1
	Median=2 Mean=5 Mode=1
	Median=7 Mean=7 Mode=*	*No value calculated

2.5 For a typical NSF grant that your institution is awarded, approximately how many hours are spent communicating directly with NSF on revisions to the original proposal?

· Do not include principal investigator hours

| | | | average number of hours per nsf grant

Median=1 Mean=2 Mode=1

After a grant is awarded, institutions are responsible for administering the grant and providing additional oversight. For the following questions, please think about grant administration.

3.1 Are there specific individuals or administrative offices assigned to administer grant awards?

96% Yes

3% No SKIP TO Q3.3a

3.2 What is the total number of each of the following assigned to administer grants:

· Do not include principal investigators

| | | individuals Median=3 Mean=6 Mode=2

| | | administrative offices Median=2 Mean=2 Mode=2

3.3a In the grid below, please identify, up to five, the key administrative offices at your institution involved in administering grant awards.

· Do not include individual academic departments or research centers

3.3b For each office, please give your best estimate of the average number of hours individuals in that office spent to administer a typical FY 2001 NSF grant award.

Administrative Office	Average Number of Hours Per NSF Grant Administration
	Median=8 Mean=18 Mode=2
	Median=5 Mean=16 Mode=1
	Median=6 Mean=14 Mode=1
	Median=6 Mean=10 Mode=1
	Median=9 Mean=13 Mode=*	*No value calculated

3.4 Approximately how many hours are spent to complete and submit NSF required reports for a typical FY 2001 NSF grant?

| | | | average number of hours per nsf grant

Median=4 Mean=8 Mode=2

NSF is considering increasing the amount and duration available for grants. Think about how these potential changes would impact how your institution applies for and administers NSF grants.

4.1 If NSF had more money to award each year, please rank from most important (1) to least important (3), the following possible actions for awards to your institution.

mark one
Ranking
1	2	3
36%	39%	12%	Increase only the amount of funding per award
6%	28%	52%	Increase only the duration per award
46%	20%	21%	Increase only the total number of awards per year

4.2 Overall, if NSF made each of the following changes, would it increase, decrease, or not make any difference in the administrative time your institution uses to mange all aspects of NSF awards?

		Increase Time Needed		Decrease Time Needed	No Difference in Time Needed
a.	Increasing the amount of funding for NSF awards................................................................		17%	5%	77%
b.	Increasing the duration of NSF awards...........		41%	20%	38%
c.	Increasing the total number of NSF awards....		85%	1%	14%

4.3 In your opinion, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average dollar amount for each grant award?

4.4 Now, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average duration for each grant award?

4.5 Please outline any suggestions you have for NSF changes that would result in a reduction of the amount of time and resources used by your institution to manage NSF grants.