APPENDIX A

 

RESEARCH METHODOLOGY



APPENDIX A: RESEARCH METHODOLOGY

The sections that follow describe the research methodology used for the National Science Foundation Principal Investigator FY 2001 Grant Award Survey and for the National Science Foundation Institutional FY 2001 Grant Award Survey.

A.   QUESTIONNAIRE DEVELOPMENT

       The initial phase of questionnaire development included two focus groups with NSF representatives who could identify key issues to be included in the two questionnaires.  A third focus group with institutional representatives was scheduled for September 2001, however the events of September 11 resulted in a cancellation.  Instead institutional representatives were contacted by telephone to discuss key issues to be included in the survey.  After draft questionnaires were developed, they were cognitively pretested with PIs and institutional representative, and revisions were made based on the findings from the pretests.  The following provides details about the steps that were followed:

Date                                 Type of Group                         Number of Participants

 

August 8, 2001                 NSF Focus Group                               12

 

August 9, 2001                 NSF Focus Group                               11

 

October 2001                   Institutional Representatives                 4

                                                (Telephone interviews)*

 

December 4, 2001            Principal Investigators

                                                Cognitive pretest/group discussion       8

 

January/February             Institutional Representatives                 4

2002                                 Cognitive pretest/individual

                                                interviews

 

*Re-scheduled from the Federal Demonstration Project Group discussion because of September 11,2001.


B.    PROCEDURES FOR PRETEST WITH PRINCIPAL INVESTIGATORS

Eight PIs of a sample of 30 potential respondents participated in the pretest for the Principal Investigator FY 2001 Grant Award Survey. The sample was randomly selected from a total of 156 PIs throughout New Jersey representing a variety of grant types and award sizes.  We decided to limit the sample selection to New Jersey because we assumed that MPR’s Princeton office in New Jersey would make it easier for the respondents to participate.

Respondents were asked to complete the draft questionnaire and comment on the questions.  When respondents had difficulty understanding a question, MPR reworded the question or divided it into parts to make it more understandable.  MPR also added some probes to better focus respondents on questions.  Because participants voiced concerns about the amount of time it took to complete the questionnaire, the length of the questionnaire was reduced.  Also, feedback about the focus of questions was implemented into a revised questionnaire. In particular, the concept “fully enabled” was discussed and rejected by the group.  A preferred concept to describe the goals was “ongoing research and educational activities.”

The final questionnaire was programmed into a Web format to be conducted as a Computerized Self-Administered Questionnaire (CSAQ).  Extensive testing was conducted on the Web questionnaire to insure compatibility with a wide range of different computers and servers that would be accessing the questionnaire. 

 

C.   SAMPLE APPROACH

 

1.    Principal Investigator Survey

The universe for the PI survey comprises all 6,180 FY 2001 NSF award grantees. NSF decided to collect data from the universe of PIs instead of a sample to ensure that the most robust information.  Since the primary mode of data collection is the World Wide Web, the additional costs associated with using the universe, instead of a sample, were minimal.  In addition, examining the universe eliminates both the additional costs needed to develop a sampling plan and the potential sampling bias associated with sampling plans.

2.    Institutional Survey

       The universe for the institutional survey comprises all 582 institutions where at least one PI received an NSF award in FY 2001.  Each institution in the universe was mailed a questionnaire and afforded the opportunity to participate.  However, a sample of 100 institutions was drawn from the universe, based on institutional size and type (for example, private research institution, academic institution), the number of grants received, the type of grants received, and the institution’s geographic region.

       The sampling design is based on the purpose and analytical objectives of the study.  The purpose of this study is to determine the burden of the grant awards on institutions receiving grants from NSF.  The analytic objective is to investigate the burden of the grant awards using both institution-level and grant-level measures.  Therefore, there is an interest in both the estimate of the proportion of institutions that have a level of burden and the estimate of the average burden per grant for specific types of grants or type of institutions.  The sampling design accounts for these two analytical objectives, which indicate somewhat different designs.  A stratified random sample of institutions was selected that included an over sampling of institutions with a larger number of grants. 

       The number grant awards per institution is highly skewed with 40 percent of institutions (233) receiving one award and 16 institutions receiving in aggregate more than 1,500 awards.  To account for both analytical objectives, sampling strata were developed that permit an over sample of the institutions with the greatest number of awards, and allocate a sufficient number of sampled institutions to the strata of the institutions with one or only a few awards.  Within each stratum, a sample of institutions with equal probability and without replacement were selected. A larger initial sample was selected and then partitioned into random sub samples called waves.  Some waves were  released for data collection at the start of the fielding period and others were held in reserve.  Three reserve waves were released because of institutions on the original data base that NSF determined to be ineligible. At the end of the data collection, sampling weights were applied to the final data file based on the inverse of the selection probabilities and computed adjustment to compensate for non-response among sampled institutions.

       The following provides a description of the universe and the sampling frame, the sampling design, sample allocation, and expected precision from the sample.

a.    Description of the Universe

       The target population and the universe for this study is a listing of current recipients of grant awards by NSF.  The population includes 582 institutions receiving a total of 6,180 grants, an average of 10.6 grants per institution.  In total, 440 institutions (75 percent) received 9 or fewer grants with 233 (40 percent) institutions receiving one award and 85 (15 percent) institutions receiving two awards.  On the other hand, 16 institutions (2.7 percent) accounted for 1,523 (25 percent) of the grant awards.

3.    Sampling Design and Allocation

       The analytical objectives indicate two variations on a stratified sampling design.  For institution-level survey estimates, the sampling design that can offer smallest sampling variance is an equal probability sample of all institutions.  For grant-level measures of the burden of the grant awards, the sampling design offering smallest sampling variance has the institutions selected with probability proportional to the number of grant awards.  The sampling approach that offered a reasonable comprise between these two designs.


       A classical process to develop sampling strata that account for the “size” (in this case, the number of awards at the institution) of a sampling unit is to use the square root of the size factor and partition a list of sampling units into strata so that the aggregate value of the square root of the size factor for institutions in each strata is equal (see Cochran 1997 for the “cumulative square root of f rule”).[1]  Using the cumulative square root of f rule, estimates of totals (in this situation grant awards) is improved over an equal probability sample of institutions.  For example, if 5 sampling strata are desired, the cumulative square root is summed over all units and then divided by 5.  This value is used to identify the units that are assigned to each stratum.  In developing the strata, there was a slight modification of this procedure to achieve better precision for institution-level estimates.

       The proposed sample size is 100 institutions. The precision available from a sample of 100 units is assessed by using an estimate of an institution-level proportion around 0.50.  The estimated half-width of a 95 percent confidence interval is 0.098, that is an interval of .402 to .598 (see Table B.1).  Using the cumulative square root of the frequency (f) rule, we looked not only at the square root but also the cube root.  When the finite population correction is accounted for, using the cumulative square root of f rule, resulted in a half-width of a 95 percent confidence interval of 0.115, whereas using the cumulative cube root of the frequency, resulted in a half-width of a 95 percent confidence interval of 0.100.  That is, the use of the cube root can achieve nearly the precision of a simple random sampling of all institutions, but includes over sampling of the institutions with the largest number of grants.   Increasing the number of strata beyond 3 had only a slight effect on the precision, and the plan was to use 5 strata for operational ease.      For grant-level estimates, the level of precision is based on the correlation between the number of grant awards at an institution and the outcome measures.  The anticipated precision will be as good and most likely better than will be available for the institution-level estimates. 

       In summary, for the institution survey there was a stratified random sample of institutions using 5 strata for respondent sample of 100 institutions.  The sampling strata were developed to achieve good precision for both institution-level estimates and grant-level estimates. 

 

TABLE B.1

SAMPLE ALLOCATION AND STRATA FOR INSTITUTION SAMPLE

 

 

 

Number of Institutions

Strata

Sample Size

Equal Size Strata

Square Root Algorithm

Cube Root Algorithm

 1

20

116

269

197

 2

20

116

154

159

 3

20

116

79

106

 4

20

117

47

70

 5

20

117

33

50

Half-Width of 95% Confidence Interval

 

0.098

0.115

0.100

 

Source:        Mathematica computations.

Note:   Half-width of 95% confidence interval = 1.96 * %variance for a stratified random sample where the variance within a stratum is computed from p * (1 – p) with p =0.50.

 

D.   DATA COLLECTION

       The PI survey was conducted using a mixed-mode format of Web and mail methods and the institution survey was a mail survey.  A database containing contact information (telephone numbers and e-mail addresses) for potential respondents was provided to MPR by NSF. 

The following provides additional detail of the data collection steps that were taken:

January 2001 NSF Director Dr. Rita R. Colwell sends PIs e-mail message announcing the survey.
January 30, 2002 MPR begins sending PI e-mail invitations with Web site access username and password on a rolling schedule.
February 4-19, 2002 MPR sends e-mail remiders to non-responders on a 3 day schedule.
February 15, 2002 MPR sends questionnaire mail packets to 778 PIs who have responded to the Web questionnaire.
March 8, 2002 Deadline for data collection

 

 

Original PI grants in NSF data file

6,180

PIs with multiple grants randomly selected a single grant for the survey (375) or questionable grant information (12)

5,793

Total completes and partials

5,221

Cases screened out during quality assurance process for criteria such as inconsistent grant award or duration information

232

Total cases used for analysis

4,989

 

         A tracking system was developed to monitor participation. Figure A-1 illustrates the PI participation in the Web mode of the questionnaire. A total of 778 mail packets were sent to insure participation from PIs who may not have had Web access or would prefer to complete the questionnaire on paper.

         The institutional survey was a mail only survey that used an e-mail approach to identify the most appropriate institutional participant. The data collection process was as follows:

January 2001 NSF Director Dr. Rita R. Colwell sends institution presidents an e-mail message announcing the two surveys.
January 24, 2002 MPR sends e-mail messages to institution contact people identified on the NSF data file to identify the appropriate person to participate in the survey.
February 15-March 6, 2002 Questionnaire mail packets are sent as institutional representatives contact information is identified.
March 8-30, 2002 MPR contacts non-responders in the institution sample by phone and e-mail.
March 30, 2002 All data collection is completed.

 

 

Total institutions with 2001 NSF grant recipients

582

No contact information

60

Total number with contact information

471(total); 105 (sample)

Total questionnaires returned

369 (total); 95 (sample)

Questionnaires not acceptable after quality assurance

359 (total); 95 (sample)

 

 

E.    INSTITUTIONAL SURVEY ESTIMATES OF STANDARD ERROR

       As described in Section D, the results from the institution survey are based on a sample, not a census of all institutions.  Therefore, the results discussed in the report have standard errors. The estimates of the standard error for the key items included in the analysis are on Table A-1.

F.    PRINCIPAL INVESTIGATOR SURVEY MEAN CALCULATIONS

       The report includes information about means that are calculated in two different ways.  There are means that are calculated for a single question in the PI questionnaire or for a single item of information from the NSF FY 2001 grant data files.  In addition, there are means that have been calculated using measures constructed from either two items in the survey data or using a combination of questionnaire items and items from the NSF FY 2001 grant data file.  The means for these constructed variables are calculated by taking the individual PI information for the included items, doing the calculation for each individual PI, and then getting an average.  The following describes the information that is based on means calculated from multiple items.  Appendix G has the central tendency distributions for these constructed variables.

 

CONSTRUCTED VARIABLES

 

CALCULATION AND DATA SOURCE

Option 1: Award Efficiency and Effectiveness

Deviation from Requested Award Amount

(FY 2001 Award Request – FY 2001 Award

Amount)/Number of FY2001 Grant Award

Years

            (Information from NSF data file)

Option 2: Award Efficient and Effectiveness

Percent of Research Being Funded

(FY 2001 Award Amount/(Q3.2¸100)-FY2001 Award Amount)

Divided by 5 Years to annualize

            (NSF information and survey question)

Option 4: Award Efficient and Effectiveness

NFS’s Contribution

Q3.3 X Q3.4 Divided by 5 Years to Annualize

            (Survey questions)

Difference in FY 2001 Award Amount

Request and Amount Awarded

FY 2001 Amount Request-FY 2001 Amount

Award

            (NSF data file)

Difference in FY 2001 Duration Request and

Duration Award

FY 2001 Duration Request-FY 2001 Duration

Award

            (NSF data file)

Additional Duration Needed

FY 2001 Duration Award + Q3.1

            (NSF data file and survey question)

 

 

G.   SURVEY MEASUREMENT ERROR

       It should be noted that in any survey there are sources of both sampling and non-sampling error.  Some examples of sources of survey measurement error are non-response to the survey, skipped questions, context effects, data collection methodology, and question wording.  In conducting this study, all efforts possible were taken to minimize survey measurement error.

 



 

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX B

 

ANNOTATED QUESTIONNAIRES


 


 

 

 

 

APPENDIX B CONTENTS

 

 

 

A.  NATIONAL SCIENCE FOUNDATION PRINCIPAL INVESTIGATOR 2001 GRANT AWARD SURVEY

 

B.  NATIONAL SCIENCE FOUNDATION INSTITUTIONAL SURVEY


 


Text Box: Conducted for NSF by:
Text Box: Please Complete and Submit Questionnaire
By March 8, 2002
Text Box: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB number of this project is 3145-0185.
 


                                                                                                                             OMB Approval Number:  3145-0185

 

 

 

 

 

                                                    Welcome to the

 

 

 

National Science Foundation

Principal Investigator

2001 Grant Award Survey

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

BY MAIL

 

Questions or Comments?

TO:

Matt Mishkind

Project Director

Mathematica Policy Research, Inc.

P.O. Box 2393

Princeton, NJ  08543

Contact Matt Mishkind at

877-236-4185

or

E-mail: nsfgrantsweb@mathematica-mpr.com

 

 

 

 

 

 

 

 


 

2001 NSF GRANT INFORMATION

 

#1

Grant Title

__________________________________

#2

Grant Effective Date

__________________________________

#3

Requested Amount

__________________________________

#4

Awarded Amount

__________________________________

#5

Amount Change 5% or Greater

__________________________________

#6

Requested Duration

__________________________________

#7

Awarded Duration

__________________________________

#8

Duration Change 1 Year or Greater

__________________________________

 

·       You will be asked to reference the information listed above throughout this questionnaire.  This information is from our database and is specific to the NSF grant you were awarded funding in 2001.

 

·       When a question asks you to think about any of the above information, a notation will be made in the questionnaire.  Therefore, it is important to keep this information attached to the rest of the questionnaire.

 

 
 


·       If this is your grant, please check the box and begin the questionnaire.

 

·       If any of this grant information is incorrect, please contact Matt Mishkind at 877-236-4185 or nsfgrantsweb@mathematica-mpr.com before you complete the questionnaire.

 

·       You may also complete this questionnaire on the Web:

Log onto

http://nsfgrants.mathematica-mpr.com

and enter the following

 

USERNAME:   xxxxxx

 

PASSWORD:  xxxxxx

 

 

 

 

 

 

 

 

 
 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 



PRINCIPAL INVESTIGATOR

2001 GRANT AWARD SURVEY

 

 


SECTION 1

 

 

REMINDER:  Please check grant information provided on back of cover page.

 

 

1.1   Was your 2001 NSF grant [#1 GRANT TITLE] awarded on [#2 GRANT EFFECTIVE DATE] a first‑time submission or a revision of a previously declined NSF proposal?

 

·   A revised proposal does not refer to changes made in your 2001 NSF grant proposal after the initial review

 

       mark one

71%    a first time submission

29%    a revision of a previously declined NSF proposal

 

 

1.2   NSF research grants can be classified along a number of different dimensions.  Which ONE of the following definitions best describes the research that is funded by this grant?

 

·   If your work involves several of these categories please choose the one that is most appropriate

 

      THEORETICAL research can be accomplished with minimal physical resources beyond the investigator’s institutional research library, computing capability and office space.

 

      LABORATORY research requires an equipped laboratory, for example, research often found in chemistry, biology or engineering university laboratories requiring research and/or testing equipment, plumbing.

 

      FIELD research requires fieldwork, specimen collection, sample survey, location of sensors, etc. away from the principal investigator’s institution, for example, some science activities in geosciences, biology, social sciences.

 

       mark one

37%    Theoretical Research

44%    Laboratory Research

18%    Field Research

 

 

1.3   Does your 2001 NSF project require the use of a national or international research facility such as access to an accelerator, a light source, a ship, major telescope or supercomputer center?

 

         16%    Yes

         83%    No

 

 

1.4    In general, would you say that this 2001 NSF grant is funding:

 

         mark one

           7%    A specific product or deliverable

         89%    A project that is part of your ongoing body of research and educational activities

           4%    Other (Please Describe)

                                                                                                                                      

 


 

1.5    For each of the following, how much advice did you get from NSF staff when you were preparing your grant proposal:

 

 

 

MARK ONE FOR EACH

 

 

A Great Deal

Some

Not Much

 

None At All

a.      

The amount of funding.............

12%

27%

17%

 

43%

b.      

The duration of the grant proposal.................................

11%

21%

16%

 

51%

c.    

The substance or focus of the grant......................................

7%

25%

19%

 

49%

 

 

 

 

1.6    Based on the advice provided by NSF staff, did you increase, not change, or decrease:

 

 

 

Increase

Not Change

Decrease

Not Asked

a. 

The amount of the award you proposed.......................................

4%

36%

27%

31%

b. 

The award duration you proposed....

3%

58%

6%

31%

 

 

 


SECTION 2

 

 

 

As part of the review process, NSF may ask principal investigators to revise their proposal before they are awarded funding.  The following questions are about your revised budget and award duration.

 

ONLY ANSWER Q2.1 IF #5 AMOUNT CHANGE >5% IS LABELED “YES.”  See inside cover.

 

2.1    In your proposal, you requested [#3 REQUESTED AMOUNT] and in your award you received [#4 AWARDED AMOUNT].

 

         Overall, how much will this change in the award amount impact your ability to do what you expected to accomplish with this 2001 NSF grant?

 

         mark one

           1%    Can do a great deal more than expected

           2%    Can do somewhat more than expected

           7%   Can do about the same as expected

         28%    Can do somewhat less than expected

         15%    Can do a great deal less than expected

           1%    Don’t know

         47%    Not asked

 

 

ONLY ANSWER Q2.2 IF #8 DURATION CHANGE >1 YEAR IS LABELED “YES.”  See inside cover.

 

(IF BOTH #5 AMOUNT CHANGE >5% AND #8 DURATION CHANGE >1 YEAR ARE LABELED “NO,” PLEASE SKIP TO SECTION 3).

 


 

2.2    In your proposal, you requested [#6 REQUESTED DURATION] and in your award you received [#7 AWARDED DURATION].

 

         Overall, how much will this change in award duration impact your ability to do what you expected to accomplish with this 2001 NSF grant?

 

         mark one

           1%    Can do a great deal more than expected

           1%    Can do somewhat more than expected

           1%    Can do about the same as expected

           4%    Can do somewhat less than expected

           5%    Can do a great deal less than expected

         88%    Not asked

 

 

IF YOU RESPONDED AS 6 “CAN’T ANSWER” OR -1 “DON’T KNOW” TO BOTH Q2.1 AND Q2.2, PLEASE SKIP TO SECTION 3.

 

 

ONLY ANSWER Q2.3 IF YOU PROVIDED A RESPONSE OF 1, 2, 3, 4, OR 5 TO EITHER Q2.1 OR Q2.2.

 

2.3    The following are some possible consequences of the changes in your NSF award funding and/or duration.  Will this change have a positive impact, no impact, or negative impact on your ability to ...

 

 

 

Positive

Impact

No

Impact

Negative

Impact

Not

Applicable

Not

Asked

 

A.  Goals and Objectives

 

a.   

Pursue innovative ideas.............................................................

4%

25%

23%

1%

47%

 

b.   

Pursue high-risk ideas..............................................................

3%

20%

26%

4%

47%

 

c.   

Obtain other funding.................................................................

6%

35%

6%

5%

47%

 

B.  Applications and Outcomes

 

d.   

Disseminate research findings...................................................

4%

32%

17%

1%

47%

 

e.   

Develop instrumentation or other enhancements for the research and education infrastructure......................................................

2%

17%

20%

13%

47%

 

f.    

Develop partnerships with industry, other educational institutions, or national laboratories..............................................................

3%

26%

13%

11%

47%

 

g.   

Integrate research activity into your teaching and training.............

4%

27%

18%

4%

47%

 

h.   

Nurture connections between research activity and its potential for:  health benefits, economic benefits, and national security benefits...................................................................................

2%

24%

10%

16%

47%

 

i.    

Develop programs with K‑12 teachers and/or students.................

1%

23%

7%

22%

47%

 

j.    

Improve public understanding of the project.................................

3%

31%

11%

9%

47%

 

C.  Process and Team Building

 

k.   

Collaborate with researchers in your area of research..................

5%

21%

26%

1%

47%

 

l.    

Broaden participation of under-represented groups in the research activity....................................................................................

3%

27%

18%

6%

47%

 

m. 

Collaborate with researchers in different areas of research............

4%

25%

21%

3%

47%

 

n.   

Achieve the research objectives within the specified time.............

4%

14%

34%

1%

47%

 

o.   

Obtain quality personnel...........................................................

3%

17%

28%

4%

47%

 

p.   

Establish mentoring or other research-based education activities..

3%

23%

21%

5%

47%

 

D.  Research Tools

 

4%

 

 

 

47%

q.   

Access state-of-the-art equipment.............................................

2%

28%

17%

6%

47%

 

r.    

Access facilities.......................................................................

2%

34%

10%

6%

47%

 

 

 

         SKIP Q2.4a IF NO POSITIVE ITEMS IN Q2.3

2.4a   Among the items you marked “Positive Impact,” please rank order (write in the number(s)), up to three, those that had the most positive impact.

 

#1

 

#2

 

#3

 

 

         SKIP Q2.4b IF NO NEGATIVE ITEMS IN Q2.3

2.4b  Among the items you marked “Negative Impact,” please rank order (write in the number(s)), up to three, those that had the most negative impact.

 

#1

 

#2

 

#3


 

2.5    Please describe any other impact(s) that resulted from the change in your 2001 NSF award or give more details on any in the list that need further explanation.

 

                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                                                                                                                                

                                                                                                                                                                                                                                                                                                

 

 

SECTION 3

 

 

The next group of questions is your assessment of how this grant fits into your ongoing body of research and educational activities.

 

         ·   Our records indicate that your 2001 NSF grant is for $[#4 AWARDED AMOUNT] over a period of [#7 AWARDED DURATION] Years.  See inside cover.

 

 

 

3.1    Thinking about the timeframe for your ongoing body of research and educational activities, about how many additional years do you think you would need to accomplish your key goals?

 

         ·   DO NOT include the years for the 2001 NSF grant

 

         ·   Enter “0” for “Do not need any additional years”

 

Median=2

Mean=3

Mode=2

Range:  0 to 40

 

 

 

3.2    If you think about your ongoing body of research and educational activities as 100 percent of what you’d like to accomplish in the next five years, about what percent of what you’d like to do will be achieved with your 2001 NSF research grant?

 

Median=30

Mean=37

Mode=20

Range:  0 to 100

 

 

 

Now, speculate on what changes, if any, you would need to accomplish all you would like to in the next five years.

 

3.3    In the next five years, how much additional funding from all sources, if any, would you need to achieve what you would like to with your ongoing body of research and educational activities?

 

         ·   Exclude funding you currently have for this NSF grant and from any other funding sources

 

         ·   Enter “0” for “Do not need any additional funding”

 

Median=$500,000

Mean=$1,149,000

Mode=$500,000

Range:  $0 to $300,000,000

 

 

 

IF YOU DO NOT NEED ADDITIONAL FUNDING OR DON’T KNOW, SKIP TO Q3.6.

 

3.4    What percent of this additional amount do you think is appropriate for NSF to fund?

 

Median=70%

Mean=67%

Mode=100%

Range:  0% to 100%

 


 

3.5    About how many additional grants do you think you would need to get this funding?

 

Median=2

Mean=2.39

Mode=2

Range:  0 to 32

 

 

 

ONLY ANSWER Q3.6 IF YOU NEED ADDITIONAL YEARS (Q3.1) AND/OR ADDITIONAL FUNDING (Q3.3).

 

IF YOU RESPONDED “0” OR “DON’T KNOW” TO Q3.1 AND Q3.3, SKIP TO SECTION 4.

 

3.6    If NSF provided this additional funding and/or duration to support your ongoing research and educational activities, would there be a positive impact, no impact, or a negative impact on each of the following:

 

 

 

Positive

Impact

No

Impact

Negative

Impact

Not

Applicable

Not

Asked

A.  Goals and Objectives

1.  

Pursue innovative ideas.............................................................

87%

2%

<1%

<1%

10%

2.  

Pursue high-risk ideas..............................................................

76%

9%

<1%

4%

10%

3.  

Obtain other funding.................................................................

54%

26%

5%

3%

10%

B.  Applications and Outcomes

4.  

Disseminate research findings...................................................

74%

14%

<1%

1%

10%

5.  

Develop instrumentation or other enhancements for the research and education infrastructure......................................................

61%

16%

<1%

13%

10%

6.  

Develop partnerships with industry, other educational institutions, or national laboratories..............................................................

62%

19%

<1%

8%

10%

7.  

Integrate research activity into your teaching and training.............

73%

13%

<1%

2%

10%

8.  

Nurture connections between research activity and its potential for:  health benefits, economic benefits, and national security benefits...................................................................................

48%

24%

<1%

16%

10%

9.  

Develop programs with K‑12 teachers and/or students.................

32%

36%

<1%

20%

10%

10.  

Improve public understanding of the project.................................

58%

25%

<1%

5%

10%

C.  Process and Team Building

11.  

Collaborate with researchers in your area of research..................

83%

6%

<1%

<1%

10%

12.  

Broaden participation of under-represented groups in the research activity....................................................................................

62%

23%

<1%

3%

10%

13.  

Collaborate with researchers in different areas of research............

76%

12%

<1%

2%

10%

14.  

Achieve the research objectives within the specified time.............

83%

6%

<1%

1%

10%

15.  

Obtain quality personnel...........................................................

76%

9%

<1%

3%

10%

16.  

Establish mentoring or other research-based education activities..

71%

14%

<1%

3%

10%

D.  Research Tools

17.  

Access state-of-the-art equipment.............................................

60%

22%

<1%

7%

10%

18.  

Access facilities.......................................................................

49%

32%

<1%

8%

10%

 

 

 

         SKIP Q3.7a IF NO POSITIVE ITEMS IN Q3.6

3.7a   Among the items you marked “Positive Impact,” please rank order (write in the number(s)), up to three, those that had the most positive impact.

 

#1

 

#2

 

#3

 

 

 

         SKIP Q3.7b IF NO NEGATIVE ITEMS IN Q3.6

3.7b  Among the items you marked “Negative Impact,” please rank order (write in the number(s)), up to three, those that had the most negative impact.

 

#1

 

#2

 

#3


 

3.8    Please describe any other impact(s) that would result if NSF provided you what you need for what you want to accomplish, or give more details on any in the list that needs further explanation:

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

 

 

3.9    If you received this additional funding and/or duration from NSF that you need for your ongoing body of research and educational activities, how likely would you increase each of the following?

 

 

 

MARK ONE FOR EACH

 

 

Very

Likely

Somewhat

Likely

Neither

Likely

Nor

Unlikely

Somewhat

Unlikely

Very

Unlikely

Not

Applicable

Not

Asked

Personnel

1.  

The number and/or months of senior personnel.....................................

29%

21%

13%

7%

13%

4%

10%

2.  

The number and/or months of post doctoral associates........................

43%

22%

7%

4%

5%

7%

10%

3.  

The number and/or months of technicians..............................................

15%

14%

14%

7%

16%

19%

10%

4.  

The number and/or months of programmers..........................................

7%

11%

14%

6%

19%

24%

10%

5.  

The number and/or months of graduate students...................................

70%

11%

2%

1%

1%

3%

10%

6.  

The number and/or months of undergraduate students..........................

45%

26%

7%

3%

3%

4%

10%

Equipments

7.  

The number of equipment purchases.....................................................

33%

31%

11%

4%

5%

5%

10%

8.  

The quality of equipment purchases........................................................

28%

22%

20%

5%

7%

7%

10%

Travel

9.  

The number of trips..................................................................................

29%

32%

18%

5%

4%

1%

10%

10.

The cost per trip.......................................................................................

3%

8%

36%

11%

29%

2%

10%

Experiments

11.

The number of experiments, tests, subjects...........................................

49%

14%

5%

1%

1%

20%

10%

12.

The size of the experiments or tests.......................................................

30%

17%

14%

3%

3%

22%

10%

13.

The quality of the experiments or tests....................................................

36%

13%

13%

2%

4%

20%

10%

Other Direct Costs

14.

Participant support...................................................................................

21%

18%

19%

4%

10%

16%

10%

15.

Consultant services.................................................................................

6%

10%

19%

7%

23%

22%

10%

16.

Computer/Publication costs....................................................................

16%

28%

21%

7%

11%

5%

10%

17.

Other (Please Specify)............................................................................

 

 

 

 

 

 

 

27% gave a response

 

                                                                                                                                                                                                                                   

 

 

 

 

 

 

 

 

 


3.10   Among the items you are “Very Likely” to increase, rank order (write in the numbers), up to three, those that would have the most impact on what you want to accomplish.

 

#1

 

#2

 

#3


 

3.11   And, if you received this additional funding and/or duration from NSF that you need for your ongoing research and educational activities, would your ability to do each of the following be:

 

 

 

MARK ONE FOR EACH

 

 

Increased

A Great

Deal

Increased

Somewhat

About

the

Same

Decreased

Somewhat

Decreased

A Great

Deal

Not

Applicable

Not

Asked

a.

Recruit post doctoral associates.......................................

47%

23%

8%

<1%

<1%

9%

10%

b.  

Recruit graduate students.................................................

56%

23%

6%

<1%

<1%

4%

10%

c.

Recruit undergraduate students........................................

27%

31%

24%

<1%

<1%

6%

10%

d.

Provide adequate support for a graduate student to shorten time to degree.................................................................

29%

26%

25%

<1%

<1%

8%

10%

e.

Provide stability for technicians........................................

17%

15%

13%

<1%

<1%

42%

10%

f.  

Provide stability for programmers......................................

8%

9%

14%

<1%

<1%

57%

10%

g.

Conduct more experiments, tests or subjects.....................

42%

22%

5%

<1%

<1%

19%

10%

h.

Have higher-quality experiments or tests............................

31%

22%

16%

<1%

<1%

20%

10%

i.

Duration of experiments...................................................

17%

19%

24%

1%

1%

26%

10%

j.

Other (Please Specify)....................................................

 

 

 

 

 

 

 

                                                                                                                                                                             

 

 

 

 

 

 

 

29% gave a response

 

 

 

 

3.12   Thinking about all the different aspects of what you would like to accomplish, which of the following would have the greatest impact on your ongoing body of research and educational activities:

 

         mark one

         54%    More funding

         35%    Longer duration

         10%    Not asked

 


 

 

SECTION 4

 

 


The following are questions about NSF funding and your general field of research.

 

4.1    In your opinion, if NSF increased the funding and the duration of the awards in your field of research, how likely would these changes …?

 

 

 

MARK ONE FOR EACH

 

 

Very

Likely

Somewhat

Likely

Neither

Likely Nor

Unlikely

Somewhat

Unlikely

Very

Unlikely

Not

Applicable

a.

Widen the focus of the research in your field......

63%

28%

7%

1%

1%

<1%

b.  

Increase the number of  proposals to NSF with innovative ideas.................................................

46%

35%

13%

3%

1%

1%

c.  

Increase the number of proposals to NSF with

high-risk ideas..................................................

37%

38%

17%

4%

2%

2%

d.  

Attract more established researchers to apply for NSF funding......................................................

37%

31%

23%

4%

2%

2%

e.  

Decrease the amount of time to answer research questions..........................................................

31%

32%

20%

6%

6%

4%

f.

Attract more graduate students...........................

65%

26%

6%

<1%

<1%

2%

g.

Attract better graduate students..........................

62%

25%

8%

1%

<1%

2%

h.

Improve access to facilities and databases..........

36%

34%

20%

1%

1%

7%

i.

Decrease interruptions in funding.......................

70%

23%

4%

<1%

<1%

1%

 

 

 

4.2    If NSF had more money to award each year, please rank in descending order of importance from (1) most important to (3) least important, the following possible actions for awards in your area of research:

 

         rank order                                                                        1st          2nd          3rd

Increase only the amount of funding per award................................... 40%         36%          23%

Increase only the length of time per award.......................................... 24%         37%          38%

Increase only the total number of awards per year.............................. 36%         26%          37%

 


 

SECTION 5

 

 


This section asks about your experience preparing this NSF grant and about some other research experiences.

 

5.1    Thinking about the proposal you submitted to NSF for this grant, what is your best estimate of the total hours of preparation for submitting this proposal?

 

         In determining your estimate, please make sure you:

 

         ·   consider all of your own time for writing the proposal, preparing the budget, completing forms, and consulting with others about your proposal

 

         ·   consider the time other people such as graduate assistants, secretaries, and budget administrators put into the preparation of this proposal

 

         ·   DO NOT include any institutional personnel who might review or internally process your proposal such as staff from the sponsored research office

 

Median=100

Mean=157

Mode=100

Range:  1 to 9,000

 

 

 

5.2    What’s your best estimate of the percent of hours that were devoted to the intellectual content of the proposal and the percent devoted to the mechanics of proposal preparation?

 

         ·   Your total must equal 100%

 

Preparation of intellectual content..........................

 

Median=75%   Mean=68%   Mode=80%   Range:  5% - 100%

 

Mechanics of proposal preparation........................

 

Median=25%   Mean=32%   Mode=20%   Range:  0% - 100%

 

 

 

5.3    How helpful is having an NSF research grant in obtaining funding from other sources?

 

         mark one

         39%    Very helpful

         33%    Somewhat helpful

         25%    Neither helpful nor unhelpful

           2%    Somewhat unhelpful

         <1%    Very unhelpful

 


 

Now, think about any other funding you may be getting for your ongoing body of research and educational activities.

 

5.4    Right now, are you getting NSF funding for any other projects for your ongoing body of research and educational activities?

 

         ·   This includes funding from grants on which you are a collaborator or subcontractor

 

         ·   DO NOT include the 2001 NSF grant identified for this survey

 

         44%    Yes

         55%    No

           1%    Don’t know

 

 

 

5.5    Not including the 2001 NSF grant identified for this survey, what is the total number of current NSF grants funding your ongoing body of research and educational activities?

 

Median=1

Mean=2

Mode=1

Range:  0 to 236

 

 

 

5.6    What is the total amount of annual funding you currently have from these other NSF grants?

 

         ·   DO NOT include the 2001 NSF grant identified for this survey

 

Median=$100,000

Mean=$207,000

Mode=$100,000

Range:  $0 to $30,000,000

 

 

 

5.7    Did you divide your ongoing body of research and educational activities into several proposals and submit them to NSF?

 

         38%    Yes

         62%    No

 

 

 

Now, think about any non-NSF funding you are getting for your ongoing body of research and educational activities.

 

5.8    In addition to your NSF funding, do you currently have other funding for your ongoing body of research and educational activities?

 

         ·   This may be funding from sources such as your institution, another federal agency, a state agency, a non-profit foundation, or a for-profit company or organization

 

         72%    Yes

         27%    No

         <1%    Don’t know

 

 

 

5.9    What is the total number of current non-NSF funding sources for your ongoing body of research and educational activities?

 

Median=2

Mean=2

Mode=1

Range:  0 to 420

 


 

5.10   And, what is the total amount of annual funding you have from non-NSF sources?

 

Median=$100,000

Mean=$199,000

Mode=$100,000

Range:  $0 to $10,000,000

 

 

 

The next set of questions are about your research activities and professional duties.

 

5.11    What’s your best estimate of the percent of your time spent conducting research in each of the following ways:

 

           ·   Your total must equal 100%

 

Work as part of a team with researchers from other disciplines..........

Median=10%   Mean=14%   Mode=0%   Range:  0% - 100%

Work as part of a team including other senior investigators in the

same discipline...............................................................................

Median=20%   Mean=25%   Mode=20%   Range:  0% - 100%

Work individually with students and post doctoral assistants................

Median=55%   Mean=54%   Mode=50%   Range:  0% - 100%

Other (Please Specify)....................................................................

Median=0%   Mean=6%   Mode=0%   Range:  0% - 100%

                                                                                                     


5.12    How many peer-reviewed articles have you published during the past 5 years where you have been the primary author?

 

Median=9

Mean=13

Mode=5

Range:  0 to 500

 

 

 

For the following question, please think about your current experience.

 

5.13    How many people in the following categories work with you on your current research projects?

 




Type of Institution

Undergraduate Students

 

Median=2     Mean=2     Mode=1     Range:  0 – 50

  4%   Non-Academic

  5%   Non-PhD

18%    Other PhD

25%    NSF Funding Top 20

26%    NSF Funding Top 21-50

22%    NSF Funding Top 51-100

Graduate Students

 

Median=3     Mean=4     Mode=2     Range:  0 – 300

Post-doctoral fellows

 

Median=1     Mean=1     Mode=0     Range:  0 – 100

 


 

 5.14   Questionnaires by their nature are limited.  Please write in any other comments you have about your experiences with the NSF grant process that you think are important.

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

                                                                                                                                                    

 

 

 

Thank you very much for completing this questionnaire!  Please return in the postage-paid envelope.


 

 

 


 

 

Text Box: Please Complete and Submit Questionnaire
By March 8, 2002


Sample Institution

 

 
Text Box: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB number of this project is 3145-0185.Text Box: Conducted for NSF by:

 
                                                                                                                              OMB Approval Number:  3145-0185

 

 

 

 

 

 

 

 

 


National Science Foundation

Institutional Survey

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Please Complete and Submit Questionnaire

By March 8, 2002

BY MAIL:

 

Questions or Comments?

TO:

Matt Mishkind

Project Director

Mathematica Policy Research, Inc.

P.O. Box 2393

Princeton, NJ  08543

Contact Matt Mishkind at

877-236-4185

or

E-mail: nsfgrants@mathematica-mpr.com

 

 

 

 

 

 

 

 



 

 

 

 

 

 

 

 

 

 

 

 

 

 

Thank you for participating in this study of institutional representatives who are responsible for applying for and administering National Science Foundation (NSF) grants.  We know that your time is valuable and we greatly appreciate your assistance.

 

Dr. Colwell, Director of the National Science Foundation, sent a letter informing your institution about this study.  Mathematica Policy Research, Inc. (MPR) is conducting this study for the National Science Foundation (NSF).  To assist in their future planning, NSF is very interested in learning more about NSF grants from the perspective of the institutional representatives responsible for NSF grants.

 

Your participation is critical to the success of the study and to the quality of the information we get about NSF grants.  If you have any questions about the background of the study you can contact Bob Abel at NSF (nsf-survey@nsf.gov).  If you have any questions or require any assistance while you are completing the questionnaire, you may contact Matt Mishkind at MPR (877‑236‑4185/nsfgrants@mathematica-mpr.com).

 

CONFIDENTIALITY

 

All of your responses to the questionnaire are strictly confidential.  We will not use your name or email for any other purposes than this study.  All information from the study will be kept in a secure place.  Only the MPR researchers directly working on the study will have access to this information.  Any reports of the results of this study will be presented in the aggregate.

 

INSTRUCTIONS

 

As you answer some of these questions, you will focus on the NSF FY 2001 grant(s) awarded to your institution.  It will include questions about the NSF proposals submitted by your institution and the NSF grants administered by your institution.  For your convenience, a list of these grants is enclosed.

 

The process of applying for and administering NSF grants may vary from institution to institution.  The purpose of this questionnaire is to get a general assessment of the resources your institution uses for this process.  Please make sure the most informed person at your institution completes each section of the questionnaire.  For some institutions, multiple people may need to respond.

 

 

Number of 2001 NSF grant awards         Median=3          Mean=12       Mode=1

 

Number of 2001 NSF grant declines      Median=7            Mean=32       Mode=1

 


 

 

 

 

 

Text Box: SECTION 1

The following questions focus on the proposal process at your institution.

 

1.1      Does your institution have a formal, standardized process that is followed to submit grant proposals?

 

·   This is only your institution’s process for grant proposals, it does not refer to others such as NSF FastLane

 

           98%     Yes

            2%     No

 

 

1.2      Are there specific individuals or administrative offices assigned to work with grant proposals?

 

·   Do not include principal investigators

 

           99%     Yes

             1%     No         SKIP TO Q1.4a

 

 

1.3      What is the total number of each of the following assigned to grant proposals:

 

           |        |        |  individuals                  Median=4     Mean=6     Mode=3

 

           |        |        |  administrative offices          Median=1     Mean=2     Mode=1

 

 

1.4a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal process for grants.

 

·   Do not include individual academic departments or research centers

 

 

1.4b    For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal.

 

Administrative Office

Average

Number of

Hours Per NSF Grant Proposal

 

Median=4        Mean=6        Mode=1

 

Median=2        Mean=4        Mode=1

 

Median=1        Mean=2         Mode=1

 

Median=3        Mean=10        Mode=1

 

Median=1        Mean=2         Mode=1

 


 

 

 

 

Text Box: SECTION 2

The following questions are about the process of negotiating grant proposal revisions.

 

2.1      Does your institution have a formal, standardized process that is followed to negotiate grant proposal revisions?

 

           72%     Yes

           28%     No

 

 

2.2      Are there specific individuals or administrative offices assigned to work with grant proposal revisions?

 

·   Do not include principal investigators

 

           87%     Yes

           14%     No        SKIP TO Q2.4a

 

 

2.3      What is the total number of each of the following assigned to grant proposal revisions:

 

           |        |        |  individuals                  Median=3     Mean=5     Mode=3

 

           |        |        |  administrative offices          Median=1     Mean=2     Mode=1

 

 

2.4a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal revision process for grants.

 

·   Do not include individual academic departments or research centers

 

 

2.4b    For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal revision.

 

Administrative Office

Average Number

of Hours Per NSF

Grant Proposal Revision

 

Median=2        Mean=3         Mode=1

 

Median=1        Mean=2        Mode=1

 

Median=1        Mean=1        Mode=1

 

Median=1        Mean=3        Mode=1

 

Median=0        Mean=<1        Mode=1

 

 

2.5      For a typical NSF grant that your institution is awarded, approximately how many hours are spent communicating directly with NSF on revisions to the original proposal?

 

·   Do not include principal investigator hours

 

           |        |        |        | average number of hours per nsf grant

 

           Median=1     Mean=2     Mode=1


 

 

 

 

Text Box: SECTION 3

After a grant is awarded, institutions are responsible for administering the grant and providing additional oversight.  For the following questions, please think about grant administration.

 

3.1      Are there specific individuals or administrative offices assigned to administer grant awards?

 

           97%     Yes

             3%     No        SKIP TO Q3.3a

 

 


3.2      What is the total number of each of the following assigned to administer grants:

 

·   Do not include principal investigators

 

           |        |        |  individuals                                Median=4     Mean=8       Mode=3

 

           |        |        |  administrative offices          Median=2     Mean=2       Mode=2

 

 

3.3a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in administering grant awards.

 

·   Do not include individual academic departments or research centers

 

 

3.3b    For each office, please give your best estimate of the average number of hours individuals in that office spent to administer a typical FY 2001 NSF grant award.

 

Administrative Office

Average Number of Hours Per

NSF Grant Administration

 

Median=8          Mean=21       Mode=5

 

Median=5          Mean=10       Mode=1

 

Median=4          Mean=11       Mode=1

 

Median=2          Mean=5         Mode=1

 

Median=10        Mean=7         Mode=<1

 

 

3.4      Approximately how many hours are spent to complete and submit NSF required reports for a typical FY 2001 NSF grant?

 

           |        |        |        | average number of hours per nsf grant

 

           Median=3     Mean=6     Mode=2


 

 

 

Text Box: SECTION 4

NSF is considering increasing the amount and duration available for grants.  Think about how these potential changes would impact how your institution applies for and administers NSF grants.

 

4.1      If NSF had more money to award each year, please rank from most important (1) to least important (3), the following possible actions for awards to your institution.

 

mark one

 

Ranking

 

1

2

3

 

         44%

38%

13%

Increase only the amount of funding per award

         9%

23%

62%

Increase only the duration per award

         50%

30%

17%

Increase only the total number of awards per year

 

 

4.2      Overall, if NSF made each of the following changes, would it increase, decrease, or not make any difference in the administrative time your institution uses to mange all aspects of NSF awards?

 

 

 

Increase

Time

Needed

Decrease

Time

Needed

No

Difference in

Time Needed

a.               

Increasing the amount of funding for NSF awards................................................................

  12%

      7%

81%

b.               

Increasing the duration of NSF awards...........

  42%

      24%

33%

c.               

Increasing the total number of NSF awards....

  86%

      --%

14%

 

 

4.3      In your opinion, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average dollar amount for each grant award?

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     

 

 

4.4      Now, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average duration for each grant award?

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     

 

 

4.5      Please outline any suggestions you have for NSF changes that would result in a reduction of the amount of time and resources used by your institution to manage NSF grants.

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     


 

 

Text Box: SECTION 5

 

 

 

The following questions will provide an overview of the grants managed by your institution.

 

5.1      Thinking about all the grant awards managed by your office in FY 2001, approximately what percent is for NSF grants?

 

           |        |        |        |  percent               Median=10%     Mean=16%     Mode=10%

 

 

5.2      And, approximately what percent of the total dollar amount of all grant awards managed by your office in FY 2001, is for NSF grants?

 

           |        |        |        |  percent                Median=12%     Mean=18%     Mode=1%

 

 

           Questionnaires by their nature are sometimes limited.  Please write in any other comments you have about your institution’s experiences with the NSF grant process.

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     

 

 

 

 

 

Thank you very much for completing this questionnaire.  Please use the enclosed postage-paid envelope to return it to:  Matt Mishkind, Mathematica Policy Research, Inc., P.O. Box 2393, Princeton, NJ  08543.

 

 

 

 

Type of Institution

11%  Non-Academic

28%  Non-PhD

44%  Other PhD

  5%  NSF Funding Top 20

  3%  NSF Funding Top 21-50

  9%  NSF Funding Top 51-100

 

 

 

 

 


 

 

 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX C

 

NONSAMPLE INSTITUTIONAL SURVEY INFORMATION


 


APPENDIX C

NONSAMPLE INSTITUTION SURVEY RESULTS

This section of the appendix provides a general overview of the results on the completed questionnaires from the 264 institutional representatives who were not selected as part of the scientific sample of institutions described in Appendix A.  These results can be categorized as a convenience sample rather than a scientific sample.  The results of the scientific institution sample discussed in the report text can be projected on the population of all FY 2001 institutions who had PIs that received NSF grants; the results from this convenience sample describes the responses from these 264 institutional representatives.

The results from these nonsample institutions follows in two forms:  (1) an annotated questionnaire with the responses and (2) tables that have the percentages of responses from the open-ended questions.  It should be noted that in Appendix G there is a table with the central tendency distributions for the nonsample institutions.



Nonsample Institution

 
Text Box: Conducted for NSF by:
Text Box: Please Complete and Submit Questionnaire
By March 8, 2002
Text Box: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB number of this project is 3145-0185.

 
 


                                                                                                                             OMB Approval Number:  3145-0185

 

 

 

 

 

 

 

 

 

National Science Foundation

Institutional Survey

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

BY MAIL:

 

Questions or Comments?

TO:

Matt Mishkind

Project Director

Mathematica Policy Research, Inc.

P.O. Box 2393

Princeton, NJ  08543

Contact Matt Mishkind at

877-236-4185

or

E-mail: nsfgrants@mathematica-mpr.com

 

 

 

 

 

 

 


 

Thank you for participating in this study of institutional representatives who are responsible for applying for and administering National Science Foundation (NSF) grants.  We know that your time is valuable and we greatly appreciate your assistance.

 

Dr. Colwell, Director of the National Science Foundation, sent a letter informing your institution about this study.  Mathematica Policy Research, Inc. (MPR) is conducting this study for the National Science Foundation (NSF).  To assist in their future planning, NSF is very interested in learning more about NSF grants from the perspective of the institutional representatives responsible for NSF grants.

 

Your participation is critical to the success of the study and to the quality of the information we get about NSF grants.  If you have any questions about the background of the study you can contact Bob Abel at NSF (nsf-survey@nsf.gov).  If you have any questions or require any assistance while you are completing the questionnaire, you may contact Matt Mishkind at MPR (877‑236‑4185/nsfgrants@mathematica-mpr.com).

 

CONFIDENTIALITY

 

All of your responses to the questionnaire are strictly confidential.  We will not use your name or email for any other purposes than this study.  All information from the study will be kept in a secure place.  Only the MPR researchers directly working on the study will have access to this information.  Any reports of the results of this study will be presented in the aggregate.

 

INSTRUCTIONS

 

As you answer some of these questions, you will focus on the NSF FY 2001 grant(s) awarded to your institution.  It will include questions about the NSF proposals submitted by your institution and the NSF grants administered by your institution.  For your convenience, a list of these grants is enclosed.

 

The process of applying for and administering NSF grants may vary from institution to institution.  The purpose of this questionnaire is to get a general assessment of the resources your institution uses for this process.  Please make sure the most informed person at your institution completes each section of the questionnaire.  For some institutions, multiple people may need to respond.

 

Number of 2001 NSF grant awards  Median=2          Mean=10       Mode=1

 

Number of 2001 NSF grant declines Median=9          Mean=27       Mode=1



 

 

 

 


Text Box: SECTION 1

The following questions focus on the proposal process at your institution.

 

1.1      Does your institution have a formal, standardized process that is followed to submit grant proposals?

 

·   This is only your institution’s process for grant proposals, it does not refer to others such as NSF FastLane

 

           94%     Yes

             5%     No

 

 

1.2      Are there specific individuals or administrative offices assigned to work with grant proposals?

 

·   Do not include principal investigators

 

           96%     Yes

             3%     No         SKIP TO Q1.4a

 

 

1.3      What is the total number of each of the following assigned to grant proposals:

 

           |        |        |  individuals                  Median=3     Mean=5     Mode=2

 

           |        |        |  administrative offices          Median=1     Mean=2     Mode=1

 

 

1.4a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal process for grants.

 

·   Do not include individual academic departments or research centers

 

 

1.4b    For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal.

 

Administrative Office

Average

Number of

Hours Per NSF Grant Proposal

 

Median=4        Mean=8        Mode=2

 

Median=2        Mean=4        Mode=1

 

Median=1        Mean=3        Mode=1

 

Median=1        Mean=4        Mode=1

 

Median=2        Mean=7        Mode=1

 


 

 

 

 

Text Box: SECTION 2

The following questions are about the process of negotiating grant proposal revisions.

 

2.1      Does your institution have a formal, standardized process that is followed to negotiate grant proposal revisions?

 

           65%     Yes

           34%     No

 

 

2.2      Are there specific individuals or administrative offices assigned to work with grant proposal revisions?

 

·   Do not include principal investigators

 

           85%    Yes

           14%    No        SKIP TO Q2.4a

 

 

2.3      What is the total number of each of the following assigned to grant proposal revisions:

 

           |        |        |  individuals                  Median=3     Mean=4     Mode=2

 

           |        |        |  administrative offices          Median=1     Mean=2     Mode=1

 

 

2.4a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in the proposal revision process for grants.

 

·   Do not include individual academic departments or research centers

 

 

2.4b    For each office, please give your best estimate of the average number of hours individuals in that office spent on a typical FY 2001 NSF grant proposal revision.

 

Administrative Office

Average Number

of Hours Per NSF

Grant Proposal Revision

 

 

Median=1        Mean=3        Mode=1

 

 

Median=1        Mean=2        Mode=1

 

 

Median=1        Mean=2        Mode=1

 

 

Median=2        Mean=5        Mode=1

 

 

Median=7        Mean=7        Mode=*

*No value calculated

 

 

2.5      For a typical NSF grant that your institution is awarded, approximately how many hours are spent communicating directly with NSF on revisions to the original proposal?

 

·   Do not include principal investigator hours

 

           |        |        |        | average number of hours per nsf grant

 

           Median=1     Mean=2     Mode=1


 

 

 

 

 

Text Box: SECTION 3After a grant is awarded, institutions are responsible for administering the grant and providing additional oversight.  For the following questions, please think about grant administration.

 

3.1      Are there specific individuals or administrative offices assigned to administer grant awards?

 

           96%     Yes

             3%     No        SKIP TO Q3.3a

 


3.2      What is the total number of each of the following assigned to administer grants:

 

·   Do not include principal investigators

 

           |        |        |  individuals                                Median=3     Mean=6       Mode=2

 

           |        |        |  administrative offices          Median=2     Mean=2       Mode=2

 

 

3.3a    In the grid below, please identify, up to five, the key administrative offices at your institution involved in administering grant awards.

 

·   Do not include individual academic departments or research centers

 

 

3.3b    For each office, please give your best estimate of the average number of hours individuals in that office spent to administer a typical FY 2001 NSF grant award.

 

Administrative Office

Average Number of Hours Per

NSF Grant Administration

 

 

Median=8          Mean=18        Mode=2

 

 

Median=5          Mean=16        Mode=1

 

 

Median=6          Mean=14        Mode=1

 

 

Median=6          Mean=10        Mode=1

 

 

Median=9          Mean=13        Mode=*

*No value calculated

 

 

3.4      Approximately how many hours are spent to complete and submit NSF required reports for a typical FY 2001 NSF grant?

 

           |        |        |        | average number of hours per nsf grant

 

           Median=4     Mean=8     Mode=2


 

 

 

 

Text Box: SECTION 4

NSF is considering increasing the amount and duration available for grants.  Think about how these potential changes would impact how your institution applies for and administers NSF grants.

 

4.1      If NSF had more money to award each year, please rank from most important (1) to least important (3), the following possible actions for awards to your institution.

 

mark one

 

Ranking

 

1

2

3

 

    36%

39%

12%

Increase only the amount of funding per award

    6%

28%

52%

Increase only the duration per award

    46%

20%

21%

Increase only the total number of awards per year

 

 

4.2      Overall, if NSF made each of the following changes, would it increase, decrease, or not make any difference in the administrative time your institution uses to mange all aspects of NSF awards?

 

 

 

Increase

Time

Needed

Decrease

Time

Needed

No

Difference in

Time Needed

a.               

Increasing the amount of funding for NSF awards................................................................

     17%

       5%

77%

b.               

Increasing the duration of NSF awards...........

     41%

      20%

38%

c.               

Increasing the total number of NSF awards....

     85%

       1%

14%

 

 

4.3      In your opinion, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average dollar amount for each grant award?

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     

 

 

4.4      Now, what, if any, would be the 2 or 3 most significant changes for your institution if NSF increased the average duration for each grant award?

 

                                                                                                                                                     

 

                                                                                                                                                     

 

                                                                                                                                                     

 

 

4.5      Please outline any suggestions you have for NSF changes that would result in a reduction of the amount of time and resources used by your institution to manage NSF grants.