NSF 11-588: Discovery Research K-12 (DRK-12)

Program Solicitation NSF 11-588

National Science Foundation Directorate for Education & Human Resources      Research on Learning in Formal and Informal Settings

Letter of Intent Due Date(s) (required) (due by 5 p.m. proposer's local time): 

     November 17, 2011

     October 04, 2012

Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

     January 10, 2012

     December 06, 2012

Important Information And Revision Notes

A revised version of the NSF Proposal & Award Policies & Procedures Guide (PAPPG), NSF 11-1, was issued on October 1, 2010 and is effective for proposals submitted, or due, on or after January 18, 2011. Please be advised that the guidelines contained in NSF 11-1 apply to proposals submitted in response to this funding opportunity.

Cost Sharing: The PAPPG has been revised to implement the National Science Board's recommendations regarding cost sharing. Inclusion of voluntary committed cost sharing is prohibited. In order to assess the scope of the project, all organizational resources necessary for the project must be described in the Facilities, Equipment and Other Resources section of the proposal. The description should be narrative in nature and must not include any quantifiable financial information. Mandatory cost sharing will only be required when explicitly authorized by the NSF Director. See the PAPP Guide Part I: Grant Proposal Guide (GPG) Chapter II.C.2.g(xi) for further information about the implementation of these recommendations.

Data Management Plan: The PAPPG contains a clarification of NSF's long standing data policy. All proposals must describe plans for data management and sharing of the products of research, or assert the absence of the need for such plans. FastLane will not permit submission of a proposal that is missing a Data Management Plan. The Data Management Plan will be reviewed as part of the intellectual merit or broader impacts of the proposal, or both, as appropriate. Links to data management requirements and plans relevant to specific Directorates, Offices, Divisions, Programs, or other NSF units are available on the NSF website at: https://www.nsf.gov/bfa/dias/policy/dmp.jsp. See
Chapter II.C.2.j of the GPG for further information about the implementation of this requirement.

Postdoctoral Researcher Mentoring Plan: As a reminder, each proposal that requests funding to support postdoctoral researchers must include, as a supplementary document, a description of the mentoring activities that will be provided for such individuals. Please be advised that if required, FastLane will not permit submission of a proposal that is missing a Postdoctoral Researcher Mentoring Plan. See Chapter II.C.2.j of the GPG for further information about the implementation of this requirement.

Revision Summary

The Discovery Research K-12 (DRK-12) program solicitation supports projects that lead to significant and sustainable improvements in STEM learning, advance STEM teaching, and contribute to improvements in the nation's formal education system. Successful DRK-12 projects emphasize both research on and development of innovative STEM resources, models, and tools. DRK-12 is interested in projects that build upon educational research (theory, knowledge, findings) and promote effective STEM practices in diverse preK-12 classrooms. DRK-12 is also interested in high risk/high return projects that have the potential to radically transform formal STEM education.

Revisions to this Solicitation:

(1) Adjusted the award amount and duration of Full Research and Development awards;

(2) Incorporated the call for highly innovative learning materials into Strand 2 and eliminated previous Challenge 5; and

(3) Conference and workshop proposals may no longer be submitted at any time during the year and are now due at the same deadline as all other DRK-12 proposals.

Summary Of Program Requirements

General Information

Program Title:

Discovery Research K-12 (DRK-12)

Synopsis of Program:

The Discovery Research K-12 program (DRK-12) seeks to significantly enhance the learning and teaching of Science, Technology, Engineering and Mathematics (STEM) by preK-12 students, teachers, administrators and parents. All DRK-12 projects should be framed around a research question or hypothesis that addresses an important need or topic in preK-12 STEM education. The emphasis in DRK-12 is on research projects that study the development, testing, deployment, effectiveness, and/or scale-up of innovative resources, models and tools. DRK-12 invites proposals that address immediate challenges that are facing preK-12 STEM education as well as those that anticipate a radically different structure and function of pre-K 12 teaching and learning. DRK-12 especially encourages proposals that challenge existing assumptions about learning and teaching within or across STEM fields, envision the future needs of learners, and consider new and innovative ways to support student and teacher learning. DRK-12 is particularly interested in projects that hold promise for identifying and developing the next generation of STEM innovators (NSB, 2010). There are four strands described in detail in the solicitation: 1) Assessment; 2) Learning; 3) Teaching; 4) Scale-up.

DRK-12 projects are based on theories of learning, prior research and development. Projects reflect the needs of an increasingly diverse population as well as national, state, or discipline priorities. Outcomes include usable and scalable resources, models, tools, and contributions to the knowledge about STEM teaching and learning. In addition, teachers and students who participate in DRK-12 studies are expected to enhance their understanding and use of STEM content, practices and skills.

The DRK-12 program is primarily concerned with the goals and effectiveness of formal education, but recognizes that learning is not limited to formal school environments and times. The program encourages projects to draw from knowledge and practice of learning in out-of-school and informal settings.

Most young people and STEM professionals today use powerful technologies in the activities of their everyday lives. New knowledge, new ways of thinking, and new ways of finding and processing information drive our society and economy. Many of the resources, models and tools researched and developed by DRK-12 will provide innovative ways to use current and emerging technologies to transform STEM education.

DRK-12 recognizes that outstanding teaching is a critical and integral component of this improvement process. While Strand 3 has a specific focus on resources, models and tools for teacher education and the impact of those models on student learning, projects submitted to the other strands may also include teacher support materials or professional development components in support of student learning. Projects submitted to the Learning strand might also include the development of assessments related to the specific goals of the project.

Some DRK-12 projects focus on a specific STEM discipline or concept, while others have cross-disciplinary, cross-grade level content, but all projects must demonstrate that the content is important from both a disciplinary and learning perspective.

Full Research and Development projects are expected to lead to successful dissemination and adoption of findings or products in the preK-12 enterprise at a scale beyond that directly supported by the grant.

Cognizant Program Officer(s):

Please note that the following information is current ;at the time of publishing. See program website for any updates to the points of contact.

• Inquiries can be made to, telephone: (703) 292-8620, email: DRLDRK12@nsf.gov

• David Campbell, telephone: (703) 292-5093, email: dcampbel@nsf.gov

• Julia Clark, telephone: (703) 292-5119, email: jclark@nsf.gov

• Edward Geary, telephone: (703) 292-4960, email: egeary@nsf.gov

• Robert Gibbs, telephone: (703) 292-5122, email: rgibbs@nsf.gov

• Edith Gummer, telephone: (703) 292-5110, email: egummer@nsf.gov

• James Hamos, telephone: (703) 292-4687, email: jhamos@nsf.gov

• David Haury, telephone: (703) 292-5102, email: dhaury@nsf.gov

• Maria Oliver-Hoyo, telephone: (703) 292-4313, email: moliver@nsf.gov

• Janet Kolodner, telephone: (703) 292-8930, email: jkolodne@nsf.gov

• Julio Lopez-Ferrao, telephone: (703) 292-5183, email: jlopezfe@nsf.gov

• Ellen McCallie, telephone: (703) 292-5115 email: emccalli@nsf.gov

• Joseph Reed, telephone: (703) 292-5187, email: jreed@nsf.gov

• Gerhard Salinger, telephone: (703) 292-5116, email: gsalinge@nsf.gov

• Elizabeth VanderPutten, telephone: (703) 292-5147, email: evanderp@nsf.gov

• Darryl Williams, telephone: (703) 292-7906, email: dnwillia@nsf.gov

Applicable Catalog of Federal Domestic Assistance (CFDA) Number(s):

• 47.076 --- Education and Human Resources

Award Information

Anticipated Type of Award: Standard Grant or Continuing Grant

Estimated Number of Awards: 35 to 45 per year. It is anticipated that about 15-20 Exploratory awards, 15-20 Full Research and Development awards, and 5 Conference/Workshop awards will be made in FY 2012, pending availability of funds.

Anticipated Funding Amount: $40,000,000 in FY 2012 for new awards made under this solicitation, pending availability of funds. Normal limits for funding requests of DRK-12 proposals are as follows: (1) Exploratory projects up to $450,000 with duration up to three years; (2a) Full Research and Development projects up to $3,000,000 with duration up to four years; (2b) Full Research and Development projects with a primary focus on learning how to take proven STEM innovations to scale, up to $4,000,000 with a duration of four years; (3) Conference/Workshop projects up to $100,000 for duration up to two years.

Eligibility Information

Organization Limit:

None Specified

PI Limit:

None Specified

Limit on Number of Proposals per Organization:

None Specified

Limit on Number of Proposals per PI:

None Specified

Proposal Preparation and Submission Instructions

A. Proposal Preparation Instructions

• Letters of Intent:Submission of Letters of Intent is required. Please see the full text of this solicitation for further information.

• Preliminary Proposal Submission: Not Applicable

• Full Proposals:
  • Full Proposals submitted via FastLane: NSF Proposal and Award Policies and Procedures Guide, Part I: Grant Proposal Guide (GPG) Guidelines apply. The complete text of the GPG is available electronically on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg.
  • Full Proposals submitted via Grants.gov: NSF Grants.gov Application Guide: A Guide for the Preparation and Submission of NSF Applications via Grants.gov Guidelines apply (Note: The NSF Grants.gov Application Guide is available on the Grants.gov website and on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=grantsgovguide)

B. Budgetary Information

• Cost Sharing Requirements: Inclusion of voluntary committed cost sharing is prohibited.

• Indirect Cost (F&A) Limitations: Not Applicable

• Other Budgetary Limitations: Not Applicable

C. Due Dates

• Letter of Intent Due Date(s) (required) (due by 5 p.m. proposer's local time): 

       November 17, 2011

       October 04, 2012

• Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

       January 10, 2012

       December 06, 2012

Proposal Review Information Criteria

Merit Review Criteria: National Science Board approved criteria apply.

Award Administration Information

Award Conditions: Standard NSF award conditions apply.

Reporting Requirements: Additional reporting requirements apply. Please see the full text of this solicitation for further information.

Agency Contacts

Other Information

I. Introduction

About the National Science Foundation and the Directorate for Education and Human Resources

The National Science Foundation (NSF) is charged with promoting the vitality of the nation's science, technology, engineering and mathematics (STEM) research and education enterprises.

The mission of the Directorate of Education and Human Resources (EHR) is to achieve excellence in U.S. STEM education at all levels and in all settings (both formal and informal). EHR programs support the development of a diverse and well-prepared workforce of scientists, technicians, engineers, mathematicians and educators and a well-informed citizenry that have access to the ideas and tools of science and engineering. The purpose of these activities is to enhance the quality of life of all citizens and the health, prosperity, welfare and security of the nation.

EHR Goals

1. Prepare the next generation of STEM professionals and attract and retain more Americans to STEM careers.

2. Develop a robust research community that can conduct rigorous research and evaluation that will support excellence in STEM education and that integrates research and education.

3. Increase the technological, scientific and quantitative literacy of all Americans so that they can exercise responsible citizenship and live productive lives in an increasingly technological society.

4. Broaden participation (individuals, geographic regions, types of institutions, STEM disciplines) and close achievement gaps in all STEM fields.

To achieve these goals, the Directorate sponsors programs in the Division of Research on Learning in Formal and Informal Settings (DRL), Division of Undergraduate Education (DUE), Division of Graduate Education (DGE), and Human Resource Development (HRD). The DRK-12 program is managed in DRL.

The Division of Research on Learning in Formal and Informal Settings invests in projects to enhance STEM learning for people of all ages in both formal and informal learning settings. Its mission includes promoting innovative and transformative research and development, and evaluation of learning and teaching in all STEM disciplines. New and emerging areas of STEM must play prominent roles in efforts to improve STEM education. The integration of cutting-edge STEM content and the engagement of scientists, engineers, and educators from the range of disciplines represented at NSF is encouraged in all DRL initiatives. DRL's role is to be a catalyst for change by advancing theory, method, measurement, development, evaluation, and application in STEM education. The Division seeks to support both development of promising new ideas and scale-up and sustainability of proven educational innovations.

The Division's programs offer a set of complementary approaches for advancing research, development, and improvement of practice.

• The Discovery Research K-12 (DRK-12) program enables significant advances in preK-12 learning of the STEM disciplines through research and development on innovative resources, models, and tools for use by students, teachers, administrators and policy makers.
• The Informal Science Education (ISE) program supports innovation in anywhere, anytime, lifelong learning, through investments in research, development, infrastructure and capacity-building for STEM learning outside formal school settings.
• The Innovative Technology Experiences for Students and Teachers (ITEST) program invests in projects designed to address the growing demand for professional and information technology workers through the design, implementation, scale-up, and testing of technology-intensive educational experiences for students and teachers, and through related research studies.
• The Research and Evaluation on Education in Science and Engineering (REESE) program advances research at the frontiers of STEM learning, education, and evaluation, in order to provide foundational knowledge for improving STEM teaching and learning at all educational levels and in all settings.
• The Transforming STEM Learning (TSL) activity is a combined effort of the programs listed above to challenge existing assumptions, encourage innovative thinking and catalyze the development of new models for future STEM learning.
• The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program supports research on evaluation with special emphasis on innovative approaches for determining the impacts of STEM projects and growing the capacity of the evaluation field.

Each of these programs is intended to improve their field's capacity to further STEM learning. They are central to NSF's strategic goals of Learning and Discovery, helping to cultivate a world-class, broadly inclusive STEM workforce, expanding the scientific literacy of all citizens, and promoting research that advances the frontiers of knowledge (NSF, 2011).

DRL and Innovation

All research and development activities within DRL aim at generating knowledge and transforming practice in STEM education. DRL's programs are designed to complement each other within a cycle of innovation and learning (see Figure 1) that forms the conceptual framework for its programs (adapted from American Statistical Association, 2007; NSF, 2005; RAND, 2003). All DRL programs are concerned with all five components of the cycle, to different degrees.

Each part of the cycle forms the vital and compelling foundation for transition to the next part of the cycle. The research, development implementation and evaluation activities need to be appropriately rigorous. Projects funded by DRL are providing the ideas, resources, and human capacity to advance STEM learning and education in the 21^st century:

• Challenging the STEM education and research communities with transformative ideas
• Conducting the pioneering and pragmatic research necessary to advance STEM learning and teaching
• Developing and studying world-class learning resources for students, teachers, and the public
• Addressing workforce needs through the design and study of technology-intensive educational experiences
• Enhancing diversity in the STEM workforce

The major distinction between the DRK-12 and REESE programs is that DRK-12 projects focus on development and study of specific resources, models and tools designed to improve STEM education in preK-12 schools, while REESE projects focus primarily on building theory and knowledge about STEM education across learning contexts and ages. The outcomes of DRK-12 projects will be STEM education innovations and products that are informed by research and tested in practice. The primary outcomes of REESE projects will be research findings, methods, and theoretical perspectives about STEM education. Proposers who are in doubt about the appropriate program for funding of their work should consult an NSF program officer.

In addition, all DRL program Principal Investigators who have created innovative and proven resources, models, and tools with commercialization potential are eligible to apply to NSF's new Innovation Corps Program (I-Corps). The purpose of I-Corps is to identify NSF-funded researchers who will receive additional support - in the form of mentoring and funding - to accelerate innovations and products that can attract subsequent third-party funding. To learn more about I-Corps, visit https://www.nsf.gov/funding/pgm_summ.jsp?pims_id=504672

II. Program Description

The NSF 2011-2016 Strategic Plan calls for the integration of research and education to ensure the Nation's capacity to generate the workforce needed to meet the challenges of a rapidly evolving workplace. This includes a commitment to drawing in and retaining a diverse population of students in STEM fields. The Discovery Research K-12 (DRK-12) program seeks to significantly enhance STEM learning for children and adolescents through research, development, testing, deployment and scale-up of exemplary resources, models and tools. The intent is to catalyze new approaches to STEM learning, develop students' 21^st century STEM workforce skills, and provide the pathways and resources to study the learning process itself.

DRK-12 invites proposals that address immediate challenges facing preK-12 STEM education as well as those that anticipate a radically different structure and function of pre-K 12 teaching and learning. DRK-12 especially encourages proposals that challenge existing assumptions about learning and teaching within or across STEM fields, envision the future needs of learners, and consider new and innovative ways to support their learning. DRK-12 is particularly interested in projects that hold promise for identifying and developing the next generation of STEM innovators (NSB, 2010).

Collectively, the four strands described below foster the creation of a new generation of resources, models, and tools that take full advantage of the rich research base on STEM learning and the capabilities of modern information and communications technologies to enhance the education of preK-12 learners and teachers. Specifically, the DRK-12 program encourages the development, study, and use of learning and teaching resources that motivate and engage learners in deep and meaningful investigations within a coherent curriculum. These resources should be dynamic, responsive, and adaptable to support the wide range of interests, abilities, languages and cultures in modern classrooms. In addition, DRK-12 encourages research and development of new STEM learning resources, models, and tools that help students become scientific and engineering problem solvers. They should also model ways of learning and doing that are personally sustainable, preparing young people, teachers, and administrators to be lifelong learners who effectively utilize cutting-edge technologies and are able to learn and adapt to a rapidly changing world.

The DRK-12 program seeks to maintain a balanced portfolio by supporting projects ranging from those with immediate applicability to those that anticipate and provide the foundation for preK-12 STEM education as it could be in future decades. Projects that address immediate and pressing challenges typically develop and study resources, models and tools that could be implemented and brought to scale in the relatively near term, albeit in highly innovative ways.

Projects that anticipate education as it could be in the future (10-20 years) will necessarily put forward ideas, concepts, theories, and development that challenge existing assumptions about STEM learning and teaching. Such projects, for example, might develop and study resources, models and tools that (1) are dramatically more effective with the diversity of learners they will serve; (2) support STEM learning with collaborative and interactive technologies; (3) help students and teachers draw on the expertise and resources of scientists and practitioners located far from the classroom or teacher education setting; or (4) link in-school and out-of-school STEM learning in new ways.

Full Research and Development projects lead to products and findings that can be successfully disseminated and adopted at a scale beyond that directly supported by the grant.

Information on current DRK-12 projects can be found at www.cadrek12.org

A. DRK-12 Program Strands

1) The Assessment Strand: projects that develop and study valid and reliable assessments of student and teacher knowledge, skills, and practices.

In an era of increased accountability in preK-12 education, resources, models and tools for assessing STEM content knowledge, effective practices, affective characteristics, and habits of mind must keep pace with and anticipate the demands of instruction and educational policy. Among the most pressing issues is the alignment of preK-12 assessments with the learning goals represented in widely used standards documents. In line with recommendations in the Common Core State Standards in Mathematics (http://www.corestandards.org/the-standards/mathematics), the K-12 Science Framework (NRC, 2011), and the report on Successful K-12 STEM Education (NRC, 2011), assessments should probe student understanding of the most important disciplinary concepts, principles, and mathematical, engineering, and scientific practices, as well as the application of that knowledge to problem-solving and decision-making.

DRK-12 is interested in supporting the development and study of innovations in (1) summative assessment of student content knowledge, skills, attitudes, beliefs, motivation, aptitudes, interests, creativity and other important objectives of STEM education; (2) formative assessment of student progress in learning STEM concepts, skills, and practices; and (3) valid and reliable assessments of STEM teacher content and pedagogical content knowledge, effective teaching practices, confidence, interest, and motivation.

Assessing the full scope of mathematical, scientific, and technological proficiency in valid and reliable ways presents conceptual, psychometric, and practical challenges. Thus, in all assessment projects, collaborations with psychometricians, STEM disciplinary experts and educational researchers are expected.

2) The Learning Strand: projects that develop and study resources, models and tools to support all students' STEM learning, enhance their knowledge and abilities, and build their interest in STEM fields.

The DRK-12 program seeks proposals framed around an important research question or hypothesis related to student learning. Successful projects in this strand will develop and study innovative resources, models, and tools that will substantially improve how and what pre-K-12 children learn. NSF seeks to support proposals that examine what is possible if the constraints of pedagogical tradition, educational policies, and limited resources are challenged by emerging research findings and the application of powerful and pervasive technologies and media. The program especially encourages projects that: (1) prepare students to understand increasingly sophisticated content in STEM subjects (Wiggins, 2005; NRC 2008); (2) engage students in meaningful scientific data collection, analysis, visualization, modeling and interpretation, (3) develop important, cross-cutting concepts and ideas needed to understand important interdisciplinary subjects like environmental sustainability, climate change, and renewable and non-renewable energy sources, (4) help students learn STEM practices, modes of inquiry, scientific investigation, and engineering design through hands-on activities, real and virtual laboratories, field experiences, and collaborations with STEM professionals and peers enabled by cyberinfrastructure and/or (5) provide substantive STEM learning activities that effectively engage and serve the diversity of learners found in contemporary U.S. classrooms. While some DRK-12 proposals may include one or more of these components, other proposals in this strand may focus on other aspects of student learning. All Full Research and Development Learning Strand projects must result in a usable product and indicate how the product could be adopted and directly support formal education.

Recent examples of successful DRK-12 projects include development and study of: adaptive tutors, scalable differentiated instruction, place-based learning, embedded formative assessments, blended learning environments (in and out of classrooms activities and projects), new laboratory designs, crowd or people sourcing, interactive digital textbooks, virtual environments and simulations, visualization tools, virtual scientific instruments, materials that are adaptable to the learner and learning environment, dynamic content that is constantly updated and improved, and ways to support instruction that transcend the limitations imposed by traditional classrooms.

3) The Teaching Strand: projects that develop and study resources, models and tools to help pre- and in-service teachers provide high quality STEM education for all students.

The DRK-12 program recognizes that a well-prepared and well-supported STEM teacher workforce is crucial to excellent preK-12 STEM education. Thus the program seeks proposals to study existing teacher pre- and inservice programs and develop innovative scalable models that support learning by preK-12 teachers at all points in their careers. For example, projects in this area might develop and study (1) innovative models to recruit, certify, induct, and retain STEM teachers; (2) new resources for helping pre- and in-service teachers develop content and pedagogical knowledge and skills; or (3) tools for sharing teaching expertise within schools and districts and across the broader national teacher community. As with all DRK-12 projects, these proposals should start with a specific research question or hypothesis, build on an explicit theory of learning, generate resources, models, and tools that are useful and usable by others, and use appropriate research and development designs.

Teachers today have unprecedented access to and experience with communication, information, and learning technologies that facilitate social networking, virtual gaming, scientific data analysis, and collaborations with scientists. At the same time, STEM fields are rapidly changing with greater emphasis on collaborative, interdisciplinary research. A major challenge in teacher preparation and professional development is in applying technological and human resources to the professional work of teaching. The DRK-12 program is especially interested in supporting projects that anticipate professional learning options and the needs of teachers who work in a global environment with powerful cyberinfrastructure.

Some examples of innovative resources, models and tools developed and studied by DRK-12 teacher education projects might include: (1) just-in-time online courses or digital library repositories and ways of using web-resources for teaching; (2) models for teacher networking and collaboration and tools to allow productive communication with peers, mentors, parents, and experts around the world; (3) tools that provide teachers with dynamic diagnostic information about student learning in real-time as well as tools that provide teachers with the ability to customize curriculum to meet the needs of diverse student populations; (4) models for helping teachers implement cutting-edge STEM content, or other teacher education innovations. All Full Research and Development projects should include an analysis of the impact of these resources, models and tools on teacher learning and practice and collected evidence on student learning.

4) The Scale-up and Sustainability Strand: projects that develop and study the factors that contribute to successful implementation, scale-up, and sustainability of proven, high-quality innovations in schools and districts in a cost effective manner.

Many studies of innovative resources, models, or tools have demonstrated positive effects on student or teacher STEM outcomes in a small number of sites under carefully controlled conditions. A key challenge is identifying conditions under which such promising innovations can be successfully implemented and sustained in a broad range of schools and districts across the country (Dede, et.al., 2005). Studies addressing this challenge could seek to understand how innovative resources, models, or tools that data have shown to be successful at a moderate scale, or in a particular grade band, can be disseminated, implemented, and scaled up in increasingly varied sites or adapted for use with additional audiences or grade levels. Alternatively, studies could focus on how STEM learning innovations can be successfully sustained in classrooms in combination with out-of-school partners or by using informal learning settings. The resources, models or technologies studied in such scale-up research may include work supported by NSF or any other source.

The DRK-12 program also accepts proposals that study the impact of resources, models, and tools when taken to scale. In such cases, the effects of specific STEM education interventions should have been established through smaller scale efficacy studies before submitting a proposal to examine if the effects are sustained when the innovation is taken to scale. These research projects should be designed to include an appropriate number and variety of sites to justify broadly generalizable results.

Because scale-up studies generally aim to attribute improvements in STEM education practice and/or results to an intervention, the research designs must involve a statistically appropriate number and nesting of individuals, classes, or schools, and should give careful attention to measures of fidelity and adaptation when the intervention is implemented. Effectiveness of the innovation should be assessed by appropriate, valid, and reliable instruments. Experimental studies with random assignment to treatment are encouraged. Longitudinal studies of student achievement may be appropriate for studies of impact at large scale.

Studies of organization and scale: Another type of scale-up study examines how a specific new resource, model, or technology is implemented, institutionalized, and sustained with the aim of understanding the organizational elements necessary for implementing the innovation successfully. Research questions for such studies might focus on implementation factors such as: (1) school or district financial investments, leadership, and organizational practices; (2) feasibility and fidelity of classroom implementation; (3) teacher professional development in support of the innovation; (4) engagement of teachers, administrators, and community representatives in adoption and implementation decisions; and/or (5) policy issues such as the innovation's alignment with state standards or assessments. Studies of the implementation and scale-up process might employ qualitative, quantitative, or mixed research methods to document, analyze, and interpret relationships between critical implementation factors and outcomes.

All scale-up studies must provide sufficient evidence that prior research on the resource, model, or technology being studied has provided efficacy data showing positive impact on student learning under specific conditions. Results of previous rigorous experimental or quasi-experimental studies or meta-analysis of related studies might provide such evidence of efficacy.

B. Additional Program Information Applicable to Proposal Types

The DRK-12 program invites proposals for three types of projects: Exploratory projects, Full Research and Development projects and Conferences and Workshops.

Exploratory projects allow researchers and developers an opportunity to undertake preliminary work needed to clarify constructs, assemble theoretical or conceptual foundations, or perform early investigations of an idea for an innovative resource, model, or tool. Exploratory projects can also focus on the innovative repurposing or adaptation of existing resources, models, or tools. These short duration projects might develop prototype educational materials or practices and conduct research in small-scale pilot tests to provide proof of concept and preliminary estimates of impact. These projects should produce empirical evidence forming the basis of anticipated further research and development work. Exploratory projects may also be synthesis projects that bring together findings on current technology-enhanced resources and models to identify new directions for research and development. DRK-12 particularly encourages synthesis projects that provide research findings and recommendations that are useful to STEM education practitioners and decision makers.

Full Research and Development projects are built on the most promising Exploratory projects or other (non-NSF funded) projects. These projects have already demonstrated effectiveness in small sets of classrooms, schools, or other learning settings. Greater funding levels and longer timelines allow researchers and developers an opportunity to undertake more in-depth product development, more targeted research, and to reach a broader, more diverse, audience. Resources, models, or tools developed in full research and development projects should result in completed products, ready for implementation by others who request them.

Conferences and Workshops related to the mission of the DRK-12 program are also supported under this solicitation. Conferences or workshops should be well focused, related to the goals of the program, and generate a product usable by researchers or practitioners.

Conference and workshop proposals must be submitted by the deadline. All conference proposals should provide for an evaluation of the impact of the conference to be conducted at least 12 months after the conference is completed.

Please see the Proposal and Award Policies and Procedures Guide/Grant Proposal Guide Section II. D. for additional information about conference and workshop proposals.

References

American Statistical Association (2007). Using statistics effectively in mathematics education research. Retrieved July 9, 2007 from http://www.amstat.org/education/pdfs/UsingStatisticsEffectivelyinMathEdResearch.pdf.

Clements, D. H. (2007). Curriculum research: Toward a framework for "Research-based curricula". Journal for Research in Mathematics Education, 38 (1): 35-70.

Cobb, P., Confrey, J., deSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9 -13.

Common Core State Standards for Mathematics (2010) http://www.corestandards.org/assets/CCSSI_Math%20Standards.pdf

Frechtling, J., Mark, M., Rog, D., Thomas, V., Frierson, H., Hood, S., Hughes, G., Johnson, E. (2010) The 2010 User-Friendly Handbook for Project Evaluation, http://www.westat.com/Westat/pdf/news/UFHB.pdf

Lamberg, T. & Middleton, J. A. (2009). Design research perspectives on transitioning from individual microgenetic interviews to a whole-class teaching experiment. Educational Researcher 38(4), 233-245.

National Mathematics Advisory Panel (2008). Foundations for Success: The Final Report of the National Mathematics Advisory Panel, U.S. Department of Education: Washington, DC. National Research Council (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.

National Research Council (2002). Scientific Research in Education. Washington, DC: National Academy Press.

National Research Council (2003). Assessment in support of instruction and learning: Bridging the gap between large-scale and classroom assessment. Washington, DC: National Academy Press.

National Research Council (2004). On evaluating curricular effectiveness: Judging the quality of K-12 mathematics evaluations. Washington, DC: National Academy Press.

National Research Council (2006). Systems for State Science Assessment. Washington, DC: National Academy Press.

National Research Council (2007), Taking Science to School: Learning and Teaching Science in Grades K-8, Washington, DC: National Academy Press.

National Research Council (2008), Research on Future Skill Demands. Washington, DC: National Academy Press.

National Research Council (2009). Learning Science in Informal Environments: People, Places, and Pursuits. Washington, DC: National Academy Press.

National Research Council (2011). Successful K-12 STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics. Washington, DC: National Academy Press.

National Science Foundation (2005). The mathematics education portfolio brief, (NSF 05-03). https://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf0503

National Science Foundation Task Force on Cyberlearning. (2008). Fostering Learning the Networked World. National Science Foundation.

National Science Foundation/National Science Board (2010), Preparing the Next Generation of STEM Innovators: Identifying and Developing our Nation's Human Capital.

National Science Foundation (2011), Empowering the Nation Through Discovery and Innovation, NSF Strategic Plan for 2011-2016.

President's Council of Advisors on Science and Technology (2010). Prepare and Inspire: K-12 Education in Science, Technology, Engineering, and Math (STEM) for America's Future. http://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-stem-ed-final.pdf

RAND Mathematics Study Panel (2003). Mathematical proficiency for all students: Toward a strategic research and development program in mathematics education. Santa Monica, CA: RAND Corporation.

Scaling Up Success: Lessons Learned from Technology-Based Educational Improvement (2005), Chris Dede, James Honan, Laurence Peters, editors, John Wiley and Sons, 176pgs.

Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W. H., & Shavelson, R. J. (2007). Estimating causal effects using experimental and observational designs (report from the Governing Board of the American Educational Research Association Grants Program). Washington, DC: American Educational Research Association.

Shadish, W., Cook, T., and Campbell, D. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton-Mifflin Company.

U.S. Department of Education, Office of Educational Technology (March 5, 2010). Transforming American Education: Learning Powered by Technology.

Wiggins, G.P., & McTighe, J. (2005). Understanding by design (2nd Edition). Upper Saddle River, NJ: Prentice Hall.

III. Award Information

Anticipated Type of Award: Continuing Grant or Standard Grant

Estimated Number of Awards: 35 to 45 per year. It is anticipated that about 15-20 Exploratory awards, 15-20 Full Research and Development awards, and 5 Conference/Workshop awards will be made in FY 2012, pending availability of funds.

Anticipated Funding Amount: $40,000,000 in FY 2012 for new awards made under this solicitation, pending availability of funds. Normal limits for funding requests of DRK-12 proposals are as follows: (1) Exploratory projects up to $450,000 with duration up to three years; (2a) Full Research and Development projects up to $3,000,000 with duration up to four years; (2b) Full Research and Development projects with a primary focus on learning how to take proven STEM innovations to scale, up to $4,000,000 with a duration of four years; (3) Conference/Workshop projects up to $100,000 for duration up to two years.

IV. Eligibility Information

The categories of proposers eligible to submit proposals to the National Science Foundation are identified in the Grant Proposal Guide, Chapter I, Section E.

Organization Limit:

None Specified

PI Limit:

None Specified

Limit on Number of Proposals per Organization:

None Specified

Limit on Number of Proposals per PI:

None Specified

V. Proposal Preparation And Submission Instructions

A. Proposal Preparation Instructions

Letters of Intent(required):

Specify the Strand that the proposal addresses, the tentative project title, the principal investigators, and the organizations involved.

Letters of Intent must be submitted via the NSF FastLane system, using the Letter of Intent module in FastLane, for all DRK-12 proposals except conferences.

Letters of Intent are limited to 2,500 characters, including spaces (approximately 350 words). Your Letter of Intent should contain a brief narrative that describes the project and provides the following information: (1) a project title; (2) clear identification of the primary Strand; (3) the STEM focus; (4) a list of proposed Principal Investigators and Co-Principal Investigators, including organizational affiliations and departments; (5) other partnering institutions; (5) STEM discipline; and (6) grade levels as appropriate.

Letter of Intent Preparation Instructions:

When submitting a Letter of Intent through FastLane in response to this Program Solicitation please note the conditions outlined below:

• Sponsored Projects Office (SPO) Submission is not required when submitting Letters of Intent
• Submission of multiple Letters of Intent is allowed

Full Proposal Preparation Instructions: Proposers may opt to submit proposals in response to this Program Solicitation via Grants.gov or via the NSF FastLane system.

• Full proposals submitted via FastLane: Proposals submitted in response to this program solicitation should be prepared and submitted in accordance with the general guidelines contained in the NSF Grant Proposal Guide (GPG). The complete text of the GPG is available electronically on the NSF website at: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg. Paper copies of the GPG may be obtained from the NSF Publications Clearinghouse, telephone (703) 292-7827 or by e-mail from nsfpubs@nsf.gov. Proposers are reminded to identify this program solicitation number in the program solicitation block on the NSF Cover Sheet For Proposal to the National Science Foundation. Compliance with this requirement is critical to determining the relevant proposal processing guidelines. Failure to submit this information may delay processing.

• Full proposals submitted via Grants.gov: Proposals submitted in response to this program solicitation via Grants.gov should be prepared and submitted in accordance with the NSF Grants.gov Application Guide: A Guide for the Preparation and Submission of NSF Applications via Grants.gov. The complete text of the NSF Grants.gov Application Guide is available on the Grants.gov website and on the NSF website at: (https://www.nsf.gov/publications/pub_summ.jsp?ods_key=grantsgovguide). To obtain copies of the Application Guide and Application Forms Package, click on the Apply tab on the Grants.gov site, then click on the Apply Step 1: Download a Grant Application Package and Application Instructions link and enter the funding opportunity number, (the program solicitation number without the NSF prefix) and press the Download Package button. Paper copies of the Grants.gov Application Guide also may be obtained from the NSF Publications Clearinghouse, telephone (703) 292-7827 or by e-mail from nsfpubs@nsf.gov.

In determining which method to utilize in the electronic preparation and submission of the proposal, please note the following:

Collaborative Proposals. All collaborative proposals submitted as separate submissions from multiple organizations must be submitted via the NSF FastLane system. Chapter II, Section D.4 of the Grant Proposal Guide provides additional information on collaborative proposals.

Information Applicable to all Proposals

Cover Sheet. Complete this form with the appropriate information. The DRK-12 Program Solicitation number must be entered on the first line of the cover page. (Grants.gov Users: The program solicitation number will be pre-populated by Grants.gov on the NSF Grant Application Cover Page.) All proposals submitted to DRK-12 are assumed to have the potential for conducting research on human subjects. Therefore, proposers must select the human subjects box on the cover sheet and should have prior or pending approval of their research from the appropriate institutional review board (IRB).

Project Summary. The first sentence of the Project Summary should specify the type of proposal (e.g., Exploratory; Full Research and Development; Workshop/Conference) and the Strand addressed. The second sentence should state the discipline or disciplines being addressed and grade level(s) if appropriate. Unless the two National Science Board criteria--intellectual merit and broader impacts--are addressed explicitly in separate statements in the project summary, the proposal will be returned without review.

Project Description. Project descriptions are limited to 15 pages and must comply with all formatting requirements of the most current Grant Proposal Guide. Proposals funded under this solicitation must begin with a research question or hypothesis about preK-12 STEM learning.

All proposals for the DRK-12 solicitation must address the following elements in the 15-page project description:

1. Importance:

The proposal must clearly show why the proposed project has an important STEM focus, addresses critical educational needs, and has the potential for broad impact. Proposals of all types (Exploratory, Full Research and Development, and Conference/Workshop) must articulate the goals of the proposed project and why the goals are important for STEM education. The proposal should provide a rationale for how the project will improve STEM education for students and advance knowledge, and it should explain how products or findings might ultimately be implemented in schools on a large scale.

2. Results from prior NSF support

The proposal must describe results of prior NSF support for related educational projects in which the PI or co-PI have been involved. Include evidence of the quality and effectiveness of the resources, models and technologies previously developed. Describe how prior work influences this proposal.

3. Research and Development Design

The design of any DRK-12 project begins with a hypothesis about how some aspect of STEM education can be improved based on theories of learning and development. The proposal then offers a plan for developing an innovative resource, model, or tool and studying the innovation's impact on STEM learning. The proposal should articulate a plan of work that describes research and development strategies appropriate for attaining its goals. Proposals must demonstrate how the work is related to similar research and development.

4. Evaluation

All DRK-12 proposals must have a plan for formative and summative evaluation of project research and development work. Exploratory projects typically have more limited evaluation plans than Full Research and Development projects. The evaluation should focus on the validation of, fidelity to, and the usefulness of the development and research processes to achieve the targeted outcomes. The objectives of the evaluation include: (1) assessing whether the project is making satisfactory progress toward its goals; (2) recommending reasonable, evidenced-based adjustments to project plans; (3) determining the effectiveness and impact of the research, resources, models, and tools developed by the project; and (4) attesting to the integrity of outcomes reported by the project. Proposals should describe the main features of the evaluation design -- the evaluation questions, the data to be gathered, the data analysis plans, and the expertise of the investigators who will be responsible for the work. Each proposal should clearly distinguish between the role of the evaluation effort and that of other critical product and/or research components.

Formative evaluation serves primarily to provide timely feedback to the development and research team. At a minimum, the formative evaluation should validate that the project activities are guided by a reasonable theory of action, are of high quality, are on schedule, and are likely to result in the attainment of the broad goals and objectives of the project. The evaluation plan should explain how appropriate feedback will be given to the project leadership so that it can make timely modifications to the project activities and address significant issues.

Summative evaluation must be conducted by capable professionals who are external to the development and research team(s) and usually external to the team's institutions. The summative evaluation substantiates that the project has collected credible evidence to answer its research questions and hypotheses and/or justify its claims, and reports on threats to the internal and external validity of the research findings. Although the project development and research teams might conduct the majority of the data-gathering, analysis and interpretation as part of the core work of the project, the evaluator would use their work and other data to evaluate success of that work in meeting project objectives. The summative evaluation must be submitted as part of the final project report.

The type and extent of the evaluation may vary by type of project (Exploratory, Full Research and Development, or Conferences/Workshops) and the monetary level of the award. For additional information about project evaluation please see the 2010 User-Friendly Handbook for Project Evaluation (http://www.westat.com/Westat/pdf/news/UFHB.pdf )

For Exploratory projects, the evaluation functions should be performed by a capable individual or an advisory board with independence from the project. The evaluation should be primarily formative, suggesting ways to improve project implementation, checking the validity of project research findings and interpretations, and/or gauging the quality of the resources, models, or tools being developed.

For Full Research and Development projects that aim to address larger issues in more depth and that have larger budgets, the evaluation must have plans for both formative and summative evaluations. The evaluators may be the same or different for each type of evaluation but must be sufficiently distant from the project to assure confidence in the objectivity of the evaluation.

For such projects that involve a complex intervention, the evaluation capacity might need to be expanded to include an expert advisory board to evaluate the project and advise the research team.

5. Dissemination

Proposals should include plans for effective dissemination of project resources, models, tools, and findings to researchers, policymakers, and practitioners. The dissemination plan should include a description of anticipated contributions of the research and development activities to teachers, schools, preK-12 administrators, teacher educators, STEM education researchers, and policymakers as appropriate. Applicants are encouraged to bring the same levels of insight and creativity to the dissemination aspect of their proposal as they do to their educational research and development design. Projects will be expected to share research and development designs, findings, and overall project information with the DRK-12 Resource Network and DRK-12 program evaluators.

6. Expertise

DRK-12 projects generally involve interdisciplinary teams. In all cases, proposals must describe the expertise needed for the work, how this expertise is incorporated in the project, and who is responsible for each component. Projects typically include STEM education researchers, development experts, experienced teachers, STEM researchers, statisticians, psychometricians, informal learning experts, and policy researchers, as appropriate. When feasible, projects should include future researchers and developers (e.g., beginning scholars, postdoctoral associates, graduate students) as part of the project team as a means of building a more diverse community of researchers and developers. Proposals should include a brief narrative describing the expertise of personnel and their contributions to the proposed work, including the project evaluator. Each proposal that requests funding to support postdoctoral researchers must include, as a supplementary document, a description of the mentoring activities that will be provided for such individuals. The mentoring plan must not exceed one page.

In addition, all resources, models and tools developed by DRK-12 projects must be reviewed by qualified experts in relevant STEM disciplines (e.g., scientists, mathematicians, engineers) and in STEM pedagogy. This may be done by an advisory committee with appropriate expertise whose members may be from the same or different institutions as the project.

7. Biographical Sketches (max. 2 pages)

All activities funded under this solicitation must include biographical sketches for all key personnel. Biographical sketches are limited to two pages each and formatting must comply with the most current Grant Proposal Guide. Biographical sketches should be sufficiently detailed to show that the necessary expertise is available to conduct the project.

8. Special Information/Supplementary Documentation:

Supplementary documentation is restricted to four document types.

(1) Brief letters of commitment or collaboration -- for example, letters from participating schools or advisory panel members, and

(2) A one-page list of staff, affiliations and partner institutions.

(3) Postdoctoral Researcher Mentoring Plan - If the proposal requests funding for a post-doc, a one page mentoring plan must be included in the supplementary documents as per the Grant Proposal Guide (https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg).

(4) Data Management Plan - As per the Grant Proposal Guide (https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg), all proposals must describe plans for data management and sharing of the products of research, or assert the absence of the need for such plans. FastLane will not permit submission of a proposal that is missing a Data Management Plan. Check the following website for additional information and a link to Frequently-Asked questions (FAQs)on this requirement: https://www.nsf.gov/bfa/dias/policy/dmp.jsp.

Proposals with other supplementary material will be returned without review.

B. Budgetary Information

Cost Sharing: Inclusion of voluntary committed cost sharing is prohibited

Budget Preparation Instructions:

A careful and realistic budget in accordance with the general guidelines contained in the NSF Grant Proposal Guide and consistent with the proposed activities of the project should be included. The budget for the total amount of money requested from NSF, with information on salaries and other expenses, including but not limited to, equipment (where allowable), participants, consultants, travel, sub-awards, and indirect costs must be provided. The Budget Justification section should include a budget narrative that describes and validates each of the expenses, including the hourly rate and effort expected from each consultant. DRK-12 proposals generally do not fund equipment that is normally found in schools, universities, and research and development organizations, such as computers.

Requests for equipment must be accompanied by justification for its importance to the operation of the project. In addition to the above budgetary items, the budget should include a request for funds to cover the cost of attendance of the Principal Investigator at each year's annual awardees meeting in the Washington, DC area.

Please note that as a general policy, NSF limits salary compensation for senior project personnel to no more than two months of their regular salary in any one year. This limit includes salary compensation received from all NSF-funded grants. If the current and pending support documents for a proposal show individual senior personnel with more than 2 months of annual compensation expected, an explanation must be provided in the budget justification.

C. Due Dates

• Letter of Intent Due Date(s) (required) (due by 5 p.m. proposer's local time): 

       November 17, 2011

       October 04, 2012

• Full Proposal Deadline(s) (due by 5 p.m. proposer's local time):

       January 10, 2012

       December 06, 2012

D. FastLane/Grants.gov Requirements

• For Proposals Submitted Via FastLane:

  Detailed technical instructions regarding the technical aspects of preparation and submission via FastLane are available at: https://www.fastlane.nsf.gov/a1/newstan.htm. For FastLane user support, call the FastLane Help Desk at 1-800-673-6188 or e-mail fastlane@nsf.gov. The FastLane Help Desk answers general technical questions related to the use of the FastLane system. Specific questions related to this program solicitation should be referred to the NSF program staff contact(s) listed in Section VIII of this funding opportunity.

  Submission of Electronically Signed Cover Sheets. The Authorized Organizational Representative (AOR) must electronically sign the proposal Cover Sheet to submit the required proposal certifications (see Chapter II, Section C of the Grant Proposal Guide for a listing of the certifications). The AOR must provide the required electronic certifications within five working days following the electronic submission of the proposal. Further instructions regarding this process are available on the FastLane Website at: https://www.fastlane.nsf.gov/fastlane.jsp.

• For Proposals Submitted Via Grants.gov:

Before using Grants.gov for the first time, each organization must register to create an institutional profile. Once registered, the applicant's organization can then apply for any federal grant on the Grants.gov website. Comprehensive information about using Grants.gov is available on the Grants.gov Applicant Resources webpage: http://www07.grants.gov/applicants/app_help_reso.jsp. In addition, the NSF Grants.gov Application Guide provides additional technical guidance regarding preparation of proposals via Grants.gov. For Grants.gov user support, contact the Grants.gov Contact Center at 1-800-518-4726 or by email: support@grants.gov. The Grants.gov Contact Center answers general technical questions related to the use of Grants.gov. Specific questions related to this program solicitation should be referred to the NSF program staff contact(s) listed in Section VIII of this solicitation.

Submitting the Proposal: Once all documents have been completed, the Authorized Organizational Representative (AOR) must submit the application to Grants.gov and verify the desired funding opportunity and agency to which the application is submitted. The AOR must then sign and submit the application to Grants.gov. The completed application will be transferred to the NSF FastLane system for further processing.

VI. NSF Proposal Processing And Review Procedures

Proposals received by NSF are assigned to the appropriate NSF program where they will be reviewed if they meet NSF proposal preparation requirements. All proposals are carefully reviewed by a scientist, engineer, or educator serving as an NSF Program Officer, and usually by three to ten other persons outside NSF who are experts in the particular fields represented by the proposal. These reviewers are selected by Program Officers charged with the oversight of the review process. Proposers are invited to suggest names of persons they believe are especially well qualified to review the proposal and/or persons they would prefer not review the proposal. These suggestions may serve as one source in the reviewer selection process at the Program Officer's discretion. Submission of such names, however, is optional. Care is taken to ensure that reviewers have no conflicts of interest with the proposal.

A. NSF Merit Review Criteria

All NSF proposals are evaluated through use of the two National Science Board (NSB)-approved merit review criteria: intellectual merit and the broader impacts of the proposed effort. In some instances, however, NSF will employ additional criteria as required to highlight the specific objectives of certain programs and activities.

The two NSB-approved merit review criteria are listed below. The criteria include considerations that help define them. These considerations are suggestions and not all will apply to any given proposal. While proposers must address both merit review criteria, reviewers will be asked to address only those considerations that are relevant to the proposal being considered and for which the reviewer is qualified to make judgements.

What is the intellectual merit of the proposed activity?
How important is the proposed activity to advancing knowledge and understanding within its own field or across different fields? How well qualified is the proposer (individual or team) to conduct the project? (If appropriate, the reviewer will comment on the quality of the prior work.) To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts? How well conceived and organized is the proposed activity? Is there sufficient access to resources?

What are the broader impacts of the proposed activity?
How well does the activity advance discovery and understanding while promoting teaching, training, and learning? How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)? To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships? Will the results be disseminated broadly to enhance scientific and technological understanding? What may be the benefits of the proposed activity to society?

Examples illustrating activities likely to demonstrate broader impacts are available electronically on the NSF website at: https://www.nsf.gov/pubs/gpg/broaderimpacts.pdf.

Mentoring activities provided to postdoctoral researchers supported on the project, as described in a one-page supplementary document, will be evaluated under the Broader Impacts criterion.