March 9, 2018

Dear Colleagues:

The National Science Foundation's (NSF) Directorate for Social, Behavioral and Economic Sciences (SBE) encourages submission of proposals that target reproducibility and replicability efforts in data-intensive domains and that specifically rely on analysis of neuroimaging or neuroelectric data, including but not limited to electroencephalography, magnetoencephalography, electrocorticography and functional neuroimaging. SBE considers these areas as targets for support across several content domains, given increased cognizance of potential concerns about analytic assumptions and derived workflows and increased community awareness of the need to define and publicize best practices for analyzing, documenting, managing and disseminating large datasets. These activities are consistent with SBE's long-term mission of stimulating research and other activities to enhance the robustness and reliability of research as mentioned in NSF's DCL 16-137, "Robust and Reliable Research in the Social, Behavioral, and Economic Sciences", which highlights the importance of reproducibility, replicability and generalizability. The proposed projects should target core questions in the content domains considered by the NSF programs mentioned below, as described in the relevant program guidelines. They should also be consistent with DCL 16-137, but address the specific interests of communities whose work involves the study of the human brain function.

The current DCL encourages the submission of proposals that use replication, reproduction and generalization for the purpose of testing new ideas. Research questions should fall within the content domain of any of the following NSF programs: Cognitive Neuroscience, Perception, Action, and Cognition, and Science of Learning. NSF expects that these activities will aid in verification of prior findings, disambiguate among alternative hypotheses and serve to build a community of practice that engages in thoughtful reproducibility and replicability efforts. The suggested research must demonstrate clear potential for generating new scientific advances and discoveries, beyond simply rejecting or corroborating prior findings.

SBE particularly encourages the submission of proposals for the following types of projects:

1. Replication of several different studies with the same individuals in order to discover shared latent structures (of brain activity and behavior) within individuals and across tasks. Collection of new behavioral, neural, physiological or other data will be viewed positively.
2. Replication of pivotal or controversial studies but with sample sizes substantially larger than in the original studies or using different sampling strategies. Such proposals should clarify the relationship between sample size and measurement stability and should enable understanding or modeling of individual differences related to psychological strategies and neural systems underlying the task. Ideally, these activities should provide insights into eventual replication failures.
3. Generalization of findings to other populations or contexts, e.g., from participants to populations, across measures, contexts, circumstances, or cultures.
4. Evaluation of the impact of parameter choice, analysis toolchain, and workflows on the results reported and conclusions drawn.

Specific guidance to proposers responding to this DCL

All proposals should: (1) make a strong case for the studies chosen for replication, reproduction or generalization; (2) substantiate the chosen analytic strategy; and (3) present a plan or template for evaluating, documenting, and communicating the lessons learned during the work. Ideally, submitted proposals should consider various analytic criteria for determining successful replication and argue which is most applicable in the context of the suggested research.

Successful research proposals will set a standard for rigor in conducting data intensive research and serve as models for further replication and reproducibility projects. Proposers should therefore describe how data, analysis code and results from these experiments will be shared with the community. Use of tools that enable automated capture and recording of experimental and analysis details, such as workflow languages or engines, is encouraged. To allow replication of the supported work, researchers are encouraged to use interoperable data formats, consider establishing interfaces, and to leverage existing open-source analysis environments. Pre-registration of data collection and analysis are also encouraged.

Proposals submitted in response to this DCL should be submitted to the June 11, 2018 target date for the Cognitive Neuroscience program (PD 15-1699). The proposal title should include the prefix NeuroDataRR. It is anticipated that awards will provide three to four years of support for projects whose total budget does not exceed $600,000. Awards are anticipated to be made in Fiscal Year 2018. Potential proposers are strongly encouraged to contact one of the Program Officers listed below prior to submission to ensure that the research topic falls within the scope of the program.

Points of Contact:
Uri Hasson (uhasson@nsf.gov), Cognitive Neuroscience Program Director
Kurt Thoroughman (kthoroug@nsf.gov) or Soo-Siang Lim (slim@nsf.gov), Science of Learning Program Directors
Betty Tuller (btuller@nsf.gov) or Larry Gottlob (lgottlob@nsf.gov), Perception, Action, and Cognition Program Directors

Sincerely,
Dr. Fay Lomax Cook
Assistant Director
Directorate for Social, Behavioral & Economic Sciences