Request for Proposals: Mephisto Dataset Collection Tool

About

Meta AI is pleased to invite university faculty to respond to this call for research proposals for Mephisto, a dataset collection tool.

Most novel machine learning research projects find themselves limited in having the right data for the project. Data collection can be tricky to get right and even with a solid task, it’s often difficult to ensure that data quality is up-to-par. Crowdsourcing best practices are often ‘tribal knowledge,’ leading researchers to rediscover ways to collect quality data across labs and domains.

Mephisto is a tool that Meta AI Research uses to standardize and codify best practices and infrastructure for research data collection and annotation. It also serves as a clear method to open source and share our methodology, such that others can reproduce or expand upon our work. Now that we’ve hit our 1.0 release, we feel like the platform is mature and stable enough for general use. We are opening Mephisto up to the community to ensure we’re building infrastructure that is useful for your use cases.

To foster further innovation in this area, aid in integration work for early adopters, and deepen our collaboration with academia, Meta AI is pleased to invite university faculty to submit research proposals pertaining to Mephisto. A total of 4 awards are available, in the range of $25,000 - $37,500 each, including overhead in an amount up to 40% of project costs. Funding will be provided to RFP winners pursuant to a Sponsored Research Agreement (SRA) containing open science terms. Please note that the terms of the SRA will not be subject to negotiation. We strongly encourage researchers from diverse backgrounds and of diverse abilities to apply.

The deadline for submissions is March 23, 11:59pm PT.

Areas of Interest

Meta AI encourages creative submissions that leverage the Mephisto tool, with areas of interest that include, but are not limited to, the following data collection tasks:

  • Entirely novel and complex tasks that may require rounds of iteration to design and experiment with.

  • Tasks that involve live interaction between multiple workers in some form, either collaborative or competitive.

  • Tasks that benefit from having a model-in-the-loop to assist workers.

  • Tasks that require intricate worker filtering specifications, for pairing, ensuring specified groups, or otherwise.

  • Tasks that can be built on existing frameworks, but may benefit from incorporating support for that framework into Mephisto.

Requirements

Prior to submitting a proposal, please confirm that your institution will agree to a contract for sponsored research from Meta. The agreement will operate under “Open Science” terms meaning that all research will be available for publication in the public sphere. Application materials include:

  • A summary of the project (1-2 pages), in English, explaining the target dataset and its value to your research area, a description of techniques, any relevant prior work, and a timeline with milestones and expected outcomes;

  • A draft budget description (1 page) including an approximate cost of the award and explanation of how funds would be spent;

  • Curriculum Vitae for all project participants; and

  • Organization details; this will include tax information and administrative contact details.

Eligibility

  • Proposals must comply with applicable US and international laws, regulations and policies;

  • Applicants must be current full-time faculty at an accredited academic institution that awards research degrees to PhD students;

  • Applicants must be the Principal Investigator on any resulting award;

  • Meta cannot consider proposals submitted, prepared or to be carried out by individuals residing in, or affiliated with an academic institution located in, a country or territory subject to comprehensive U.S. trade sanctions; and

  • Government officials (excluding faculty and staff of public universities, to the extent they may be considered government officials), political figures, and politically affiliated businesses (all as determined by Meta in its sole discretion) are not eligible.

Terms and Conditions

Please read these terms carefully before proceeding.

Meta’s decisions will be final in all matters relating to Meta RFP solicitations, including whether or not to grant an award and the interpretation of Meta RFP Terms and Conditions. By submitting a proposal, applicants affirm that they have read and agree to these Terms and Conditions.

  • Meta is authorized to evaluate proposals submitted under its RFPs, to consult with outside experts, as needed, in evaluating proposals, and to grant or deny awards using criteria determined by Meta to be appropriate and at Meta’s sole discretion. Meta’s decisions will be final in all matters relating to its RFPs, and applicants agree not to challenge any such decisions.

  • Meta will not be required to treat any part of a proposal as confidential or protected by copyright, and may use, edit, modify, copy, reproduce and distribute all or a portion of the proposal in any manner for the sole purposes of administering the Meta RFP website and evaluating the contents of the proposal.

  • Personal data submitted with a proposal, including name, mailing address, phone number, and email address of the applicant and other named researchers in the proposal may be collected, processed, stored and otherwise used by Meta for the purposes of administering Meta’s RFP website, evaluating the contents of the proposal, and as otherwise provided under Meta’s Privacy Policy.

  • Neither Meta nor the applicant is obligated to enter into a business transaction as a result of the proposal submission. Meta is under no obligation to review or consider the proposal.

  • Feedback provided in a proposal regarding Meta products or services will not be treated as confidential or protected by copyright, and Meta is free to use such feedback on an unrestricted basis with no compensation to the applicant. The submission of a proposal will not result in the transfer of ownership of any IP rights.

  • Applicants represent and warrant that they have authority to submit a proposal in connection with a Meta RFP and to grant the rights set forth herein on behalf of their organization. All awards provided by Meta in connection with this RFP shall be used only in accordance with applicable laws and shall not be used in any way, directly or indirectly, to facilitate any act that would constitute bribery or an illegal kickback, an illegal campaign contribution, or would otherwise violate any applicable anti-corruption or political activities law.

  • Funding for winning RFP proposals will be provided to the academic institution with which the applicant is affiliated pursuant to a sponsored research agreement (“SRA”) between Meta and such academic institution. Applicants understand and acknowledge that their affiliated academic institution will need to agree to the non-negotiable terms and conditions of the SRA to receive funding.

  • Applicants must disclose any current or past collaborations with Meta researchers that are related to their RFP project proposal..

To apply

Please submit proposal materials here.

Important Dates:

  • Launch date: February 16th

  • Deadline: March 23rd, 11:59pm PT

  • Est. notification date: June 1st

Please reach out to mephisto-aiproposals@fb.com if you have any questions.