List of Stages

Although the specific needs of each Working Group (WG) and Task Force (TF) may vary, the following table describes the recommended stages in the development of a ToIP deliverable (e.g., a specification, guide, template, or white paper):

† Stages officially recognized in the Linux Foundation Joint Development Foundation process: "Pre-Draft", "Draft", "Working Group Approved", and "ToIP Approved" deliverable. 

Stages 1 thru 6 (Prior to Working Drafts)

It is understandable that many WGs or TFs (collectively, "groups") may want to get right to work, i.e., pick up the pen and start drafting. Some may even have been started based on a contributed document or specification. However we strongly recommend that the group still take the time to deliberately go through stages 1 through 6 in order to make sure all group members are aligned and working from the same conclusions about the mission, scope, problem definition, and requirements. It is also highly recommended that all group members have the opportunity to present their own proposed solutions. This is the best way to build towards consensus in the Consolidation stage prior to formally commencing Working Drafts.

Stage 7: Working Drafts

When the group is ready to begin drafting, it must decide which collaboration tools it will use. ToIP offers two choices:

Google Docs

This option is recommended if the group wants to minimize barriers for contributors. It can work well for non-technical deliverables, where intellectual property rights (IPR) for contributions are usually not an issue. In addition, Google docs is a low-friction tool for the early stages of drafting technical specifications—though it is still important to keep a clear record of which ToIP members make which contributions. Note, however, that this option is not yet supported by our Terminology Engine tools (produced by our Concepts and Terminology WG) for automatic linking/annotation of glossary terms (see below).

Guidelines:

1. Any Google Docs used should be stored in the ToIP Shared Drive for your WG (or in a subfolder of the WG for your TF).  This allows you to assure that any contributions are being made by ToIP Members who have access to these shared drives. 
2. No Google Docs used for the creations of ToIP deliverable should be stored outside of the ToIP Shared Drives.
3. Any deliverable whose entire production uses Google docs should be converted into the <TODO—add link>ToIP Word Template<end-link> before submission for being submitted for approval as a Working Group Approved Draft (stage 10).
4. If a WG or TF member has difficulty accesses the ToIP Shared Drive, please contact Michelle Janata, ToIP Program Manager or Judith Fleenor, ToIP Executive Director, for assistance.

GitHub Markdown documents

This option MUST be used for the final form of all ToIP technical specifications. It is also RECOMMENDED for all other ToIP deliverables whenever possible because It has the advantage of keeping a strict, verifiable date-stamped record of all contributions by author. It is also RECOMMENDED as the final form for any ToIP deliverable that wishes to take advantage of our Terminology Engine tools (produced by our Concepts and Terminology WG) for automatic linking and annotation of glossary terms.

Guidelines:

1. Contact your Working Group chair(s) or Michelle Janata, ToIP Program Manager ,or Judith Fleenor, ToIP Executive Director, for assistance in setting up the GitHub repo you will need for managing your deliverable.
2. See <TODO—add link>these instructions<end-link> which explain how to set up your repo and begin using the ToIP Markdown Template for your deliverable.

Working Draft Subtitle and Versioning

Regardless of whether Google Docs or GitHub Markdown is used, the document should carry the subtitle "Working Draft" followed by a two-digit number representing the Working Draft version, starting with 01. For example: "Working Draft 01", "Working Draft 02", "Working Draft 03", etc.

Stage 8: Implementers Drafts (Optional)

If a deliverable is a technical specification, it is recommended to follow the IETF dictum of "rough consensus and running code", i.e., provide evidence of completeness by demonstrating multiple interoperable implementations. The best way to do this is to publish at least one Implementers Draft that developers can actively use to implement and provide feedback. Depending on the complexity of the spec, more than one version of the Implementers Draft may be published, and the cycle will typically last at least few months, or even a year or more.

Note that explicitly labeling a specification as an Implementers Draft sends a clear signal that the spec is ready for implementation feedback—and that the group values that feedback before the spec goes final.

Implementers Draft Subtitle and Versioning

The document should carry the subtitle "Implementers Draft" followed by a two-digit number representing the Implementers Draft version, starting with 01. For example: "Implementers Draft 01", "Implementers Draft 02", "Implementers Draft 03", etc. The Implementers Draft version numbering is entirely independent of the Working Draft version number. For example, your group could transition from "Working Draft 07" to "Implementers Draft 01".

Stage 9: Public Review Drafts

Before a Working Draft or Implementers Draft can be submitted for approval as a Working Group Approved Draft, it MUST be subject to at least one public review of at least 15 days (30 to 60 days is RECOMMENDED). Either a Working Group or Task Force can trigger a public review draft. If the deliverable is a technical specification, it is RECOMMENDED to wait until the group has received sufficient implementation feedback. For any other deliverable, it is up to the group to decide when the deliverable is ready for public review.

Guidelines:

1. The document MUST carry the subtitle "Public Review Draft" followed by a two-digit number representing the Public Review Draft version, starting with 01. For example: "Public Review Draft 01", "Public Review Draft 02", "Public Review Draft 03", etc. The Public Review Draft version numbering is entirely independent of the Working Draft or Implementers Draft version number. For example, your group could transition from "Working Draft 11" to "Implementers Draft 02" to "Public Review Draft 01".
2. The deliverable MUST be posted to a ToIP GitHub repo (even if it is a PDF of a Google doc or Word doc).
3. The public review process MUST use GitHub Issues.
4. All feedback from non-ToIP members MUST use the Feedback license as specified by the Joint Development Foundation.
5. The group MUST publish a ToIP blog post announcing the public review, explaining the background and purpose of the deliverable, and providing instructions on how anyone (both ToIP members and non-members) can provide feedback during the public review period. See this wiki page for guidance on writing the blog post.
6. The ToIP Working Group Chair or Executive Director MUST send a notice of the public review, including a synopsis of the deliverable and a link to the blog post, to the ToIP All-Members mailing list and to the ToIP Steering Committee.
7. The official Public Review period MUST begin no earlier than the day the blog post is published and the email is sent.
8. The group MUST process all feedback received by either incorporating it into the deliverable or explaining why it was not incorporated by posting one or more responses to the GitHub issue.
9. If feedback is incorporated into the deliverable, the group MAY decide to hold one or more additional rounds of public review. If so, those additional rounds MUST follow the same rules as the first one.

Stage 10: WG Approved Draft

Once public review is complete and the Working Group or Task Force has responded to all issues raised, the deliverable is ready to be submitted to the Working Group for approval as a Working Group Approved Draft. This approval MUST be given via a formal call for consensus using one of two options:

1. A meeting of the Working Group for which at least one week's notice has been given of the planned call for consensus, OR
2. An email sent to the Working Group's mailing list that allows at least seven days for any WG member to reply with an objection. This email MUST state:
   1. The formal name for the deliverable which MUST conform to these requirements.
   2. A link to the final Public Review Draft to be approved. This MUST be a link to the WG or TF GitHub repo. If the deliverable is a Google doc or Word doc, that doc MUST be posted to the repo as a PDF. Otherwise, it MUST be a link to the Markdown document.
   3. A synopsis of the deliverable (which can be the same as the synopsis for the public review).
   4. A link to the blog post announcing the public review.
   5. A link to a GitHub issue that summarizes the resolution of all feedback received during the public review process.
   6. A link to this wiki page explaining the deliverable process.
   7. A call for consensus stating: "This is a formal call for consensus of the members of the <name of WG> for approval of <name of deliverable> at <link to final Public Review Draft> as a Working Draft Approved Deliverable."
   8. Instructions stating that any WG member who wishes to object MUST respond to the email publicly stating their objection, and that if no objections are received in seven calendar days, the deliverable shall be approved.

If no objections are received during the call for consensus, the deliverable SHALL be approved. If a call for consensus receives an objection which cannot be resolved via discussion, then the Working Group MUST hold a formal ballot following the same rules above. To be approved at a meeting of the Working Group, a majority (51%) of Working Group members present MUST vote in favor of approving the deliverable. To be approved via an email ballot, a majority (51%) of Working Group members on the mailing list MUST vote in favor of approving the deliverable.

Stage 11: ToIP Approved Draft

Once a deliverable is approved as a Working Group Approved Draft, the Working Group MAY elect to submit the deliverable to the ToIP Steering Committee (SC) for approval as a ToIP Approved Draft.

If a majority of the Working Group is in favor of that step:

1. The Working Group MUST prepare a PDF of the Working Group Approved Deliverable in the proposed form to become a ToIP Approved Deliverable. <TODO: add link>See these preparation guidelines.<end-link>
2. The Working Group chair(s) MUST send a copy of the PDF to the ToIP Executive Director together with a request for approval as a ToIP Approved Deliverable and a link to either:
   1. The Meeting Notes for the meeting where the deliverable was approved as a Working Group Approved Deliverable, or
   2. A copy of the email call for consensus that was approved by the WG.
3. If the request meets these requirements, the ToIP Executive Director MUST forward the request to the ToIP (SC) at least two weeks in advance of the next ToIP Steering Committee (SC) meeting.

After receiving notice, the SC MAY choose to take up to a total of 30 days to hold a call for consensus for approving the deliverable.

If consensus is not achieved, the Steering Committee MUST hold a formal ballot. Formal ballot rules for an SC vote follow the same rules as for a WG vote.

Key Process Flows

The following diagrams are intended to help understand the process by which the various deliverables move through the JDF and ToIP deliverable processes.

High-Level

The LF/JDF system recognizes the following deliverables:

• Draft Deliverable
• Working Group Approved Deliverable 
• ToIP Approved Deliverable

The "Pre-Draft" stage does not form a deliverable yet, but the stage is critical in forming a deliverable. 

Draft to Working Group Approved

This is the workflow that has the longest period of work - both technically and socially. It is all about getting the deliverable into a formal state. Depending on the deliverable type this process may be relatively lightweight (e.g. whitepaper) or very heavyweight (e.g. specification). 

Working Group Approved to ToIP Approved