Vocabularies and "format"

[handrews]

We have numerous requests for additional formats, or ideas that could be implemented with formats (see #152, #312, json-schema-org/json-schema-vocabularies#45, json-schema-org/json-schema-vocabularies#49, #542).

There have also been a number of discussions around the implementation requirements, particularly

they SHOULD implement validation for [all] attributes defined below

which becomes more burdensome as we add more formats. One idea has been to say that you only need to implement certain subsets (e.g. you could implement the date and time formats but skip the URI/IRI formats, but you shouldn't just implement uri-reference and not uri). But there's no good way to convey support levels. Which brings us to:

Save for agreement between parties, schema authors SHALL NOT expect a peer implementation to support this keyword and/or custom format attributes.

which makes interoperability very challenging. format is really only reliably useful as a semantic annotation, not as validation.

We need a better story here, if for no other reason than to figure out how to manage the endless stream of requests for standardized formats. Currently, that's the only way to achieve any level of interoperability, so there is a high motivation to push for inclusion.

---

If we go with vocabulary support along the general lines of #561, I feel like this should help manage the variations. A vocabulary could include specific format values (and likewise for contentType and contentEncoding, I suppose).

I've split this out from #561 because it's not clear how it would work. Using a meta-schema, as #561 proposes, doesn't work well for format values because enum is difficult to combine. The typical allOf used to compose vocabularies produces the intersection of enums rather than the union. But anyOf, which would work, would often produce unexpected behaviors otherwise.

Some other mechanism may be required. Therefore this is filed in the future milestone as a question. If we come up with a broadly supported proposal quickly, it can be added to draft-08, but otherwise it's fine as a follow-on in a later draft.

[ralfhandl]

Formats usually serve the dual purpose of a programming-language-independent code-generation hint for the recipient and a production/validation instruction for the sender or an intermediate validator, so pulling format out of validation seems to be the right direction.

Allowing vocabularies to define specific format values would certainly reduce the pressure for adding new formats to a central list.

It would also remove the single point of reference for looking up existing formats and their meaning.

Unless that vocabulary mechanism is accompanied by a central "registry" or "repository" for format definitions.

The "endless stream of requests for standardized formats" just shows how important interoperability is.

[handrews]

The "endless stream of requests for standardized formats" just shows how important interoperability is.

Agreed. I think for a central list, we should look to the IANA registry model. For now, we'll keep a small standard set in one of the major specification drafts.

But if vocabularies are identified by URIs, and we define some clear way for a vocabulary to define format values (and similar extensible value sets), then we can probably use vocabulary URIs with fragments to completely identify a format term.

[ralfhandl]

Sounds like a plan: 👍

[awwright]

The link relations IANA registry has a nice model, either:

• expert review for new entries; or
• anyone define a format by a URI within a namespace they own

Either form can be used by anyone.