Use multipart/mixed for notating a sequence of documents

[awwright]

The document at https://w3c-cg.github.io/rsp/spec/messages and especially many of the issues within it seems to struggle with how to concatenate multiple documents together while still preserving the EOF (end of file) semantics of each of them. (Issue 9)

It also appears to struggle with how to refer to the resulting media type (Issue 10), unfortunately you cannot just mint new media type identifiers on a whim. In particular, the identifiers such as application/rdfm+xml and application/x-ld+ndjson are not legal. For experimental use, use the "personal" or perhaps "vendor" subtrees as described in RFC 3838.

For concatenating together multiple documents into a sequence, use the multipart/mixed media type. This is a very natural format for RDF+XML and JSON-LD because these documents cannot start with the -- delimiter that multipart encoding uses, so you can use an empty boundary name. It is even technically possible to mix together media types:

---
Content-type: application/ld+json

{"@context": "https://schema.org/", "@type": "Product", "name": "Hairdryer DRY 2000", "color": "red"}
---
Content-type: application/ld+json

{"@context": "https://schema.org/", "@type": "Product", "name": "27-inch 4K Monitor 2700P", "color": "black"}
---
Content-type: application/rdf+xml

<?xml version="1.0" encoding="utf-8"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:schema="http://schema.org/"><rdf:Description rdf:nodeID="b3"><rdf:type rdf:resource="http://schema.org/Product"/><schema:name>Hairdryer DRY 2000</schema:name><schema:color>red</schema:color></rdf:Description></rdf:RDF>
---
Content-type: application/rdf+xml

<?xml version="1.0" encoding="utf-8"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:schema="http://schema.org/"><rdf:Description rdf:nodeID="b3"><rdf:type rdf:resource="http://schema.org/Product"/><schema:name>27-inch 4K Monitor 2700P</schema:name><schema:color>black</schema:color></rdf:Description></rdf:RDF>
-----

[Ostrzyciel]

Hey @awwright, thank you for your interest in our work! :)

Please bear in mind that this document is a work in progress, and we haven't yet had time to properly discuss these issues.

It also appears to struggle with how to refer to the resulting media type (Issue 10), unfortunately you cannot just mint new media type identifiers on a whim. In particular, the identifiers such as application/rdfm+xml and application/x-ld+ndjson are not legal. For experimental use, use the "personal" or perhaps "vendor" subtrees as described in RFC 3838.

I hope that when the IANA police comes to arrest me for these highly illegal acts, they will consider the fact that this is a draft document which is (for now) just a way to organize a discussion. :) The document does not claim that it is a standard, and we do realize that real media type identifiers must be registered. Please let me know if there are any specific things we should do to keep the rest of the Internet community happy with our little experiment.

We haven't yet seriously discussed media types. These identifiers are just something we made up quickly during a discussion.

For concatenating together multiple documents into a sequence, use the multipart/mixed media type. This is a very natural format for RDF+XML and JSON-LD because these documents cannot start with the -- delimiter that multipart encoding uses, so you can use an empty boundary name.

Thank you for the suggestion! We will look into that. However, for other formats (e.g., Turtle, N-Triples) we will likely simply propose extensions of these formats with added support for message boundaries – so this would concern only RDF/XML and JSON-LD.

It is even technically possible to mix together media types:

That's interesting, but not at all something that we want to do. We want to have only either a series of JSON-LD documents or RDF/XML documents, not both at the same time.

Do I understand correctly that with the multipart/mixed media type we would have to restate the Content-type header for every document in the stream? That sounds wasteful, and it would complicate parsing such a stream. Our initial shot was at line-delimited formats because they are so utterly simple.

[awwright]

Please let me know if there are any specific things we should do to keep the rest of the Internet community happy with our little experiment.

The RFC I linked to discusses how to approach this. Work affiliated with a standards organization can register provisional media types in the standards tree though I'm not sure if community groups qualify. In any event, the unregistered tree doesn't require any coordination.

Do I understand correctly that with the multipart/mixed media type we would have to restate the Content-type header for every document in the stream?

The multipart encoding has no technical reason to prohibit mixing of media types, though you're not required to accept that if you don't want to.

Repeating the header isn't significantly more overhead than e.g. scanning for an @message or LF delimiter; it's possible to express as a regular expression the same way @message is. The multipart format makes it possible to begin reading from any location in the stream, even if you don't know where the next part begins or anything about the content before it.